double blind testing...

A succesful DBT would allow us to put distance between reports of cable directionality and alien abduction, (where the number of the latter rather outnumber the former...)

IMO this would be a good thing for the mental wellbeing of the world.

Paul

Paul, how would you measure success

laurie S

Ross appears to believe that we are unable to reliably report our own experiences of hearing in such a way that would allow researchers to gather meaningful data. He seems to have faith in 'the numbers'. Moreover, whilst he can accept that low end speaker manufacturers may have done research into buyers listening preferences, and concluded that loud and boomy is the way to go, he doesn't seem to have considered that perhaps hi-end manufacterers may do the same, but get different results. I am also lost as to why people would be able to report differences in speakers, but not racks.

Ross concludes that people just love buying snake oil, poorly evidenced, or not. If the aim is to stop people wasting money on trinkets, conducting double blind randomised controlled trials of hi fi components in such a way as to not violate ceteris paribus assumptions would seem to be a monumental waste of time. If on the other hand, Ross is wrong, and we do have access to our perception of music reproduction in such a way that allows us to report it reliably, you would think it would allow interested parties to bring products to market that people actually enjoy listening to.

Or maybe it is all in the numbers, I have no idea.

"If on the other hand, Ross is wrong, and we do have access to our perception of music reproduction in such a way that allows us to report it reliably"

My understnading is that Ross does believe this is possible but it would still be a waste of time as either the results would show that the "obvious differences" really do exist or else, if the differences didn't show up, the vast majority would ignore the experiments and carry on as before. Ie. when he says its pointless or a waste of time he is not questioning whether the science would be valid but rather is there any point in doing it at all.

Also if one were to beleive that our perception of the music couldn't be reliably accessed in a carefully designed experiment its not clear how anyone would be able to rely upon it in real life. Consequently one might just as well choose cables on the basis of colour, hosepipe like width, cool looking heat shrink, marketing bumf telling you about how the cable was annointed with lobster sperm while beig subjected to articfical gravity equivalent to be bing on the surface of Jupiter, etc. Which -- in the case of the more fanicful claims and some proportion of the hi-fi public -- is what I suspect may well be happening.

Matthew

quote:

---

Paul, how would you measure success

---

A statistician would say 'this test shows that these people could hear that this cable sounds different depending on direction of connection.'

It's possible that this would qualify for James Randi's $1000000 wager.

Paul

My point is, Paul, is what difference would this result make to a prospective purchaser, say, who could not then detect a difference?

laurie S

quote:

---

Originally posted by Matthew Robinson:
Consequently one might just as well choose cables on the basis of colour, hosepipe like width, cool looking heat shrink, marketing bumf telling you about how the cable was annointed with lobster sperm while beig subjected to articfical gravity equivalent to be bing on the surface of Jupiter, etc.

Matthew

---

That reminds me, I must get around to applying my Rainbow Foil to my LPs.

Paul, they'd say 'this falsifies the null hypothesis that cable direction cannot be detected, and that there is only a 5%, 1%, or .1% chance that rejecting the null hypothesis is an error.

Laurie, all things being equal, it would mean that they are in the 95th> percentile of being rubbish at hearing cable differences.

Mekon said "Laurie, all things being equal, it would mean that they are in the 95th> percentile of being rubbish at hearing cable differences"

Is that true? I would have thought such a result meant there was a 5% (or less) chance that the differences cannot be detected despite the experiment showing that they could (ie through an extraordinary sequence of radnom chance) rather than meaning that there are 5% of the population who cannot detect the difference.

Or conversely, if the test shows that the differences canot be detected then the interpretation is that its very likely the differences do not exist although there is a small chance that they do (e.g. we might have randomly selected a significant majority of people who are bad at testing cables).

Matthew

Sorry, getting carried away whilst my mind is elsewhere running ANCOVAs on exercise behaviour.

I've inadvertantly illustrated why you shouldn't let psychologists play with statistics, especially when they skipped lunch. Yes, what it shows is that if we ran the same test a 100 times, we'd get the same result 95 times.

Also, given we're talking about a dichotomous outcome variable, wasn't me talking about percentiles is daft in the extreme? I rarely use dichotomous outcome measures, I'd have to go back to my textbooks before I'd have much confidence in doing more than a chi-squared or a logistic regression on them. Actually, I'd prolly even sneak a look before I did the latter.

Mekon, in that case would it matter to them that they might have, say purchased the "wrong" cable?

laurie S

Of course not, it's their money, and it's it's only hi-fi, after all. If people want to buy magic stones, and whatnot, that's their lookout.

quote:

---

Originally posted by Mekon:
Jeremey, it's Pevensey (1)- it's the building that would be behind the Meeting House if you were standing at the top of the library stairs, looking out over library square. It's also to the right of Chichester 1 lecture theatre. Got it?

---

Thanks. Yes, that's MAPS 1. Chichester 1 is also an unfamiliar name so I imagine it was also called something else. [Sorry if this post is uninteresting to anyone else].

--J

Mekon, for the record all my purchases have been based solely on my own extensive (at home) appraisal,and I only write the cheque when I am sure something has something genuine to offer, regardless of what others say about it. Perhaps that is why I have never bought a CDS2 or LP12..... I find it surprising (perhaps I am naive) that anyone should act any differently

Laurie S

i need to catch up on all the postings on DBT. in the meantime, a question comes to mind and i would welcome some comments.

do you guys believe that there are people who are more "blessed" at telling differences between hifi components than others (the "cloth-eared" ones). the answer seems to be obviously YES, but (a) i am not too sure how one would demonstrate (prove?) this and (b) whether this has implications for DBT methodology. of course, one could presumably use DBT to establish which group one belongs to... and implications for reports we read here from fellow contributors, (including mine)

i used to be a customer of a very successful hifi shop in south london where one particular individual in that shop was reputed as being of "good ear"...

enjoy

ken

Laurie,

If you have a number of well desinged and executed tests that show, say, that the cable directionality of NAC5 cannot be reliably detected then you can say something like "it is almost certain that NACA5 direction cannot be reliably detected and you can therefore plug it in anyway you like as it doesn't really matter".

If an individual believes they can still hear this difference despite this evidence then you can say that it is much more likely that this is caused by an effect like expectation bias than by a real difference caused by cable directionality.

There is -- Ken -- the possibility that this person has astonishing bat like hearing and is therefore hearing a real difference that cannot be detected in the general population. To completely remove this possibility in an individual you would have to test that inidividual directly. Since its a rare ability though, if you found a lot of people claiming the ability -- say on a hi-fi forum -- it probably indicates some other effect (assuming non-directionality of A5 was well evidenced).

As always there exists the possibilty of poor design or execution of the experiment not elminiating some other factor. Which is why you need a number of indpendent experiments and proper peer review before you can start making anything other than tentative statenents.

Matthew

matthew, many thanks. i find it interesting that such "super-hearers" may exist, even on this forum. conversely, that some of us are cloth-eared, and we waste money buying stuff that is not (to us) audibly better than what we had before. frightening...!!!

any more thoughts?

enjoy

ken

DBT in action:

i found the following an interesting (if not cryptic) read. if i find any more links, i will post here.

http://www.pcavtech.com/abx/abx_pwr.htm

enjoy

ken

ps: after posting, i found another one -- from stereophile. probably the one mekon was referring to early on in this thread? its:

http://www.stereophile.com//features/141/index.html

You know, I never figured I would get to use signal detection theory. Anyone with bat-like ears fancy being tested on the direction of a cable?

Funnily enough, whilst I was refreshing my memory of ROC curves, I came across the following warning:

"If you wish to compare the area under two or more ROC curves it is best to consult with a statistician."

quote:

---

Originally posted by Mekon:
You know, I never figured I would get to use signal detection theory. Anyone with bat-like ears fancy being tested on the direction of a cable?

Funnily enough, whilst I was refreshing my memory of ROC curves, I came across the following warning:

"If you wish to compare the area under two or more ROC curves it is best to _consult with a statistician_."

---

mekon, what's an ROC curve?

enjoy

ken

It's the receiver operating characteristic curve. There are loads of useful guides to signal detection theory on the web. This one gives a nice summary.

mekon, many thanks...

enjoy


ken

quote:

---

'this falsifies the null hypothesis that cable direction cannot be detected, and that there is only a 5%, 1%, or .1% chance that rejecting the null hypothesis is an error.

---

Talk dirty some more!

Paul

i have just finished a quick read through of the stereophile articles and letters on DBT. fascinating stuff!!!

interesting quote from a contributor: The things the listener hears are as much a part of the listener as they are a part of the equipment. Withholding the identity of the equipment breaks the bond with the listener and the differences disappear.

the to-and-fro between dr leventhal and mr clark on type 1 and type 2 errors is worth looking at. i am more in sympathy with leventhals position -- although the issue of experimental design mentioned briefly by another contributor is quite important.

i now understand the background to the ABX device -- seems like a good idea -- but i would have a niggling feeling about its possible detrimental effects. there is a hint of protecting vested interests here and there in the various discussions.

so, no DBT-difference between amplifiers heh?? so, guys, never mind all these discussions on 250MkI, 250 MkII, 300, even 500?? wow !!! why even bother with mains and multiple spurs?? apparently, unless i have misread, there is even a claim that there is no statistically audible difference between cartridges!!!

i like the idea of DBT, i am not so sure i am happy with uses of it to date. in the same article you can see references to some DBT's done by Hifi News and RR -- i will try to dig these up.

i urge anyone who is interested in DBT to have a look at the stereophile articles and letters. if anything, it makes for an interesting read.

enjoy...

ken

andrewthomas:

Ken, it looks like you will soon abandon Hifi porn and become and associate member of the AES ($60/annum, online Journal included, discount on pre-prints too!)

why exchange free hifi porn for another, which i have to pay for?

enjoy

ken