double blind testing...

laurie, would you not find it at least interesting(just a tinny weeny bit...?) if DBT actually also confirmed your genuine findings that multiple spurs are better? i would, regardless of what i think/feel about the technique. you see, this sort of result is definitely not ruled out, especially if the differences are as "night and day" as reported. this will then be yet another source of "independent" evidence, if you like...

if DBT says "no difference" (or a more convoluted statement involving an appropriate null hyposthesis), then SO WHAT!!! it doesnt make you or anyone else who believes in multiple spurs more or less clever than anyone else contributing here.

i am finding both your, AND andrew's, contributions very interesting and thought provoking. there is nothing wrong with you being sceptical about DBT. and there is nothing wrong with andrew being sceptical about multiple spurs, or any other exotic claims we make in this lovely hobby of ours.

for me, for the time being, i am more interested in understanding what DBT IS, and then time allowing, using it, and making my own conclusions as what i think about it. in fact, if i ever get a chance to install multiple spurs, i will do the two tests -- normal "sighted" test, and then, provided i know how to proceed, a double blind test. if these two tests yield different conclusions --well, isn't life grand!!!


enjoy

ken

Matthew

At times Andrews stance is'nt just skeptical but down right dogmatic. If you cant prove to him in a DBT that there is a difference then that's the subject closed. A difference can't possibly exist.

He and yourself have been invited to members homes( in a previous thread on this subject)who have fitted spurs to listen for yourselves changing between single spur, multiple spurs and domestic ring but you both it seems have consistently refused the offer. This is a closed mind attitude and certainly not healthy skeptisism. I consider myself skeptical and have tried many tweaks reported on this forum and in the HIFI press and sometimes I detect no change, other times a change but not an improvement and thankfully at times a real improvement in the musical involvement that I get from my sistem. I would be hard pressed at times to say exactly what the improvement was in sonic terms, whether a change in frequency response, less distortion or whatever but I know that I prefer listening with the sistem setup one way than another. Most of the changes I have made over the years are all reversible and often when going back I realise that the change really was a positive one.

The simplest change I have made was suggested a few months ago utilising the electric spur to the cooker in the kitchen. It's never been used as I have a gas cooker. I used a 5mtr extension cable and plugged in the amplifiers, at the time still using the standard cables and plugs into a multiplug and into the extension socket. The change was IMHO quite noticeable and I spent the rest of the evening changing back and forth between Cooker spur and standard ring to elimanate the differences between a cold sistem and a warmed up one.

But unfortunately keeping it permanent isn't really a consideration for aesthetic/safety reasons and as reported recently I made up a hydra Home Made Hydrawhich was an improvement over the ring but not as much as the seperate spur. If I wasn't in a council house I would have gone down the seperate spur route by now and I may just do it one day as I am convinced that at least one spur would be quite an improvement and I now have no doubt that multiple spurs would be better still. I apologise for the length of this contribution but i cant think of a way to make my point in a less detailed way.

Yours in Music

Syd

There's a dbt for cable directionality being organised on diyaudio.com at the moment.

The thread is http://www.diyaudio.com/forums/showthread.php?s=&threadid=21828 but there's obviously quite a lot of fluff as well as the good stuff.

Paul

quote:

---

Originally posted by Mekon:
Syd, I presume you are specifically interested in experiences of DBTs related to Hi-Fi, right? If you are interest in what it's like taking part in DBTs generally, I have participated in a whole load of them. PM me if you are interested.

---

as you can see in the very first posting of this thread, it goes without saying that i am also very interested to hear accounts of someone who has actually taken place in a DBT. if you can post here for the benefit of everyone, better -- if not possible, i can discuss this with you "offline" ( ) if you like...

enjoy

ken

quote:

---

Originally posted by Mekon:
Syd, I presume you are specifically interested in experiences of DBTs related to Hi-Fi, right? If you are interest in what it's like taking part in DBTs generally, I have participated in a whole load of them. PM me if you are interested.

---

I have invited you to A Private Topic as I am very interested in DBTs.

Yours in Music

Syd

quote:

---

Originally posted by Paul Ranson:
There's a dbt for cable directionality being organised on diyaudio.com at the moment.

..... but there's obviously quite a lot of fluff as well as the good stuff.

Paul

---

paul, many thanks for that link. will check it out in a mo.

fluff and good stuff i guess are the reality of life... yin and yang and all that...

enjoy

ken

quick question to dbt experts:

are the sorts of conclusions one can reach as a result of dbt of the type:

" so and so cannot tell the difference between A and B"

or are they of the type:

"there is no statistical difference between A and B"


enjoy


ken

andrew, would you have hard copies of these papers that you could snail mail to me, if you dont mind?

"Fifty Years of Loudspeaker Developments as Viewed through the Perspective of the Audio Engineering Society" J. Audio Eng. Soc. Vol 46, No 1/2, 1998 Jan/Feb. Section 23 has a summary of the major works in "Subjective Evaluation and Assessment" and references to where you can find them.

Floyd Toole has done some excellent work in this area, you might want to start with "Subjective Measurements of Loudspeaker Sound Quality and Listener Performance" Floyd Toole JAES Vol 33 No 1/2 1985 Jan/Feb.

many thanks. if you prefer, you can email me on kenchakahwata at first-derivative dot com

enjoy

ken

Laurie -- I was suggesting that by having such a tenacious sceptic as Andrew you have been obliged to make your case more thoroughly than you might otherwise have done. Which is quite possibly to our benefit.

Syd -- My position (and my understanding of Andrew's position) is not that these effects do not exist but that the claims are quite extraordinary you can't just assume they exist on the basis of the subjective reports of listeners with obvious potential for bias. This fact is staggeringly obvious and reasonable to me and I am consistently astonished that it causes such controversy. Note that the exceptional nature of the claims is an essential part of this and you would not, for example, run into the same arguments if you claimed a more powerful amplifier produces better bass (or somesuch).

FWIW Its my impression that Andrew is no more dogmatic than the people who disagree with him.

"but you both it seems have consistently refused the offer. This is a closed mind attitude and certainly not healthy skeptisism"

Its not closed minded at all -- I am perfectly willing and open to the possibiliy that these effects are very real and are caused by a interesting new phenomenon as yet unknown to science. My point was that I am not that interested in going all the way to the other side of London to do some mains experiments. And, with particular reference to Tom's offer, if I did go to his home I'd much rather drink some wine, eat some cheese and play some music than sit around with Tom scuttling around behind the sofa swapping mains plugs over.

Matthew

I do not feel the need for DBTs(?) since the changes were so immediately obvious to me when I made them that I am not sure what extra benefit I would achieve from any DBT. Perhaps to confirm or deny what I already have experienced So what? As I said, I am not trying to prove anything. Just reporting what I did, and the results I percieved. I am not twisting anyones arm.

Laurie S
And Ken, it would not really make one iota of difference to me what the outcome of such tests were. The evidence I have (despite the scepticsm of others) is sufficient for me.

quote:

---

Originally posted by Laurie Saunders:
I do not feel the need for DBTs(?) since the changes were so immediately obvious to me when I made them that I am not sure what extra benefit I would achieve from any DBT. Perhaps to confirm or deny what I already have experienced So what? As I said, I am not trying to prove anything. Just reporting what I did, and the results I percieved. I am not twisting anyones arm.

Laurie S
And Ken, it would not really make one iota of difference to me what the outcome of such tests were. The evidence I have (despite the scepticsm of others) is sufficient for me.

---

great!!

i sense you feel that we are discussing DBT in the context of your experience with multiple mains spurs in particular. actually, that wasnt the intention when i started the thread. i wanted to (a) understand DBT in general, (b) find out where it has been used successfully in the area of hifi and (c) find out from people who have actually taken part in DBT.

of course, DBT could (presumably) be applied to multiple spurs testing, and given what has been reported here, my guess is that the results would confirm what you have found.

this thread was really meant as a (i hope) healthy exchange of ideas/challenges and not an opportunity to devalue other people's contributions. DBT is interesting to me. so are your reported experiences with multiple spurs, (for which there is a separate thread) independent of DBT. perhaps, given your disinterest in the subject of the thread, you should consider giving it a miss?? of course, i have no right to suggest this -- but just a suggestion to save you time and aggravation...

fair???

enjoy

ken

mekon:

(b) ... In the mean time, this is interesting.

i have now read all that thread on hydrogenaudio. basically about DBT on original vs copied CD. simple enough, one would have thought. but there doesnt seem to be universal acceptance of WHAT the results actually mean. is this typical? would be useful to see an example where the methodology and interpretation wasnt so problematical...

thanks for the reference anyway, but i fear i didnt get much out of it. of course, i may not have read it "properly"...

enjoy

ken

Ken,

Perhaps it might help you to understand the arguments about double blind testing and related issues of experimental methodology and bias if we consider a famous example from the 1940s.

In this experiment children were asked to adjust a knob which varied the size of a disc of light projected on a screen until they beleived the disc was the same size as a common demonination of coin. Consistently in this experiement children from poor backgrounds exaggerated the size of the coin compared to children from wealthy backgrounds. This is the classic demonstration of how expectation bias can dramatically alter even the simplest of physical perceptions.

Now project this notion into the area of some of the more outlandish claims made for hi-fi. Here you have extraordinary claims (say, 10mm spurs are massively better than 6mm spurs) and the only evidence we have is anecdotal and subjective reports from people who have spent considerable time, effort and money in their hifi and the changes in question. Obviously there is clear scope here for expectation bias even before we get into other dubious aspects of the evidence like many of the reports being subjective estimations of differences between a hi-fi now and a hi-fi some weeks previously, or the very nature of the reports being highly subjective and variable, etc.

I cannot stress enough about how all of this is *basic* science that has been validated again and again for all sorts of purposes. And that the various objections raised to DBT (about how hi-fi is special or different or incredibly subtle, etc.) are all easily addressed by well established scientific methods. As I've said before I, personally, care less about whether or not there really are extraordinary differnences between various spur arrangements than the fact that much of the arguments on this issue show a (frankly) ignorant, psuedo-scienctific view of the world that makes me despair of some of my fellow forumites.

Matthew

PS For the record I shall state once again that I am completely open minded to the idea that a proper experiment would demonstrate that there were very real differences at work. But this will make no odds -- people will jump in and repeat the same fuzzy headed nonsense that frankly would get you an instant F in even a secondary school science exam and acuse me of being a closed minded science bigot because I excercise an element of caution about these extraordinary claims.

Ross, I presume you skipped the class on sampling methods?

Ross,

"but that doesn't mean that the results will have any predictive value"

Of course it does! It does not mean the results of the experiment "proved" the hypothesis or that cause and effect is established but it will mean (assuming good design and execution) that you can say with a certain degree of confidence (as defined by the experiment) that the hypothesis is correct.

If this weren't the case then lots of things in life wouldn't "work". Certainly not drugs and you might just as well not bother taking that blood pressure drug the doctor gives you to stave off the coronary from too may lawyerly style lunches with that bottle of 58 Margaux. And all those studies which show a statisical connection between smoking and lung cancer don't predict you will get lung cancer but you'd be a fool to start smoking now.

Its also possible, although obviously less useful, to design an experiment to cover your very particular circumstances and its also possible to prove by experiment that a small section of highly trained observers are capable of reliably detecting certain stimuli (e.g. the cardas cable) but 98% of the population isn't.

Matthew

Ross, Matthew has already answered, but your 'missed the class...' come back made me giggle, as I one of the few classes I still teach on our research methods course is the one on multiple regression.

Damn, that is a dorky post.

Ross,

"The fact that you can show a linkage between two statistical occurrences does not mean that one causes the other"

Yes I know and indeed that was what I said.

FWIW I don't claim to be an expert of PUWO PhD quality (anything but) but I have done an introductory undergraduate course in statistics and experimental methodology and used those techniques in real experiments on applied psychology and IT courses (specifically on how people interact with computers).

"It’s not double blind experiments that make things “work”"

Give me *some* credit please!

"In hi fi, it is virtually impossible in practice to obtain this type of simplicity"

This is the "hi-fi is special" arguments again. Science, experiment and statistics can (and is) be used to test theories about things far more demanding and complex in experimental terms than hi-fi. Fields like Cosmology and Quantum physics spring to mind for complexity, psychology obviously has huge difficulties with human perceptions and subjectivity, etc. The medical sciences are also rather more complex than you imply with your "give drug see if patient gets better" and regularly deal with issues like patients who cannot communicate fully, situations where its unethical to have a control group, etc. To suggest that hi-fi is somehow novel or even particularly demanding is not tenable in my view.

"In cases where no difference is detectable (e.g. two groups of test subjects unable to tell the difference between two cables) this can always be dismissed as a problem with the sample, or the methodology"

Well only if there *is* a problem with the sample or methodology. I agree there are some people who would never be convinced (either way) but that doesn't alter the validity of the test.

"if I can clearly hear a difference between the two cables, the fact that 100 psychology students couldn’t is irrelevant to me"

"It’s not a particularly useful result, and this type of experiment is really only designed to satisfy sceptics"

If we do a series of well designed and controlled studies on a popular herbal remedy for some illness and find it consistently has no effect is that useful?

Suppose similar studies show, say, Valhalla at a gazillion £s a foot to be indistinguishable from a stock cable with the same electrical characteristics? Is that useful?

I agree if you are absolutely convinced about the results before hand and are determined to ignore them whatever the outcome then it has little relevence for you but it doesn't stop it having huge relevance for others.

"As to experiment (2), aside from the difficulty of devising such an experiment, ... What conclusions of general or specific applicability could you draw from such an experiment?"

Again its not particulary hard to devise experiments to test for people's subjective experiences. With a few years post-graduate study at a prestiguous institution even a lawyer could manage it

"I’d be interested to know exactly what sort of proposition you would want to test and how you would test it"

As I've said before I'm not actually that interested in testing such things -- it would seem a peculiar waste of resources of such a trivial issue. My motivation in these sorts of threads is more to do with defending Science's corner from woolly thinking.

Matthew

quote:

---

Originally posted by Kenwood O'Breezy:
I would be interested to know if anyone has EVER been able to distinguish between two 'normally' constructed cables (i.e. not ones that are designed to modify frequency response) in a proper double blind test?

I believe there is a reward on the Google uk hi-fi group of £1,000 (so far unclaimed!) for anyone who can...

---

and I guess the reason for the reward is that a statistically convincing result would "prove" the existence of a difference, even if that difference is below most mortals' threshold of discrimination.

cheers, Martin

E-mail:- MartinPayne (at) Dial.Pipex.com. Put "Naim" in the title.

Ross, although I am a doctoral candidate in psychology, it's late, and I am not sure my uni is considered sufficiently prestigious. Is what you are saying that you think that if it could be demonstrated that (for instance) certain sonic characteristics are associated with stable preferences amongst potential buyers, it would not be considered important by the manufacturer?

I have no idea whether such characteristics can be operationalised effectively, but if they can, it would be possible to examine both their predictive ability, and whether there are characteristics particularly associated with buying behaviour (retrospectively or prospectively) using between groups structural equation modelling.

I know from mates who've left academia for proper jobs in companies that provide research consultancy that there are plenty of companies who consider understanding of the predictive ability of social cognitions related to their products very valuable.

[This message was edited by Mekon on SUNDAY 16 November 2003 at 23:09.]

Ross,

"I’m merely suggesting that any results would be trivial. You are either testing for obvious differences, or you are testing for preferences."

And I was merely suggesting that proper testing may possibly demonstrate that "obvious differences" were not dectectable which would, at least to some people, be far from trivial.

"In my view, it is unlikely because the results would not change anyone’s preferences or the way hi fi is designed, bought or used, and I have yet to see an example of such a result which would suggest otherwise"

I think a consistent body of evidence indicating that cable differences and similar effects were essentially undetectable would stop many people buying expenisive cables and drive the people who make such things out of business. Conversely, a few experiments showing the existence of such effects would provide an interesting area for more research and pretty much instantly convert the sceptics into believers. For sure there would be some people in both canps for whom the evidence would make no difference whatsoever but they are a minority I suspect.

The only reason the debate drags on AFAICT is because nobody with the know how and resources to do the science has much of a reason to do it.

Matthew

Slightly off-topic, but wandering through the physics labs this morning, I spied a poster for an open lecture at 7pm tomorrow night entitled 'Hair Cells: The Hi-Fi Biological Microphones Inside Your Ears' to be given by Dr Corne Kros. It's hosted by The Institute of Physics, and I note that past talks have included 'The Physics of Music' and 'Loudspeaker System Design' (by John Drib of B&W). I'll keep an eye out for future talks.

If anyone in Brighton was thinking of going, and needs directions, PM me. It's in PEV1 1A7, but thats not too easy to find, unless you know the uni.

Loudspeaker design by soandso of B&W - was that a comedy slot?

Anyway, I'm still waiting for scientific proof that Ross is in Singapore.

1A7 sounds strangely familiar - I think it's where I had my first lecture as a physics student 23 years ago. But what does "PEV" stand for? [It used to be called MAPS].

--J

Jeremey, it's Pevensey (1)- it's the building that would be behind the Meeting House if you were standing at the top of the library stairs, looking out over library square. It's also to the right of Chichester 1 lecture theatre. Got it?

Ross...thank you for taking the time and effort to express elequently, what I perhaps have not the patience to sit down and type. I reiterate, in my sometimes rather blunt way, Ken, that I think that double blind testing in the context most of the readers here would be interested in, would be of academic interest mostly and ultimately, in practice, it would be far more time - effective for each prospective "user" to "sample" whatever is being offered, and make their own decisions based on their own subjective judgements.

DBT, I presume is intended to establish an "absolute truth" as far as that is possible, I understand.

With all respect to the above contributors, I really do wonder what it would really achieve?
(in the hi-fi context)

Please enlighten me

Laurie S