blind listening...

Posted by: ken c on 30 June 2001

blind listening has no place in auditioning hifi. discuss.

and enjoy

ken

Posted on: 30 June 2001 by Steve Toy
The above would suggest a title to an essay, so I am tempted to answer it thus, according to the convention of thesis - antithesis - synthesis.
Doing just that, I can begin by agreeing with the statement. That's the easy bit. We listen to music because we enjoy it. I, for one, am more able to make effective evaluations of different components (or other changes that one may make to a system) if I feel relaxed. In order to feel relaxed I need to be comfortable with my environment, i.e: no blindfolds, no veils, and no darkness - all the above place you under too much pressure to make an effective judgement of something that is, essentially, highly subjective. It's a bit like a doctor saying that s/he is going to take your blood pressure reading using some very sensitive equipment - and saying that the consequences for not relaxing before the measurement is taken are dire...
Obviously you can't relax and take in all the detail and/or emotional message of the music if you are forced to do so in such a contrived situation which provokes a degree of anxiety.
Decent hands-on listening is more likely to yield consistent results over time. Moreover, given the added tension of a blind audition, listeners are more likely to over-exaggerate any differences they may hear, on the basis that the fact that they can actually hear a difference at all must make it a significant difference. (The latest issue of Hi-fi Choice makes reference to this particular phenomenon.)
However, where hands-on tests are perhaps inconclusive, a follow-up blind test may be the decider, especially where differences are extremely slight, and possibly prone to bias in their perception. The best example of this was when I was auditioning the differences between connecting a Flatcap 2 to the CD5, Nac 112 or both. There were two of us performing the blind test - we couldn't see how the connections were being made at the back. Consensus between us - that CD5-only connection was best, makes the verdict quite difficult to refute, given that the test was blind. However, if I were required to do blind auditioning regularly, I doubt I would have the inclination to bother with the process at all.
As a general point, I agree with the above statement on the basis that we are not in it simply to analyse, but also to enjoy. In order to evaluate the performance fully, the enjoyment factor must be paramount. If you thing is cerebral detail, listen to high-end American stuff (or Linn)- with your eyes closed. If you want emotional involvement in the music, listen to Naim - with your eyes wide open! cool

[This message was edited by Steven Toy on SUNDAY 01 July 2001 at 05:40.]

Posted on: 01 July 2001 by Steve Toy
Absolutely. Do you remember the experiment with the Skoda Octavia? They covered up all the badges. Everyone said they would buy it - until the badges were uncovered!
But then cars are a bit different. We buy cars for the image they give us and for their appearance, as well as for their performance.
I think (most) people buy hi-fi just with performance in mind. Let's face it, whilst Naim products ooze quality in terms of their appearance, they are not going to win any beauty contests! wink
Posted on: 01 July 2001 by graphoman
To the debate between ken c (“blind listening has no place in auditioning hifi. discuss.and enjoy”) vs Steven Toy

The last thing I want to state that I was an authority though I may join discussion on that unfortunate field. As the first audio journalist in an audio vacuum (Budapest, 1979–1992) at first I tried to follow mainly German models and insisted to make blind tests on the most rigorous manner. Then we came across the English press (HFN/RR and HFN Answers, later Audiophile), we gradually changed our methods and at the end we realised that actually anything we have learned from the Germans from the time of Helmholtz till up to now was nothing more than a) wasting of time b) sheer bullshit. Since then we made no more blind listening tests any more. As for me, I refuse to take part not only in blind testing but in rapid A-B testing as well.

It does, of course, not prove that blind or, generally speaking, A-B listening would not be reliable. Maybe we made it the wrong way.
graphoman

Posted on: 01 July 2001 by Nigel Cavendish
I chose my nait 3 in an audition where I knew what was being used, and having read What Hi-Fi, I expected the Arcam to be better(at that time What Hi-Fi did not test naim gear).

But blind testing does not have to mean that the listener cannot see only that the kit cannot be seen. So you could have only a single set of speakers visible and the other kit behind a screen for example.

I suspect that if you are a fan of a certain manufacturer then it is almost inevitable that your objectivity is compromised in a test where you can see what is playing. There would be some interesting experiments around what people could see and what is actually being played to them e.g. you could have a naim CD player with the display off apperaing to play a CD but have the amp connected to an unseen CD player and so on.

Any student psychologists after a PhD? Any forum members brave enough to give it a go?

I am.

cheers

Nigel

Posted on: 01 July 2001 by ebirah
I do a lot of medical research - noone would take any trial seriously if it were unblind. If you want 'the truth' it has to be blind or you'll be sucking on bias all day. Same with wine tasting - why do we cover labels and bottles? - in our quest for truth I'm afraid. I often encounter people who are 'afraid' of blind listening, usually because they fear they won't 'perform' up to scratch (ever mistaken a Sauternes for a St. Julien - you 'take the shame' forever!). In my experience, differences are there to be heard or not, as the case may be.

Steve

Posted on: 01 July 2001 by Sproggle
quote:
I do a lot of medical research - noone would take any trial seriously if it were unblind.

While that makes some sense, I have often wondered why these trials seem to be mainly [wholly?] placebo controlled rather than existing treatment controlled. Is there a reason for it - other than saving money?

--Jeremy

Posted on: 01 July 2001 by Chris Dolan
I just lost a long and erudite post on this topic as I got up to change the record and I knocked the line connection out!!!!!

So to keep it short as I can't type it all again, blind listening does have a place, but not a big one. I prefer extended home dems - hands on.

Watch your preconceptions and don't buy something you don't like just coz someone else does - even if you normally respect their views.

What I did not get round to typing previously is how my best blind dems are the dems I do on others to get their opinions - usually my wife's opinion on cables, supports, Lingo plugged in etc.

My wife thinks the system is superb and does not understand why I occasionally want to improve it. Because she does not care which is better and does not know how much it costs, I fully respect her view - especially when she agrees with me.

Chris wink

Posted on: 01 July 2001 by ken c
many interesting views expressed. let me just throw in a few observations from my own experience.

(a) in almost all the dems i have had, the upgrade unit was so obviously better that there was no need for a blind test. since i buy only naim, perhaps i am pre-disposed to prefer the upgrade component? in which case, (contradiction) perhaps a blind test might introduce some objectivity and save the money burning a hole in my pocket?

(b) in some dems where i hadnt preferred the upgrade component, it does appear that other factors were involved. my first dem of cdx cs cdsii using b&w nautiluses was inconclusive -- no need for a blind test as neither system was sounding particularly magical to me -- till we swapped the 250 for a pair of 135's.

(c) whenever there has been something wrong with my system, i have usually found it difficult to pin-point where the problem is straightaway -- certainly a blind test in this case would serve no purpose and would just be stressful that's all. in these situations, i find an extended listening session tells me loads of things. the system might sound OK'ish on one record and you ask yourself "what am i on about"? then later on, you play another record, and you go "hey, this record usually sounds much better than that!!!" and so on, and so on. in the limit, you call your dealer, or naim audio for help.

i think blind listening might have a place when you are comparing equivalent units from rival manufacturers (say lingo vs armageddon). but even then, as chris d and stephen t suggest -- preferences are more reliable with extended listening at home -- where cumulative differences yield more reliable results.

much food for thought...

enjoy

ken

Posted on: 02 July 2001 by ken c
stephen toy:

Decent hands-on listening is more likely to yield consistent results over time. i agree wholeheartedly. an overall performance picture over time is likely to discount the effects of normal system variability.

Moreover, given the added tension of a blind audition, listeners are more likely to over-exaggerate any differences they may hear, on the basis that the fact that they can actually hear a difference at all must make it a significant difference.

i have done a blind test at a dealers before -- and i can tell you its very stressful, especially if the sounds you are trying to compare are both substandard and you are trying to pinpoint some fault.

i can agree that non-blind tests can easily introduce biases, but would contend that the effect of these biases is much smaller than perhaps other problems that may arise -- poor records, poor installation, poor mains, poor supports, etc...

enjoy

ken

Posted on: 02 July 2001 by ken c
maurice, i suspect the difference between linn and equivalent naim system is large enough not to require a blind test, but then perhaps this is thevery bias you are referring to.

one of these days when i have some time, i might do a linn vs naim blind test, if possible, and see if the blind test might swing things the other way.

thanks for your thoughts

enjoy

ken

Posted on: 02 July 2001 by ken c
...we realised that actually anything we have learned from the Germans from the time of Helmholtz till up to now was nothing more than a) wasting of time b) sheer bullshit. Since then we made no more blind listening tests any more. As for me, I refuse to take part not only in blind testing but in rapid A-B testing as well.

what specifically made you turn 180 degress on blind tests? was it anything in the british hifi press?? or something that you suddenly could provide was wrong in the German approach??

enjoy

ken

Posted on: 02 July 2001 by ken c
nigel cavendish:

But blind testing does not have to mean that the listener cannot see only that the kit cannot be seen.

good point. easily forgotten sometimes.

enjoy

ken

Posted on: 02 July 2001 by graphoman
Hi Ken,
(you asked: what specifically made you turn 180 degress on blind tests? was it anything in the british hifi press?? or something that you suddenly could provide was wrong in the German approach??)

Since 1992 I’m absolutely out of competence (if, indeed, I was in?) and I have no reason to flatter anybody’s national vanity but I found that “go british” is the only way if we want to live together with music. There is no space here to explane the whole progression of 14 years, not to mention the situation in Hungary in the late Kadarism. While the sortiment in the shops was a Nil, as Editor I had the opportunity a) to get and read the western magazines b) to get my hands on some modest equipment for reference (we started with Thorens TD124, Spendor BC1, then Nait I and II...), c) to borrow more expensive machines from friends for the sake of A-B tests (e.g. LP12, NAP250, Audio Research etc.) d) to manage that all our measurements would be made free of charge in the main state institute, where one of the leading experts, my co-editor worked and, finally, e) to manage acoustic(?) tests to carry out with the personal co-operation of as named musicians as the members of the Liszt Ferenc Chamber Orchestra. (It was about the audibility of the PWB phenomena – can you still remember Peter W. Belt?)

Our first Bible was Hi-Fi Stereophonie (with Herr Karl Breh, president of the German Hi-Fi Association, in the editorial chair) so I know what I’m speaking about. Later we made trials with several western mags that we was able to understand (british, american, via friends even french ones) and we found two british ones reliable. We had respect for their reviewers, tried their methods, and came up with the result that nobody of them was infallible, though the overall picture was convincing and educational. We came to the conclusion that the ideal reviewer must fall anywhere inbetween Martin Colloms and Ken Kessler, I think. (As for blind listening tests, once we published a two-part report from “Martin Colloms’ workshop”, carried out one of my colleagues.)

To make a long story short, I exercise audio (now as private person) the british way. I maintained good connection to the few Hungarian dealers who do well so I don’t think that Naim is the only way to Nirvana but one must choose his own way and that’s what I have done.
graphoman

Posted on: 02 July 2001 by ebirah
All this reminds me of Dick Shahinian's experiments where he used identical sets of Obelisk speakers, except that one pair were obscured behind an acoustically transparent screen - listeners inevitably preferred the hidden speakers, assuming them to be newer, bigger and better...I was one of the few people at the Linn Grosvenor demo brave enough to put my hand in the air and admit to thinking the same piece of music on the same kit sounded better the second time I heard it - remembered musical events and all that psychology.

Sproggle: Re Medial research - although some stuff is against placebo, most is vs standard/accepted 'best practice' treatments. A lot of the placebo work is reserved for situations where there is no clear best practice or where you really aren't sure the 'treatment' being tested works at all - eg homeopathy (ducks slings and arrows)...and while I'm on the subject I don't work with nice, furry little animals (saves brick through window scenario).

Steve

Posted on: 02 July 2001 by ken c
graphoman, thanks for taking the time to respond in such detail. i read your posting with some interest. i have one more question -- when you say I exercise audio (now as private person) the british way. what, specifically is this "british" way?? am i correct that you now shun measurements and blind listening?

from my own limited experience, i find that in order to successfully audition hifi -- i need a very well defined base line -- this could be my own system or something very similar. interestingly, i find it much easier to compare 2 pieces of kit that both sound good than those that both dont. the case where one does and the other doesnt is not worthy of concern of course.

thanks for your thoughts...

enjoy

ken

Posted on: 02 July 2001 by ken c
ebirah,

...admit to thinking the same piece of music on the same kit sounded better the second time I heard it - remembered musical events and all that psychology.

interesting.... presume this is because anything that makes the brain work less hard in making sense of the music must be a good thing? this is a wild (and probably wrong) guess. any acoustic psychologists in the house??

enjoy

ken

Posted on: 02 July 2001 by Rob Doorack
There's a thread on this subject on www.audioasylum.com currently, in the "tweaks / DIY" asylum. The most interesting posting is by engineer Jon Risch who presented a paper on subjective testing at the '91 AES convention.
Posted on: 02 July 2001 by ken c
rob d: many thanks for that link -- will have a look at it soon. whats your view?

enjoy

ken

Posted on: 02 July 2001 by Andrew L. Weekes
I dont believe that blind testing makes any difference in the long term I do believe it's quite easy to try too hard in any kind of listening test, and I've found this to be a mistake I still fall into, even now.

My recent experiences with a certain equipment support, where I wanted it to work well and had convinced myself that it did - for a while, is a good case in point.

Ultimately though, once one starts to listen to the music as a whole, and here I have to say the hum along tune test really is a good starting point, it becomes obvious.

I find I can only really determine subtle differences over an extended dem, in a relaxed environment. This is often at home, but I've experienced good dems at dealers too.

I also find it very easy to focus on one particular aspect of the sound, and a second set of ears can be so useful. My original dem of the 5 series kit was partly to determine whether a CD5 was better than my own modified player. It was, and when discussing differences between them with my dealer I pointed out aspects I'd noticed, he pointed out an aspect he'd noticed, but I hadn't (vocals were more expressive). I think the bass player in my head had been concentrating on those aspects, and almost ignoring other equally obvious changes (when highlighted).

I feel part of the problem is at home I just listen to music, it's a credit to the system that it allows me to do this it getting in the way. At leats one friend of mine is always analysing his system, the reason being, I believe, that it isn't good enough for him.

A recent visit with some old Naim kit was very enlightening - a big grin and a floor littered with CD's.

I tend to switch into 'analytical' mode at dems, when I should just do what I normally do and see if the system stands the test of time or if my attention wanders.

Andy.

Andrew L. Weekes

Posted on: 02 July 2001 by ken c
I tend to switch into 'analytical' mode at dems

i consciously try not to be analytical at dems. when the system is "good", i find this very easy to achieve. however, sometimes i fail. and when i do, i just blame the dem system...

enjoy

ken

Posted on: 02 July 2001 by Sproggle
quote:
although some stuff is against placebo, most is vs standard/accepted 'best practice' treatments.

I'm glad to find I was mistaken. smile

--Jeremy

Posted on: 03 July 2001 by Rob Doorack
The only blind test that I've ever participated in was a single blind one at a Stereophile Show in the mid - Nineties. The idea was to see if CD tweaks such as green painted edges and so forth were detectable. I don't recall the gear in the system but the speakers were a pair of B & W 801s. The test was held in a fairly large meeting room holding perhaps 100 audiophile guinea pigs. I correctly identified 5 out of 8 trials. I recall it as quite a stressful experience.

Personally I'm firmly convinced that amplifiers and other components frequently sound different (I wouldn't own Naim gear if I didn't think that!). It seems to me that the ABX zealots who strenuously argue that they're applying science to prove that differences don't really exist have it backwards. There's a huge group of people who report that they can in fact hear such differences. Wouldn't a true application of the scientific method be to investigate what those witnesses are hearing, rather than starting from a premise that they're all delusional?

Posted on: 03 July 2001 by graphoman
To: ken c
(“I have one more question -- when you say I exercise audio (now as private person) the british way. what, specifically is this "british" way?? am i correct that you now shun measurements and blind listening?)

The british way for me does mean avoiding A-B tests if possible (not always possible, I must admit) and try to live together with some equipment instead, as long as you know its merits and limitations by heart.

Though I have, even now, the possibility to let measure actually anything by my friends, with Naims I don’t really need it. Not because everything made by Naim would be absolutely correct. I remember our test from the ’80s where a 250 proved nothing short to be perfect and my technical friends was all enthusiastic about its technology. Then came a Nait 1 and the poor thing was as imperfect as it could be, however, it had the same sound character as the 250 and when a very good expert made some MOD then the parameters all improved but the sound got spoiled. In the recent two years or so I had to realize some facts that I did not want accept before, namely, that to the 250 I do need Naim’s own preamp as well as own cable, then the funny story with the cableing between preamp and HiCap etc. Under circumstandes like these there remains little or no reason to measure anything.

In addition to it, I am fully aware of all the limitations of the usual labour work. But this would be a very long story.
graphoman

Posted on: 03 July 2001 by ken c
Then came a Nait 1 and the poor thing was as imperfect as it could be, however, it had the same sound character as the 250 and when a very good expert made some MOD then the parameters all improved but the sound got spoiled.

fascinating story!!!
as a matter of interest, are there any measurements that you were able to indentify that correlated with your listening tests? (aside from the really obvious ones, such as wrong mains voltage, and such like...)

interesting posting. many thanks graphoman.

enjoy

ken

Posted on: 04 July 2001 by graphoman
To: ken c, regarding Nait 1’s measurements

Just as PS: I think it was clear from my letter that it was not me who had carried out the measurements since I have never been a technical man. After 15 years, the Nait’s story is not a too actual one but since my co-editor was absolutely German oriented/minded, he registrated all posssible data in the magazin’s test column. I can assure you that no part of the amp had a correct frequency range. Phase was spoiled, too. They were the most primitive faults all about and the amp still had a pleasant sound. Later, the Nait’s second generation was mainly free of these evident faults.