blind listening...

[This message was edited by Steven Toy on SUNDAY 01 July 2001 at 05:40.]

The last thing I want to state that I was an authority though I may join discussion on that unfortunate field. As the first audio journalist in an audio vacuum (Budapest, 1979–1992) at first I tried to follow mainly German models and insisted to make blind tests on the most rigorous manner. Then we came across the English press (HFN/RR and HFN Answers, later Audiophile), we gradually changed our methods and at the end we realised that actually anything we have learned from the Germans from the time of Helmholtz till up to now was nothing more than a) wasting of time b) sheer bullshit. Since then we made no more blind listening tests any more. As for me, I refuse to take part not only in blind testing but in rapid A-B testing as well.

It does, of course, not prove that blind or, generally speaking, A-B listening would not be reliable. Maybe we made it the wrong way.
graphoman

But blind testing does not have to mean that the listener cannot see only that the kit cannot be seen. So you could have only a single set of speakers visible and the other kit behind a screen for example.

I suspect that if you are a fan of a certain manufacturer then it is almost inevitable that your objectivity is compromised in a test where you can see what is playing. There would be some interesting experiments around what people could see and what is actually being played to them e.g. you could have a naim CD player with the display off apperaing to play a CD but have the amp connected to an unseen CD player and so on.

Any student psychologists after a PhD? Any forum members brave enough to give it a go?

I am.

cheers

Nigel

Steve

quote:

---

I do a lot of medical research - noone would take any trial seriously if it were unblind.

---

While that makes some sense, I have often wondered why these trials seem to be mainly [wholly?] placebo controlled rather than existing treatment controlled. Is there a reason for it - other than saving money?

--Jeremy

So to keep it short as I can't type it all again, blind listening does have a place, but not a big one. I prefer extended home dems - hands on.

Watch your preconceptions and don't buy something you don't like just coz someone else does - even if you normally respect their views.

What I did not get round to typing previously is how my best blind dems are the dems I do on others to get their opinions - usually my wife's opinion on cables, supports, Lingo plugged in etc.

My wife thinks the system is superb and does not understand why I occasionally want to improve it. Because she does not care which is better and does not know how much it costs, I fully respect her view - especially when she agrees with me.

Chris

(a) in almost all the dems i have had, the upgrade unit was so obviously better that there was no need for a blind test. since i buy only naim, perhaps i am pre-disposed to prefer the upgrade component? in which case, (contradiction) perhaps a blind test might introduce some objectivity and save the money burning a hole in my pocket?

(b) in some dems where i hadnt preferred the upgrade component, it does appear that other factors were involved. my first dem of cdx cs cdsii using b&w nautiluses was inconclusive -- no need for a blind test as neither system was sounding particularly magical to me -- till we swapped the 250 for a pair of 135's.

(c) whenever there has been something wrong with my system, i have usually found it difficult to pin-point where the problem is straightaway -- certainly a blind test in this case would serve no purpose and would just be stressful that's all. in these situations, i find an extended listening session tells me loads of things. the system might sound OK'ish on one record and you ask yourself "what am i on about"? then later on, you play another record, and you go "hey, this record usually sounds much better than that!!!" and so on, and so on. in the limit, you call your dealer, or naim audio for help.

i think blind listening might have a place when you are comparing equivalent units from rival manufacturers (say lingo vs armageddon). but even then, as chris d and stephen t suggest -- preferences are more reliable with extended listening at home -- where cumulative differences yield more reliable results.

much food for thought...

enjoy

ken

Decent hands-on listening is more likely to yield consistent results over time. i agree wholeheartedly. an overall performance picture over time is likely to discount the effects of normal system variability.

Moreover, given the added tension of a blind audition, listeners are more likely to over-exaggerate any differences they may hear, on the basis that the fact that they can actually hear a difference at all must make it a significant difference.

i have done a blind test at a dealers before -- and i can tell you its very stressful, especially if the sounds you are trying to compare are both substandard and you are trying to pinpoint some fault.

i can agree that non-blind tests can easily introduce biases, but would contend that the effect of these biases is much smaller than perhaps other problems that may arise -- poor records, poor installation, poor mains, poor supports, etc...

enjoy

ken

one of these days when i have some time, i might do a linn vs naim blind test, if possible, and see if the blind test might swing things the other way.

thanks for your thoughts

enjoy

ken

what specifically made you turn 180 degress on blind tests? was it anything in the british hifi press?? or something that you suddenly could provide was wrong in the German approach??

enjoy

ken

But blind testing does not have to mean that the listener cannot see only that the kit cannot be seen.

good point. easily forgotten sometimes.

enjoy

ken

Since 1992 I’m absolutely out of competence (if, indeed, I was in?) and I have no reason to flatter anybody’s national vanity but I found that “go british” is the only way if we want to live together with music. There is no space here to explane the whole progression of 14 years, not to mention the situation in Hungary in the late Kadarism. While the sortiment in the shops was a Nil, as Editor I had the opportunity a) to get and read the western magazines b) to get my hands on some modest equipment for reference (we started with Thorens TD124, Spendor BC1, then Nait I and II...), c) to borrow more expensive machines from friends for the sake of A-B tests (e.g. LP12, NAP250, Audio Research etc.) d) to manage that all our measurements would be made free of charge in the main state institute, where one of the leading experts, my co-editor worked and, finally, e) to manage acoustic(?) tests to carry out with the personal co-operation of as named musicians as the members of the Liszt Ferenc Chamber Orchestra. (It was about the audibility of the PWB phenomena – can you still remember Peter W. Belt?)

Our first Bible was Hi-Fi Stereophonie (with Herr Karl Breh, president of the German Hi-Fi Association, in the editorial chair) so I know what I’m speaking about. Later we made trials with several western mags that we was able to understand (british, american, via friends even french ones) and we found two british ones reliable. We had respect for their reviewers, tried their methods, and came up with the result that nobody of them was infallible, though the overall picture was convincing and educational. We came to the conclusion that the ideal reviewer must fall anywhere inbetween Martin Colloms and Ken Kessler, I think. (As for blind listening tests, once we published a two-part report from “Martin Colloms’ workshop”, carried out one of my colleagues.)

To make a long story short, I exercise audio (now as private person) the british way. I maintained good connection to the few Hungarian dealers who do well so I don’t think that Naim is the only way to Nirvana but one must choose his own way and that’s what I have done.
graphoman

Sproggle: Re Medial research - although some stuff is against placebo, most is vs standard/accepted 'best practice' treatments. A lot of the placebo work is reserved for situations where there is no clear best practice or where you really aren't sure the 'treatment' being tested works at all - eg homeopathy (ducks slings and arrows)...and while I'm on the subject I don't work with nice, furry little animals (saves brick through window scenario).

Steve

from my own limited experience, i find that in order to successfully audition hifi -- i need a very well defined base line -- this could be my own system or something very similar. interestingly, i find it much easier to compare 2 pieces of kit that both sound good than those that both dont. the case where one does and the other doesnt is not worthy of concern of course.

thanks for your thoughts...

enjoy

ken

...admit to thinking the same piece of music on the same kit sounded better the second time I heard it - remembered musical events and all that psychology.

interesting.... presume this is because anything that makes the brain work less hard in making sense of the music must be a good thing? this is a wild (and probably wrong) guess. any acoustic psychologists in the house??

enjoy

ken

enjoy

ken

My recent experiences with a certain equipment support, where I wanted it to work well and had convinced myself that it did - for a while, is a good case in point.

Ultimately though, once one starts to listen to the music as a whole, and here I have to say the hum along tune test really is a good starting point, it becomes obvious.

I find I can only really determine subtle differences over an extended dem, in a relaxed environment. This is often at home, but I've experienced good dems at dealers too.

I also find it very easy to focus on one particular aspect of the sound, and a second set of ears can be so useful. My original dem of the 5 series kit was partly to determine whether a CD5 was better than my own modified player. It was, and when discussing differences between them with my dealer I pointed out aspects I'd noticed, he pointed out an aspect he'd noticed, but I hadn't (vocals were more expressive). I think the bass player in my head had been concentrating on those aspects, and almost ignoring other equally obvious changes (when highlighted).

I feel part of the problem is at home I just listen to music, it's a credit to the system that it allows me to do this it getting in the way. At leats one friend of mine is always analysing his system, the reason being, I believe, that it isn't good enough for him.

A recent visit with some old Naim kit was very enlightening - a big grin and a floor littered with CD's.

I tend to switch into 'analytical' mode at dems, when I should just do what I normally do and see if the system stands the test of time or if my attention wanders.

Andy.

Andrew L. Weekes

i consciously try not to be analytical at dems. when the system is "good", i find this very easy to achieve. however, sometimes i fail. and when i do, i just blame the dem system...

enjoy

ken

quote:

---

although some stuff is against placebo, most is vs standard/accepted 'best practice' treatments.

---

I'm glad to find I was mistaken.

--Jeremy

Personally I'm firmly convinced that amplifiers and other components frequently sound different (I wouldn't own Naim gear if I didn't think that!). It seems to me that the ABX zealots who strenuously argue that they're applying science to prove that differences don't really exist have it backwards. There's a huge group of people who report that they can in fact hear such differences. Wouldn't a true application of the scientific method be to investigate what those witnesses are hearing, rather than starting from a premise that they're all delusional?

The british way for me does mean avoiding A-B tests if possible (not always possible, I must admit) and try to live together with some equipment instead, as long as you know its merits and limitations by heart.

Though I have, even now, the possibility to let measure actually anything by my friends, with Naims I don’t really need it. Not because everything made by Naim would be absolutely correct. I remember our test from the ’80s where a 250 proved nothing short to be perfect and my technical friends was all enthusiastic about its technology. Then came a Nait 1 and the poor thing was as imperfect as it could be, however, it had the same sound character as the 250 and when a very good expert made some MOD then the parameters all improved but the sound got spoiled. In the recent two years or so I had to realize some facts that I did not want accept before, namely, that to the 250 I do need Naim’s own preamp as well as own cable, then the funny story with the cableing between preamp and HiCap etc. Under circumstandes like these there remains little or no reason to measure anything.

In addition to it, I am fully aware of all the limitations of the usual labour work. But this would be a very long story.
graphoman

fascinating story!!!
as a matter of interest, are there any measurements that you were able to indentify that correlated with your listening tests? (aside from the really obvious ones, such as wrong mains voltage, and such like...)

interesting posting. many thanks graphoman.

enjoy

ken

Just as PS: I think it was clear from my letter that it was not me who had carried out the measurements since I have never been a technical man. After 15 years, the Nait’s story is not a too actual one but since my co-editor was absolutely German oriented/minded, he registrated all posssible data in the magazin’s test column. I can assure you that no part of the amp had a correct frequency range. Phase was spoiled, too. They were the most primitive faults all about and the amp still had a pleasant sound. Later, the Nait’s second generation was mainly free of these evident faults.