SQ drop through NDX when streaming when compared to CD

jmtennapel

I made NO statement about musical enjoyment or musical quality.

The last paragraph of my post was:

"The tests were valid within the parameters of the tests.  It is not valid to draw conclusion outside these parameters."

I stand by this statement.

The tests were conduced using seven subjects and a binary (pass/fail) criterion.

It is valid (by deductive logic) to conclude that six of the seven test subjects could not hear a difference and that one test subject could hear a difference.

It is NOT valid to use inductive logic to conclude that there is no difference or that no person will be able to hear a difference.  Since seven subjects were tested using a binary criterion, the result is not considered statistically significant at a probability level of two standard deviations (this being the normally accepted condition for statistical significance).

If you look back at my posts there are some conditions where I have declared that, within my own limits of confidence, I couldn't distinguish a difference.

So not "Always".

I'm always quite amazed when reading double blind placebo controlled trials to read something like - headache drug 14%, placebo 10%.

"some" people will always perceive a "difference"

SJB

I read the article and it's very strange. The whole shebang is undermined by the conditions of the test itself. If you prime the audience with a skeptical view and then draw users from that audience, you have clearly biased the results.They might have ameliorated that a fraction by forcing an A/B answer, but they allowed a 'don't know'. This was presumably a statistical 'trick' to get a meaningful result from a smaller population, but given the audience priming it rendered the whole test silly. The writer presents this as if he deduced it during the exercise, but all of it was knowable in advance.

Furthermore they say they'll test 20 people but only test eight, on the basis that seven of the first eight couldn't tell. But why not carry on anyway? Testing 20 people must give a more significant result than testing eight.

Honestly, I don't personally believe that a $340 cable sounds better than a decently-made regular one, but I haven't done any proper listening so of course I might be wrong. But this kind of showmanship-style testing does no-one any favours, and certainly doesn't help to answer the question.

Hi Solid Air,

In general terms I agree with your assessment, with a couple of caveats:

The use of a binary test ('not sure' = 'no difference').

If the test is being used to prove that a detectable difference exists; then this is a statistically valid approach used to disprove the hypothesis that "no detectable difference exists".  However the test isn't being used that way, so the approach isn't appropriately applied in this case.

Using binary criteria assessment using 20 test subjects could achieve statistical significance (p=0.05) to disprove the hypothesis that "significant differences exist", even if one person did consistently detect differences provide all others detected no differences and there were no 'not sure' results.

Using eight test subjects (seems I misread this as seven) was never going to achieve the necessary statistical certainty (i.e. when probability of a different result occurring by chance is less than 5%) to reach a valid conclusion one way or the other.

As you say, bad statistical analysis being used to masquerade as a valid proof.  The test was reasonably OK (even if much too small), but the analysis presented is flawed.  Not helpful to anyone.

Also, surely any test designed to detect subtle differences between cables would use a very high resolution audio output. I struggle to believe that the headphone socket on a pretty run of the mill Dell laptop meets the required standard.

Hi Huge,

I think that 20 people is enough to disprove the hypothesis that "significant differences exist" (p=0.05) if the hypothesis means "generally people can detect a difference". In that case, if 19 people can't tell but one can, you'd say it was disproved. This is the Coke versus Pepsi analysis - asking if a general claim can be made. 

But if the hypothesis literally means "significant differences exist" then you might need a different statistical method. Then, if 19 people cannot tell but one can - consistently, 100% of the time - then you have to accept that "significant differences exist" is proven, even if 95% of the test subjects couldn't detect the difference.

Otherwise, you're at risk of a false negative. If you want to test the hypothesis "significant differences exist" between a real painting and a forgery, the fact that 19 people couldn't tell wouldn't disprove the hypothesis if one person consistently could. The danger is that you're not testing the cable/painting, but the expertise of the general population.

If their was any SQ difference between Ethernet cables then why are we continually discussing & trying to prove it to be so.  If such a difference did exist then surely by now we would be disussing the various levels of differences & VFM. 

Sorry, but this all gets cyclical to me..there is nothing directly in the Ethernet conveyance that directly affects the SQ of transferred  audio media as used by UPnP etc. There  just isn't, there are simply no direct associations other than complete physical connection failure or data errors causing the throughput to drop below the rate required... I think in our home LAN environments if this became an issue we would consider the Ethernet cable broken.

However Ethernet cables do carry (with 100 Mbps) multiple carrier frequencies of around 31 MHz. Although the conductors are balanced and twisted and some shielded, they will radiate and cross talk. This energy will almost certainly couple to varying degrees to any connected equipment that is sensitive to such RF energy such as an Ethernet enabled DAC or streamer. And therefore I say that the varying leakage, cross talk and shielding characteristics almost certainly interact with connected audio equipment with varrying effects. If you can hear a difference between FLAC and WAV decoding it defies reason to me that you would not be likely sensitive to hearing the effects of differing ethernet cables on connected equipment because of the different coupled RF cross talk produced.

However I suggest such differences are just as likely to be audible between different regular  professional/industrial cables as well as with, and  between different consumer boutique / 'audiophile' cables. The cable need not be expensively priced to hear such a difference or any perceived benefit.

Simon

+1 Simon,

As I said previously:

"The conclusion of some other technically informed persons is that different Ethernet cables can result in different tuning of the RFI introduced into the following analogue electronics, rather than differences in the data transmission; and that these differences may result in perceived differences in the sound."

Solid Air,

We're getting off topic, but the scientific method relies on disproof of the null hypothesis rather than any attempt at proof positive (we leave that to lawyers).

So, to prove a hypothesis, we seek to prove that nothing else can statistically be considered to be the case at the p=0.05 level.  As such to test the hypothesis "significant differences in Ethernet Cables exist", then the required process is to disprove the hypothesis "no significant differences in Ethernet Cables exist". If just one person detects a change even 100%, then that tests the person, not the Ethernet Cables.  As you say, in this circumstance, you'd then afterwards need to set-up another test using a different experimental protocol.