The Resolution test

Posted by: DrPo on 02 April 2018

The title is deliberately provocative; there is a simple test to check whether higher / lower resolution effects are audible which eliminates variables like mastering etc and which, to my knowledge, is not used as much as it ought (in my view at least) to be used:

1. take any high resolution  track and make transfers of the same track in lower resolution. I would take a 24 96 well recorded track and make a CD quality, MP3 320 and MP3 192 "copy" of it. Being a PC user I would do that on Foobar.

2. Do an ABX comparison of those tracks (I would again do it in foobar) either in stepwise mode (24/96 vs CD, CD vs MP3 320, MP3 320 vs MP3 192) or (for "training the ear") directly between extremes (24/96 vs MP3 192)

3. Record the results in terms of hit rate (or equivalently, confidence level) of the ABX test.

Personally, I haven't managed to go beyond 7/10 success (which statistically is not admissible as signifying audible differences) in comparing a high rez from CD FLAC resolution in ABX tests with my headphone set up (this could be due to my ears, my ability to discern subtle changes, my set up resolving capability etc but it remains a fact to reckon with).

I would be quite interested in having some concrete inputs / feedback on this topic from forum members who have run such tests (including which tracks).

Thanks

 

Posted on: 02 April 2018 by Innocent Bystander

Audible in this context of course means to the individual and in the system compared (including any room effects). Of course, at the end of the day that is all that really matters - unless you may at some time in the future upgrade your system (or improve the room).

Related to that, if you’re playing the outcome on the computer with Foobar then your conclusions will be affected by whatever may be the limitations of Foobar and whatever may be the effect of the computer it is on. Better to transfer the files back to your normal store and play with normal player to assess any difference.

A question, though, to which I don’t know the answer: can the software doing the downsampling affect the outcome - i.e does the lowe res file produced by one converter differ in any way from one produced by another, e.g Foobar vs DBPoweramp?  If so then any audible difference heard comparing files produced with one will need reassessing with others. 

Posted on: 02 April 2018 by DrPo

hi, this is precisely the point: if you cannot hear differences btw HR and lower resolution it is irrelevant whether the s/w you used to do the "downsampling" was good or best or what not. The point is: who is able to hear differences confirmed (i.e at 95% confidence level) with ABX testing? 

Posted on: 02 April 2018 by hungryhalibut

Just out of interest, what is the motivation to do this testing? Is it for some type of Golden Ears competition. Isn’t having a nice stereo about enjoying music, rather than analysing sound? I may have got completely the wrong end of the stick of course. I can’t listen to Spotify on my system as it sounds so awful, and I have many CDs that sound much better than other albums bought as high res. So much seems to be about the original recording. 

Posted on: 02 April 2018 by GraemeH

ABX is not how we listen to music though.

G

Posted on: 02 April 2018 by DrPo

the motivation could be very simple: the hi-res download is btw 50% and 80% more expensive than a CD quality download. Anyway, I am curious about the results of this simple test.

Posted on: 02 April 2018 by Innocent Bystander
DrPo posted:

hi, this is precisely the point: if you cannot hear differences btw HR and lower resolution it is irrelevant whether the s/w you used to do the "downsampling" was good or best or what not. The point is: who is able to hear differences confirmed (i.e at 95% confidence level) with ABX testing? 

I’ve only compared a couple of albums where I happen to have more than one version - leading to me discovering that mastering was different. Since then I haven’t bothered comparing. My choice would simply be that which sounds better - though in practice it is rarely possible to try all the different versions that of an album that might exist before deciding which to buy, so unless anything suggests otherwise to me, I tend to buy the higher res version on the assumption that it is likely to be no worse sounding, and that even if I might not be able to tell the difference with my present system and room, it may simply take a house move and I might.

What you suggest may be a way for people interested to compare, as long as they recognise it is subject to limitations as my previous post alluded.

Posted on: 02 April 2018 by Innocent Bystander
Hungryhalibut posted:

 Is it for some type of Golden Ears competition. 

That would be a novel idea! The “winners” presumably could find a good career in comparative equipment or recording assessments, while the “losers” would have the significant consolation that they can save the most money on recordings and system. 

Posted on: 02 April 2018 by dave4jazz

Some people say they can’t listen to Spotify on their system as it sounds so awful. CD quality it's not but is it really that bad?

Maybe there's something wrong with their system set-up or perhaps it's just that I haven't got those golden ears.

Dave

Posted on: 02 April 2018 by DrPo

same here, that is why I post this. I have created 3 copies of CD quality Bach's Jesus bleibe meine Freunde Cantata in 320 mp3, 190 mp3, 65 mp3 resolutions, unfortunately cannot upload them due to size limits to make it easier for people to take the test :-)

Posted on: 02 April 2018 by Massimo Bertola

Premise: I don't stream seriously – meaning that HD- or SSD-based files are not my main source. I have a CD player, a turntable, several internet radios around the home and a couple of smartphones (an iPhone4 only used as iPod and a iPhone 6s used as phone and with most of the music I have on the computer. The iPhone4 is used via docking station on a Marantz Consolette and on a Sony stereo Bluetooth active speaker or with earplugs, in bed or when commuting.

On my Mac I only have a) Amazon autorips and, b), iTunes purchases. The iTunes purchases are mostly single songs I wanted to listen to, from my youth. iTunes files are AACs (Mastered for iTunes, whatever that may mean). So the very few rips I have of CDs of mine are also made in AAC, to have an homogeneous rendition of music on the phone(s).

Long premise done, I have I believe reached a point where the concept of definition has become vague. No doubts that in theory my CDX2/PS/SuperNait/PS/Pro-Ject RPM Carbon3/Stageline/PS/NAC A5/S-400s system has more 'definition' than what I replay from my iPhones, but the fact is that lately I am listening to a lot of LPs and to a lot of iPhone with a pair of humble Asus earplugs survived from my former cellphone. At this point, whatever I hear with earphones sounds more organic and 'musical' than whatever I replay on the 'real' system; and the reason is quite obvious to me: with earphones I hear close to 100% of what's in the file, phase and ambience data included, whilst when having to deal with the infinite list of issues coming from the system/room/speakers interactions, I don't.

Definition is not only in the content of the file; it's a lot also in what is preserved or lost by the ambient, and by the fact that speakers themselves are not – safe for a fistful of very specific designs, like some electrostatics – able to retain and give back to air whatever reaches the drivers.

I presently do my best to ignore the siren call of the concept of definition, because it can distract me from music more than anything else in the complex web of a system's aspects. Then there is the problem of expectations, and this opens up a continent of further problems. So, everything about definition is about sound, and sound can actually distract from music.

My range of personal and repeated experience goes, so, from AAC on an iPhone4 with earplugs to CDX2, XPS-DR, 282, SuperCap, 250.2, Vertere cables, SBLs. I am in no way able, today, to tell which, each in its proper listening conditions and terms, gives the most simple and immediate musical satisfaction.

Was it all off-topic?

M

Posted on: 02 April 2018 by Bert Schurink

The tests have been executed on multiple audio shows. And so far the 2 times I got involved in a stereo magazine test it was very clear to me what the benefits are of more bits. More space around the notes, more tonal correctness and more relaxed......, so I don’t have to be convinced anymore.

Posted on: 02 April 2018 by DaveBk

Long ago I gave up on resolution testing, as I found far more differences in the attention to detail in mastering, rather than the final resolution. On a really excellent master, resolution makes a difference, so I typically buy a new album in the highest resolution available on Qobuz, but I have some excellent 44.1/16 material.  

Posted on: 02 April 2018 by Bert Schurink
DaveBk posted:

Long ago I gave up on resolution testing, as I found far more differences in the attention to detail in mastering, rather than the final resolution. On a really excellent master, resolution makes a difference, so I typically buy a new album in the highest resolution available on Qobuz, but I have some excellent 44.1/16 material.  

I agree with you, while I do buy standard the highest quality I get of every album. So if it is in high res quality I will never buy the standard quality....

Posted on: 02 April 2018 by DrPo
DaveBk posted:

... On a really excellent master, resolution makes a difference...

Dave, this thread is precisely about such cases... any example where such differences were more easily audible?

Posted on: 05 April 2018 by Rich 1

I downloaded from a well respected competitor up North a small selection of test tracks at various resolutions. My wife and I swapped remotes to see if we could hear a difference and see which we preferred, piano, organ and accompanied singing. Piano and singing was easy for both of us to consistently choose the highest resolution as best. Whilst my wife also had no trouble with the organ piece. I on the other hand I had to really study that piece before I came anywhere near consistent. My wife and I still can enjoy lower resolution music. My conclusion is it's not just resolution, it's also the quality of the musicianship. As far as high resolution is concerned, yes it's usually better but not always. It depends on the quality of the original recording, cd may sound better because the lower resolution might suppress some issues. High quality a analogue also has the ability to to show defects in the recording. So don't shoot the messenger when it might be short comings in the recording process. Rich 

Posted on: 05 April 2018 by Innocent Bystander

I suppose the point the OP was making is that in this example you can’t KNOW that the different resolutions are simply that, with no differences in mastering, and it is conceivable that a vendor could have a vested interest in persuading you that the higher res, usually more expensive, one sounds better. If you take the highest res files and make lower res copies using something like dBPoweramp you could verify whether you still hear differences (though you wouldn’t know if they are due directly to the resolutions, or to the process of downsampling, but that is perhaps irrelevant as it would apply whenever the copy is not the original recording resolution)

This reminds me of a pet gripe: music released at the original recording/mastering resolution should be the cheapest, because there has been no need for a further process to make a lower res copy...

 

Posted on: 05 April 2018 by Rich 1

Couldn't agree more with your last paragraph, does it cost that much more to download an admittedly larger Hi res file? And yes, you can argue that using a recording companies versions, they have an interest in promoting higher cost recordings. However I have to put my trust somewhere, I have to trust the likes of Naim and Linn to play fair, if for one minute I thought they weren't I would be onto Trading standards as unsubstantiated claims by manufacturers is illegal in the UK. Yesterday I tried, using my laptop, to downgrade Hi res to cd quality, admittedly only using two pieces. Result is inconclusive at the moment, need more time and effort but not sure if I'll follow up as I'm enjoying my system as it is! Rich 

Posted on: 05 April 2018 by Huge

That same organisation has released some wonderful recordings where both the musicianship and the recording are first rate.

The Dunedin Consort's Brandenburgs, and SCO's recordings of the Weber wind concertos and Berlioz Symphonie Fantastique spring to mind.

I re-encoded one movement of the  Weber at lower sampling resolution (24/48), lower bit depth as well (16/48) and at MP3 (16/44.1), specified the files as A, B, C, D (not in order) and wrote down which was which, then I left it long enough for me to forget.  I played them back to some friends (yes I've still got some), who consistently identified the 24/192 and the MP3. The 24/48 and 16/48 were a bit harder to distinguish when listening at moderate levels (we didn't try at higher volumes, only at the normal listening levels at which we were comfortable), but they still got it right more often than not.

OK, it's not a true double blind test (I knew which file was being played even if I didn't know the content of that file); but it's very close to it.

Posted on: 05 April 2018 by DrPo

Thanks Huge, this is indeed very close to what I had in mind. Incidentally, it was some test tracks (CD Qual vs higher resolution) from the Northern (UK) competitor’s homepage  (an aria excerpt, don’t recall details) that got me thinking in the first place about the topic as I could not reliably differentiate between them. In fact, I would imagine that someone would offer for such a test a track where differences would be most easily identified!

Posted on: 05 April 2018 by Innocent Bystander

Of course, hw easily identified differences may be depends on mamy factors, including the listener’s ears (physical, and mental interpretation), system (all parts thereof) and room (often overlooked, though really just as much a part of the system as anything else).

Posted on: 05 April 2018 by SongStream

While I must confess I've never done any blind testing, mainly due to the fact the my family, and the few friends I have, would have further cause to officially declare me a lunatic, I do have a conclusion in my own mind on this.  Same master, MP3 at 192, or even 320kbs, easily detectable vs full fat lossless compression, or original CD ripped to WAV / purchased via download; easily audible.  Uncompressed CD quality vs Hi-Res 24bit - and blah, I am pretty sure I would fail to pick the higher resolution in a blind test, other than by equally blind luck.

I buy albums via download in hi-res from Qobuz, because there is often a discount as a HiFi Subime subscriber, so why not. but otherwise I wouldn't bother.  The recording, and mix, makes a far bigger difference to the overall value and quality of a recording, than any amount of bit depth, or sample rate, once you're beyond the typical lossy formats.  I reserve judgement on MQA as I've never it, but in principle, based on my conclusions, it seems like a waste of time and money too.

Posted on: 05 April 2018 by MangoMonkey

Spotify (paid) sounds perfectly fine in my system. It even has lower noise floor than from uniticore..

Posted on: 05 April 2018 by Bob the Builder

Interesting thread as I have just aquired a ripping solution in preference to buying Hi Res downloads, partly this was due to the fact that you can buy very good quality CD's at a fraction of the cost of downloads and also because I couldn't hear any real difference.

I'm starting from scratch as prior to last week the only CD's I owned was a small box of sentimental titles that I couldn't bring myself to give to charity a few years back. I wanted to build a smallish system that I could pack in my car for longer stays abroad  and the vortex box >> 2Qute >> small active speakers seemed to fit the bill.