Lossless? Really?
Posted by: madgerald on 29 April 2014
Not sure if this is the right place to ask this Q but pretty sure someone will be able to help...
Following the principle that original is best (I've been brainwashed by vinylheads) and if you mess with something you make it worse then if you are going to listen to digital music then CD must be best format (unless you can get your hands on the original uncompressed file). 
A good friend of mine disagrees (yes he is in IT) and says that ripped "lossless" will be as good as the original CD since its all just 1's and 0's anyway. The only way to settle the argument would be to do a blind test streaming a ripped "lossless" CD against the original played on my CDX2 through the same DAC, amp and speakers to see if we can hear the difference. Trouble is I don't have a separate DAC and am not about to buy one just to prove him wrong.
Has anyone conducted such a test and if so what were the results? Feel free to point me at a previous post if this has been discussed before.
Thanks if you can prove me righteous 
Bill
Big Bill - ok lets beg to differ on some of those points - any way i hope i am right or my job as a professional voice/network design engineer will disappear on Tuesday 
and i would have wasted the last 20 years.. but at least my customers have paid me well for it 
PS if you are really interested I can point to you texts on data transmission/encoding and entropy which is of course a grounding principle in ICT. Ok I haven't gone into the deep theory and mathematics since being an undergraduate but it was one of my favourite areas of study, and i certainly use the principles now. You are questioning the words I am using to try and describe the basics in a non technical way - and it is beyond my capability clearly to explain it to you via this forum - perhaps if we had a white board - I usually find it helps.
Of course Morse was quite inefficient in information rate, but it was a digital or even a quantised mapping of the alphabet/numbers and certain special codes, and indeed in use you would / do send sentences/messages and seek positive confirmation.. or resend.
Data comms that use TCP use dropping algorithms to regulate data flow, so data is deliberately lost in transmission and needs to be resent. WRED is an example of this in class of service managed networks. Of course lower layers such as link frames are discarded if checksums fail due to data transmission error - and the above transport/session mechanism will manage the resend or the data is lost.
UDP is connectionless or often fire and forget - and so if corrupted it is lost for ever - which is why in class of service managed networks with give special consideration to UDP when carry realtime info such as encoded voice datagrams - as we know networks which manage class of service across a layer 3 boundary will some times need to deliberately drop traffic.
Interestingly UDP uses sequence numbers and can arrive out of order - from a data transmission point of view the datagram is digitally received 100% - however if used in realtime - its position and relationship to other datagrams is important with respect to time, therefore an application might chose to discard the successfully received data - as its information has become invalid.
Therefore one needs to look at the combination of the information and not just rely on the data transmission, and this is my point - as effectively digital information transmission is not simply a case of sending digital data.
In the same way the data transmission can be unreliable and lost but the information recovered using transport methods - and of course this was the basis of the development of the OSI 7 layer model that data/info comms has been largely based on over the 40 years.
PPS - being an engineer I like to design optimum efficient systems, and wasteful inefficient data consumption for no benefit niggles me - you could argue it is sloppy lazy design and it is a lot easier to be inefficient in data transmission rather than be efficient.
PPPS Graeme - no salutes necessary please
Here is one example. I think Graeme was saluting Bill btw.
I have no idea what UDP is, and probably don't need to know. What I do know is that last year, when I was setting up my streaming system, Simon was a total lifesaver, giving me really helpful advice when many others would have given up, and that he continues to be supportive of us numpties. So, Bill, please stop being confrontational and join the Forum spirit of mutual advice and support. It's easy to be negative, and harder to be positive, and at the end of the day it's only networking, not world peace. Let's all hold hands and feel the Ethernet love.
+ 1 to the sage advice of Lord Emsworth.
As I asked the original question just a quick note of thanks for all the info - I have learned a huge amount from all of the people who have taken time to post!
Cheers 
Bill
As I asked the original question just a quick note of thanks for all the info - I have learned a huge amount from all of the people who have taken time to post!
Cheers
Bill
Hi - Bill you are welcome. You certainly kicked off some debate and interesting views from some - hopefully you read enough to satisfy your original post
Simon
Shame the thread has gone somewhat off track - I have to admit i do enjoy Simons contributions although a lot of the more interesting subjects discussed on here could be more easily debated over a couple of pints...
But back to the original question, i still prefer lossless, in this case AIFF. Every time i've changed my computer audio solution - whether Mac 'n' Dac, Linn and Naim streamers and now the Devialet, i re-check my preference as i could gain quite a bit of space back on my hard drive. I take a few of my favourite tracks, convert them to ALAC and listen to them over the next week or two, comparing them to the AIFF versions and always find myself going back to the AIFF versions. I'm quite happy that when decompressed, the ALAC file is the same as the AIFF file and so assume there is some mechanism in the playback chain which favours the AIFF version. As Naim themselves point out in their white papers, processor loading when decompressing increases the power supply noise floor in their streamers, hence the preference for uncompressed WAV.
It's amazing, and I'm sorry I'm going to use technical words for brevity.
These posts always deteriorate into the people like Simon, who do not understand the TCP IP stack and imagine all sorts of issues with imperfect digital information transfer, arguing with people who really do understand it, like Big Bill. It's a shame that people deny science and 50 years of research in this area so readily.
It comes down to this - No matter what happens, the digital information in a signal is either transmitted with 100% success or it is not considered to be transmitted at all. The file format and the cables that you use are completely transparent to the information in the signal (see the morse code analogy above). The only thing that matters before decoding is the information in the signal. Whether that is lossless compressed or uncompressed makes no difference to the PCM/DSD bits that come out the end of the process ready to be decoded by the DAC. It's that simple, no snake oil needed.
For those who want more information, this is a good read
Thanks Conrad, needless to say I agree with everything you say.
BananaHead I think, like me, Wat may also be a big classical fan to say that, I am also in love with Maria Callas and Renata Tebaldi!
Wat I will try and explain, so keep with me please. What you say about TCP/IP is right , I suppose; it ain't perfect. But there was an interesting point in that article referenced by Conrad, where he states that it wasn't so much a grand plan but that it evolved, I had never looked at it like that before. Furthermore, it was this evolution of the standards that made it fit into so many niches.
Onto Upnp and servers in general. One of the things that has developed most in the time I have been involved in IT is Application Servers. Before PC networks we had one massive computer (wrt the times) and a load of dumb (maybe slightly intelligent with some screen rendering) terminals. But such systems, although they sound really archaic, had one great advantage over PC networks and that was that the newly developed database platforms, like SQL, ran on a single machine - don't sound like much does it? When PC networks started to replace monolithic IBM mainframes (90% of the market) people wanted to run applications that required a database and this is where the problems started. Peoples started developing applications using 4GLs or newly developed database systems like dBase, DataMaster, Access etc and these were all OKish on a single machine but were rubbish on networks. This was because they all were trying to open, at the same time, the database files which were stored on a file server and regular as clockwork these files got corrupted. The answer came in the form of the database server and I believe the first successful implementation was developed by Amin Gupta who was on the original IBM team that developed SQL. This server sat on a machine (i think a Novell Netware file server) and it was the only machine that opened the database files, which resided on the same machine. Corruption of data gone in an instance. Now the software we developed did not open database files but sent messages to the database server, for example, "Send all the invoice records for company A" and back came what come to be known as a result set. btw the message "Send all...etc" would have been sent using the SQL language after your application had connected to the database server. Just like your NAIM streamers connect to a Upnp server - see I am getting there. This totally revolutionised the IT world and today virtually all data-centric applications use this approach. The actual database server products of today, are incredibly sophisticated, like MS SQL server and Oracle, or even the free db server MySQL. They can do things like span the databases across different machine, have redundancy built in - Oh I could go on and on and often do.
Now this success I suppose sparked a whole load of other server applications, Web Servers, Application servers - where you use code not resident on the local PC but on some remote server, eq ASP pages, JSP pages etc. And a whole host of other things.
The use of servers like those described above give freedom of use, they allow the use of different PC platforms - an Apache server can be used by PCs, Apples, Androids etc, for example and they make the implementation on new devices relatively simple.
Upnp is an example of this. You can run Upnp servers on a NAS and you can have a 'Control Point' (Is it only me who thinks that is a silly name?) on a PC, Apple etc. The actual music files you play on your streamer are never opened on your streamer. Take my setup as an example: my NAS is password protected but my Upnp server (minim) has no extra passwording. To access my music files as actual files you would have to know the password but to stream them you don't. btw if that doesn't make a lot of sense then please ask and I will try again.
The other benefit we saw from using database servers instead of file sharing was in speed. I remember a client who had developed an Access database which worked fine on a single machine, was noticeably slower on 2 machines and stopped on 3. So Upnp servers give you the possibility having devices all round your house to play music and they make delivery timely and implementation very easy.
It's amazing, and I'm sorry I'm going to use technical words for brevity.
These posts always deteriorate into the people like Simon, who do not understand the TCP IP stack and imagine all sorts of issues with imperfect digital information transfer, arguing with people who really do understand it, like Big Bill. It's a shame that people deny science and 50 years of research in this area so readily.
It comes down to this - No matter what happens, the digital information in a signal is either transmitted with 100% success or it is not considered to be transmitted at all. The file format and the cables that you use are completely transparent to the information in the signal (see the morse code analogy above). The only thing that matters before decoding is the information in the signal. Whether that is lossless compressed or uncompressed makes no difference to the PCM/DSD bits that come out the end of the process ready to be decoded by the DAC. It's that simple, no snake oil needed.
For those who want more information, this is a good read
Thanks Conrad, needless to say I agree with everything you say.
BananaHead I think, like me, Wat may also be a big classical fan to say that, I am also in love with Maria Callas and Renata Tebaldi!
Wat I will try and explain, so keep with me please. What you say about TCP/IP is right , I suppose; it ain't perfect. But there was an interesting point in that article referenced by Conrad, where he states that it wasn't so much a grand plan but that it evolved, I had never looked at it like that before. Furthermore, it was this evolution of the standards that made it fit into so many niches.
Onto Upnp and servers in general. One of the things that has developed most in the time I have been involved in IT is Application Servers. Before PC networks we had one massive computer (wrt the times) and a load of dumb (maybe slightly intelligent with some screen rendering) terminals. But such systems, although they sound really archaic, had one great advantage over PC networks and that was that the newly developed database platforms, like SQL, ran on a single machine - don't sound like much does it? When PC networks started to replace monolithic IBM mainframes (90% of the market) people wanted to run applications that required a database and this is where the problems started. Peoples started developing applications using 4GLs or newly developed database systems like dBase, DataMaster, Access etc and these were all OKish on a single machine but were rubbish on networks. This was because they all were trying to open, at the same time, the database files which were stored on a file server and regular as clockwork these files got corrupted. The answer came in the form of the database server and I believe the first successful implementation was developed by Amin Gupta who was on the original IBM team that developed SQL. This server sat on a machine (i think a Novell Netware file server) and it was the only machine that opened the database files, which resided on the same machine. Corruption of data gone in an instance. Now the software we developed did not open database files but sent messages to the database server, for example, "Send all the invoice records for company A" and back came what come to be known as a result set. btw the message "Send all...etc" would have been sent using the SQL language after your application had connected to the database server. Just like your NAIM streamers connect to a Upnp server - see I am getting there. This totally revolutionised the IT world and today virtually all data-centric applications use this approach. The actual database server products of today, are incredibly sophisticated, like MS SQL server and Oracle, or even the free db server MySQL. They can do things like span the databases across different machine, have redundancy built in - Oh I could go on and on and often do.
Now this success I suppose sparked a whole load of other server applications, Web Servers, Application servers - where you use code not resident on the local PC but on some remote server, eq ASP pages, JSP pages etc. And a whole host of other things.
The use of servers like those described above give freedom of use, they allow the use of different PC platforms - an Apache server can be used by PCs, Apples, Androids etc, for example and they make the implementation on new devices relatively simple.
Upnp is an example of this. You can run Upnp servers on a NAS and you can have a 'Control Point' (Is it only me who thinks that is a silly name?) on a PC, Apple etc. The actual music files you play on your streamer are never opened on your streamer. Take my setup as an example: my NAS is password protected but my Upnp server (minim) has no extra passwording. To access my music files as actual files you would have to know the password but to stream them you don't. btw if that doesn't make a lot of sense then please ask and I will try again.
The other benefit we saw from using database servers instead of file sharing was in speed. I remember a client who had developed an Access database which worked fine on a single machine, was noticeably slower on 2 machines and stopped on 3. So Upnp servers give you the possibility having devices all round your house to play music and they make delivery timely and implementation very easy.
The usual obfuscation going on here. I think it is worthwhile looking at the basic facts rather than dusty academic papers. At the risk of generalisation and repetition, I might describe the 'challenge' as:
1) FLAC file (a track of 5 mins length) = 40MB
2) Throughput on home network = 10MB per second
3) Network cards / hubs throughput = 10MB per second
Given the above, I might venture the following high-level solutions:
1) transfer entire file to DAC memory in 4 seconds then play in entirety
2) smooth stream over 5 mins at 0.13 MB per second (approx 1/80th of operating throughput) adding a bit on for buffering.
3) some hybrid of the above
Whichever I do, decoding is a very, very, very small job. An in-memory operation that could be completed in milliseconds, whether all at once or in dripfeed fashion, for a complete FLAC file with negligible processing. Sure there may be an processing side-effect - if you're using a ZX81.
As mentioned on here a long time ago, a Raspberry Pi could be adapted for streaming duties. Certainly it could hold the track above in memory. As could many smartphones and even some wrist-watches.
Perhaps you are right, perhaps you are wrong. No one solution for any one set-up or any one set of ears. as for obfuscation of facts .......
There is no perhaps, the above simplistic measures of capacity and throughput are a perfectly sound example. You might work out what the specific parameters are for your specific situation if you so desired but I doubt it makes much odds. This is about focusing on real issues rather than making a song-and-dance about non-issues.
Regarding the right solution for anyone's ears, that isn't easy to compare notes on without everyone having the device\s in question available to listen to and all being in the same shared space. That is not going to happen on a forum.
Marky, thanks for your input. I'll make appropriate use of it.
Agree that the right solution is not easy to compare on the forum and that everyone is free to use the forum and other sources to increase their understanding.
Jude
