Servers - a complicated matter

Huge posted:

Adam Zielinski posted:

I will reply with one of my favourite quotes

'I've seen things you people wouldn't believe. Attack ships on fire off the shoulder of Orion. I watched C-beams glitter in the dark near the Tannhäuser Gate. All those moments will be lost in time, like tears...in...rain...

Just a thought:  Is the quote from the film only, or from Philip K. Dick's "Do Androids Dream of Electric Sheep"?

(Which has to be one of the best book titles ever.)

I believe that it was from the film only and that the tears bit was ad libbed although I could be wrong.  Didn't think much of the book to be honest, one of those rare times when the film is better in my view

I hear there is a sequel in the works - and Mr Ford is in it - but he is going to be definitely showing his age.

Based on the responses here, there should be a Blade Server Runner film made.  

Thank you. . . . I'm here all week!!

None of you geeks liked my Blade Server pun? What's wrong with you?? This is HUMOR people.  

Gi'uz a chance!
Wot...  Have you got no empathy?

Bart posted:

None of you geeks liked my Blade Server pun? What's wrong with you?? This is HUMOR people.  

We were all waiting for the offers to pour in to congratulate you....

Bart posted:

None of you geeks liked my Blade Server pun? What's wrong with you?? This is HUMOR people.  

Someone's got it in for me they're planting stories in the press!

Jonas Olofsson posted:

While streaming opens up a fantastic way of enjoying music, servers seems to add a bit of complexity to the equation. 

To be able to produce a server, working for a long time with out any problems, and, if problems show up, being able to repair, is a complicated machine. 

I do think audio business in general isn't set up for that and that serious server producers with a heart for audio is what you have to look for over time. 

Personally I wouldn't go back to CD or LP since the conviens is to big. 

So, rebility could be first choice 

//Jonas

I personally think that audio / video streaming services are ready for prime time. However, it may require some basic skills to drive it properly and well,  just like driving a car, you need to learn how to drive it.

Having owned a number of streamers, from the Logitech Transporter to Linn DS, I have come across no issues at all except some issues at sources (mostly streaming radio servers with inadequate infrastructures).

Bart posted:

None of you geeks liked my Blade Server pun? What's wrong with you?? This is HUMOR people.  

Ahh that explains it - some of us here perhaps better relate to HUMOUR ...  

And for those of you needing a little bit more storage

https://news.samsung.com/globa...rise-storage-systems

Might have to wait a little while though

Simon-in-Suffolk posted:

Hi Jason, indeed, it's not so much simpler network, but a simpler media server / NAS setup seems to, in my examples, provide a more consistent inter packet timing within bursts of transfers and when the data transfer is of a fast enough rate encourages the Naim streamer to use the TCP ZeroWindow flow control mechanism which seems to get the best SQ out of the Naim streamer. 

Simon

OK, so that's pique'd my interest... Doesn't TCP ZeroWindow mean that the receiver in this case the Naim, has filled it's TCP receive buffer and is refusing more data until it can process what it has? Which would imply you're seeing that the Naim works best with a full buffer? And that some media servers can't supply data fast enough to the Naim to keep the buffers full if they're doing transcoding etc? 

Jason posted:

Simon-in-Suffolk posted:

Hi Jason, indeed, it's not so much simpler network, but a simpler media server / NAS setup seems to, in my examples, provide a more consistent inter packet timing within bursts of transfers and when the data transfer is of a fast enough rate encourages the Naim streamer to use the TCP ZeroWindow flow control mechanism which seems to get the best SQ out of the Naim streamer. 

Simon

I think I'm with you, in essence a consistent flow making the streamers job easier, OK. Don't want to stray to far from the OP's original point, but would be interesting to see if that 'flow' differs when simply using a different upnp server.

Simon,

Thanks for sharing your (preliminary) impressions/findings. Regarding your "simpler media server" comment, may I ask if you are generally recommending NAS units with:

a) fewer bays

b) no hardware transcoding

c) less powerful CPU chips (say, Marvel vs. Intel)

d) lower idle power draw

e) some of the above/all of the above

Just an anecdotal data point: I use a simple 1-bay Synology NAS that is not totally bottom of the line but close. It runs Logitech Media Server and feeds a microRendu in SqueezeLite mode. I've found this to be superior to the same hardware running Minimserver outputing with the mRendu's DLNA mode. I've been toying with the idea of getting Sonore's Sonic Transporter as a replacement for the NAS (and perhaps someday to run Roon although I have no interest in Roon now), but it seems like I'm well into diminishing returns territory here (and should instead get/wait for an LPS or the forthcoming Uptone LPS-1).

Anyway, I'm interested in your opinion. Thanks again for sharing your NAS findings.

Lorenzo

Lorenzo, unfortunately my extra curricular investigations haven't concluded whether those attributes make a difference (apart from transcoding - see below) - I suspect it is not they per se that drives the variability.

However if you keep the processing required to stream simple - i.e. no transcoding and keep the other applications on the NAS to a minimum and certainly whilst streaming then I am seeing improvements in terms of more consistent network timing. I am finding a very basic Netgear 102 NAS with 2x2TB of hard disk on it with no transcoding sound pretty optimal for me now - and certainly when I look at its behaviour on the line its what I would expect in terms of  network dynamics and timing for such a good sounding server playing through to my NDX.

Now TCP stack offloading could possibly  mitigate the effects of transcoding - and this will be worth exploring in higher end servers. On typical NAS machines I don't think many support TCP stack offloading.

Personally I think if you want un-compromised music serving then you have to look at what else you're asking your NAS/server to do... 

At the bandwidth required for serving music, I'm not sure I'm in agreement around esoteric solutions like TCP offloading. 24 bit stereo at 192KHz only needs just over 1 MB/s of bandwidth, or around 10Mbits. These are't horrific networking speeds, and should not saturate most reasonably modern NAS or server devices. The time to worry about TCP overhead is when you're pushing much, much higher volumes. 

So if your NAS's only purpose is to serve music files then there really should be no major strain to it.

Given that virtually no-one uses ethernet hubs anymore, then if your NAS/Server and streamer are hardwired and connected via a switch, then I struggle to see how you are going to hit network and bandwidth issues in normal usage. 

WiFi is best thought of though as a hub IMO... it's not point to point like a switch and everything on it has to share the bandwidth. Multiple channels using MIMO etc can help, but it's amazing now how many wireless devices we actually end up with. One or two smart TV's using Netflix over Wifi and you're starting to shift a lot more data than just music, and therefore if your NAS or streamer is one of those devices then you increase the chance of issues. 

On your NAS itself, then consider the number of disks/bays. Consumer SATA (non-SSD) drives are good at doing one thing, and hopeless at doing more than one thing. Given that most two bay NAS drives are probably mirrored, it's best to assume they act like one drive. If that one drive serves your DVD/Blu-ray rips and music, and you're not the only person in the house, then there's a good chance they will struggle. In Enterprise level storage arrays, a RAID array of 8 fast enterprise SAS disks is only rated for around 1100-1500 IOPS. The same number of SATA drives is rated at about one tenth of that... they really are that rubbish. (Enterprise SAS are 15,000 rpm and more SCSI based, whereas most consumer SATA drives in NAS's are 5400 rpm without the more advanced controller algorithms to optimize I/O access.)

So what can you do? Look at a 4 bay NAS and have your music library on a dedicated pair of drives, and everything else on the other pair. Personally I prefer mirrored pairs as it's a simpler recovery and far better *if* a drive fails. Avoid RAID-5/6. SATA drives aren't great at it, the controllers usually rely on software RAID in cheap (read home) NAS units, and performance when a drive fails is awful. Unless you have to use the newest large capacity drives then the cost of more established capacities is usually far better, and will deliver better performance all round.

If you have the money then yes you can use 2TB SSD drives which will mitigate pretty much all I/O issues but it is not cheap and you still have to consider redundancy.

Which sort of comes back to Simon's point on a simple 2 drive NAS being best. I think that this is more to do with it being dedicated to a simple solitary task than it is anything else. 

The more complex our NAS/Servers get, the more tempted we are to get them to do lots of other things, but the sad, simple fact is, once you go down that road you start hitting all the problems servers in datacenters have grappled with for ages, for which there aren't really any simple solutions.

If you dedicate a single NAS to just serving your music, hard-wired on the same switch as your streamer, and don't stress the CPU with transcoding, I'm not sure what else you can do to optimise it's ability to serve music. (multi-room starts to stress disks if you have lots of rooms all playing together, bit still nowhere near as badly as video). And it may be the cheapest solution. A simple 2 drive NAS with 4TB (or now even greater) drives, dedicated to just music may well be cheaper than trying to get a fancier multi-bay NAS/server to do lots of things including music working well. 

Just my thoughts. 

Ian_S posted:

Personally I think if you want un-compromised music serving then you have to look at what else you're asking your NAS/server to do... 

At the bandwidth required for serving music, I'm not sure I'm in agreement around esoteric solutions like TCP offloading. 24 bit stereo at 192KHz only needs just over 1 MB/s of bandwidth, or around 10Mbits. These are't horrific networking speeds, and should not saturate most reasonably modern NAS or server devices. The time to worry about TCP overhead is when you're pushing much, much higher volumes. 

So if your NAS's only purpose is to serve music files then there really should be no major strain to it.

Given that virtually no-one uses ethernet hubs anymore, then if your NAS/Server and streamer are hardwired and connected via a switch, then I struggle to see how you are going to hit network and bandwidth issues in normal usage. 

WiFi is best thought of though as a hub IMO... it's not point to point like a switch and everything on it has to share the bandwidth. Multiple channels using MIMO etc can help, but it's amazing now how many wireless devices we actually end up with. One or two smart TV's using Netflix over Wifi and you're starting to shift a lot more data than just music, and therefore if your NAS or streamer is one of those devices then you increase the chance of issues. 

On your NAS itself, then consider the number of disks/bays. Consumer SATA (non-SSD) drives are good at doing one thing, and hopeless at doing more than one thing. Given that most two bay NAS drives are probably mirrored, it's best to assume they act like one drive. If that one drive serves your DVD/Blu-ray rips and music, and you're not the only person in the house, then there's a good chance they will struggle. In Enterprise level storage arrays, a RAID array of 8 fast enterprise SAS disks is only rated for around 1100-1500 IOPS. The same number of SATA drives is rated at about one tenth of that... they really are that rubbish. (Enterprise SAS are 15,000 rpm and more SCSI based, whereas most consumer SATA drives in NAS's are 5400 rpm without the more advanced controller algorithms to optimize I/O access.)

So what can you do? Look at a 4 bay NAS and have your music library on a dedicated pair of drives, and everything else on the other pair. Personally I prefer mirrored pairs as it's a simpler recovery and far better *if* a drive fails. Avoid RAID-5/6. SATA drives aren't great at it, the controllers usually rely on software RAID in cheap (read home) NAS units, and performance when a drive fails is awful. Unless you have to use the newest large capacity drives then the cost of more established capacities is usually far better, and will deliver better performance all round.

If you have the money then yes you can use 2TB SSD drives which will mitigate pretty much all I/O issues but it is not cheap and you still have to consider redundancy.

Which sort of comes back to Simon's point on a simple 2 drive NAS being best. I think that this is more to do with it being dedicated to a simple solitary task than it is anything else. 

The more complex our NAS/Servers get, the more tempted we are to get them to do lots of other things, but the sad, simple fact is, once you go down that road you start hitting all the problems servers in datacenters have grappled with for ages, for which there aren't really any simple solutions.

If you dedicate a single NAS to just serving your music, hard-wired on the same switch as your streamer, and don't stress the CPU with transcoding, I'm not sure what else you can do to optimise it's ability to serve music. (multi-room starts to stress disks if you have lots of rooms all playing together, bit still nowhere near as badly as video). And it may be the cheapest solution. A simple 2 drive NAS with 4TB (or now even greater) drives, dedicated to just music may well be cheaper than trying to get a fancier multi-bay NAS/server to do lots of things including music working well. 

Just my thoughts. 

Nice well thought out post, which I agree  with almost wholeheartedly. Having messed around with my Synology NAS this week trying to get music out of it via various means, am not convinced it is particularly any way forward for myself. As a technical exercise it has been interesting but that's about it for myself.

As to the pros and cons of WiFi versus Ethernet, as I achieve 55 Mbps available to any of my devices, the constraint is with the receivers own processing capabilities eg  the Muso's maximum of 55 Mbps via WiFi.

Now, you can argue as much as you want about realistic throughputs but 55 Mbps is easily enough for music and probably video as well in my own environment, but I know and understand many will totally disagree.

I now know, however, I can deliver Hi Def files from my NAS to my devices via WiFi if that is what I want to do. 

Ian - regarding your point of networks, and bandwidths - my point is nothing to do with throughput and bandwidth - that kind of is irrelevant unless you reach some sort of hard limit  - its about inter packet timing consistency - and I have undertaken some tests and measurements and with an NDX I have found a clear correlation between my subjective SQ and TCP packet timing consistency. By having the media server perform other tasks such as transcoding I have seen a small but noticeable in terms of SQ deterioration  and definitely measurable deterioration in timing consistency. My point of offloading is that one should isolate the transcoding processor timing from the TCP stack timing.. I have yet to explore and measure this formally - not least because of the platforms i am testing with I have not been able to configure the offloading of the TCP stack - assuming if its possible.

So yes - so far my measurements have shown and my subjective assessment using my ears support a media server doing minimum duties appears to provide the best SQ - certainly with an NDX

Simon-in-Suffolk posted:

Ian - regarding your point of networks, and bandwidths - my point is nothing to do with throughput and bandwidth - that kind of is irrelevant unless you reach some sort of hard limit  - its about inter packet timing consistency - and I have undertaken some tests and measurements and with an NDX I have found a clear correlation between my subjective SQ and TCP packet timing consistency. By having the media server perform other tasks such as transcoding I have seen a small but noticeable in terms of SQ deterioration  and definitely measurable deterioration in timing consistency. My point of offloading is that one should isolate the transcoding processor timing from the TCP stack timing.. I have yet to explore and measure this formally - not least because of the platforms i am testing with I have not been able to configure the offloading of the TCP stack - assuming if its possible.

So yes - so far my measurements have shown and my subjective assessment using my ears support a media server doing minimum duties appears to provide the best SQ - certainly with an NDX

Hi Simon - is that really the case though for the timing? All the studies I've seen suggest that the biggest issue with TCP is around the transfer of data between processor memory & cache and the network card. Intel have their own alternative in their Xeon processor line which is all about trying to optimise that flow. (I/O acceleration technology)

No home based NAS device and even most home media servers won't use Xeon's or drive their CPU's to the point where any of this matters so much with music transfer IMO. 

What's more interesting is the behaviour of the Naim device. It's unlikely to have the processing grunt of a NAS or server, and is more likely to get stressed from a CPU perspective. The fact that you're seeing TCP Zero Window flows indicates that it's relatively easy to overload the Naim on the network. What's more interesting is that when it's in this mode, you find the SQ improves. Is this down to for whatever reason a more optimised processing flow around it's TCP receive buffers in that state? Is there a general theme on these boxes that says the less processing they do internally the less digital noise is created and therefore the less pollution of the resulting analogue audio signal? 

Ian it certainly is in my findings  - i developed some bespoke filters on Wireshark to quite nice graphically show this - I was rather pleased with my findings and kind of supports and might well explain some of the SQ differences we hear reported on this forum and elsewhere  on lossless transfer and also transcoded transfer from different media servers such as Melco etc... i thought there is no smoke without fire.. - but how so? as the data is identical or bit perfect - Now I know from my professional engineering world network inter frame timing variation can be pivotal in subjective sound quality assessment  - and so I set about applying my professional engineering approaches to our humble domestic replay systems - and yes there it was a correlation. Now the reasons for the SQ differences are  slightly different on our home media systems using TCP as opposed to low latency UDP where I come across this professionally  - so the timing variation I believe affects the TCP/IP stack crosstalk  to connected audio systems. This crosstalk idea fits rather nicely with with conventional systems theory as well. The RFI reason per se is a bit blunt and doesn't really address the issues and differences.. and being an engineer that isn't good enough for me 

andarkian posted:

 

Now, you can argue as much as you want about realistic throughputs but 55 Mbps is easily enough for music and probably video as well in my own environment, but I know and understand many will totally disagree.

I now know, however, I can deliver Hi Def files from my NAS to my devices via WiFi if that is what I want to do. 

The point I'd make there is that will be the case only if your Naim is the only device on your WiFi... only one device at a time can use the WiFi bandwidth. So even your iPhone/iPad/android device running the Naim control app will compete with the Naim for time on the network if both are using Wifi... likewise any other device on the WiFi looking at emails, you-tube, sending tweets, just polling for updates etc. etc. None of which is great or ideal for anything that is timing critical. Which is why I say it's best to think of Wifi connections as an old fashioned hub connection. 

Further to Ian's observation, you can't directly relate a Wifi synchronisation speed with IP data throughput.... Wifi is rather inefficient - it has to be to work in an inherently un reliable medium - there is much overhead. ITs also is half duplex single collision domain. So in an ideal perfect 55Mbps wifi connection the best IP throughput you could  relsitically expect would be around 28Mbps or hereabouts - and if there is anything else connected  on the SSID (wifi network)  or sharing those channels in the neighbourhood that will fall away quickly - you can quite easily see that Hidef 192/24/2 (approx 10Mbps) transfer on a current Naim streamer wifi is pushing it other than in perfect idealised circumstances.

Simon

Simon-in-Suffolk posted:

Ian it certainly is in my findings  - i developed some bespoke filters on Wireshark to quite nice graphically show this - I was rather pleased with my findings and kind of supports and might well explain some of the SQ differences we hear reported on this forum and elsewhere  on lossless transfer and also transcoded transfer from different media servers such as Melco etc... i thought there is no smoke without fire.. - but how so? as the data is identical or bit perfect - Now I know from my professional engineering world network inter frame timing variation can be pivotal in subjective sound quality assessment  - and so I set about applying my professional engineering approaches to our humble domestic replay systems - and yes there it was a correlation. Now the reasons for the SQ differences are  slightly different on our home media systems using TCP as opposed to low latency UDP where I come across this professionally  - so the timing variation I believe affects the TCP/IP stack crosstalk  to connected audio systems. This crosstalk idea fits rather nicely with with conventional systems theory as well. The RFI reason per se is a bit blunt and doesn't really address the issues and differences.. and being an engineer that isn't good enough for me 

Yes, I have to say I hate the whole RFI in HiFi debate. It's far too vague. 

My curiosity lies in the fact that many believe WAV performs better than FLAC due to less processing. So could the SoC used to provide the TCP connectivity/processing perform more efficient processing when it has full buffers than when it's fed on a busier network where it spends more time managing the incoming packets in a bittier (is that a word?) fashion? That busy-ness could be down to either network (more likely on WifI) or music server which is either underpowered (must be getting rarer) or overworked, the latter becoming more likely in a world where more and more devices are active concurrently in the average household. 

andarkian, it is nice if you can achieve 55 mbps thru wifi, but to make sure the performance is consistent, you may want to dedicate that wifi channel for music streaming only and no collisions with any other wifi channels out there.

Well yes Ian - you do raise another interesting point - and I have noticed with Tidal as well as local streaming - when the throughput is in fast regular bursts - where each burst has low inter frame timing variation I hear the best SQ. Now when the transfer is fast, the Naim streamer issues a Zero Window command which halts the media server TCP windowing mechanism - until the Naim streamer has emptied its higher level buffers somewhat and then the streamer reestablishes  the TCP windows with the server and another burst of frames ensue - and as I say this seems to result in best SQ. When the transfer is slower there is a more dynamic interchange using TCP windowing - this produces a subjectively subtly inferior SQ. Now transcoding on some servers does reduce the throughput rate due to processing load and so the more dynamic TCP  (less good SQ) transfer response is initiated by the Naim streamer. 

S

Ian_S posted:

andarkian posted:

 

Now, you can argue as much as you want about realistic throughputs but 55 Mbps is easily enough for music and probably video as well in my own environment, but I know and understand many will totally disagree.

I now know, however, I can deliver Hi Def files from my NAS to my devices via WiFi if that is what I want to do. 

The point I'd make there is that will be the case only if your Naim is the only device on your WiFi... only one device at a time can use the WiFi bandwidth. So even your iPhone/iPad/android device running the Naim control app will compete with the Naim for time on the network if both are using Wifi... likewise any other device on the WiFi looking at emails, you-tube, sending tweets, just polling for updates etc. etc. None of which is great or ideal for anything that is timing critical. Which is why I say it's best to think of Wifi connections as an old fashioned hub connection. 

Ian,

This is true when, for example, I am using my iPad for music and other applications, such as reading The Times,  which definitely provides contention. However, if I run my music on one of my other devices, no contention and no problem. The contention within the iPad is actually more disruptive than any WiFi contention which is probably a reflection of Apple's operating system's ability to multitask or even manage memory on that device.

However, I do agree that there is potentially contention amongst all active devices for access to the available WiFi 'bandwidth', just not that much in my house. If the packaged music was not hitting my HiFi device it would not produce music or at least there would be distortion. This week the only distortion was attempting to use the primitive tools available to myself to extract the music from the NAS. My 'hub' is definitely not overloaded and the buffers designed to collect the data cycles made accessible by the WiFi seem to be filling up quite nicely.

As for Simon, I fear he is heading down to a very dark subjective place if he believes that the TCP zero window or stack or whatever has an impact on musical output. If the window, or buffer is under or overloaded the consequences would be immediate and terminal for the content, assuming that the normal package sending rules are somehow destroyed at the same time, which I seriously doubt. I do accept, of course, that if the WiFi receiver cannot handle the volume of data that would definitely also cause distortion. 

By the way, whatever Naim do with the musical content received from the ether they do a very fine job. 

Peter Dinh posted:

andarkian, it is nice if you can achieve 55 mbps thru wifi, but to make sure the performance is consistent, you may want to dedicate that wifi channel for music streaming only and no collisions with any other wifi channels out there.

Peter,

Again, this is true. As it happens, in my house this is more than possible but not necessarily the fact. I have an extender that can isolate a WiFi network for music if that is what I should choose. To be honest, that is not the case as all my HiFi equipment is in three rooms under the umbrella of one WiFi network. However, the main AV equipment and NAS are all attached to either the router or a switch via Ethernet cable because they are all in the same location. As for pollution from another source, my house stands in its own ground with the nearest possible sources being 50 metres away. 

Clearly, this may not be the solution for everyone as I am fortunate in having a fast internet service, Sky, though I could also have Virgin's 200 mb service if I wanted it, and am isolated from other potential conflicts. It works, I know it works and it is mature technology that is flexible, reliable and very cost effective.