How do hard disk music servers mitigate some concerns

Posted by: Guido Fawkes on 13 May 2008

Talking to another forum member, we discussed some concerns about storing music on hard disk, particular if the disk was formatted as a FAT disk like it is on some Windows devices (and indeed my trusty old Yamaha HD-CDR 1300).

Firstly, how does the music server overcome the problem of disk fragmentation - does the system ensure that music files are contiguous on the disk and does it affect sound quality if they are not?

Hard disks are necessarily writable. What is to stop something undesirable writing to the disc, either something on a CD you're ripping (memories of the Sony root kit) or, if you connect it to the Internet, a worm or something else? I feel my Yamaha is safe as it drives the disk from a read-only OS on a chip, but not all systems use this technique.

So far I have not found any hard disk system as enjoyable as a CD player or vinyl (not that I'm expending much energy looking) and wonder if this is really a technology here to stay or the latest incarnation of the Elcasset story.

Any thoughts?

ATB Rotf
Posted on: 13 May 2008 by Paul Hutchings
With issues like fragmentation it will depend entirely on the way the OS writes to the disk.

For example Windows seems to take a "dump it anywhere" approach, OS X and various *nix/BSD systems will try and write to a contiguous block of disk space with files up to a certain size (20mb with OS X I believe).

Either way, with modern hard drive transfer speeds it should be inconsequential for the sort of transfer levels you need to maintain for simple listening/recording so if it did affect sound quality you'd probably be listening using something broken.

Equally the only thing stopping "something undesirable" from writing to the drive is whatever OS is safeguarding access to the drive - the answer is to ensure you have a backup.
Posted on: 13 May 2008 by Chris Kelly
quote:
the answer is to ensure you have a backup.


Like a CD for example! Roll Eyes
Posted on: 13 May 2008 by Goldstar
quote:
So far I have not found any hard disk system as enjoyable as a CD player or vinyl


So why not stick with what is tried & tested?

I find the present system more than adequate.

Bob
Posted on: 13 May 2008 by Steve S1
quote:
Originally posted by Chris Kelly:
quote:
the answer is to ensure you have a backup.


Like a CD for example! Roll Eyes


Ha. How about making two copies of your entire music collection on attached storage (pretty much drag & drop) - then keep one off site.

You have better back-up than CDs in the event of fire or theft. Big Grin
Posted on: 13 May 2008 by Steve S1
quote:
So why not stick with what is tried & tested?





Because "Times, they are a changin'".
Posted on: 13 May 2008 by Gianluigi Mazzorana
quote:
Originally posted by ROTF:
I have not found any hard disk system as enjoyable as a CD player or vinyl


The new era is beginning now.
Posted on: 13 May 2008 by Guido Fawkes
quote:
Originally posted by Goldstar:
quote:
So far I have not found any hard disk system as enjoyable as a CD player or vinyl


So why not stick with what is tried & tested?

I find the present system more than adequate.

Bob


Hi Bob

Agreed.

That's my intention, but a lot my favourite artists new releases may move to download only.

However, it was really a question raised out of curiosity - I wondered if music servers like the HDX had ways of overcoming what I perceived as concerns or if I had hit on the reason why I'd found traditional CD players better than the music servers I'd heard.

ATB Rotf
Posted on: 13 May 2008 by Guido Fawkes
quote:
Originally posted by Paul Hutchings:
With issues like fragmentation it will depend entirely on the way the OS writes to the disk.

For example Windows seems to take a "dump it anywhere" approach, OS X and various *nix/BSD systems will try and write to a contiguous block of disk space with files up to a certain size (20mb with OS X I believe).

Either way, with modern hard drive transfer speeds it should be inconsequential for the sort of transfer levels you need to maintain for simple listening/recording so if it did affect sound quality you'd probably be listening using something broken.

Equally the only thing stopping "something undesirable" from writing to the drive is whatever OS is safeguarding access to the drive - the answer is to ensure you have a backup.


Hi Paul

I wondered if some more advanced (expensive) music servers had disk controllers that removed the fragmentation problem. Yes you are right with OS X, HFS+ always tries to create contiguous files, whereas Windows doesn't when using FAT or NTFS. I find the effects noticeable when doing my day to day work on various computers (i.e. not playing music) and was curious if this could have a detrimental effect on a music server (get your after market Russ Abbott Music DeFrag here, sort of thing).

Interested in your answer to my second question - as it may mean that many music servers would require anti-virus, firewalls and so on - all of which slow down performance. Mirrored disks wouldn't do much good as the undesirable content would go to both - so as Chris says you'd need CD or similar as a back-up.

Designing a music server certainly introduces new challenges that were not there for designers of CDPs and TTs.

ATB Rotf
Posted on: 13 May 2008 by Paul Hutchings
I think it depends on the purpose.

If the purpose is to have a computer as a dedicate music server I'd go so far as to say you could take a Windows machine (and certainly a Mac) and connect it to the Internet and so long as all you were using it for was playing music, and allowing it outbound access to do CD lookups you wouldn't have much risk at all (this assumes a hardware router with a firewall inbuilt).

I guess what I'm suggesting is that if you isolate a computer and treat it as a source and nothing more there's very little risk, the problem is when you start to mix roles because, as you rightly say, you have to think about antivirus and antispyware and all the stuff that the average PC user will feel they need to install to be protected against nasties.
Posted on: 13 May 2008 by BigH47
Do these HDX thingies have an OS? Perhaps the question should be what OS do these HDX thingies have?
Posted on: 13 May 2008 by James Lehmann
quote:
Originally posted by ROTF:
how does the music server overcome the problem of disk fragmentation - does the system ensure that music files are contiguous on the disk and does it affect sound quality if they are not

Certainly with Mac OS-X, nowadays disk fragmentation is scarcely an issue anymore.
Posted on: 13 May 2008 by Guido Fawkes
quote:
Originally posted by BigH47:
Do these HDX thingies have an OS? Perhaps the question should be what OS do these HDX thingies have?


My understanding is it is Windows XP Embedded.
Posted on: 13 May 2008 by Steve S1
quote:
Windows XP


Eek
Posted on: 13 May 2008 by BigH47
I'll pass then!
Posted on: 13 May 2008 by Guido Fawkes
quote:
Originally posted by James Lehmann:
quote:
Originally posted by ROTF:
how does the music server overcome the problem of disk fragmentation - does the system ensure that music files are contiguous on the disk and does it affect sound quality if they are not

Certainly with Mac OS-X, nowadays disk fragmentation is scarcely an issue anymore.


True - unless, notes taken from Apple Developer Connexion

Mac OS X uses hundreds of thousands of small files, many of which are rarely accessed. Optimising is a major effort for little practical gain. There is a chance a file placed in the "hot band" for rapid reads during system start-up might move during defragmentation, which would decrease performance.

If disks are almost full, and you often modify or create large files (such as editing video), there's a chance the disks could fragment. You might benefit from defragmentation using third-party disk utilities. Another option is to back up important files, erase the hard disk, reinstall OS X and backed up files.


also

HFS Plus avoids reusing space from deleted files, as much as possible, to avoid prematurely filling small areas of recently-freed space. Mac OS X has delayed allocation, which combines small allocations into a single large allocation in one area of the disk.

Fragmentation is often caused by continually appending data to existing files. With faster hard drives and better caching, many applications rewrite the entire file each time.


So the moral is don't fill your disk up.
Posted on: 13 May 2008 by gary1 (US)
Yes, but what no one has mentioned yet is what happens when you use up the HD storage/raid on the HDX or NSO1. You then need to go to either USB or ethernet attached storage. Naim has built an all in one which it feels is necessary to control all aspects of the process from ripping,storage, processing, playback to obtain the highest quality playback from the system. What happens when the source file comes from an attached NAS? Does the system have the ability to play with the same quality as the files stored on the HDX/NS01 or is there a loss in quality? Major Question!! If the quality is the same then why have the HD storage on the same box with all the potential inherent problems with HD failure etc... If the quality and playback of the files stored on the NAS are not equivalent,which is somehow implied by the need to have an all in one then the product is only good enough for 600CDs. If the quality is equivalent then the HD storage was put there for those who want the Naim high quality sound, but don't want to fuss with a computer driven NAS, but they are still left in this position once the stoage is used! I'm still confused Roll Eyes
Posted on: 13 May 2008 by Guido Fawkes
I am confused too. I trust Naim to get it right, but I still can't fathom out quite why it is like it is.
Posted on: 13 May 2008 by gary1 (US)
quote:
Originally posted by ROTF:
I am confused too. I trust Naim to get it right, but I still can't fathom out quite why it is like it is.


Yes,but not matter what the technology used, ultimately you will need to use a NAS in the system. Furthermore, if you are downloading files from Naim Music whether 16 or 24 bit it still needs to be stored either in the HD or NAS and if on the NAS then it needs to be transferred over to the HDX for playback? Same issues apply.
Posted on: 13 May 2008 by prowla
Chances are that the modern music you listen to was mastered on a hard disk system, so there are no issues with data retrieval times (and disks can handle speeds waaaaaayyyy beyond audio - they are faster than 48x CD-ROM drives, which are, ahem, 48x faster than an audio CD).

The reason you may not get the quality you expect may have to do with the format you are using to store the data (ie. a lossy compression) and/or the quality of the DAC and/or the integrity of the signal path & power supplies.

Reports in other threads suggest that Naim's HDD option sounds pretty good, and as I say, a lot of studio stuff is on HDD anyway.

So it's only a matter of time.

However, the disk storage is likely to be obsoleted by solid state in the next not many years (you can get laptop computers already), and then we'll have a no moving parts solution, which may help power supplies. They may well be direct replacement slot-in components that supersede the HDD, and so all of the surrounding technology remains valid.
Posted on: 13 May 2008 by gary1 (US)
Prowla, I agree that seems to be where technology is moving, but still doesn't answer the question I raised and that is the question wrt to the any quality difference between the music stored on the NAS vs. the HDD and if not why the HDD in the first place? If so then the units usefulness is limited to the storage space on the HDD. I can't imagine Naim R&D coming up with a product of limited use, but this still begs answers to the above questions.
Posted on: 14 May 2008 by Rockingdoc
Since the days when records went round at 78rpm, then 45rpm, then 33rpm, then onto cassettes, then CDs, the quest of the hi-fi industry has been to deliver your recorded music source in the smallest possible package, not the one delivering the best sound.
Posted on: 14 May 2008 by daddycool
Exactly Rockingdoc, that was why they were so upset that the ultimate portable package, MP3, happened completely outside them Cool

(as in without control or cash milking) Some observations: You forget one important point, they had a quest to come up with new formats that is compelling enough to make you buy your music again. Whereas many think "I paid for my music" so I'm entitled to use that again on any format, that is not at all how Big Content sees it. Actually they rather had you don't buy any content at all anymore, but rent it with a subscription, so they can cash everytime you play it.

An industry that has lost all contact with society, sad. They do a good job of enforcing their out of date business model through legislation, though, thus hindering progress for decades to come. See the Arstechnica website (search "RIAA") for enlightening articles on the subject. At least the UK stood up to lengthening copyright from 50 years to 95 years to indefinitely just to protect Mickey Mouse from ever entering the public domain.

24/96 (DVD-A) and SACD didn't make it for two reasons: First, they took the price of CD as base, which consumers have found too high for 25 years already. Then they put a premium on that for sound quality, making it very expensive. Secondly, at the same time, they used the new format to introduce fierce DRM to finally combat the digital transferability the CD had made possible.
They were unaware or ingnored that the rest of society had moved to an ultimately portable new convenience format (computers, internet and MP3) with total DRM anarchy. "Buy all your music again, at a much higher price than what you already found too expensive, and with strict copy protection so you don't have fair use, and we give you even better sound quality in return." Guess not, consumers saw it for what it was, a new invention devised primarily for the convenience of the industries. It might have stood a chance if they would have put it out at the same price-point as CDs, without any DRM, and cutting price of CDs in half, but, as ever, greed got the best of them. Management in Big Content is still living in the world where the introduction of the CD and later the DVD made them extremely rich. Now they demand big cash for a lesser effort (downloads).

CDs are much cheaper all of a sudden this year. Perhaps these 24/96 masters will become available again someday...
For now, mainstream music is lowest quality MP3 and AAC and we here are finding ways to get our CDs in a music server...
Posted on: 14 May 2008 by David Dever
quote:
the question wrt to the any quality difference between the music stored on the NAS vs. the HDD and if not why the HDD in the first place? If so then the units usefulness is limited to the storage space on the HDD.


The file must be retrieved and cached into memory, whether from (local) hard disk or network-attached storage.

As for the all-in-one approach–it's been discussed in other threads, but this offers a complete end-to-end solution that might appeal to some, perhaps not to others, but definitely (in the custom integration space) to installers who want to get up and running quickly (streaming audio out onto the network within five minutes of installation).
Posted on: 14 May 2008 by daddycool
David,

So, if I understand correctly, the internal HDs are the caching solution and/or storage solution if necessary. So it is possible to rip on the HDX, have it cached there and then transfer it to the NAS. And when retrieving from NAS it is cahed again in the HDX and then played. So there would be no difference sound-wise to files ripped by the HDX stored on the internal disc or on a NAS. Correct?
Posted on: 14 May 2008 by Phil Harris
quote:
Originally posted by ROTF:
Firstly, how does the music server overcome the problem of disk fragmentation - does the system ensure that music files are contiguous on the disk and does it affect sound quality if they are not?


There are several factors that need to be taken into account here, however the main one is the usage of the servers themselves.

The way that data flows around the servers is very different to the way that data flows around a PC ... the embedded OS build partition remains stable in its structure and therefore fragmentation of that partition is not an issue, the music partition has been found to not be an issue either in our testing here.

As music is ripped to the servers it is written to the internal storage in a way that inherently tries to ensure that the data is contiguous - once a CD is ripped then it is unusual for that CD to be deleted again. So whereas the read-write-delete-rewrite cycling on a PC hard disc can be in the order of many times a minute as applications write out temporary data, undo buffers and swap files the actual usage of the servers music partitions is - in comparision - very different and in our own testing was found not to require defragmenting.

In this respect - and because we do buffer data well ahead of playback - then for all practical purposes fragmentation of the music drive is a non-issue.

Remember that the HDX is only one server in a range of servers and other models of our servers currently can source up to 10 streams of simultaneous audio (4 played back locally on the server and an additional 6 that can be output via a network to be used in a NaimNet multiroom system) and we have done extensive testing and research to ensure that the vibration induced by the seek loadings imposed by the extreme case of serving 10 sources simultaneously has a minimal effect in terms of physical vibration influencing any other critical components within the system.

The practical upshot of this - and the brief answer to your question - is that fragmentation of a music drive is not an issue.

Phil