Distributed music storage.
Posted by: Roy T on 20 March 2010
All this talk of (the new) SSD, streaming, NAS and such has got me thinking about quality and fitness for purpose of the hardware used when it comes to storing and accessing valuable and sometimes irreplaceable copies of much loved music. I'll start with the assumption that six hundred albums (an HDX full) at say 10ukp an album and 15min an album to rip soon add up to a sizeable investment in both cash and time. I'll make a further assumption that the hardware used is much nearer the consumer end than the enterprise end of the market and this may be seen by time spent fiddling, tweaking, arranging backups and if unlucky fixing and swapping out broken bits and bobs.
So folks;
How do you manage your store of music?
Do you have any/many hardware based problems - is your kit reliable?
Are you planning any hardware upgrades?
Are you at the consumer or enterprise end of the market?
If you hardware failed how long would it be before you are once more listing to your tunes?
I may ell be dipping my toe into this vast and ever changing digital ocean in the near future hence my questions.
So folks;
I may ell be dipping my toe into this vast and ever changing digital ocean in the near future hence my questions.
Posted on: 25 March 2010 by SC
quote:Originally posted by DaveBk:
Nice rack Steve!
I agree !
Steve, can I ask 2 questions:
1. I can see your rack is essentially home made. Where did you get the front rails - I have never been able to find them separate like that before ? The costs of a 'proper' server case always seem so high and I would be quite happy enough with a simple MDF box of sorts...But those rails would be very useful!
2. This might be a silly question with an obvious answer I haven't thought of, but what's the need for the patch panel ? Why can't all the Cat6 connections simply make their way direct to the switch..?
Thanks in advance
Steve.
Posted on: 25 March 2010 by intothevoid
Cheers guys!
@Aleg, yeah, I guessed as much. I was just trying to look for ways to avoid the extra expense, but I'm going to have to bite the bullet on this. More servers, more hard disks; it's a slippery slope y'all!
@DaveBK, I just have a way to go before I catch up with your front end electronics
@SC,
1. Yes the rack is home made. I could not justify the expense of a proper commercial rack. Mine is a simple timber frame. I salvaged the front rails from a couple of rack cabinets that were in a skip Waste not, want not. It worked out brilliantly. Before I found them I found the studioracks web site. They must be able to supply the rails only, and their prices seem very keen. Give them a shot.
2. The patch panel is there for convenience and flexibility. You're quite right though, I could have wired directly into the switch, but that would have meant 21 dangly cables. By using the PP it's neat and tidy (they cost peanuts-ish), and I can swap bits of equipment around very easily. This came in really useful recently as I had a problem with my 3com switch; being able to swap individual ports was a real benefit and time saver. Turns out I the 3com was filtering uPNP traffic, so I replaced it with the D-Link and can now stream video to the PS3's seamlessly.
@Aleg, yeah, I guessed as much. I was just trying to look for ways to avoid the extra expense, but I'm going to have to bite the bullet on this. More servers, more hard disks; it's a slippery slope y'all!
@DaveBK, I just have a way to go before I catch up with your front end electronics
@SC,
1. Yes the rack is home made. I could not justify the expense of a proper commercial rack. Mine is a simple timber frame. I salvaged the front rails from a couple of rack cabinets that were in a skip Waste not, want not. It worked out brilliantly. Before I found them I found the studioracks web site. They must be able to supply the rails only, and their prices seem very keen. Give them a shot.
2. The patch panel is there for convenience and flexibility. You're quite right though, I could have wired directly into the switch, but that would have meant 21 dangly cables. By using the PP it's neat and tidy (they cost peanuts-ish), and I can swap bits of equipment around very easily. This came in really useful recently as I had a problem with my 3com switch; being able to swap individual ports was a real benefit and time saver. Turns out I the 3com was filtering uPNP traffic, so I replaced it with the D-Link and can now stream video to the PS3's seamlessly.
Posted on: 25 March 2010 by SC
Cheers Steve, many thanks - I'll look them up.
Yes, I agree about the racks etc - I looked into a sound-proof glass door one once, but it cost almost as much as one of my Mac towers...!
I understand what you say re the patch panel - all the cable nest can be at the rear..!
Good job there mate.
Steve.
Yes, I agree about the racks etc - I looked into a sound-proof glass door one once, but it cost almost as much as one of my Mac towers...!
I understand what you say re the patch panel - all the cable nest can be at the rear..!
Good job there mate.
Steve.
Posted on: 25 March 2010 by Nathaniel
I'm in the process of recovering from disaster! I've spent a little over a year re-ripping CDs in lossless format, and had got most of the way through my CD collection (all cover artwork loaded and scanned, all tagging done consistently and so on), and had about 400GB of music stored on a single disk. Then my server's power supply blew (the second power supply had died years ago and I never replaced it!) and, what I can only guess was a surge caused the server to freeze.
I extracted the disk, connected it to other computers using caddies or directly onto the ata chain, and it wouldn't spin up.
So, in the hope that it was just a power issue on the disk controller board, I bought an identical disk, with same part numbers et al, and tried the new controller board on the old disk. It spun up!
But did nothing else...
Besides opening up the disc and swapping platters around (a job, that all lines of investigation tell me shouldn't be done unless you have the tools and experience), I think I've done all I can.
My music collection is very valuable to me, but not valuable enough to pay hundreds, or possibly thousands of pounds to a data recovery expert--I reckon that money would be better spent buying more music, so I've just started re-ripping......
I knew I should have had a backup, and while I had important documents stored in SCSI raid on the same server (those disks survived okay) and backed-up elsewhere, I had always put off backing-up my music, thinking that if I lost the lot it wouldn't be the end of the world--I could just re-rip.
And so it turned out. It's not the end of the world, but it's a real drag.
So:
I've got a new, old server obtained from work when it fell out of contractual support (an old dell poweredge 2600) in my garden shed/office.
I'm running VMWare ESXi on it, so I have the following virtual machines running on the one physical box:
1. FreeBSD 8
2. Windows 7
3. Ubuntu 9.10
The onboard SCSI RAID disks are already nearly filled up with other stuff (the virtual machines' OS'es and all my 'important' documents). I'm using an additional PCI-based SATA raid controller, so will have two SATA disks mirrored (RAID 1), where my music will live. I've only got one disk hooked up at present, so I'm tempting fate.
FreeBSD is my preferred server platform, so I'm serving my (soon to be mirrored) media drive up as an NFS share from my FreeBSD virtual machine.
This server is connected to the house via a gigabit fibre-optic 'bridge' of about 50m. Unfortunately, the main ethernet switches in both the garden office and the house are both 100base-T, not gigabit.
Once in the house, most things run over wireless (I've got 2 wireless networks: a 'g' and an 'n', so the older stuff won't slow down the newer stuff).
Besides the server-meltdown that caused me to lose my entire computer-based music collection, my set-up still isn't great. Whether it's the 100base-T switches, using wireless in the house to get the music to the stereo, the networking cost of running a virtual server inside a hypervisor platform like VMWare ESXi, or an accumulation of all the above, the network's a bit slow, and while music is fine most of the time, streaming video judders a bit too frequently, and the music occasionally glitches.
Yes. I plan to upgrade all my switches/router to gigabit ethernet.
I plan on adding a second SATA disk to my server to form a raid-1.
I plan on having another disk in my house, on which I'll backup my media library from my server, perhaps using a fancy back-up tool, or more likely just via a simple cron-job.
As others have said, RAID covers you in the event of disk failure, but not necessarily in the event of server failure or environmental catastrophe.
While VMWare ESXi, FreeBSD, the remote-server management facilities offered by proper server boxes (rather than desktop workstations) et al are enterprise-level tools, my use of them is most definitely consumer.
I can answer from experience. A couple of months of dithering around!
As my new solution 'comes on line', I hope the answer will be:
1. In the event of a disk failure: as long as it takes to buy a new disk and rebuild a RAID array for a big whopping disk. A day or two.
2. In the event of a server meltdown: Plug the remote back-up disk in as an external USB drive and listen to mournful music from my collection as I consider how to go about replacing the server.
I extracted the disk, connected it to other computers using caddies or directly onto the ata chain, and it wouldn't spin up.
So, in the hope that it was just a power issue on the disk controller board, I bought an identical disk, with same part numbers et al, and tried the new controller board on the old disk. It spun up!
But did nothing else...
Besides opening up the disc and swapping platters around (a job, that all lines of investigation tell me shouldn't be done unless you have the tools and experience), I think I've done all I can.
My music collection is very valuable to me, but not valuable enough to pay hundreds, or possibly thousands of pounds to a data recovery expert--I reckon that money would be better spent buying more music, so I've just started re-ripping......
I knew I should have had a backup, and while I had important documents stored in SCSI raid on the same server (those disks survived okay) and backed-up elsewhere, I had always put off backing-up my music, thinking that if I lost the lot it wouldn't be the end of the world--I could just re-rip.
And so it turned out. It's not the end of the world, but it's a real drag.
So:
quote:
How do you manage your store of music?
I've got a new, old server obtained from work when it fell out of contractual support (an old dell poweredge 2600) in my garden shed/office.
I'm running VMWare ESXi on it, so I have the following virtual machines running on the one physical box:
1. FreeBSD 8
2. Windows 7
3. Ubuntu 9.10
The onboard SCSI RAID disks are already nearly filled up with other stuff (the virtual machines' OS'es and all my 'important' documents). I'm using an additional PCI-based SATA raid controller, so will have two SATA disks mirrored (RAID 1), where my music will live. I've only got one disk hooked up at present, so I'm tempting fate.
FreeBSD is my preferred server platform, so I'm serving my (soon to be mirrored) media drive up as an NFS share from my FreeBSD virtual machine.
This server is connected to the house via a gigabit fibre-optic 'bridge' of about 50m. Unfortunately, the main ethernet switches in both the garden office and the house are both 100base-T, not gigabit.
Once in the house, most things run over wireless (I've got 2 wireless networks: a 'g' and an 'n', so the older stuff won't slow down the newer stuff).
quote:
Do you have any/many hardware based problems - is your kit reliable?
Besides the server-meltdown that caused me to lose my entire computer-based music collection, my set-up still isn't great. Whether it's the 100base-T switches, using wireless in the house to get the music to the stereo, the networking cost of running a virtual server inside a hypervisor platform like VMWare ESXi, or an accumulation of all the above, the network's a bit slow, and while music is fine most of the time, streaming video judders a bit too frequently, and the music occasionally glitches.
quote:
Are you planning any hardware upgrades?
Yes. I plan to upgrade all my switches/router to gigabit ethernet.
I plan on adding a second SATA disk to my server to form a raid-1.
I plan on having another disk in my house, on which I'll backup my media library from my server, perhaps using a fancy back-up tool, or more likely just via a simple cron-job.
As others have said, RAID covers you in the event of disk failure, but not necessarily in the event of server failure or environmental catastrophe.
quote:
Are you at the consumer or enterprise end of the market?
While VMWare ESXi, FreeBSD, the remote-server management facilities offered by proper server boxes (rather than desktop workstations) et al are enterprise-level tools, my use of them is most definitely consumer.
quote:
If you hardware failed how long would it be before you are once more listing to your tunes?
I can answer from experience. A couple of months of dithering around!
As my new solution 'comes on line', I hope the answer will be:
1. In the event of a disk failure: as long as it takes to buy a new disk and rebuild a RAID array for a big whopping disk. A day or two.
2. In the event of a server meltdown: Plug the remote back-up disk in as an external USB drive and listen to mournful music from my collection as I consider how to go about replacing the server.
Posted on: 25 March 2010 by SC
quote:Originally posted by intothevoid:
Before I found them I found the studioracks web site. They must be able to supply the rails only, and their prices seem very keen. Give them a shot.
A pair of 12U front rails - £16. Result !
Thanks again Steve
Posted on: 25 March 2010 by intothevoid
Cool. Glad to be of service
Now we want to see the photos...
Steve
Now we want to see the photos...
Steve
Posted on: 25 March 2010 by SC
In time...! Have enough on my hands at present doing the house...!
Actually, for anyone interested in a small server rack, in the UK, Maplins do a basic and reasonably priced 9U unit...Here.
Steve.
Actually, for anyone interested in a small server rack, in the UK, Maplins do a basic and reasonably priced 9U unit...Here.
Steve.
Posted on: 26 March 2010 by Alonso
quote:How do you manage your store of music?
I (try to) buy about 4 CD's per month. They come through the post and the process is as follows:
- Rip them in Apple lossless format and in 256Kbps. The Lossless format files go to a Ready NAS Duo in RAID1. This is accessed via a Timecapsule router and then by a Sonos system, which sends music to 3 zones (bathroom, dining room and living room, where my DAC and Naim Nait XS amp reside). The NAS has 2 main folders MEDIA & BACK UP.
- I make 2 lossless copies of each album, one goes to MEDIA; where the Sonos reads it from, the other copy goes to BACK UP. This is just as a first line of defense against accidental deleting (there seems to be no UNDO with Network Attached Servers, no trash bin either!), the RAID1 protects me against mechanical failure. I guess the last line of defense are the physical CDs themselves...
- The 256Kbps copies are for casual/on the move listening (2 iphones, 2 Macbooks, 2 XP Laptops at home) these 256kbps files also have a 'spare' copy in the BACK UP folder on the NAS.
quote:Do you have any/many hardware based problems - is your kit reliable?
- Never experienced any hardware problems. Just silly mistakes of mine (deleting accidentally) I feel relatively safe. My Time Capsule takes care of data loss from Macs (no music) just home stuff and individual USB attached HDD take care of work stuff from the XP Laptops..... SH*T! How could I forget!? Of course, My Timecapsule died on month 18 after purchase (known issue with overheating PS) but it was replaced under one of the mac's warranties. And because all the data on the TC was in the macs, no biggie.
quote:Are you planning any hardware upgrades?
- BIG ONE: Im building a Media Centre around a Mac Min (recently sold my Panansonic DVD/HDD recorder/player, my Denon AV amp, my surround speakers and returned the Virgin+ box. All will be Mac Mini centred. One box. 2.0 sound for Music & AV. Simplicity HEAVEN
- I will also need to replace the 2 500GB discs inside the NAS for at least 2 1.5TB ones. The NAS will also start being the storage for my movies (all ripped without compression!). That is another 'philosophical' issue that I need to sort out... Considering we get 6 movies a month via LoveFilm and the music
quote:Are you at the consumer or enterprise end of the market?
- Consumer
quote:If you hardware failed how long would it be before you are once more listing to your tunes?
- Probably 24 hours (assuming both NAS discs didnt fail at the same time!)