Check and Test Your Backups!

Posted by: trickydickie on 05 March 2018

Folks

Just a little story about my experience this weekend.

On Saturday I had downloaded some music to my QNAP NAS and was editing the metadata and lost connection to the NAS.  At a similar time the music stopped too. After a visit to our understairs cupboard I established that the QNAP had restarted itself and was booting up.

After some minutes it was up and running and I received a warning that I should do a file system check which I did without issue.  Everything seemed OK.

Overnight I received some warnings about a synchronization job to Google cloud wasn't working correctly (I have this so my phone pictures sync with the NAS) so I decided to check on this.   When I logged on to the QNAP I observed that the desktop was almost completely bare and was introducing me to my 'new' NAS and none of my backup jobs were showing.

I tried to recreate my backup jobs but they would briefly save and then disappear.  Researching this seemed to suggest that I reboot the NAS which I did.

This time the NAS did not boot up and there was a Pool Volume error.  Research showed that the storage pool was corrupt.

I tried many things and after a while admitted defeat and reset the NAS and my disks back to factory defaults, losing all the data on the NAS.

Fortunately my backup strategy was robust.  I have a replica of all data on 2 other NAS drives as well as a USB drive connected to the NAS and I also backup to Azure.  It's taken a while to restore but I now have all my data back except for the albums I downloaded on Saturday which were not backed up Saturday night.  This is not an issue as I can redownload these this evening.

The NAS seems fine, something must have corrupted it, we did have a power cut a week ago so maybe this contributed to the problem.  

So please, check you backups and don't put them off.  Whilst it took me an afternoon to set the NAS backup and many hours of copying data from my other devices no harm has been done apart from an afternoon lost listening to music.  Without these backups I wouldn't have been so lucky.

Richard

Posted on: 05 March 2018 by nickpeacock

Wise words, and what a relief for you - spurred on by forum members, I now keep portable backup drives in two separate locations (though I am not quite as assiduous as you in keeping them topped up)!

Posted on: 05 March 2018 by ChrisSU

I like to keep my main NAS (Unitiserve) backed up to a second NAS, on which I run Minimserver, which reassures me that the automated backup contains fully fully functioning music files, as I can then play them. People used to complain about the complexity of the Naim auto backup system, compared to just a USB drive, but actually, I think a pretty good setup.

Posted on: 06 March 2018 by Huge

Ironically, yesterday I had what I thought was a software problem corrupting the metadata in the music on my NAS.  So I did a complete restore.  Nothing changed (which at least showed that my backup and restore worked OK!).

I now realise it was yet another fault in the Naim App V2.5 which now won't display album art on my devices if it's larger than some undeclared value (larger image dimensions? larger data size? who knows; the developers aren't telling) 

Posted on: 06 March 2018 by Mike-B

Frightening when that kind of stuff happens.  I had something like that a long while back,  heart stopping etc, while I was editing in the NAS,  since then I've avoided editing in the NAS.   I download (or rip) & check/edit on the laptop,  once I test run & verified its correct,  then I upload to NAS.       That said I downloaded a Qobuz last weekend & was shocked at the mess that appeared on the iPad when I played it,  I did edit it on the NAS this time,  but reading this thread   I should have copied to laptop to do the editing.     And yes portable backups every time.

Posted on: 06 March 2018 by Rich 1

Never to many backups, Nas and USB for me.

Posted on: 06 March 2018 by Huge

In my case it was "Album Art" from Illustrate - you don't get any choice on where it acts - it scans your libraries and acts on the files in the active libraries.

Further information of the App V2.5 - the problem my be that it no longer requests Album Art at all, only ever looking at embedded art in the individual tracks.

Posted on: 06 March 2018 by robgr

Were you using RAID?

Would the data not have been intact?

Could you have recreated the storage pool?

Posted on: 06 March 2018 by trickydickie
robgr posted:

Were you using RAID?

Would the data not have been intact?

Could you have recreated the storage pool?

Yes, I was using Raid 5.  

The problem was that the NAS wouldn't boot up so there was no web interface to access, just the error message on the front panel display.  Connecting a monitor to the unit showed that it was inviting setup.  The device didn't even have an IP address so it was early on in the boot up I suspect.  I suspect that someone from QNAP may have had a back door that a console user could use to undertake recovery.

It's been fine since, whilst I had a backup of the settings they couldn't be applied.  I suspect that you should take one of these backups at each firmware upgrade which i hadn't done.

No harm done, just a Sunday afternoon lost, all data is back now.  

Posted on: 06 March 2018 by simes_pep
Huge posted:

Ironically, yesterday I had what I thought was a software problem corrupting the metadata in the music on my NAS.  So I did a complete restore.  Nothing changed (which at least showed that my backup and restore worked OK!).

I now realise it was yet another fault in the Naim App V2.5 which now won't display album art on my devices if it's larger than some undeclared value (larger image dimensions? larger data size? who knows; the developers aren't telling) 

The Naim App should and has always displayed the Album art is being 'served' by the UPnP server, and scale appropriately depending on the various different grid or list views.
Occasionally it can take a moment for an album artwork to load, but that's when it is huge, in the Mbytes, but there is also caching in the device, to buffer some of that behaviour.

What UPnP server are you using?

Simon.

Posted on: 06 March 2018 by Phil Harris
Huge posted:

<<SNIP>>

I now realise it was yet another fault in the Naim App V2.5 which now won't display album art on my devices if it's larger than some undeclared value (larger image dimensions? larger data size? who knows; the developers aren't telling) 

Hi Huge,

I asked and I was told...

The "limit" is a little over 6 megapixels (2,500 x 2,500 pixels assuming a square image).

Cheers

Phil

Posted on: 06 March 2018 by Huge

I'm using Synology Media Server.

When <Album Art> is requested at a data structure tree level that isn't an individual track (i.e. when it's an album) the server returns the first of "cover.jpg,Cover.jpg,COVER.jpg,folder.jpg,Folder.jpg,FOLDER.jpg".

When <Album Art> is requested as a metadata tag from an individual track, it returns the embedded Album Art.


The App now only displays Album Art when individual tracks are listed, and then it displays the <Album Art> metadata obtained from the first track in the listing.  It will no longer display the Album Art from a Cover.jpg or Folder.jpg file (it used to do this correctly, but not any more).

This behaviour is replicated identically on both my Android devices and the change occurred on the updating to V2.5.  The server hadn't changed until I updated it to try to fix this issue (to which it made no difference).

Posted on: 06 March 2018 by Mike-B

Hi Huge,  I'm using Syn's "Media Server" on the iOS app & don't have the problem,  i.e. it seems most likely to be your app

On the same theme:   I had some albums downloaded with .png art embedded,  that did some strange things with art.  I've added .png to the cover.jpg///folder.jpg line in the Syn DMA  & I also e-mailed the vendor & he admitted it was an error. 

Finally, I assume you've already run a cache clear,  & have you run a reindex on the Synology & did that change anything.

Posted on: 06 March 2018 by Huge
Mike-B posted:

Hi Huge,  I'm using Syn's "Media Server" on the iOS app & don't have the problem,  i.e. it seems most likely to be your app

<snip> 

Finally, I assume you've already run a cache clear,  & have you run a reindex on the Synology & did that change anything.

OK...

Yes, both; & yes and no!

Also tried UPnP Compatibility mode: folder icons look different and it's slower, but same overall result.

Posted on: 06 March 2018 by Phil Harris
Huge posted:

I'm using Synology Media Server.

When <Album Art> is requested at a data structure tree level that isn't an individual track (i.e. when it's an album) the server returns the first of "cover.jpg,Cover.jpg,COVER.jpg,folder.jpg,Folder.jpg,FOLDER.jpg".

When <Album Art> is requested as a metadata tag from an individual track, it returns the embedded Album Art.


The App now only displays Album Art when individual tracks are listed, and then it displays the <Album Art> metadata obtained from the first track in the listing.  It will no longer display the Album Art from a Cover.jpg or Folder.jpg file (it used to do this correctly, but not any more).

This behaviour is replicated identically on both my Android devices and the change occurred on the updating to V2.5.  The server hadn't changed until I updated it to try to fix this issue (to which it made no difference).

Hi Huge,

You say "The App now only displays Album Art when individual tracks are listed, and then it displays the <Album Art> metadata obtained from the first track in the listing.  It will no longer display the Album Art from a Cover.jpg or Folder.jpg file (it used to do this correctly, but not any more)" but the app itself doesn't (and can't) access indivdual files on the NAS - the only images or data that it can access are the details that the UPnP server gives it, it can't bypass the UPnP server and pull files directly from the storage device - even when playing the audio file itself it only ever gets a handle to a virtual file not teh actual file itself.

Similarly you say that in UPnP compatibility mode you get the same (but with slightly different folder artwork) - well that 'mode' is the streamer browsing the UPnP server and the app pulling the information that it displays from the streamer instead of the app browsing the UPnP server so it does look like the UPnP server itself is behaving differently...

Best

Phil

Posted on: 06 March 2018 by Huge

Phil, I'm aware that the app doesn't read any file directly.  The Cover.jpg or Folder.jpg (the JPEG data, not the file) are returned by the server when the current browse object is a node on the tree rather than a leaf (i.e. it's a list containing other entities at the next level down the tree, rather than a playable media item).

In the specific case of the Synology Media Server it actually only does this when the current node is an album.

I'm currently trying to test another hypothesis that could also account for the lack of displayed artwork.

Posted on: 06 March 2018 by Hmack

Huge,

No idea why, but I had continual issues with the display of artwork on my apps (both Naim and Linn) when I used Synology's Media Server. Even when displayed correctly, the artwork always took an age to load. I never got to the bottom of the problem.

Since switching to MinimServer a year or two ago (initially on a temporary basis to check out FLAC to WAV transcoding on the fly) , I have had absolutely no problems with artwork on any of the apps I use (Naim, Linn Kazoo & Lumin), and the artwork loads extremely quickly. From a music perspective the Synology Media Server was fine, but I am now very happy with Minimserver.  

Posted on: 06 March 2018 by Mike-B

Hmack,  Synology Media Server sucks on two counts IMO:   its slo-o-o-w loading art.  But it doesn't bother me because I browse 'By Folder' (I look for artist & composer by name)    On the odd occasion I do browse through the art I just put up with its inadequacies.    I tried Minim but ended up deleting it (as good as it is) because it had some issues when Synology auto loaded an update,  life's too short to be bothered over a few faster loading thumbnail pics. The second sucks point is it doesn't play gapless transcoding,  but I'm all WAV & DSD so don't need transcoding.  

Posted on: 06 March 2018 by Huge

I'm (personally) running a bit too hot at the moment, so I'm going to put my own biological OS into a sleep state for a bit.

Posted on: 06 March 2018 by Hmack

Mike,

Funnily enough, although I (think I) did hear a marginal difference between FLAC and WAV with my ND5XS/Hugo setup, I am not convinced I hear a difference when using my microRendu/Hugo or Klimax DS/1 setups. However, I have retained transcoding on the fly just in case.

The lack of gapless whilst transcoding on the fly with MinimServer doesn't bother me since classical music is a very marginal part of my music collection, and when I do play classical music I don't really notice the lack of gapless playback.

Minimserver is pretty much perfect for me, and artwork loads almost instantaneously.

Incidentally, I don't think that the artwork issues were always present with the Synology Media Server. I am pretty sure that the 'slow artwork' was not an issue when I first started using the Synology Media Server.      

Posted on: 06 March 2018 by Filipe
Huge posted:

In my case it was "Album Art" from Illustrate - you don't get any choice on where it acts - it scans your libraries and acts on the files in the active libraries.

Further information of the App V2.5 - the problem my be that it no longer requests Album Art at all, only ever looking at embedded art in the individual tracks.

Huge, Yep, a bit over a month ago I was demoing an NDX and the same happened. I had about 1600 albums spread over half a dozen libraries on my NAS. Decided not to buy, and continue enjoying my CDX2! It affected the Core I was demoing as well, but not the UnitiServe!

Phil

Posted on: 06 March 2018 by Big Bill
trickydickie posted:

Folks

Just a little story about my experience this weekend.

On Saturday I had downloaded some music to my QNAP NAS and was editing the metadata and lost connection to the NAS.  At a similar time the music stopped too. After a visit to our understairs cupboard I established that the QNAP had restarted itself and was booting up.

After some minutes it was up and running and I received a warning that I should do a file system check which I did without issue.  Everything seemed OK.

Overnight I received some warnings about a synchronization job to Google cloud wasn't working correctly (I have this so my phone pictures sync with the NAS) so I decided to check on this.   When I logged on to the QNAP I observed that the desktop was almost completely bare and was introducing me to my 'new' NAS and none of my backup jobs were showing.

I tried to recreate my backup jobs but they would briefly save and then disappear.  Researching this seemed to suggest that I reboot the NAS which I did.

This time the NAS did not boot up and there was a Pool Volume error.  Research showed that the storage pool was corrupt.

I tried many things and after a while admitted defeat and reset the NAS and my disks back to factory defaults, losing all the data on the NAS.

Fortunately my backup strategy was robust.  I have a replica of all data on 2 other NAS drives as well as a USB drive connected to the NAS and I also backup to Azure.  It's taken a while to restore but I now have all my data back except for the albums I downloaded on Saturday which were not backed up Saturday night.  This is not an issue as I can redownload these this evening.

The NAS seems fine, something must have corrupted it, we did have a power cut a week ago so maybe this contributed to the problem.  

So please, check you backups and don't put them off.  Whilst it took me an afternoon to set the NAS backup and many hours of copying data from my other devices no harm has been done apart from an afternoon lost listening to music.  Without these backups I wouldn't have been so lucky.

Richard

It just goes to show, that although making backups takes a bit of time, costs money and is generally a pain, there is no alternative to this strategy.  I also do the following:

(i) Use an online backup service costs about £40 a year but it is another level of safety,

(ii) I also use an Uninterruptible Power Supply (UPS) to power my NAS.  The model I have has a USB port which connects to my QNAP NAS and sends messages whenever there is a power outage and my NAS is configured to shut down after a particular time period of no mains power.  What surprised me with this combo at first was the number of small power outages (a fraction of a second) that occur, the lights may dip, the TV fades and then come up again - it tracks all of these - and we gave to ask what affect these might have on our NAS's.

In the last couple of months 2 of my friends have lost all of their digital photos and one thought he had (but hadn't) because I had convinced him to use an online backup system and he simply didn't know how to get stuff back until I showed him.  But the first two lost the lot.  Did they take any backups?  NO!

Well done Richard.

Posted on: 06 March 2018 by jsaudio

All this has got me somewhat worried and so I would like opinions on my back-up strategy.

I have a QNAP NAS with 2 drives that I think have a RAID configuration. I also have a My Book drive connected for scheduled back -ups right next to the NAS. I assume that its working as planned. I also had someone help do a back-up about 6 months ago to an additional external drive that I keep off site. Have ripped and downloaded a lot of music since then so should try to update that. 

Is this enough? Is there a better way? Cloud based BU?

thanks

Posted on: 06 March 2018 by Eoink
Mike-B posted:

Hmack,  Synology Media Server sucks on two counts IMO:   its slo-o-o-w loading art.  But it doesn't bother me because I browse 'By Folder' (I look for artist & composer by name)    On the odd occasion I do browse through the art I just put up with its inadequacies.    I tried Minim but ended up deleting it (as good as it is) because it had some issues when Synology auto loaded an update,  life's too short to be bothered over a few faster loading thumbnail pics. The second sucks point is it doesn't play gapless transcoding,  but I'm all WAV & DSD so don't need transcoding.  

I don't transcode, so never notcied the second point. I've  not seen the 1st one at all, art loading is snappy on my IPhone and IPad, from SMS on a DS115j and from a DS216j, odd how what you'd think would be identical setups work differently.

Posted on: 06 March 2018 by trickydickie
jsaudio posted:

All this has got me somewhat worried and so I would like opinions on my back-up strategy.

I have a QNAP NAS with 2 drives that I think have a RAID configuration. I also have a My Book drive connected for scheduled back -ups right next to the NAS. I assume that its working as planned. I also had someone help do a back-up about 6 months ago to an additional external drive that I keep off site. Have ripped and downloaded a lot of music since then so should try to update that. 

Is this enough? Is there a better way? Cloud based BU?

thanks

I would suggest that you do the following

  1.  Check that the backup to the My Book drive is actually working.  It's worth setting your email details up on the QNAP so it can email you any errors or warnings but I would also recommend that you manually check from time to time.  
  2. Your external offsite backup should be refreshed regularly.  Most QNAPs allow you to setup a backup job that you can start by plugging in the drive and pressing a button on the front panel.

 

Cloud based backup is also good, you need to select a provider that works with the QNAP ideally so the process can be automated.  I backup to Microsoft Azure but you do need a bit of knowledge to do this.  Bespoke backup providers can be easier but can be quite expensive.

Posted on: 06 March 2018 by jsaudio

Thanks Richard will look into cloud based options, possibly apple I-cloud?