Jun 24th, 2014, 4:31 pm
Ever since I can remember, books have been a hugely important part of my life. There's no doubt that other than food and shelter, reading material is the one material item I couldn't live without. Everything else is extra, but books are essential.

Now, like a dream come true, and despite the fact that I am family man of very modest means, and thanks to (primarily) amazing people on this site, I have literally more than enough to read for the rest of my life.

So how do I protect my library? Obviously I have to backup, but where do I keep my backup? What's the way they will be most safe, and how safe can they be?

As you can tell, I'm not especially IT savvy, so forgive me if these are stupid questions but as fellow book lovers and generous folk, I thought surely someone would be willing to give me a tip or two.

Thanks in advance and thank you all for participating on this site.
Jun 24th, 2014, 4:31 pm
Jun 24th, 2014, 8:20 pm
Being a constant computer user, and having lost terabytes of data over the last ten years to hard drive crashes, I'd recommend backing up your library on at least two different drives: a large external drive with a good access speed, and a second backup on a 16 or 32 G flash drive, that may have a slower access rate, but doesn't get used as often. use the external HD on a daily or weekly basis, to supplement what's on your computer, and the flash drive monthly, as a safety measure. that way, if your computer goes belly up, or your external HD gets kicked across the room (trust me, it's happened to me), then you have an alternative site to recover from, and you're less likely to lose all those thousands of ebooks.
Jun 24th, 2014, 8:20 pm
Jun 24th, 2014, 10:52 pm
I would suggest a third alternative for storage: cd/dvd backup. If you have the options on your computer, you could backup files on rewritable discs, and sync them with the files on your computer. These files will last a very long time, and it's inexpensive enough that you could make multiple copies that you could keep in a safe place away from your home.
Jun 24th, 2014, 10:52 pm
Jun 25th, 2014, 11:28 am
A good NAS with good expensive/enterprise grade drives in hardware raid + some other backup on single external drives
Jun 25th, 2014, 11:28 am
Jun 26th, 2014, 1:14 am
I use a range of storage for my files, depending on what they are - external drive, DVD/RW and flash drive. One other thing I've been fiddling with using an enclosure on old hard disk drives. Lap top drives are very handy as they as enclosures are cheap and tend not to need a power supply other than the USB, so they are portable enough to stick in a pocket.

You also need to keep in mind that if all your data solutions are stored in one location then a house fire will take out all but the cloud storage. You'd be wise to store one of the more portable alternatives (flash drive, DVD/RW or enclosure) at a friend or family members house (work might be a bit tricky given the content you are storing). If you have two you can alternate them, rather than bringing your single device home, backing up and taking it back - you just need to back up before you go, leave the storage device and bring home the other one. I'd look into some kind of encryption, as you don't want people snooping through your documents. Something like DiskCryptor should do the trick.
Jun 26th, 2014, 1:14 am

Links dead? Need re-up? PM me.

Strange is the night where black stars rise,
And strange moons circle through the skies,
But stranger still is Lost Carcosa.
Jun 26th, 2014, 4:56 am
You guys are great, thanks! (Even mbmbo, whose post was incomprehensible to me!)

What about storage of storage - in other words, any particular way to store hard drives, discs, flashdrives etc that is better than another? (Not sure what I mean but I suppose I'm thinking of heat, cold, moisture or any other environmental factors that could be deleterious over years).
Jun 26th, 2014, 4:56 am
Jun 26th, 2014, 5:24 am
wallyjack wrote:What about storage of storage - in other words, any particular way to store hard drives, discs, flashdrives etc that is better than another? (Not sure what I mean but I suppose I'm thinking of heat, cold, moisture or any other environmental factors that could be deleterious over years).


Nothing much beyond common sense - they are designed to work in a human-friendly environment, so don't store it anywhere there are extremes of temperature or humidity. Keep away from sources of strong electrical fields. It might be an idea to bag up any storage you don't have permanently connected (it is a good idea to keep perhaps your largest storage permanently hooked up and sycing - perhaps using something like SyncBack, which will also help you set up back-ups). In addition to the basics, I keep my second largest storage with a full file back-up in a fireproof safe with legal and important family documents.

It works nicely - my main PC actually blew-up and I was back up and running within a day with perhaps only the loss of my browser tabs and if I cared enough about them, I'd sync those too.

At this stage my data is pretty safe this side of a nuclear blast or solar flare, I'm more likely to get erased than it is. :shock:
Jun 26th, 2014, 5:24 am

Links dead? Need re-up? PM me.

Strange is the night where black stars rise,
And strange moons circle through the skies,
But stranger still is Lost Carcosa.
Jun 26th, 2014, 5:42 am
Thanks for a prompt, informative and funny reply...

You've given me good plan as my goals are twofold: one to maintain a synced/up to date back up of everything but I'm also interested in just having a sort of fail safe of what I have currently that I can keep for the rest of my life (30 years? hopefully) - obviously in that time both software and hardware changes will be beyond what I can imagine and conversions etc would have to take place, but my hope is that somehow, the thousands of ebooks I already have will last me in some form or another...

Hence my thoughts about physical storage of storage devices. Thanks again.
Jun 26th, 2014, 5:42 am
Jun 26th, 2014, 6:05 am
The big developments, and the most useful to me, has been in cloud storage. I keep all active files in the cloud, so as soon as you save a file off it syncs with the cloud. It is very satisfying to plug in a new computer, install Dropbox, log in and sit back watching all your files starting to slot into place. As I've now got two or three different types of cloud storage on PC, laptop and phone I can just wonder off to watch a film and the file is already waiting for me on the laptop. Evernote is also very useful in that regard.

wallyjack wrote:You've given me good plan as my goals are twofold: one to maintain a synced/up to date back up of everything but I'm also interested in just having a sort of fail safe of what I have currently that I can keep for the rest of my life (30 years? hopefully) - obviously in that time both software and hardware changes will be beyond what I can imagine and conversions etc would have to take place, but my hope is that somehow, the thousands of ebooks I already have will last me in some form or another...


I'd be wary of having a fail safe, as anything can fail - it'd be better to stay flexible and keep duplicate copies on different storage in different locations. As one fails, you can just roll on to slotting the next one in as a replacement. That way your storage solutions will last as long as you do, even if ever single one of the devices has been replaced numerous times (and in 30 years we'll probably be doing the same thing with out internal organs - it'll be like your grandmother's broom that has has used for 50 years, despite the dozens of replacements to the head and handle ;) ).

Perhaps a more important choice is format - Calibre should always be able to switch formats, but each time you degrade the quality. Even looking back a few years on here and you'll find formats people tend not to release these days, like .lit, and you can see how quickly things can change. Even though recent developments suggest everyone is resigned to the fact that the Nook will never be a true competitor to the Kindle, the .ePub format it uses looks to be here to stay and is a very useful "base" format for converting into others. That is why I've tried to get that format for my core library (the one loaded into Calibre, as opposed to the random other ebooks I have sloshing around) and will happily update to the retail version as they become available. However, that is just my opinion and YMMV.
Jun 26th, 2014, 6:05 am

Links dead? Need re-up? PM me.

Strange is the night where black stars rise,
And strange moons circle through the skies,
But stranger still is Lost Carcosa.
Jun 26th, 2014, 2:34 pm
wallyjack wrote:but I'm also interested in just having a sort of fail safe of what I have currently that I can keep for the rest of my life (30 years? hopefully) - obviously in that time both software and hardware changes will be beyond what I can imagine and conversions etc would have to take place, but my hope is that somehow, the thousands of ebooks I already have will last me in some form or another...



Sure, print them out on paper and store them in a fire-proof safe...

You know even burned CD/DVD discs really don't have a fool-proof shelf life storage of that long. Your best bet is to do what others have suggested and not try to "permanently" store them anywhere. Keep your archive on a folder on your personal computer; get a large capacity flash drive and store a copy of your archive on it and hook it to your key chain; create a cloud account on gDrive/DropBox/MsOne/etc., and store copies there.

Myself, I have an archive folder on my PC, a couple of 32GB flash drives, and a small 150GB usb hard drive where I keep backup copies loaded. Periodically, as newer and better quality versions of some of the books appear or newer formats come out, I replace the older copies across the storage devices.
Jun 26th, 2014, 2:34 pm
Jun 26th, 2014, 2:43 pm
Carcosa wrote:Perhaps a more important choice is format - Calibre should always be able to switch formats, but each time you degrade the quality. Even looking back a few years on here and you'll find formats people tend not to release these days, like .lit, and you can see how quickly things can change. Even though recent developments suggest everyone is resigned to the fact that the Nook will never be a true competitor to the Kindle, the .ePub format it uses looks to be here to stay and is a very useful "base" format for converting into others. That is why I've tried to get that format for my core library (the one loaded into Calibre, as opposed to the random other ebooks I have sloshing around) and will happily update to the retail version as they become available. However, that is just my opinion and YMMV.



I tend to keep:

Original image scans (if it my own conversion of a personal paper book).
Text file (usually simple html with bare formatting for headers, paragraphs, italics).
Complied ebook format (currently ePub).

I can't comment on Calibre as I do not use it because I find it a bit cumbersome and the output generated from the program tends to do more damage than good. Of course this second opinion is based primarily on the ebooks I have come across that were almost unreadable and overloaded with "class=calibre" formatting ... so your results may vary.
Jun 26th, 2014, 2:43 pm
Jun 26th, 2014, 3:03 pm
Carcosa wrote:Perhaps a more important choice is format - Calibre should always be able to switch formats, but each time you degrade the quality. Even looking back a few years on here and you'll find formats people tend not to release these days, like .lit, and you can see how quickly things can change. Even though recent developments suggest everyone is resigned to the fact that the Nook will never be a true competitor to the Kindle, the .ePub format it uses looks to be here to stay and is a very useful "base" format for converting into others.


While the Nook might be suffering the newer ebooks for modern Kindles (AZW/AZW3/KF8) is basically a zipped file containing a .ePub for the newer ereaders and a .mobi for the older ones. So Amazon has also seen the power of the .epub and is moving over to using it, even if it isn't obvious from the outside. Calibre will convert to AZW3, which will preserve most of the formatting (I'm still experimenting to see what the differences are between that and using Kindle Preview for the conversion). All of which means if you focus on collecting the retail .ePubs (and AZW or AZW3 where available) you should have a pretty future-proofed set of ebooks no matter which ereader you buy over the years. At least until Amazon come up with a way to make their ebooks 3D to work with their new device and then they develop a way to make them holograms that the Kindle Fire 23 will beam directly down your optic nerve into your brain. Even then, I bet Calibre will figure out a way to convert an .ePub into that format ;)
Jun 26th, 2014, 3:03 pm

Links dead? Need re-up? PM me.

Strange is the night where black stars rise,
And strange moons circle through the skies,
But stranger still is Lost Carcosa.
Jun 26th, 2014, 3:12 pm
Steven522 wrote:
Carcosa wrote:Perhaps a more important choice is format - Calibre should always be able to switch formats, but each time you degrade the quality. Even looking back a few years on here and you'll find formats people tend not to release these days, like .lit, and you can see how quickly things can change. Even though recent developments suggest everyone is resigned to the fact that the Nook will never be a true competitor to the Kindle, the .ePub format it uses looks to be here to stay and is a very useful "base" format for converting into others. That is why I've tried to get that format for my core library (the one loaded into Calibre, as opposed to the random other ebooks I have sloshing around) and will happily update to the retail version as they become available. However, that is just my opinion and YMMV.



I tend to keep:

Original image scans (if it my own conversion of a personal paper book).
Text file (usually simple html with bare formatting for headers, paragraphs, italics).
Complied ebook format (currently ePub).

I can't comment on Calibre as I do not use it because I find it a bit cumbersome and the output generated from the program tends to do more damage than good. Of course this second opinion is based primarily on the ebooks I have come across that were almost unreadable and overloaded with "class=calibre" formatting ... so your results may vary.


It can require some clean-up but a bit of find and replace in Sigil tends to do the trick - if you have an AZW/AZW3 to convert there is a Calibre plug-in that unpacks the file, giving you direct access to the .ePub, so Calibre wins there too. I prefer not to convert to .ePub there though if it can't be helped, but I'll store the best quality .ePub and use that as the base format.

I tend to find that the "class=calibre" either comes from people converting older formats (.mobi or .lit) or not knowing how to get at the .ePub in an AZW/AZW3 file (and I'm including myself in that category before I realised what Amazon were doing inside the file). Hopefully, we'll see less of it in the future.
Jun 26th, 2014, 3:12 pm

Links dead? Need re-up? PM me.

Strange is the night where black stars rise,
And strange moons circle through the skies,
But stranger still is Lost Carcosa.
Jun 26th, 2014, 4:45 pm
Carcosa wrote:If you have an AZW/AZW3 to convert there is a Calibre plug-in that unpacks the file, giving you direct access to the .ePub, so Calibre wins there too. I prefer not to convert to .ePub there though if it can't be helped, but I'll store the best quality .ePub and use that as the base format.

I tend to find that the "class=calibre" either comes from people converting older formats (.mobi or .lit) or not knowing how to get at the .ePub in an AZW/AZW3 file (and I'm including myself in that category before I realised what Amazon were doing inside the file). Hopefully, we'll see less of it in the future.

[My emphasis]

Didn't know such a plugin existed. Are you referring to the KindleUnpack plugin listed here? - http://plugins.calibre-ebook.com


More help on How To Unpack AZW/AZW3/KF8 w/ Calibre (KEEP RETAIL VERSIONS)
Jun 26th, 2014, 4:45 pm
Jun 26th, 2014, 10:59 pm
wallyjack wrote:You guys are great, thanks! (Even mbmbo, whose post was incomprehensible to me!)

What about storage of storage - in other words, any particular way to store hard drives, discs, flashdrives etc that is better than another? (Not sure what I mean but I suppose I'm thinking of heat, cold, moisture or any other environmental factors that could be deleterious over years).


Lol

http://www.amazon.com/Synology-DiskStat ... 007JLE84C/

4 drive NAS box + 4x of these
http://www.amazon.com/WD-Red-NAS-Hard-Drive/dp/B008JJLW4M/

In RAID6 2 of the 4 drives could die and you be ok before replacing it,
RAID5 one of the 4 could die and you be ok before replacing it.

Mind you ebooks dont take up terrabytes of space, but the above be good for your family photos and videos etc
Jun 26th, 2014, 10:59 pm