= 27d567bc2d48d9f40e9ccc20ea370b15)
iceman comments on Open Thread, January 4-10, 2016 - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (430)
PSA: I had a hard drive die on me. Recovered all my data with about 25 hours of work all up for two people working together.
Looking back on it I doubt many things could have convinced me to improve my backup systems; short of working in the cloud; my best possible backups would have probably lost the last two weeks of work at least.
I am taking suggestions for best practice; but also a shout out to backups, and given it's now a new year, you might want to back up everything before 2016 right now. Then work on a solid backing up system.
(Either that or always keep 25 hours on hand to manually perform a ddrescue process on separate sectors of a drive; unplugging and replugging it in between each read till you get as much data as possible out, up until 5am for a few nights trying to scrape back the entropy from the bits...) I firmly believe with the right automated system it would take less than 25 hours of effort to maintain.
bonus question: what would convince you to make a backup of your data?
Use RAID on ZFS. RAID is not a backup solution, but with the proper RAIDZ6 configuration will protect you against common hard drive failure scenarios. Put all your files on ZFS. I use a dedicated FreeNAS file server for my home storage. Once everything you have is on ZFS, turn on snapshotting. I have my NAS configured to take a snapshot every hour during the day (set to expire in a week), and one snapshot on Monday which lasts 18 months. The short lived snapshots lets me quickly recover from brain snafus like overwriting a file.
Long lived snapshotting is amazing. Once you have filesystem snapshots, incremental backups become trivial. I have two portable hard drives, one onsite and one offsite. I plug in the hard drive, issue one command, and a few minutes later, I've copied the incremental snapshot to my offline drive. My backup hard drives become append only logs of my state. ZFS also lets you configure a drive so that it stores copies of data twice, so I have that turned on just to protect against the remote chance of random bitflips on the drive.
I do this monthly, and it only burns about 10 minutes a month. However, this isn't automated. If you're willing to trust the cloud, you could improve this and make it entirely automated with something like rsync.net's ZFS snapshot support. I think other cloud providers also offer snapshotting now, too.
I feel that this is too complicated a solution for most people to follow. And it's not a very secure backup system anyway.
You can just get an external hard drive and use any of the commonly-available full-drive backup software. Duplicity is a free one and it has GUI frontends that are basically just click-to-backup. You can also set them up to give you weekly reminders, etc.