I have a 2 bay NAS, and I was planning on using 2x 18tb HDDs in raid 1. I was planning on purchasing 3 of these drives so when one fails I have the replacement. (I am aware that you should purchase at different times to reduce risk of them all failing at the same time)
Then I setup restic.
It makes backups so easy that I am wondering if I should even bother with raid.
Currently I have ~1TB of backups, and with restics snapshots, it won’t grow to be that big anyways.
Either way, I will be storing the backups in aws S3. So is it still worth it to use raid? (I also will be storing backups at my parents)
Keep in mind that if you set up raid using zfs or btrfs (idk how it works with other systems but that’s what I’ve used) then you also get scrubs which detect and fix bit rot and unrecoverable read errors. Without that or a similar system, those errors will go undetected and your backup system will backup those corrupted files as well.
Personally one of the main reasons I used zfs and now btrfs with redundancy is to protect irreplaceable files (family memories and stuff) from those kinds of errors, as I used to just keep stuff on a hard drive until I discovered loads of my irreplaceable vacation photos to be corrupted, including the backups which backed up the corruption.
If your files can be reacquired, then I don’t think it’s a big deal. But if they aren’t, then I think having scrubs or integrity checks with redundancy so that issues can be repaired, as well as backups with snapshots to prevent errors or mistakes from messing up your backups, is a necessity. But it just depends on how much you value your files.
Note that you do not need any sort of redundancy to detect corruption.
Redundancy only gains you the ability to have that corruption immediately and automatically repaired.
While this sounds nice in theory, you have no use for such auto repair if you have backups handy because you can simply restore that data manually using your backups in the 2 times in your lifetime that such corruption actually occurs.
(If you do not have backups handy, you should fix that before even thinking about RAID.)It’s incredibly costly to have such redundancy at a disk level and you’re almost always better off using those resources on more backups instead if data security is your primary concern.
Downtime mitigation is another story but IMHO it’s hardly relevant for most home users.backups in the 2 times in your lifetime that such corruption actually occurs.
What are you even talking about here? This line invalidates everything else you’ve said.
I was thinking whether I should elaborate on this when I wrote the previous reply.
At the scale of most home users (~dozens of TiBs), corruption is actually quite unlikely to happen. It’ll happen maybe a handful of times in your lifetime if you’re unlucky.
Disk failure is actually also not all that likely (maybe once every decade or so, maybe) but still quite a bit more likely than corruption.
Just because it’s rare doesn’t mean it never happens or that you shouldn’t protect yourself against it though. You don’t want to be caught with your pants down when it does actually happen.
My primary point is however that backups are sufficient to protect against this hazard and also protect you against quite a few other hazards. There are many other such hazards and a hard drive failing isn’t even the most likely among them (that’d be user error).
If you care about data security first and foremost, you should therefore prioritise more backups over downtime mitigation technologies such as RAID.
It depends on your uptime requirements.
According to Backblaze stats on similarly modern drives, you can expect about a 9% probability that at least one of those drives has died after 6 years. Assuming 1 week recovery time if any one of them dies, that’d be a 99.997% uptime.
If that’s too high of a probability for needing to run a (in case of AWS potentially very costly) restore, you should invest in RAID. Otherwise, that money is better spent on more backups.
Raid 1 has saved my server a couple of times over from disaster. I make weekly cold backups, but I didn’t have to worry about it when my alert came in notifying me which drive went dead - just swap, rebuild, move along. So yeah I’d say it’s definitely worth it. Just don’t treat raid as a backup solution - and yes, continue to use an external cold storage backup solution as you mentioned. Fires, exploding power supplies, ransomware, etc don’t care if you’re using raid or not.
It’s up to you. Things to consider:
- Size of data
- Recovery speed (Internet speed)
- Recovery time objective
- Recovery point objective (If you’re backing up once per day, is it okay to lose 23 hours of data when a disk fails?)
If your recovery objectives can be met with the anticipated data size and recovery speed, then you could do RAID 0 instead of RAID 1 to get higher speeds and capacity. Just know that if you do that, you better be on top of your backups because they will be needed eventually.
Yes, I would still do raid. Because a disk fail will not cause a blackout. Much better than have your server offline waiting to replace disk and restore backup.
And no way you can backup 18tb in 1tb, restic or no restic.
i was also thinking like this, then i had to restore everything from a backup when the ssd suddenly died. I wasted so much time setting everything back as before
If you needed to spend any time “setting everything back as before”, you didn’t have a full backup.
the reason OP was thinking of doing this, was saving disk space and avoiding buying another hdd. So if it’s a 1:1 full disk image, then there’s almost no difference with the costs of raid1. Setting exclusions, avoiding certain big files, and so on. In this case he’s talking about restic, which can restore data but very hard to do a full bootable linux system - stuff needs to be reinstalled
if it’s a 1:1 full disk image, then there’s almost no difference with the costs of raid1
The problem with that statement is that you’re likening a redundant but dependant copy to a backup which is a redundant independent copy. RAID is not a backup.
As an easy example to illustrate this point: if you delete all of your files, they will still be present in a backup while RAID will happily delete the data on all drives at the same time.
Additionally, backup tools such as restic offer compression and deduplication which saves quite a bit of space; allowing you to store multiple revisions of your data while requiring less space than the original data in most cases.
In this case he’s talking about restic, which can restore data but very hard to do a full bootable linux system - stuff needs to be reinstalled
It’s totally possible to make a backup of the root filesystem tree and restore a full system from that if you know what you’re doing. It’s not even that hard: Format disks, extract backup, adjust fstab, reinstall bootloader, kernels and initrd into the boot/ESP partition(s).
There’s also the wasteful but dead simple method to backing up your whole system with all its configuration which is full-disk backups. The only thing this will not back up are EFI vars but those are easy to simply set again or would just remain set as long as you don’t switch motherboards.
I’m used to Borgbackup which fulfils a very similar purpose to restic, so I didn’t know this but restic doesn’t appear to have first-class support for backing up whole block devices but it appears this can be made to work too: https://github.com/restic/restic/issues/949
I must admit that I also didn’t think of this as a huge issue because declarative system configuration is a thing. If you’re used to it, you have a very different view on the importance of system configuration state.
If my server died, it’d be a few minutes of setting up the disk format and then waiting for a ~3.5GiB download after which everything would work exactly as it did before modulo user data. (The disk format step could also be automatic but I didn’t bother implementing that yet because of https://xkcd.com/1205/.)
Yes yes yes yes yes
Raid1 that thing and sleep easier. Good on you for having a cold spare, and knowing to buy your drives at different locations/times to get different batches. Your head is in the right place! No reason to leave that data unprotected if you have the underlying tech and hardware.
RAID is a great backup alternative.
/s
