This is unnecessarily complex, you can just offline a drive, remove the disk, insert the new disk, and replace the offline drive with the new disk. ZFS will resilver the new disk.
You gradually replace one disk at a time, resilver then repeat. ZFS automates most or all the work mentioned in this article and it comes with built-in bitrot detection.
Btw I have run a NAS similar to this machine for 6 years now using FreeNAS. It’s awesome.
If you have space you can “zfs attach” the new drives to make mirror vdevs with the disks they’re replacing, and then “zfs detach” the old drives to break the mirror when done.
This is preferable because you’re never exposed to any additional risk of data loss and can replace more disks at once than original pool redundancy would allow.
Something that's important to consider is if your drives can handle a resilver. Each resilver is essentially reading the entire disk (with lots of asterisks) while setting up a parallel zfs system to provision your disks and using the transfer I described will only read your systems fs once.
You don't even need to use the same system. You can setups a different system, run the transfer, turn off the main system, swap the disks.
It's a hastle but it's an option at your disposal.
I believe there are no problems with your scenario, there's zpool replace command which does this:
zpool replace [-f] pool device [new_device]
Replaces old_device with new_device. This is equivalent to attaching
new_device, waiting for it to resilver, and then detaching
old_device.
The size of new_device must be greater than or equal to the minimum
size of all the devices in a mirror or raidz configuration.
new_device is required if the pool is not redundant. If new_device is
not specified, it defaults to old_device. This form of replacement
is useful after an existing disk has failed and has been physically
replaced. In this case, the new disk may have the same /dev path as
the old device, even though it is actually a different disk. ZFS
recognizes this.
-f Forces use of new_device, even if its appears to be in use.
Not all devices can be overridden in this manner.
The problem is that ZFS doesn't have an offline repair tool. A (granted unlikely) bit flip in an important data structure that gets written to disk makes the whole fs unmountable and that's it (idk if it has a tool to rescue file data from a unmountable pool? Maybe we should ask Gandi...).
With e.g. ext4 you can get back to a mountable state pretty much guaranteed with e2fsck. You might loose a few files, or find them in lost+found, etc. but at least you have something.
The reason ZFS doesn't have a offline repair tool is pretty convincing. Once you have zettabytes (that's the marketing) of data, running that repair tool will take too long, so you'd have to do everything to prevent that in the first place anyway. Including checksumming everything, storing everything redundantly and using ECC RAM.
ZFS is great. But one thing scares me away from using it. With ZFS, you can't just simply add a new disk raidz set. You have to either build a new pool to send your data to, or :
1. Buy the same amount of (but larger) hard-disks
2. Replace them one by one
3. After the last disk is done, the size will grow.
4. Sell your old hard-disks.
You can, although you need to replace all the drives in the pool. You can swap one, wait for resilver (or for a month, if you're in a budget), do the next one... And once you replaced every drive, you do like this:
> That's the easy thing with ZFS, just add the disks as new VDEVs.
Careful though, depending on how you're setting up the VDEVs this could introduce fragility to the pool. For example, if you have a raidz2 partition and then add a single disk vdev, that single disk going down would make the entire pool unavailable.
Of course that ZFS supports any sized drives in a pool. ZFS expands very easily, in fact, the user doesn't have to do anything, it automatically expands.
Of course you can remove drives in ZFS. How else could you replace disks?
I downvoted your post because every statement you made about ZFS is wrong.
ZFS has open my eyes to /dev/disk/by-id/. It's the only way when you have many of the exact same make/model/size drives per machine. You will learn by fire the first time you need to determine which drive you need to replace.
Can I add another gotcha - if you have a failed device in zfs and you can't import the pool anymore, don't think you can walk the drive and put the blocks back together manually.
You can do this in FAT and mostly in NTFS and UFS - especially if your drive is defragmented.
But the complicated data structures of ZFS makes this really hard if not close to impossible - there are tools and ways of using ZFS code to help you do this, but they are exceedingly hard to use.
Source: took me 52 hours to retrieve a 16k file from a failed device. Granted it would take me less time now, but I now think of devices the have failed as if they has completely disappeared from our universe.
We are talking about something resembling adding an extra disk to raid5. Can be easily done in mdadm raid, and then you just need to resize lvm, or whatever you run over it. Can not be done in zfs, not in raid5/6 mode
I've had a few drives fail on my ZFS for Linux fileserver and wondered why my hot spares weren't automatically kicking in, and this is why.
On Linux, if you don't use the zed script that's referenced in that Github issue above and just replace a failing drive manually, a hot spare is worse than useless, because you need to remove the hot spare from the array before you can use it with a manual replace operation.
reply