Hacker Read top | best | new | newcomments | leaders | about | bookmarklet login

This is unnecessarily complex, you can just offline a drive, remove the disk, insert the new disk, and replace the offline drive with the new disk. ZFS will resilver the new disk.


sort by: page size:

zfs replace does not require you to offline the drives. TFA only suggests it as a workaround for the reslivering being slowed down by a faulty drive.

You gradually replace one disk at a time, resilver then repeat. ZFS automates most or all the work mentioned in this article and it comes with built-in bitrot detection.

Btw I have run a NAS similar to this machine for 6 years now using FreeNAS. It’s awesome.


If you have space you can “zfs attach” the new drives to make mirror vdevs with the disks they’re replacing, and then “zfs detach” the old drives to break the mirror when done.

This is preferable because you’re never exposed to any additional risk of data loss and can replace more disks at once than original pool redundancy would allow.


ZFS doesn't do it as you phase in disks, but once you've replaced all of them you get the extra capacity.

Something that's important to consider is if your drives can handle a resilver. Each resilver is essentially reading the entire disk (with lots of asterisks) while setting up a parallel zfs system to provision your disks and using the transfer I described will only read your systems fs once.

You don't even need to use the same system. You can setups a different system, run the transfer, turn off the main system, swap the disks.

It's a hastle but it's an option at your disposal.


I believe there are no problems with your scenario, there's zpool replace command which does this:

     zpool replace [-f] pool device [new_device]

         Replaces old_device with new_device.  This is equivalent to attaching
         new_device, waiting for it to resilver, and then detaching
         old_device.

         The size of new_device must be greater than or equal to the minimum
         size of all the devices in a mirror or raidz configuration.

         new_device is required if the pool is not redundant. If new_device is
         not specified, it defaults to old_device.  This form of replacement
         is useful after an existing disk has failed and has been physically
         replaced. In this case, the new disk may have the same /dev path as
         the old device, even though it is actually a different disk.  ZFS
         recognizes this.

         -f      Forces use of new_device, even if its appears to be in use.
                 Not all devices can be overridden in this manner.

The problem is that ZFS doesn't have an offline repair tool. A (granted unlikely) bit flip in an important data structure that gets written to disk makes the whole fs unmountable and that's it (idk if it has a tool to rescue file data from a unmountable pool? Maybe we should ask Gandi...).

With e.g. ext4 you can get back to a mountable state pretty much guaranteed with e2fsck. You might loose a few files, or find them in lost+found, etc. but at least you have something.

The reason ZFS doesn't have a offline repair tool is pretty convincing. Once you have zettabytes (that's the marketing) of data, running that repair tool will take too long, so you'd have to do everything to prevent that in the first place anyway. Including checksumming everything, storing everything redundantly and using ECC RAM.


Hm?

It automatically replaces the dead drive with a hot spare over a couple of days. You can also do it in a couple of hours with dRAID: https://openzfs.github.io/openzfs-docs/Basic%20Concepts/dRAI...

You just need a couple of parity drives per vdev and one hot spare for the whole array


ZFS is great. But one thing scares me away from using it. With ZFS, you can't just simply add a new disk raidz set. You have to either build a new pool to send your data to, or :

1. Buy the same amount of (but larger) hard-disks 2. Replace them one by one 3. After the last disk is done, the size will grow. 4. Sell your old hard-disks.

Much more painful then other raid solutions.


You can, although you need to replace all the drives in the pool. You can swap one, wait for resilver (or for a month, if you're in a budget), do the next one... And once you replaced every drive, you do like this:

https://www.ateamsystems.com/tech-blog/expand-zfs-to-use-lar...


Forgive me but I'm completely missing what you're saying here. Are you saying that ZFS has a write hole when reslivering?

Thanks, I've been wondering about the "proper" way of doing this in ZFS and this method hadn't come up in my searches.

> That's the easy thing with ZFS, just add the disks as new VDEVs.

Careful though, depending on how you're setting up the VDEVs this could introduce fragility to the pool. For example, if you have a raidz2 partition and then add a single disk vdev, that single disk going down would make the entire pool unavailable.


How exactly did you do that? I'm learning ZFS still and I'm planning to move some of the disks in my first NAS to a better machine.

Which ZFS commands?


Of course that ZFS supports any sized drives in a pool. ZFS expands very easily, in fact, the user doesn't have to do anything, it automatically expands.

Of course you can remove drives in ZFS. How else could you replace disks?

I downvoted your post because every statement you made about ZFS is wrong.


ZFS has open my eyes to /dev/disk/by-id/. It's the only way when you have many of the exact same make/model/size drives per machine. You will learn by fire the first time you need to determine which drive you need to replace.

Can I add another gotcha - if you have a failed device in zfs and you can't import the pool anymore, don't think you can walk the drive and put the blocks back together manually.

You can do this in FAT and mostly in NTFS and UFS - especially if your drive is defragmented.

But the complicated data structures of ZFS makes this really hard if not close to impossible - there are tools and ways of using ZFS code to help you do this, but they are exceedingly hard to use.

Source: took me 52 hours to retrieve a 16k file from a failed device. Granted it would take me less time now, but I now think of devices the have failed as if they has completely disappeared from our universe.


We are talking about something resembling adding an extra disk to raid5. Can be easily done in mdadm raid, and then you just need to resize lvm, or whatever you run over it. Can not be done in zfs, not in raid5/6 mode

FYI for those running ZFS on Linux or FreeNAS (and maybe FreeBSD, as well):

The nice zfs built-in autoreplace functionality doesn't work on Linux or FreeNAS. You need some scripting/external tooling to do the equivalent. See

https://github.com/zfsonlinux/zfs/issues/2449

https://forums.servethehome.com/index.php?threads/zol-hotspa...

I've had a few drives fail on my ZFS for Linux fileserver and wondered why my hot spares weren't automatically kicking in, and this is why.

On Linux, if you don't use the zed script that's referenced in that Github issue above and just replace a failing drive manually, a hot spare is worse than useless, because you need to remove the hot spare from the array before you can use it with a manual replace operation.

next

Legal | privacy