For years my homelab ran mdadm RAID with ext4 on top, the same setup I'd recommend to anyone wanting straightforward redundancy without learning a new filesystem. Then a SATA cable started intermittently flipping bits on one drive, and I learned exactly what "silent corruption" means when nothing in the stack is checksumming your data.
The incident that started this
Photos from a trip came back from a backup looking subtly wrong — color banding in places that hadn't been there originally. RAID had faithfully mirrored corrupted data across both drives, because mdadm has no concept of whether the bytes it's writing are correct, only that both drives have the same bytes. The cable was eventually the culprit, but the real problem was that nothing in my stack would have caught it earlier, regardless of the cause.
Why ZFS over mdadm + ext4
ZFS checksums every block, on every read and write, and compares it against a stored checksum from when the block was written. If a drive lies about what it actually stored — which happens more often than most people assume, especially on consumer hardware — ZFS catches it immediately rather than serving you quietly corrupted data.
Beyond checksumming, the other practical wins were:
- Snapshots that cost almost nothing at creation time and let me roll back a dataset to any prior point.
- Compression (lz4 by default) that's fast enough to leave on everywhere, typically saving 20–40% of space on general-purpose data with no noticeable CPU cost.
- Send/receive for incremental backups that transfer only the blocks that changed, rather than re-syncing entire datasets.
Setting up the pool
Debian 12 doesn't ship ZFS in the default kernel due to licensing, but it's well-supported via the contrib repository and DKMS:
Installing ZFS on Debian 12
apt install -y linux-headers-amd64 zfsutils-linux zfs-dkmsWith four 4TB drives, I went with RAIDZ1 — tolerating one drive failure, with about 75% of raw capacity usable. RAIDZ2 would tolerate two failures at the cost of more overhead; for a homelab that's backed up elsewhere, RAIDZ1 was the right tradeoff.
Creating the pool
zpool create -o ashift=12 tank raidz1 \
/dev/disk/by-id/ata-DRIVE1 \
/dev/disk/by-id/ata-DRIVE2 \
/dev/disk/by-id/ata-DRIVE3 \
/dev/disk/by-id/ata-DRIVE4Always reference drives by their /dev/disk/by-id/ path, not /dev/sda style names. Device letters can shuffle on reboot, and building a pool on the wrong assumption about which disk is which is a bad way to find that out.
Snapshots as a safety net
Snapshots are near-instant and only consume space for blocks that change afterward, which makes "snapshot before doing anything risky" essentially free:
zfs snapshot tank/containers@pre-upgrade
# ... do the risky thing ...
zfs rollback tank/containers@pre-upgradeI run automated hourly snapshots on the dataset holding container volumes, pruned down to daily after a week and weekly after a month, using sanoid rather than hand-rolled cron scripts.
Scheduling scrubs
A scrub reads every block in the pool and verifies its checksum, catching the kind of corruption that started this whole story before it has a chance to spread to a backup. I run one monthly via systemd timer rather than cron — partly preference, partly covered in a separate post on why I made that switch generally.
zpool scrub tankEighteen months in, the pool has caught two checksum mismatches on aging drives, both resolved automatically from redundant data before I noticed anything. That's the entire point — the failures still happen, but they stop being silent.