# How to Set Up ZFS on FreeBSD: Complete Guide 2026
ZFS is the single most compelling reason to run FreeBSD as a storage platform. While Linux treats ZFS as a third-party add-on constrained by licensing disputes, FreeBSD ships ZFS in its base system, integrated into the kernel, tested in the release cycle, and supported by the core project. If you are building a server that manages data -- a NAS, a backup target, a database host, a virtualization node -- FreeBSD with ZFS is the production-grade choice.
This guide covers everything from your first pool to advanced performance tuning. Every command is real. Every output block reflects what you will actually see on a FreeBSD 14.x system running OpenZFS 2.2.x.
Table of Contents
1. [Why ZFS on FreeBSD](#why-zfs-on-freebsd)
2. [ZFS Concepts](#zfs-concepts)
3. [Creating Your First Pool](#creating-your-first-pool)
4. [Dataset Management](#dataset-management)
5. [Snapshots and Clones](#snapshots-and-clones)
6. [ZFS Send/Receive](#zfs-sendreceive)
7. [Boot Environments with bectl](#boot-environments-with-bectl)
8. [Performance Tuning](#performance-tuning)
9. [Monitoring ZFS Health](#monitoring-zfs-health)
10. [Common Maintenance](#common-maintenance)
11. [Best Practices and Gotchas](#best-practices-and-gotchas)
12. [FAQ](#faq)
---
Why ZFS on FreeBSD
ZFS arrived on FreeBSD in 2007 with version 7.0 and has been a first-class citizen ever since. FreeBSD was the first operating system outside of Solaris to ship ZFS, and the project has maintained that integration for nearly two decades.
Here is what that means in practice:
- **Kernel-native integration.** ZFS ships as a loadable kernel module in the FreeBSD base system. No DKMS. No out-of-tree builds. No license workarounds. It is built, tested, and released alongside the kernel.
- **Boot from ZFS.** FreeBSD supports root-on-ZFS out of the box. The installer offers it as a default option. Boot environments let you snapshot your OS before upgrades and roll back in seconds.
- **OpenZFS upstream alignment.** FreeBSD tracks the OpenZFS project directly. Features like block cloning, dRAID, and the latest compression algorithms land in FreeBSD shortly after upstream merges them.
- **Mature tooling.** bectl for boot environments, zfsd for automated drive replacement, and deep integration with poudriere, iocage, and bhyve make ZFS the backbone of the FreeBSD ecosystem.
If you are building a [FreeBSD NAS](/blog/freebsd-nas-build/), ZFS is not optional -- it is the entire point.
---
ZFS Concepts
Before touching a single command, you need a mental model of how ZFS organizes storage.
Pools and Vdevs
A **pool** (zpool) is the top-level storage container. A pool is made of one or more **vdevs** (virtual devices). Each vdev provides a specific redundancy level:
| Vdev Type | Min Disks | Redundancy | Usable Capacity (4 disks) |
|-----------|-----------|------------|---------------------------|
| stripe | 1 | None | 4x |
| mirror | 2 | N-1 disks | 1x (2-way) or 1x (4-way) |
| raidz1 | 3 | 1 disk | 3x |
| raidz2 | 4 | 2 disks | 2x |
| raidz3 | 5 | 3 disks | 1x (with 4) |
A pool can contain multiple vdevs. Data is striped across vdevs. If a single vdev is lost entirely, the pool is lost. This is the most critical thing to understand: **redundancy is per-vdev, not per-pool.**
Datasets
A **dataset** is a filesystem within a pool. Datasets share pool capacity but can have individual properties: compression, quotas, record size, mount points. Datasets are cheap -- create as many as you need for organizational separation.
Properties
Every dataset and pool has **properties** that control behavior. Properties are inherited by child datasets unless explicitly overridden. Key properties include compression, atime, recordsize, quota, reservation, copies, and dedup.
ARC (Adaptive Replacement Cache)
The **ARC** is ZFS's read cache, held in RAM. It is the single most important performance factor. ZFS uses a sophisticated eviction algorithm that balances recently-used and frequently-used data. More RAM means a larger ARC, which means fewer disk reads.
Checksums and Self-Healing
Every block in ZFS is checksummed. On redundant vdevs (mirror, raidz), ZFS automatically repairs corrupted blocks by reading the correct copy and rewriting the damaged one. This happens transparently during normal reads and during scrubs.
---
Creating Your First Pool
Prerequisites
Enable ZFS at boot by adding this to /etc/rc.conf:
# sysrc zfs_enable="YES"
zfs_enable: -> YES
Load the ZFS kernel module immediately (if not already loaded):
# kldload zfs
Verify it is loaded:
# kldstat | grep zfs
6 1 0xffffffff8284a000 5d8b78 zfs.ko
Identify Your Disks
List available disks with geom:
# geom disk list | grep -E "Geom name|Mediasize|descr"
Geom name: ada0
Mediasize: 256060514304 (238G)
descr: Samsung SSD 870 EVO 250GB
Geom name: ada1
Mediasize: 2000398934016 (1.8T)
descr: WDC WD2003FZEX-00SRLA0
Geom name: ada2
Mediasize: 2000398934016 (1.8T)
descr: WDC WD2003FZEX-00SRLA0
Geom name: ada3
Mediasize: 2000398934016 (1.8T)
descr: WDC WD2003FZEX-00SRLA0
Geom name: ada4
Mediasize: 2000398934016 (1.8T)
descr: WDC WD2003FZEX-00SRLA0
In this example, ada0 is the boot SSD. ada1 through ada4 are four identical 2TB drives for the storage pool.
Mirror Pool (2 Disks)
A mirror is the simplest redundant configuration. Two disks, one copy of your data on each:
# zpool create tank mirror /dev/ada1 /dev/ada2
# zpool status tank
pool: tank
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
ada1 ONLINE 0 0 0
ada2 ONLINE 0 0 0
errors: No known data errors
You lose half your raw capacity but gain the best read performance and the fastest resilver times of any redundant layout.
RAIDZ1 Pool (3+ Disks)
RAIDZ1 tolerates one disk failure. With four disks, you get three disks of usable capacity:
# zpool create storage raidz1 /dev/ada1 /dev/ada2 /dev/ada3 /dev/ada4
# zpool status storage
pool: storage
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
storage ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
ada1 ONLINE 0 0 0
ada2 ONLINE 0 0 0
ada3 ONLINE 0 0 0
ada4 ONLINE 0 0 0
errors: No known data errors
RAIDZ2 Pool (4+ Disks)
RAIDZ2 tolerates two simultaneous disk failures. With four disks, you get two disks of usable capacity. This is the recommended minimum for drives larger than 2TB, where resilver times make a second failure during rebuild a real risk:
# zpool create vault raidz2 /dev/ada1 /dev/ada2 /dev/ada3 /dev/ada4
# zpool list vault
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
vault 7.27T 564K 7.27T - - 0% 0% 1.00x ONLINE -
Stripe Pool (No Redundancy)
A stripe uses all disk capacity with zero redundancy. Only use this for scratch space or data that is fully backed up elsewhere:
# zpool create scratch /dev/ada1 /dev/ada2
Using GPT Labels (Recommended)
Using raw device names (ada1, ada2) works but ties your pool to specific SATA port assignments. GPT labels survive controller changes and cable swaps:
# gpart create -s gpt ada1
# gpart add -t freebsd-zfs -l disk1 ada1
# gpart create -s gpt ada2
# gpart add -t freebsd-zfs -l disk2 ada2
# zpool create tank mirror /dev/gpt/disk1 /dev/gpt/disk2
This is the production approach. Always label your disks.
---
Dataset Management
Creating Datasets
Create a dataset hierarchy for a file server:
# zfs create tank/data
# zfs create tank/data/documents
# zfs create tank/data/media
# zfs create tank/data/backups
# zfs list -r tank
NAME USED AVAIL REFER MOUNTPOINT
tank 444K 1.76T 96K /tank
tank/data 96K 1.76T 96K /tank/data
tank/data/backups 96K 1.76T 96K /tank/data/backups
tank/data/documents 96K 1.76T 96K /tank/data/documents
tank/data/media 96K 1.76T 96K /tank/data/media
Each dataset is automatically mounted at a path matching its name under the pool mountpoint. You can override this with the mountpoint property.
Setting Properties
Set compression on the parent dataset -- children inherit it:
# zfs set compression=lz4 tank/data
# zfs get compression tank/data/documents
NAME PROPERTY VALUE SOURCE
tank/data/documents compression lz4 inherited from tank/data
For media files that are already compressed (video, JPEG, MP3), use zstd or disable compression:
# zfs set compression=off tank/data/media
Disable access time updates globally (reduces write amplification significantly):
# zfs set atime=off tank
Quotas and Reservations
Set a hard quota on a dataset to prevent it from consuming more than its share:
# zfs set quota=500G tank/data/media
# zfs set quota=100G tank/data/documents
Set a reservation to guarantee minimum available space:
# zfs set reservation=200G tank/data/backups
Check quota usage:
# zfs get quota,used,available tank/data/media
NAME PROPERTY VALUE SOURCE
tank/data/media quota 500G local
tank/data/media used 12.3G -
tank/data/media available 487G -
Setting Record Size Per Workload
The default record size is 128K, which is good for sequential I/O (large files, streaming). For databases and VMs, smaller record sizes reduce write amplification:
# zfs create -o recordsize=16K tank/data/postgresql
# zfs create -o recordsize=64K tank/data/vms
For large sequential workloads like video editing or backups:
# zfs create -o recordsize=1M tank/data/backups
Changing Mount Points
# zfs set mountpoint=/srv/media tank/data/media
# zfs mount tank/data/media
# df -h /srv/media
Filesystem Size Used Avail Capacity Mounted on
tank/data/media 500G 12G 487G 3% /srv/media
---
Snapshots and Clones
Snapshots are the killer feature that makes ZFS invaluable for any [FreeBSD backup strategy](/blog/freebsd-backup-guide/). A snapshot is a point-in-time, read-only copy of a dataset. It is instantaneous, costs no additional space at creation time, and only grows as the active dataset diverges.
Creating Snapshots
# zfs snapshot tank/data/documents@2026-03-29
# zfs list -t snapshot
NAME USED AVAIL REFER MOUNTPOINT
tank/data/documents@2026-03-29 0 - 2.1G -
Create recursive snapshots across all child datasets:
# zfs snapshot -r tank/data@daily-2026-03-29
# zfs list -t snapshot -r tank/data
NAME USED AVAIL REFER MOUNTPOINT
tank/data@daily-2026-03-29 0 - 96K -
tank/data/backups@daily-2026-03-29 0 - 156G -
tank/data/documents@daily-2026-03-29 0 - 2.1G -
tank/data/media@daily-2026-03-29 0 - 12.3G -
Browsing Snapshot Contents
Every dataset has a hidden .zfs/snapshot directory:
# ls /tank/data/documents/.zfs/snapshot/
2026-03-29 daily-2026-03-29
# ls /tank/data/documents/.zfs/snapshot/2026-03-29/
contracts/ invoices/ proposals/ reports/
You can copy individual files out of a snapshot without performing a full rollback.
Comparing Snapshots (Diff)
See what changed between a snapshot and the live dataset:
# zfs diff tank/data/documents@2026-03-29
M /tank/data/documents/reports/
+ /tank/data/documents/reports/q1-summary.pdf
M /tank/data/documents/contracts/
- /tank/data/documents/contracts/old-vendor.pdf
M means modified, + means added, - means removed.
Rolling Back
Rollback discards all changes since the snapshot and restores the dataset to that exact state:
# zfs rollback tank/data/documents@2026-03-29
If intermediate snapshots exist between the target and the current state, you must use -r to destroy them:
# zfs rollback -r tank/data/documents@2026-03-29
**Warning:** Rollback is destructive. All data written after the snapshot is lost. Take a fresh snapshot before rolling back if you want to preserve the current state.
Clones
A clone creates a writable copy of a snapshot. It is instantaneous and shares blocks with the original:
# zfs clone tank/data/documents@2026-03-29 tank/data/documents-test
# zfs list tank/data/documents-test
NAME USED AVAIL REFER MOUNTPOINT
tank/data/documents-test 18K 1.76T 2.1G /tank/data/documents-test
Clones are useful for testing changes before applying them to production data, or for creating development environments from production snapshots.
Destroying Snapshots
Remove a single snapshot:
# zfs destroy tank/data/documents@2026-03-29
Remove all snapshots matching a pattern:
# zfs destroy tank/data/documents@daily-%
Automating Snapshot Rotation
Install zfs-auto-snapshot from ports or use a cron job. A simple approach with cron:
# crontab -e
Add:
15 * * * * /sbin/zfs snapshot -r tank/data@hourly-$(date +\%Y\%m\%d-\%H)
0 0 * * * /sbin/zfs snapshot -r tank/data@daily-$(date +\%Y\%m\%d)
0 0 * * 0 /sbin/zfs snapshot -r tank/data@weekly-$(date +\%Y\%m\%d)
Then a cleanup script to remove old snapshots (keep 24 hourly, 30 daily, 12 weekly).
---
ZFS Send/Receive
ZFS send/receive is the native replication mechanism. It serializes a snapshot (or the difference between two snapshots) into a byte stream that can be piped to another pool, another machine, or a file.
Local Replication
Send a snapshot to a different pool on the same machine:
# zfs send tank/data/documents@2026-03-29 | zfs receive backup/documents
Incremental Send
After the initial full send, incremental sends transfer only the changes between two snapshots:
# zfs snapshot tank/data/documents@2026-03-30
# zfs send -i tank/data/documents@2026-03-29 tank/data/documents@2026-03-30 | \
zfs receive backup/documents
The -i flag specifies the incremental source. This is dramatically faster and smaller than a full send.
Remote Replication via SSH
Send to a remote FreeBSD host:
# zfs send tank/data/documents@2026-03-29 | \
ssh backup-server zfs receive offsite/documents
Incremental remote replication:
# zfs send -i tank/data/documents@2026-03-29 tank/data/documents@2026-03-30 | \
ssh backup-server zfs receive offsite/documents
Resumable Send
For large initial sends over unreliable links, use resumable send. If the transfer is interrupted, you can pick up where it left off:
# zfs send -v tank/data@2026-03-29 | ssh backup-server zfs receive -s offsite/data
If interrupted, get the resume token on the receiving side:
backup# zfs get receive_resume_token offsite/data
NAME PROPERTY VALUE SOURCE
offsite/data receive_resume_token 1-bf31...a7e3 -
Resume from the sending side:
# zfs send -t 1-bf31...a7e3 | ssh backup-server zfs receive -s offsite/data
Compressed and Raw Send
Use -c to send compressed blocks without recompressing, and --raw to send encrypted datasets without decrypting:
# zfs send -c --raw tank/data@2026-03-29 | \
ssh backup-server zfs receive offsite/data
Replication Script Example
A practical daily replication script for your [backup workflow](/blog/freebsd-backup-guide/):
sh
#!/bin/sh
# /usr/local/bin/zfs-replicate.sh
POOL="tank/data"
REMOTE="backup-server"
DEST="offsite/data"
TODAY=$(date +%Y-%m-%d)
YESTERDAY=$(date -v-1d +%Y-%m-%d)
# Create today's snapshot
zfs snapshot -r "${POOL}@${TODAY}"
# Incremental send
zfs send -Ri "${POOL}@${YESTERDAY}" "${POOL}@${TODAY}" | \
ssh "${REMOTE}" zfs receive -Fdu "${DEST}"
# Prune snapshots older than 30 days on both sides
for snap in $(zfs list -H -t snapshot -o name -r "${POOL}" | grep '@20' | head -n -30); do
zfs destroy "$snap"
done
The -R flag sends a recursive replication stream (all child datasets). The -F flag on receive forces a rollback if the destination has diverged.
---
Boot Environments with bectl
Boot environments are one of FreeBSD's most powerful features for system administration. A boot environment is a ZFS clone of your root filesystem. You can create one before an upgrade, boot into it, and roll back instantly if something breaks.
Listing Boot Environments
# bectl list
BE Active Mountpoint Space Created
default NR / 4.2G 2026-01-15 10:30
N means active now. R means active on reboot.
Creating a Boot Environment Before an Upgrade
# bectl create pre-upgrade-2026-03-29
# bectl list
BE Active Mountpoint Space Created
default NR / 4.2G 2026-01-15 10:30
pre-upgrade-2026-03-29 - - 8K 2026-03-29 09:00
Now run freebsd-update fetch install or pkg upgrade. If something goes wrong:
Rolling Back
# bectl activate pre-upgrade-2026-03-29
# reboot
After reboot, you are back on the exact system state from before the upgrade. No reinstallation. No restoring from backups.
Mounting a Boot Environment for Inspection
# bectl mount pre-upgrade-2026-03-29 /mnt
# ls /mnt/etc/rc.conf
/mnt/etc/rc.conf
# bectl umount pre-upgrade-2026-03-29
Destroying Old Boot Environments
# bectl destroy pre-upgrade-2026-03-29
Boot environments are cheap. Create one before every system change. There is no reason not to.
---
Performance Tuning
ZFS performance depends primarily on RAM (for the ARC), followed by the choice of special vdevs (L2ARC, SLOG), and then dataset properties. For a deep dive into system-level optimization, see our [FreeBSD performance tuning guide](/blog/freebsd-performance-tuning/).
ARC Size
By default, ZFS claims up to a large portion of system RAM for the ARC. On a dedicated file server, this is exactly what you want. On a multi-purpose machine running jails, databases, or VMs, you may need to cap it.
Check current ARC usage:
# sysctl vfs.zfs.arc_summary | head -5
... (use arc_summary for full output)
# sysctl kstat.zfs.misc.arcstats.size
kstat.zfs.misc.arcstats.size: 6845210624
That is approximately 6.4GB of ARC.
Set ARC maximum to 8GB in /boot/loader.conf:
vfs.zfs.arc_max="8589934592"
Or at runtime (does not persist across reboots):
# sysctl vfs.zfs.arc_max=8589934592
Set ARC minimum (prevents ZFS from releasing too much cache under memory pressure):
vfs.zfs.arc_min="2147483648"
**Rule of thumb:** Give ZFS 1GB of ARC per TB of storage as an absolute minimum. More is always better. On a dedicated NAS, let ZFS have 80% or more of physical RAM.
L2ARC (Level 2 ARC)
L2ARC extends the read cache to a fast SSD. It is a second-tier cache -- data evicted from the ARC goes to L2ARC before being discarded entirely.
Add an L2ARC device:
# zpool add tank cache /dev/ada5
# zpool status tank
pool: tank
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
ada1 ONLINE 0 0 0
ada2 ONLINE 0 0 0
cache
ada5 ONLINE 0 0 0
errors: No known data errors
**When L2ARC helps:** Workloads with a working set larger than RAM but smaller than the L2ARC device. Random read-heavy workloads.
**When L2ARC does not help:** Sequential reads (streaming video, backups). Working sets that fit entirely in the ARC. Write-heavy workloads.
L2ARC metadata consumes ARC space. On systems with less than 64GB of RAM and large L2ARC devices, the metadata overhead can actually reduce effective cache. Size your L2ARC at roughly 5-10x your ARC size.
SLOG (Separate Intent Log)
ZFS uses a write intent log (ZIL) to guarantee synchronous write durability. By default, the ZIL resides on the pool disks. A SLOG is a dedicated, fast, power-loss-protected device (like an Optane SSD or an enterprise NVMe with capacitors) that holds the ZIL.
Add a SLOG device:
# zpool add tank log mirror /dev/nvd0 /dev/nvd1
**Always mirror your SLOG.** Losing a SLOG device can lose the last few seconds of synchronous writes.
**When SLOG helps:** NFS with sync=standard, databases with fsync, ESXi datastores, any workload that issues synchronous writes.
**When SLOG does not help:** Workloads using async writes (most local file operations). Bulk data transfers.
**Critical requirement:** Your SLOG device must have power-loss protection. A consumer NVMe without capacitors defeats the purpose entirely -- data in the SLOG's volatile write cache will be lost on power failure, which is the exact scenario the ZIL is designed to protect against.
Compression
Compression is almost always a net win on ZFS. It reduces the amount of data written to disk and read from disk, which means less I/O. On modern CPUs, LZ4 compression is effectively free:
# zfs set compression=lz4 tank
Check compression ratio:
# zfs get compressratio tank/data
NAME PROPERTY VALUE SOURCE
tank/data compressratio 1.82x -
A ratio of 1.82x means your data occupies 55% of the space it would otherwise need. This is extra usable capacity and extra performance for free.
For cold storage or archival data where CPU time is cheap:
# zfs set compression=zstd tank/data/backups
zstd achieves significantly better ratios than LZ4 at the cost of higher CPU usage.
Record Size Tuning
Default is 128K. Tune per dataset based on workload:
| Workload | Recommended recordsize |
|----------|----------------------|
| General files | 128K (default) |
| PostgreSQL | 16K |
| MySQL/InnoDB | 16K |
| MongoDB | 64K |
| VM images (bhyve) | 64K |
| Large media files | 1M |
| Backup streams | 1M |
# zfs set recordsize=16K tank/data/postgresql
Special Allocation Class
On pools with both spinning disks and SSDs, you can add SSDs as a **special** vdev for metadata and small blocks. This dramatically accelerates metadata-heavy operations (directory listings, ls -la, find):
# zpool add tank special mirror /dev/nvd2 /dev/nvd3
# zfs set special_small_blocks=32K tank
Blocks smaller than 32K will be stored on the fast special vdev. Always mirror the special vdev -- losing it without redundancy loses the pool.
Key Tunables Summary
Add these to /boot/loader.conf as needed:
# ARC size limits
vfs.zfs.arc_max="8589934592"
vfs.zfs.arc_min="2147483648"
# Prefetch tuning (disable if random I/O dominant)
vfs.zfs.prefetch_disable="0"
# Transaction group timeout (seconds) - lower means more frequent writes
vfs.zfs.txg.timeout="5"
# Scrub speed - increase for faster scrubs (default 0, unlimited)
vfs.zfs.scrub_delay="0"
# L2ARC population speed
vfs.zfs.l2arc_write_max="67108864"
vfs.zfs.l2arc_write_boost="134217728"
---
Monitoring ZFS Health
Pool Status
The most important command you will run:
# zpool status -v
pool: tank
state: ONLINE
scan: scrub repaired 0B in 03:24:18 with 0 errors on Sun Mar 28 03:24:18 2026
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
ada1 ONLINE 0 0 0
ada2 ONLINE 0 0 0
errors: No known data errors
Any non-zero value in READ, WRITE, or CKSUM columns demands investigation. CKSUM errors indicate data corruption on that device. WRITE errors indicate the device is failing to accept writes.
Pool I/O Statistics
Real-time I/O monitoring:
# zpool iostat tank 5
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
tank 1.23T 547G 245 128 30.2M 15.6M
tank 1.23T 547G 312 89 38.9M 10.8M
tank 1.23T 547G 198 156 24.7M 19.4M
Add -v for per-vdev breakdown:
# zpool iostat -v tank 5
ARC Statistics
The arc_summary tool provides a comprehensive view of ARC performance:
# arc_summary
------------------------------------------------------------------------
ZFS Subsystem Report Sat Mar 29 09:15:22 2026
------------------------------------------------------------------------
ARC Summary: (HEALTHY)
Memory Throttle Count: 0
ARC Misc:
Deleted: 142.3k
Mutex Misses: 89
Evict Skips: 12
ARC Size: 64.12% 6.41 GiB
Target Size: (Adaptive) 100.00% 10.00 GiB
Min Size (Hard Limit): 12.50% 1.25 GiB
Max Size (High Water): 8:1 10.00 GiB
ARC Size Breakdown:
Recently Used Cache Size: 50.00% 3.21 GiB
Frequently Used Cache Size: 50.00% 3.21 GiB
ARC Hash Breakdown:
Elements Max: 312.4k
Elements Current: 78.32% 244.7k
ARC Efficiency: 459.2k accesses
Cache Hit Ratio: 97.43% 447.4k
Cache Miss Ratio: 2.57% 11.8k
...
A hit ratio above 95% is good. Above 99% is excellent. Below 90% suggests your working set exceeds your ARC and you should add more RAM or an L2ARC.
Disk Health with SMART
Always monitor the physical health of your drives alongside ZFS:
# pkg install smartmontools
# smartctl -a /dev/ada1 | grep -E "Reallocated|Current_Pending|Offline_Uncorrectable"
5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0
Non-zero values on these three attributes mean the drive is developing bad sectors. Replace it proactively.
---
Common Maintenance
Scrub Schedule
A scrub reads every block in the pool and verifies its checksum. On redundant pools, it repairs any corruption it finds. Run scrubs at least monthly, weekly for critical data.
Add to cron:
# crontab -e
0 2 * * 0 /sbin/zpool scrub tank
This runs a scrub every Sunday at 2:00 AM.
Check scrub status:
# zpool status tank | grep scan
scan: scrub in progress since Sun Mar 29 02:00:01 2026
1.23T scanned at 456M/s, 987G issued at 367M/s, 1.23T total
0B repaired, 80.24% done, 00:12:34 to go
Replacing a Failed Drive
When a drive fails:
# zpool status tank
pool: tank
state: DEGRADED
status: One or more devices has been removed by the user.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
config:
NAME STATE READ WRITE CKSUM
tank DEGRADED 0 0 0
mirror-0 DEGRADED 0 0 0
ada1 ONLINE 0 0 0
ada2 UNAVAIL 0 0 0
errors: No known data errors
Physically replace the failed drive. If the new drive appears as ada2:
# zpool replace tank ada2 /dev/ada2
If the new drive has a different device name (e.g., you swapped it into a different bay):
# gpart create -s gpt ada5
# gpart add -t freebsd-zfs -l disk2 ada5
# zpool replace tank ada2 /dev/gpt/disk2
Monitor the resilver:
# zpool status tank
scan: resilver in progress since Sat Mar 29 10:15:00 2026
456G scanned at 234M/s, 123G issued at 145M/s, 1.23T total
123G resilvered, 10.00% done, 02:15:30 to go
Expanding a Pool
ZFS pools can be expanded by adding new vdevs. You cannot add individual disks to an existing vdev (with the exception of the relatively new RAIDZ expansion feature in OpenZFS 2.3+), but you can add entirely new vdevs to the pool:
# zpool add tank mirror /dev/ada5 /dev/ada6
This adds a second mirror vdev. The pool now stripes data across both mirrors, roughly doubling throughput and capacity.
**Warning:** Match your vdev types. Adding a stripe vdev to a mirrored pool means your pool is only as reliable as the stripe. If the stripe vdev dies, the entire pool is lost.
Upgrading Pool and Filesystem Features
After upgrading FreeBSD or OpenZFS, new features may be available:
# zpool upgrade
This system supports ZFS pool feature flags.
The following pools are formatted using feature flags, but do not
have all supported features enabled:
tank
The following features are supported but not enabled:
...
Upgrade when ready:
# zpool upgrade tank
# zfs upgrade -a
**Note:** Pool upgrades are one-way. Once upgraded, older versions of ZFS cannot import the pool. Ensure all systems that need to access this pool support the new features before upgrading.
---
Best Practices and Gotchas
Do
- **Use mirrors for anything you care about.** Mirrors resilver in hours instead of days, have the best read performance, and tolerate up to N-1 disk failures.
- **Enable compression everywhere.** lz4 is the minimum. zstd for cold data. The CPU overhead is negligible and you gain both capacity and performance.
- **Set atime=off on every pool.** Access time updates generate a write for every read. Disable this unless you have a specific reason to track access times.
- **Use ECC RAM.** ZFS checksums detect corruption on disk but cannot detect corruption in RAM. Non-ECC RAM can silently corrupt data in the ARC before it is written to disk. This is not ZFS-specific -- ECC RAM protects all workloads -- but ZFS users tend to care more about data integrity.
- **Create datasets, not directories.** If you would create a top-level directory, create a dataset instead. Datasets give you per-directory snapshots, quotas, compression settings, and send/receive capability.
- **Snapshot before every system change.** Boot environments for the OS, manual snapshots for data. Make it reflexive.
- **Scrub monthly at minimum.** Scrubs are the only way to detect latent corruption (bit rot) before it becomes unrecoverable.
- **Monitor with zpool status daily.** Automate this with a cron job that emails you on non-ONLINE state.
Do Not
- **Do not use RAIDZ1 with large drives (4TB+).** Resilver times on large drives can exceed 24 hours. A second failure during resilver kills the pool. Use RAIDZ2 or mirrors.
- **Do not enable deduplication unless you have measured your DDT size.** Dedup requires approximately 5GB of RAM per TB of stored data for the deduplication table (DDT). If the DDT spills out of the ARC, performance collapses catastrophically. Most workloads do not benefit from dedup. Use compression instead.
- **Do not use consumer SSDs as SLOG devices.** Without power-loss protection (PLP), the SLOG cannot guarantee write durability. Use Intel Optane, enterprise NVMe with PLP capacitors, or skip the SLOG entirely.
- **Do not mix vdev types in a pool.** A pool with one mirror vdev and one stripe vdev has the reliability of a stripe.
- **Do not use ZFS on systems with less than 8GB of RAM.** ZFS works with less, but the ARC will be too small to cache meaningfully and performance will suffer.
- **Do not put swap on ZFS.** Use a separate partition or GEOM device for swap. ZFS and swap interact poorly under memory pressure -- ZFS needs RAM to free RAM, creating deadlock potential.
- **Do not resize or repartition disks in an active vdev.** Always replace the entire disk using zpool replace.
---
FAQ
How much RAM does ZFS actually need?
The common claim is "1GB per TB." This is a rough minimum. ZFS itself needs at least 1-2GB for basic operation. The ARC (read cache) is what scales with data size. For a 20TB pool, 16GB of RAM is comfortable. 32GB or more is ideal. On a dedicated NAS, give ZFS as much RAM as you can afford -- the ARC turns spinning disks into what feels like SSD-class read performance.
Can I add a single disk to an existing RAIDZ vdev?
Historically, no. RAIDZ vdevs were fixed-width at creation. However, OpenZFS 2.3 introduced RAIDZ expansion, which allows adding one disk at a time to an existing RAIDZ vdev. FreeBSD 14.2+ with OpenZFS 2.3 supports this:
# zpool attach tank raidz1-0 /dev/ada5
This process rewrites all data in the vdev to redistribute across the new width. It is slow but it works. Check your OpenZFS version with zpool --version before relying on this feature.
Should I use RAIDZ1, RAIDZ2, or mirrors?
For fewer than 6 disks: mirrors. For 6-12 disks: RAIDZ2. For 12+ disks: RAIDZ3 or multiple mirror vdevs. RAIDZ1 is appropriate only for small drives (under 2TB) where resilver completes quickly. The cost of a second parity disk is trivial compared to the cost of data loss.
Is ZFS native encryption production-ready on FreeBSD?
Yes. OpenZFS native encryption (added in 0.8.0) is stable and production-ready. It encrypts data at the dataset level before it reaches disk, which means ZFS send/receive can replicate encrypted datasets to untrusted remote hosts without exposing plaintext:
# zfs create -o encryption=aes-256-gcm -o keyformat=passphrase tank/encrypted
Enter new passphrase:
Re-enter new passphrase:
Load the key at boot by adding to /etc/rc.conf:
zfs_keys="YES"
How do I check if my pool is healthy?
Run zpool status. If the state is ONLINE and all error counts are zero, the pool is healthy. Any state other than ONLINE (DEGRADED, FAULTED, UNAVAIL) requires immediate attention. Also check the last scrub date -- if it has been more than 30 days, run zpool scrub tank now.
Can I convert an existing UFS root to ZFS?
Not in-place. The cleanest path is to reinstall FreeBSD with ZFS root (the installer supports this directly) and restore your data. Alternatively, you can migrate a running system by creating a ZFS pool on separate disks, copying the root filesystem, installing the bootcode, and adjusting the bootloader. This is complex and error-prone -- a fresh install is almost always faster.
What is the difference between ZFS send and rsync?
zfs send operates at the block level, not the file level. It serializes the exact block-level difference between two snapshots. This makes it faster than rsync for most replication tasks because it does not need to scan directory trees to find changes. It also preserves all ZFS properties, snapshots, and metadata. Use zfs send for ZFS-to-ZFS replication. Use rsync when the destination is not ZFS or when you need file-level filtering.
---
Conclusion
ZFS on FreeBSD is a mature, battle-tested storage stack. With first-class kernel integration, boot environments, native encryption, and the full OpenZFS feature set, it provides capabilities that no other general-purpose operating system matches out of the box.
Start with a mirror pool, enable LZ4 compression, set atime=off, create datasets for each logical data group, snapshot regularly, replicate offsite with zfs send, and scrub monthly. This baseline covers 90% of storage use cases and protects your data against the failures that will inevitably occur.
For a complete server build around ZFS, see our [FreeBSD NAS build guide](/blog/freebsd-nas-build/). For system-level performance optimization beyond ZFS, see [FreeBSD performance tuning](/blog/freebsd-performance-tuning/).