Root on ZFS with partitions

It is important to have a clean separation of concerns, both in your code, as in your machine’s setup. If you are looking for information on how to get a ZFS FreeBSD setup on one hard-drive with multiple partitions, this is what you want.

This guide assumes you want to install FreeBSD and manually create the ZFS pool, to change default settings, or to not use the entire disk.

If you just want to setup ZFS on the entire disk, use the ’ZFS’ option in the bsdinstall partitioning menu.

  1. Boot FreeBSD install DVD or USB Memstick
  2. Select Install, and answer questions such as keyboard layout and hostname
  3. When prompted to partition the disk, choose the shell option. In this mode you are expected to create a root file system mounted under /mnt that the installer will install the operating system into.

Now to the installation steps, start by determining which disks you wish to use:

camcontrol devlist
# or alternatively
geom disk list

That should return something like TOSHIBA HDWN180 GX2M at scbus7 target 0 lun 0 (ada0,pass1)

SATA disks usually start with ada, followed by a number. SAS and USB disks start with da, followed by a number. The rest of this page assumes the disk is ada0, replace it with your disk devices name in all commands below.

Proceed to create a fresh partition table, erasing ALL data on the disk!:

# The first command is not needed if you are working on an empty new disk
gpart destroy ada0
gpart create -s gpt ada0

Then create the boot code partition:

for Legacy Boot:

# Confirmed working on FreeBSD 13
gpart add -a 4k -s 512K -t freebsd-boot ada0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0

for UEFI Boot we will need more space and some more manual steps:

# Confirmed working on FreeBSD 13
gpart add -t efi -s 40M ada0
newfs_msdos -F 32 -c 1 /dev/ada0p1
mount -t msdosfs /dev/ada0p1 /mnt
mkdir -p /mnt/EFI/BOOT
cp /boot/loader.efi /mnt/EFI/BOOT/BOOTX64.efi
umount /mnt

We will now be creating the needed partitions to setup the system as we wish. I will be working with a 20GB hard drive, which I want to split in 3 partitions. 2GB for Swap, 10 GB for the FreeBSD installation and the remaining (8GB) for the Jails storage, in a secondary pool which will later be created.

Note that while a ZFS Swap Volume can be used instead of the freebsd-swap partition, it is not recommended, and crash dumps can’t be saved to the ZFS Swap Volume.

Some of the gpart options explained: * -a {number}: controls alignment. * -s {size}: sets the size of the partition, if not set, it will default to the remaining space on the disk. * -l {name}: sets the label for the partition

gpart add -a 1m -s 4G -t freebsd-swap -l swap0 ada0
gpart add -a 1m -s 10G -t freebsd-zfs -l disk0 ada0
gpart add -a 1m -t freebsd-zfs -l disk1 ada0

Find the device for the freebsd-zfs partition created above with gpart show -p ada0

=>      40  41942960  ada0  GPT  (20G)
        40      1024    1  freebsd-boot  (512K)
        1064       984       - free -  (492K)
        2048   4194304    2  freebsd-swap  (2.0G)
        4196352  20971520    3  freebsd-zfs  (10G)
        25167872  16773120    4  freebsd-zfs  (8.0G)
        41940992      2008       - free -  (1.0M)

Our FreeBSD installation will be on ada0p3.

For proceeding with the install we will mount a tmpfs filesystem to /mnt with:

mount -t tmpfs tmpfs /mnt

We will assume zroot as the name of the ystem’s zpool, it could be anything (i.e. tank, data, mypool), and to create it we do:

zpool create -o altroot=/mnt zroot ada0p3

We will use the lz4 compression algorithm, since it is fast enough to be used by default, even for uncompressible data. It can be enabled with:

zfs set compress=on zroot

We will then create a Boot Environment hierarchy with:

zfs create -o mountpoint=none zroot/ROOT
zfs create -o mountpoint=/ -o canmount=noauto zroot/ROOT/default
mount -t zfs zroot/ROOT/default /mnt

and then proceed to create the rest of filesystems required for FreeBSD to be installed to this ZFS Pool:

zfs create -o mountpoint=/tmp -o exec=on -o setuid=off zroot/tmp
zfs create -o canmount=off -o mountpoint=/usr zroot/usr
zfs create zroot/usr/home
zfs create -o exec=off -o setuid=off zroot/usr/src
zfs create zroot/usr/obj
zfs create -o mountpoint=/usr/ports -o setuid=off zroot/usr/ports
zfs create -o exec=off -o setuid=off zroot/usr/ports/distfiles
zfs create -o exec=off -o setuid=off zroot/usr/ports/packages
zfs create -o canmount=off -o mountpoint=/var zroot/var
zfs create -o exec=off -o setuid=off zroot/var/audit
zfs create -o exec=off -o setuid=off zroot/var/crash
zfs create -o exec=off -o setuid=off zroot/var/log
zfs create -o atime=on -o exec=off -o setuid=off zroot/var/mail
zfs create -o exec=on -o setuid=off zroot/var/tmp

then we will set a symlink to home, and set the correct permissions with:

ln -s /usr/home /mnt/home
chmod 1777 /mnt/var/tmp
chmod 1777 /mnt/tmp

We can then finalize the zpool creation, by setting the boot environment configuration:

zpool set bootfs=zroot/ROOT/default zroot

To finish the installation, we will create a new entry in the fstab of the installer, which later will be stored in our system. Now is your opportunity to attach other devices if needed as well, but for this tutorial, simply adding the swap is enough:

cat << EOF > /tmp/bsdinstall_etc/fstab
# Device Mountpoint FStype Options Dump Pass#
/dev/gpt/swap0 none swap sw 0 0
EOF

We can then exit the shell Partitioning mode, at which point bsdinstall will continue and complete the installation.

In the post installation step, when asked if you would like to drop into the installed system, choose yes, and execute the following:

sysrc zfs_enable="YES"
echo zfs_load="YES" >> /boot/loader.conf

You are now ready to reboot into your newly installed FreeBSD system. Simply type exit at the current prompt, and eject the installation media, when the machine restarts.

Welcome to the magical world of FreeBSD!