DEV Community 👩‍💻👨‍💻

Roy Keene
Roy Keene

Posted on • Originally published at rkeene.org

BtrFS, ZFS, and More

Butter Filesystem. Hold the toast.

I've started experimenting with BtrFS which aims to provide an "advanced and modern filesystem" (heavily compared to ZFS) on Linux. With my new workstation I've started using BtrFS for my home directories (/home) and my build directories (/mnt/slackbuilds) to gain exposure to the filesystem and compare it to ZFS and EXT4 on LVM (all of my other data, including my root disk is on EXT4 on LVM).

I have used ZFS heavily in the past, and using BtrFS is significantly different as many of the fundamental concepts are different. BtrFS has no concept of "pools" or "volume groups" -- instead there are "volumes." BtrFS has no concept of "datasets" or "logical volumes" -- instead there are "subvolumes".

Here's a comparison between ZFS, BtrFS, and EXT4 on LVM:

Comparison

ZFS BtrFS EXT4 and LVM
Commands Involved zpool, zfs mkfs.btrfs, btrfs pvcreate, vgcreate, lvcreate, mkfs.ext4
Pool of disks "zpool" "volume" "volume group"
Mountable unit "dataset" "volume" and "subvolume" "logical volume"
License CDDL GPL GPL
Can be Boot filesystem Yes Yes (grub 2.00) No
Can be Root filesystem Yes Yes Yes
Can provide swapspace Yes (zvols) No Yes (lvm)
OSes with Implementations Solaris, OpenSolaris, Nexenta, FreeBSD, Mac OS X, Linux Linux Linux
Stability Stable Stable Stable
CLI-System Integration [1] Strong Weak Mild
Grow Online Yes Yes Only when there are no snapshots
Shrink Pool No Online Online
Shrink Filesystem Online (reduce quota) Online Offline
Replace Disk (without parity) Yes (must be compatible size disk) Yes Yes (copies space allocated on disk)
Filesystem Level Storage Pooling Yes Yes No
Re-balance No Yes Can be done manually (pvmove)
Checksumming Yes Yes No
Autocorrect Checksum Errors Yes ??? No
Compression Yes Yes No
De-duplication Yes (only synchronous) Yes (only asynchronous) No
Ditto Blocks Yes ??? No
Tiered Caching Yes No No
Writable Snapshots Yes (clone) Yes Yes
Copy-on-Write Fast, space-efficient Fast, space-efficient Slow, requires pre-allocating an LV
Redundancy Mirroring and Parity (x1, x2, x3) Mirroring Mirroring, though the PVs can be RAID devices
Maximum Volume Size 16 Exabytes 16 Exabytes 1 Exabyte
Maximum File Size 16 Exabytes 16 Exabytes 16 Terabytes
Maximum Number of Snapshots Unlimited Unlimited Effectively 32

[1] For lack of a better term -- how well the command line interface integrates with the system as a whole, this might be subjective.

For a more complete, but less focused comparison see Wikipedia's Comparison of Filesystems

The Rosetta Stone

  1. Task: Create a pool of storage from disks /dev/A, /dev/B, and /dev/C (striped or linear concat)
    1. Using ZFS:
      1. # zpool create TESTPOOL A B C
    2. Using BtrFS:
      1. # mkfs.btrfs -d raid0 /dev/A /dev/B /dev/C
    3. Using EXT4 on LVM:
      1. # pvcreate /dev/A /dev/B /dev/C
      2. # vgcreate TESTPOOL /dev/A /dev/B /dev/C
  2. Task: Make storage from pool available to system
    1. Using ZFS:
      1. # zfs set mountpoint=/data TESTPOOL
    2. Using BtrFS:
      1. # mkdir /data
      2. # mount -t btrfs /dev/A /data
      3. Update /etc/fstab
    3. Using EXT4 on LVM:
      1. # mkdir /data
      2. # lvcreate -L _SizeOfVolume_ -n DATA TESTPOOL
      3. # mkfs -t ext4 /dev/TESTPOOL/DATA
      4. # mount /dev/TESTPOOL/DATA /data
      5. Update /etc/fstab
  3. Task: Add an additional disk to the pool
    1. Using ZFS:
      1. # zfs add TESTPOOL D
    2. Using BtrFS:
      1. # btrfs device add /dev/D /data
      2. # btrfs filesystem balance /data
    3. Using EXT4 on LVM:
      1. # pvcreate /dev/D
      2. # vgextend TESTPOOL /dev/D
  4. Task: Add additional space to a filesystem
    1. Using ZFS:
      1. No action needed
    2. Using BtrFS:
      1. No action needed
    3. Using EXT4 on LVM:
      1. # lvextend -L _SizeToIncreaseTo_ /dev/TESTPOOL/DATA
      2. # resize2fs /dev/TESTPOOL/DATA
  5. Task: Remove a disk from the pool
    1. Using ZFS: N/A
    2. Using BtrFS:
      1. # btrfs device delete /dev/A /data
      2. # btrfs filesystem balance /data
    3. Using EXT4 on LVM:
      1. # pvmove /dev/A
      2. # vgreduce TESTPOOL /dev/A
  6. Task: Replace operational disk
    1. Using ZFS:
      1. # zfs replace TESTPOOL A D
    2. Using BtrFS:
      1. # btrfs device add /dev/D /data
      2. # btrfs device delete /dev/A /data
      3. # btrfs filesystem balance /data
    3. Using EXT4 on LVM:
      1. # pvcreate /dev/D
      2. # vgextend TESTPOOl /dev/D
      3. # pvmove TESTPOOL /dev/A /dev/D
      4. # vgreduce TESTPOOL /dev/A
  7. Task: Take a snapshot of a filesystem
    1. Using ZFS:
      1. # zfs snapshot TESTPOOL@snapshot1
    2. Using BtrFS:
      1. # btrfs subvolume snapshot /data /data/snapshot1
    3. Using EXT4 on LVM:
      1. # lvcreate -s /dev/TESTPOOL/DATA -L _SizeToAllocate_ -n snapshot1
  8. Task: Rollback a snapshot
    1. Using ZFS:
      1. # zfs rollback TESTPOOL@snapshot1
    2. Using BtrFS:
      1. Not sure...
    3. Using EXT4 on LVM:
      1. # lvconvert --merge /dev/TESTPOOL/snapshot1
  9. Task: Delete a snapshot
    1. Using ZFS:
      1. # zfs destroy TESTPOOL@snapshot1
    2. Using BtrFS:
      1. # btrfs subvolume delete /data/snapshot1
    3. Using EXT4 on LVM:
      1. # lvremove /dev/TESTPOOL/snapshot1

Top comments (4)

Collapse
 
jebotz profile image
Jürgen Botz

For Task 8: Rollback a snapshot, you say "not sure" for btrfs. The answer is that snapshots are completely equivalent to subvolumes, so you just start using (and mounting, if appropriate) the snapshot instead of the "original" subvolume (and delete that one if you want).

Collapse
 
jebotz profile image
Jürgen Botz

Oh, and you can put a swapfile on a btrfs filesystem nowadays, as well as any kind of other file that is meant to serve as storage for a block device (such as VM images, etc.). The key is to set the NoCOW attribute on the subvolume or directory those files live in; that done, today's btrfs is no worse for this usecase than any other filesystem.

Collapse
 
jebotz profile image
Jürgen Botz • Edited on

I'm really curious what you mean by CLI integration... the footnote doesn't help. Please explain your "subjective" understanding of this.

Collapse
 
petteyg profile image
Gordon

GRUB2 has been able to boot to LVM for a long time.

Join us at DEV
Yes, this is technically an “ad”, but really we just want to ask if you want to join DEV. We have 900k+ developers reading, posting, and enjoying community, and would love to have you.   Create an account and continue your coding journey.