fstrim Said 2 GiB Again, and Nothing Was Freed

This line was sitting in my blog server's weekly journal:

fstrim[1000031]: /: 159.6 GiB (171393343488 bytes) trimmed on /dev/sda1

The disk is 678 GB, half full, 330 GB free. The timer runs every Monday morning and writes that line again. The question that came to mind was simple: was 159.6 GiB really freed, or does that number not count what I think it counts?

It's the second one. More than that: the same line means three different things on ext4, on XFS, and on the FAT filesystem at /boot/efi — and in only one of the three does the number actually tell you something. This article is the record of chasing that difference down: measurements from a lab I built on a loop device, the reasoning I read in the kernel source, and finally a piece of duplicated work I found in my own fleet that I did not enjoy discovering.

rm tells nobody

To see the chain I built the most honest rig I could measure: a sparse file, a loop device on top of it, and ext4 on top of that. The beauty of a sparse file is that du reports exactly how many blocks are really held underneath. Whatever df says upstairs, I know beyond argument what is happening downstairs.

Ubuntu 26.04, kernel 7.0.0-31, util-linux 2.41.3. I created a 2 GiB image, ran mkfs.ext4, mounted it, wrote 400 MiB of random data, then deleted it:

Step	df (used)	Real size of the image file
empty sparse file	—	0
after mkfs.ext4	—	66M
400 MiB written	401M	466M
rm + sync	536K	466M
after fstrim	536K	66M

The fourth row is this entire article. The filesystem says "I'm practically empty," while the storage underneath still holds 466 MiB. rm did nothing beyond clearing bits in the block bitmap; it said not a word to the block layer below it, let alone to the physical device. Nor does it have to — POSIX never asked it to.

As for who does the telling, every layer along the chain is free to swallow the message:

The two paths at the top of that chain — the mount option and fstrim — arrive at the same place but run at completely different rhythms. And all of the confusion lives in what fstrim reports back.

What does the number count?

I ran fstrim three times in a row on the same filesystem:

$ fstrim -v /mnt          # 1st run
/mnt: 1.9 GiB (2039824384 bytes) trimmed
$ fstrim -v /mnt          # 2nd
/mnt: 0 B (0 bytes) trimmed
$ fstrim -v /mnt          # 3rd
/mnt: 0 B (0 bytes) trimmed

Even the first number is odd: not the 400 MiB I deleted, but 1.9 GiB. Because fstrim doesn't care about the blocks you freed — it pushes the filesystem's entire current free space downwards. To measure that properly I repeated the test without deleting the 400 MiB file: df said 401M used, 1.4G available; fstrim reported 1,620,393,984 bytes. The extra ~100 MiB is ext4's 5% reserve for root — space df leaves out of its "available" column but which is perfectly free as far as the device is concerned. So on ext4 the number is the sum of the free space that was walked.

The util-linux man page says so outright. Its definition of --verbose is: the number of bytes passed from the filesystem down the block stack to the device for potential discard — and it goes on to call that number a maximum from the storage device's perspective, because a FITRIM ioctl called repeatedly will keep sending the same sectors for discard over and over. The warning that follows explains why most people see the same number forever: fstrim will report the same potential discard bytes each time, but only sectors written to between the discards would actually be discarded.

So why did I get 0? Because ext4 behaves differently here from most filesystems.

ext4's one-bit memory

ext4_trim_all_free in fs/ext4/mballoc.c asks a single question before touching a block group:

if (!EXT4_MB_GRP_WAS_TRIMMED(e4b.bd_info) ||
    minblocks < EXT4_SB(sb)->s_last_trim_minblks)
    ret = ext4_try_to_trim_range(sb, &e4b, start, max, minblocks);
else
    ret = 0;

EXT4_GROUP_INFO_WAS_TRIMMED_BIT is bit 1 of the bb_state field in the group info struct. It is set once the whole group has been trimmed, and cleared when blocks are freed in that group — the comment in the source states the reason without ceremony: clear the trimmed flag so that the next ext4_trim_fs can trim it. In other words ext4 says "I already told them about this region, I won't repeat myself." It skips the pointless work and reports 0.

But this memory lives only in RAM. bb_state is not stored in any on-disk structure. I tested it:

1st fstrim after a fresh mount : 1.9 GiB (2039824384 bytes)
immediately after, 2nd fstrim  : 0 B
umount + mount, then fstrim    : 1.9 GiB (2039824384 bytes)

A single umount wipes the entire memory. After every reboot, the first fstrim narrates everything to the device from scratch.

There's also a subtle rule the source promises: even with the flag set, the group is walked again if a smaller minlen is requested (minblocks < s_last_trim_minblks). Measuring that was fun. On a freshly mounted ext4 holding 400 MiB, in order:

Command	Reported
fstrim -m 64M	1,605,722,112
fstrim -m 64M (again)	0
fstrim -m 4M (smaller)	1,620,393,984
fstrim -m 4M (again)	0
fstrim -m 64M (large again)	0
fstrim -m 1K	1,620,393,984

The ratchet only turns downwards: lower the threshold and the door opens again, raise it and it stays shut. The last row is a nice detail too — 1 KB is below the device's 4 KB discard granularity, and in that case ext4_trim_fs quietly raises your value to the granularity ("No point to try to trim less than discard granularity"). Because the previous line returned 0 without an error, s_last_trim_minblks had been set to 64 MiB (the source asks "was there an error", not "did it trim anything"); 4 KB is smaller than that, so the ratchet drops again.

The 14 MiB gap between the first two numbers carries information as well: free holes smaller than 64 MiB but larger than 4 MiB. Raising -m makes the job finish faster but leaves fragmented free space behind.

To see that properly I deliberately fragmented a filesystem: 1500 files of 1 MiB each, then deleted every other one. e2freefrag shows the shape of the free space precisely — 750 extents in the 1–2 MiB range (63% of the free blocks), one in the 64–128 MiB range and one in the 256–512 MiB range. Remounting before every measurement, so no flag carries over:

Command	Reported
fstrim -m 1M	1.2 GiB
fstrim -m 4M	440.3 MiB
fstrim -m 16M	440.3 MiB
fstrim (no threshold)	1.2 GiB

440.3 MiB is exactly the sum of the two large extents in the histogram (24,145 + 88,576 blocks). The 750 one-megabyte holes drop out entirely the moment the threshold reaches 4 MiB — so raising -m on a backup server "to make the run faster" can quietly cost you two thirds of the space you were going to reclaim.

On XFS the number is your own request

I built the same rig with XFS. The result made me laugh:

xfs fstrim #1      : 2 GiB (2147483648 bytes) trimmed
xfs fstrim #2      : 2 GiB (2147483648 bytes) trimmed
xfs fstrim -m 64M  : 2 GiB (2147483648 bytes) trimmed

2,147,483,648 bytes — exactly the size of the device. The same number with 471 MiB of data on the filesystem. The same number when -m 64M is given. The end of fs/xfs/xfs_discard.c says why:

range.len = min_t(unsigned long long, range.len,
          XFS_FSB_TO_B(mp, max_blocks) - range.start);

XFS never reports what it trimmed; it takes the range you asked for, clips it to the filesystem size recorded in the superblock, and hands it back (on a loop device the two are equal, which is why the number matches the device size). The comment just above it sums up the attitude: trimming blocks is an advisory interface anyway. True. But in the meantime the output of fstrim -v on XFS stops being a measurement altogether.

I found the third behaviour out in the field. The /boot/efi partition on vps5 is FAT, and the line doesn't change across a month of journal:

Aug 19: /boot/efi: 98.1 MiB (102828032 bytes) trimmed on /dev/sda15
Aug 24: /boot/efi: 98.1 MiB (102828032 bytes) trimmed on /dev/sda15
Aug 31: /boot/efi: 98.1 MiB (102828032 bytes) trimmed on /dev/sda15
Sep 07: /boot/efi: 98.1 MiB (102828032 bytes) trimmed on /dev/sda15
Sep 14: /boot/efi: 98.1 MiB (102828032 bytes) trimmed on /dev/sda15
Sep 21: /boot/efi: 98.1 MiB (102828032 bytes) trimmed on /dev/sda15

Six runs, identical to the byte — eight counting the two I ran by hand today. FAT counts its free space honestly but keeps no "already told them" flag like ext4's; and since almost nothing changes on the ESP from week to week, it narrates the same 98.1 MiB every time.

Don't miss that the first line falls on a Wednesday: the machine was rebooted that morning, and Persistent=true caught up the missed run at boot. The other two lines of that run are exactly what I'd expect: 903.6 MiB for /boot and 93.7 GiB for the root. The root's six-run series reads: 93.7 · 56.6 · 7.5 · 7.4 · 42.9 · 5.8 GiB. The largest is the post-boot run, the one where every flag was clear.

Looking at the same machine's ext4 /boot shows the flag's real-world lifetime: 827.2 MiB on 24 August, 0 B on 31 August, 206.1 MiB on 7 September, 833.1 MiB on 14 September, 129.4 MiB on 21 September. The zero week is the week nothing was freed on /boot. The bit in memory stayed put for seven days, because the partition was never remounted. The root series peaking on the post-boot run is the same bit seen from the other side: a reboot wipes every flag, and the first run narrates everything from scratch.

The layer that cuts the chain: LUKS

So far I've been describing it as "the filesystem talks, the device listens." Put a layer in between and the story changes.

I set up a LUKS2 encrypted volume (cryptsetup 2.8.4) and opened it twice. The only difference is one flag:

	discard_granularity	discard_max_bytes	fstrim	Real usage after deleting 400 MiB
cryptsetup open	0	0	the discard operation is not supported	450M
cryptsetup open --allow-discards	4K	4G	reported 1.9 GiB	50M

The kernel's ABI documentation is unambiguous about discard_granularity: a value of 0 means the device does not support discard functionality. But that isn't actually the field holding the door — the code that receives FITRIM checks bdev_max_discard_sectors(), i.e. whether discard_max_bytes is zero, and returns EOPNOTSUPP before ever reaching the filesystem (fs/ext4/ioctl.c, and the same line in XFS). dm-crypt zeroes both, so in practice the distinction is invisible; but the column to read when diagnosing is DISC-MAX. In the second case dmsetup table shows an extra 1 allow_discards field; that's the whole difference.

The baselines in the table come from this: of the 50M, 16 MiB is the LUKS2 header and the rest is ext4's own metadata. Compare within the rig — 450M against 50M.

cryptsetup chose this default on purpose. Ask its man page why and it speaks plainly: the option can have a negative security impact because it makes filesystem-level operations visible on the physical device; information leaking filesystem type and used space may be extractable if the discarded blocks can be located later. In its own words: if in doubt, do not use it.

So on an encrypted disk the choice is a genuine trade-off. In my view it depends on where the device physically sits: on a server in a data centre whose disposal is covered by contract, I consider it reasonable to turn --allow-discards on and rescue both the SSD's endurance and the thin pool's utilisation. On a laptop that can be stolen, or a cloud disk I will hand back, I leave it off — because there the threat model is precisely "whoever gets the disk reads the pattern of free space and learns how much data there was and which filesystem it held." One persistence detail too: pass the flag with --persistent and it is written into the LUKS2 header, so you don't have to add it by hand on every open. To remove it, you run --persistent again, this time without --allow-discards.

ext4 stays quiet, btrfs speaks for itself

fstrim isn't the only route. The filesystem can also report at deletion time; on ext4 that's the -o discard mount option. I measured it: on a filesystem mounted with discard, deleting the 400 MiB file and running sync was enough — the image file dropped from 466M to 66M without fstrim being called at all.

btrfs has been making this decision for you since 6.2. A btrfs volume I mounted with no options at all had discard=async among its effective options — as the documentation says, since kernel 6.2 it is enabled automatically on devices that support it. The interesting part is the timing. I deleted the 400 MiB file and watched second by second:

After deletion	Real size of the image	discardable_bytes
5 s	402M	654,311,424
15 s	1.7M	0
60 s	2.0M	0

I had taken my first reading at the 3-second mark and was about to write "async discard isn't working." Good thing I waited. The delay isn't accidental either: fs/btrfs/discard.c holds a fully emptied block group for BTRFS_DISCARD_UNUSED_DELAY — 10 seconds — before queueing it, and the comment in the source gives the reason: to give the blocks some chance of being reused. For groups still in use the delay is 120 seconds. Seeing nothing at 5 seconds and everything finished by 15 is exactly that. The send rate is capped too: on my system iops_limit was 1000 and max_discard_size 64 MiB. (The 2.0M at 60 seconds is btrfs metadata written in between.) That design is the answer to what the documentation says about synchronous trim: it could have a severe performance hit, it is not recommended, and the ranges to be trimmed could be too fragmented.

So there are three distinct policies: ext4 stays quiet by default and relies on the weekly fstrim, btrfs speaks on its own and in moderation, and -o discard speaks immediately on every deletion.

Once a week — but to exactly where?

The route distributions chose is the weekly timer. The unit shipped in Ubuntu 26.04 is identical to what util-linux ships upstream:

[Timer]
OnCalendar=weekly
AccuracySec=1h
Persistent=true
RandomizedDelaySec=100min

systemd expands the weekly shorthand to Mon *-*-* 00:00:00. Monday midnight, plus up to 100 minutes of random delay. On 21 September 2026 all seven servers in my fleet ran between 00:28 and 01:22, all inside the window. (AccuracySec=1h can stretch that window by another hour, putting the real ceiling at 02:40.) That randomness exists to stop dozens of virtual machines sharing one physical pool from starting a discard storm in the same second, and to my eye it is the most elegant line in the unit.

The unit's ExecStart, on the other hand, hides a trap:

ExecStart=/sbin/fstrim --listed-in /etc/fstab:/proc/self/mountinfo --verbose --quiet-unsupported

At first glance it reads as "both fstab and all mounts." It isn't. The man page says this about --listed-in: evaluation of the list stops after the first non-empty file. Since /etc/fstab is non-empty on every normal system, /proc/self/mountinfo is never read.

I measured the consequence. I mounted an ext4 over a loop device at /mnt/trimtest, added no fstab entry, wrote and deleted 300 MiB, then ran the unit's command verbatim:

/boot/efi: 98.1 MiB (102828032 bytes) trimmed on /dev/sda15
/boot: 0 B (0 bytes) trimmed on /dev/sda13
/: 626.9 MiB (657399808 bytes) trimmed on /dev/sda1

/mnt/trimtest isn't in the list, and its image file stayed at 333 MiB. Targeting it by hand brought it down to 33 MiB. A data disk that isn't in fstab — attached by a systemd .mount unit, mounted by hand, or added later by automation — gets no benefit from the weekly run at all. The reverse is possible too: put X-fstrim.notrim on an fstab line and fstrim skips that filesystem. And the unit carries ConditionVirtualization=!container; inside a container this never runs, it's the host's job.

The duplicated work I found in my own fleet

Once I understood the mechanism I went back to all seven servers with the same question: how is the root mounted, is the timer on, and what did it report last week?

Server	Ubuntu	Root mount options	discard_granularity	Reported on 21 Sep
vps1	25.04	relatime,discard	4096	16.4 GiB
vps2	25.04	relatime,discard	4096	10.3 GiB
vps3	24.04	relatime,discard	4096	159.6 GiB
vps4	24.04	relatime	512	77.8 GiB
vps5	26.04	relatime,discard	4096	5.8 GiB
vps6	26.04	relatime,discard	4096	8.6 GiB
vps7	26.04	relatime,discard,quota	4096	4.4 GiB

Six of the seven mount the root with discard and also run the weekly fstrim. I didn't write that; it comes from the provider's image. So these servers send a discard to the device on every deletion, and then on Monday morning tell the same device "by the way, there's another 159 GiB free over here." On the blog server that work burns 13 seconds of CPU. Not a disaster — but entirely redundant, and I had no idea it was there until I looked at that line.

vps4, the sole exception, makes an interesting control: no discard, granularity 512 bytes, and the weekly run reports 77.8 GiB. There the timer really is the only trim path.

The meaning of that 159.6 GiB in this table is clear now too: it is about half of the 330 GiB of free space. During that week at least one block was freed in roughly half of vps3's block groups, clearing their WAS_TRIMMED bit, and those regions got walked again. The machine hasn't been rebooted since 10 September, so the number reflects genuine churn — expected on a build server. But it does not say "159.6 GiB was reclaimed."

Checklist

If you're going to look at your own systems, this is the order I'd use:

1. Run lsblk -D and look at the DISC-MAX column. If it is 0 the chain is broken; the cause is usually LUKS, an old virtual disk controller, or a mapping layer in between.
2. Check whether the root got both discard and the timer (findmnt -no OPTIONS / and systemctl is-enabled fstrim.timer). You don't need both. I prefer the distribution's own default, the weekly timer — the kernel documentation still keeps discard off by default on ext4 and gives its reason as "off by default until sufficient testing has been done." If you go the other way, add X-fstrim.notrim to the fstab line and turn the weekly run off for that filesystem.
3. List the volumes that aren't in fstab. The weekly unit can't see them; write your own timer or put them in fstab.
4. On encrypted volumes, decide by threat model, then make it stick with --persistent — adding the flag by hand on every open is something you will forget.
5. Verify the timer isn't failing silently. The --quiet-unsupported in the unit doesn't just suppress the warning; in the man page's words it also cleans the exit status. So when LUKS or an intervening layer cuts the chain, fstrim.service keeps looking green. Once a month, check journalctl -u fstrim.service and confirm the mount points you expect are there and not permanently reporting 0 B.
6. Trust the number even less under RAID or LVM. The man page says this too: the block layer reserves the right to adjust discard ranges to fit raid stripe geometry or non-trim-capable devices in an LVM setup, and those reductions never show up in the report. If you run LVM, review the issue_discards setting separately.
7. Don't read the real gain from fstrim's output. On a VM look at the hypervisor's thin disk usage, in a home lab at the image file's du, on an SSD at the vendor's tools.

Closing thought

The line fstrim -v prints is not a result report, it's a transcript of a conversation. It tells you what the filesystem said to the device; nobody tells you what the device did, because nothing reports that part back. ext4 keeps this conversation in memory and refuses to repeat itself, XFS echoes the question back at you, FAT starts over every week, and LUKS steps in and hangs up the phone.

This pattern is common in the storage stack: every layer tells the one above it its own truth, and none of them is lying. I ran into the same thing while tracing read_ahead_kb's jump to 8192 back to what the disk claimed about itself, and again when Docker ate 56 GB in a day. The problem isn't that layers report wrong values; the confusion comes from us treating answers from two different layers as answers to the same question.

So before I put a number on a dashboard now, I ask two things: which question does the code producing this value actually answer, and which question do I think I asked it? The 159.6 GiB in this article is a good example of what happens when those two don't line up.

Official Sources

• util-linux — fstrim(8) man page source
• Linux kernel — discard sysfs interface for block devices (ABI/stable/sysfs-block)
• Linux kernel source — fs/ext4/mballoc.c
• Linux kernel source — fs/xfs/xfs_discard.c
• cryptsetup — documentation and security warning for --allow-discards
• systemd — systemd.time(7) calendar shorthands
• Linux kernel — ext4 mount options (admin-guide/ext4.rst)
• Linux kernel source — fs/btrfs/discard.c (delay and IOPS constants)
• util-linux — the fstrim.timer and fstrim.service units