DEV Community

Cover image for Stop Trusting rotate 14 to Bound Your Disk: Practical logrotate Semantics on Linux
Lyra
Lyra

Posted on

Stop Trusting rotate 14 to Bound Your Disk: Practical logrotate Semantics on Linux

A service starts writing 2 GB of logs a day instead of 200 MB. The logrotate rule has not changed in a year. It still says rotate 14, so it must be fine.

It is not fine. rotate 14 is a count of files, not a disk budget. Fourteen files at 2 GB each is 28 GB. Nothing in that directive caps how large a single file grows before rotation, how often the rotator runs, or what happens when the writer ignores the rotation entirely.

Most logrotate incidents I have seen are not broken syntax. They are correct configs that mean something different from what the operator assumed. Here is the mental model, the three size directives that are easy to confuse, and a way to test a rule before it matters.

The number that looks like a quota

rotate N means: keep N old versions, then remove the oldest. That is all.

  • rotate 0 removes old versions rather than keeping them.
  • rotate -1 never removes old logs, except through maxage.
  • rotate 14 with a 5 GB file still keeps up to 70 GB of rotated data.

The actual disk ceiling is roughly rotate × size at rotation time, plus whatever the live file has grown to since the last run. If you want a bound, you have to bound the file size and make sure rotation runs often enough to enforce it.

Two decision layers: the timer and the rules

logrotate does not sit in the background watching files. Something invokes it — traditionally a daily cron job, on systemd systems usually logrotate.timer. Then logrotate reads its config and decides, per file, whether this invocation is a rotation.

Two consequences follow.

First, daily does not mean midnight. It means the first invocation of the day that considers the file. The upstream logrotate.timer ships with:

[Timer]
OnCalendar=daily
RandomizedDelaySec=1h
Persistent=true
Enter fullscreen mode Exit fullscreen mode

So on a stock setup the run lands somewhere in a one-hour window after midnight, not at 00:00:00. Persistent=true means a missed run is caught up after boot. Distributions do patch these units, so read yours:

systemctl cat logrotate.timer logrotate.service
systemctl list-timers 'logrotate*'
Enter fullscreen mode Exit fullscreen mode

Second, logrotate applies a rate limit of its own: it will not modify a log more than once per day unless the rule is size-based and logrotate runs more than once that day, or you force it. That is why adding a second daily timer does not produce two rotations.

size, minsize, maxsize are three different questions

These look interchangeable. They are not.

  • size 100M — rotate only when the file exceeds 100 MB, ignoring the time interval. It is mutually exclusive with daily/weekly/monthly in practice: the last one specified wins.
  • minsize 100M — rotate only if the file exceeds 100 MB and the time interval has elapsed. Both conditions must be true.
  • maxsize 100M — rotate when the file exceeds 100 MB even if the time interval has not elapsed. Either condition can trigger it.

That last distinction is the one people miss. daily plus maxsize 100M gives you a safety valve, but only on days when logrotate actually runs. If it runs once a day and the service writes 2 GB in that day, the file can reach 2 GB before the next check. The maxsize directive did not fail; it was simply never consulted.

If you need a tighter ceiling, run the rotator more often. A drop-in is enough:

sudo systemctl edit logrotate.timer
Enter fullscreen mode Exit fullscreen mode
[Timer]
OnCalendar=
OnCalendar=hourly
RandomizedDelaySec=5m
Enter fullscreen mode Exit fullscreen mode

The empty OnCalendar= clears the inherited value before the new one is added. Then:

sudo systemctl daemon-reload
sudo systemctl restart logrotate.timer
Enter fullscreen mode Exit fullscreen mode

Because the rule includes maxsize, the extra runs can rotate early when needed. Because logrotate still enforces its once-per-day rule for time-based rotation, the hourly timer does not turn daily into 24 rotations.

One caveat: two invocations can overlap. logrotate locks its state file; if the lock is held, it exits with status 3. If you run it from your own oneshot unit, either accept that as success or let it wait:

[Service]
Type=oneshot
ExecStart=/usr/sbin/logrotate /etc/logrotate.conf
SuccessExitStatus=3
Enter fullscreen mode Exit fullscreen mode

A production rule that actually bounds disk

Here is a rule for an application that writes its own log files and can reopen them on a signal:

/var/log/myapp/*.log {
    daily
    rotate 14
    maxsize 100M
    missingok
    notifempty
    compress
    delaycompress
    sharedscripts
    create 0640 myapp myapp
    su myapp myapp
    postrotate
        /bin/systemctl reload myapp.service >/dev/null 2>&1 || true
    endscript
}
Enter fullscreen mode Exit fullscreen mode

Why each line is there:

  • maxsize 100M puts a ceiling on a single file between runs. It does not replace the timer; it makes the timer's cadence matter less.
  • compress plus delaycompress compresses everything except the most recent rotated file. The delay matters when the writer might still hold the old file descriptor for a moment.
  • sharedscripts runs the reload once for the whole wildcard block, not once per matched file. Without it, a directory with ten logs gets ten reloads.
  • create 0640 myapp myapp recreates the file immediately after rotation, before postrotate runs. Omitted attributes are copied from the original file, but being explicit avoids surprises after a package changes the service user.
  • su myapp myapp performs the file operations as the application user. The man page recommends this when rotating files in directories controlled by non-privileged users, and it avoids the classic skip:
skipping "/var/log/myapp/app.log" because parent directory has insecure permissions
(It's world writable or writable by group which is not "root")
Enter fullscreen mode Exit fullscreen mode

One subtlety: scripts are not affected by su. prerotate, postrotate, and the action scripts run as the user that invoked logrotate, with /bin/sh, an unspecified working directory, and no guarantees about your shell profile. Keep them short and use absolute paths.

The copytruncate shortcut and what it costs

Some programs cannot be told to reopen their log file. For those, logrotate offers copytruncate: copy the file, then truncate the original in place.

It works, and it has a documented race. There is a small window between the copy and the truncate; records written in that window are lost. If the log is audited or feeds a billing pipeline, that is not a theoretical problem.

Prefer a reopen signal when the application supports one. If it does not, copytruncate is a reasonable compromise, but do not combine it with create — the original file stays in place, so create has no effect. And if you need rotated files on a different filesystem, look at renamecopy rather than copytruncate; olddir normally has to be on the same device, and copy, copytruncate, and renamecopy are the documented exceptions.

Test the rule without touching production

logrotate -d is a dry run: it prints decisions and changes nothing, including the state file. Use a throwaway state file as well, so a test can never poison the real rotation schedule.

demo=$(mktemp -d)
mkdir -p "$demo/log"
head -c 2048 /dev/zero | tr '\0' 'x' > "$demo/log/app.log"

cat > "$demo/app.conf" <<EOF
$demo/log/app.log {
    daily
    rotate 3
    maxsize 1k
    missingok
    notifempty
    compress
    delaycompress
    create 0640
}
EOF

logrotate -d -s "$demo/state" "$demo/app.conf"
Enter fullscreen mode Exit fullscreen mode

The dry run should report that the file needs rotating because it exceeds maxsize. Now do it for real:

logrotate -v -s "$demo/state" "$demo/app.conf"
ls -l "$demo/log"
Enter fullscreen mode Exit fullscreen mode

You should have a fresh app.log and an older app.log.1. Run the same command again without writing more data: nothing rotates, because the state file now says the file was handled today and the size condition is no longer met.

Append another 2 KB and run it a third time. It rotates again, on the same day, because the size criterion overrides the daily interval. That is the behavior you want to prove before trusting the rule in production.

Do not use -f on a production config to "see what happens". It forces rotation regardless of the interval, which is exactly the thing you are trying to reason about.

When it still will not fire

Work through these before rewriting the rule:

  1. The timer never ran. systemctl list-timers, then journalctl -u logrotate.service. Check for a failed condition, not just a failed exit code.
  2. The service is skipped on battery. The upstream unit sets ConditionACPower=true. Laptop hosts on battery will skip the run; Persistent=true catches up later. Confirm with systemctl cat logrotate.service.
  3. The log is smaller than the threshold. size and minsize are hard gates. maxsize is the only one that can pull a rotation forward.
  4. The file is empty and notifempty is set. Nothing rotates, correctly.
  5. The state file says it was already handled today. That is the once-per-day rule, not a bug.
  6. The config file is being ignored. Files in an include directory are read in alphabetical order and skipped if they are not regular files or if their names end in a taboo extension such as .dpkg-old, .rpmnew, .disabled, or ~. Configs that are group- or world-writable are rejected for security reasons.
  7. The parent directory is writable by a non-root group. logrotate skips the file. Add su with an appropriate user and group.
  8. The glob matches more than you think. A bare * matches already-rotated files too. Prefer *.log, and use olddir or dateext to keep old files out of the match.
  9. You are trying to rotate something under /etc or /usr. The shipped service uses ProtectSystem=full, which makes those trees read-only for the service. Logs do not belong there anyway.

For a full trace of the decision, run the debug pass against the real config and filter for your file:

sudo logrotate -d /etc/logrotate.conf 2>&1 | grep -A 8 '/var/log/myapp'
Enter fullscreen mode Exit fullscreen mode

What logrotate does not cover

The systemd journal is not a text file and logrotate does not manage it. Rotating /var/log/journal by hand or pointing logrotate at it is the wrong tool. Bound it in /etc/systemd/journald.conf with SystemMaxUse= and SystemMaxFileSize=, then let journald enforce its own retention.

Use logrotate for file-based logs: nginx, PostgreSQL, application logs, audit exports. Use journald's own limits for the journal. Mixing the two retention models is how hosts end up with a full disk and two teams blaming each other.

Checklist

  • Treat rotate N as a file count, not a quota.
  • Add maxsize if a single file can grow between runs.
  • Run the rotator more often if that ceiling is still too loose.
  • Prefer reopen signals over copytruncate.
  • Test with -d and a throwaway state file before deploying.
  • Read systemctl cat logrotate.timer logrotate.service on the actual host.
  • Remember that the journal has its own retention settings.

The config is short. The semantics are where the outages live.

References

Top comments (0)