Linux Doctor, heal thyself: 20 ways my Linux health checker was wrong

7sh1d0w7x — Thu, 01 Oct 2026 17:22:03 +0000

A diagnostic tool has exactly one job: tell the truth about the machine in
front of it. So when I built Linux Doctor — a read-only checker that
reports what is wrong and prints the fix without ever running it — I did the
thing a doctor dreads. I audited it against reality: five distro families,
minimal container images, and machines that had been running for years.

It lied about twenty times. None of them were exotic. Every one is a class of
bug you can hit in any script that reads the output of system tools.

Here they are, grouped by what they teach.

1. A pipeline exits with the status of its last command

df -P /boot | tail -n 1     # a failed df looks successful
journalctl -p err | grep MCE # grep exits 1 when nothing matches

The boot check read /boot's usage through tail. If df failed, the exit
status still belonged to tail, which succeeded — so a failed probe looked
like a good one.

Worse was hardware. It used the exit status of journalctl … | grep … as its
readability gate. But grep exits 1 when there is nothing to match —
so "no hardware errors" (good news) was read as "I could not read the log", and
the check stayed silent on a healthy machine instead of saying "No hardware
errors logged". A benign EDAC banner made grep exit 0 on my own box, which
hid the bug for weeks.

Lesson: never gate readability on a pipeline's status. Take the raw output
and decide in code.

2. "Nothing found" and "the command is missing" are not the same

Minimal Debian, Fedora and Ubuntu images do not ship ip (iproute2). The probe's failure was read as an empty route table — a medium "No default network route" on a machine with a perfectly good route.
Minimal openSUSE has no awk. zypper … | awk produced nothing, so Tumbleweed reported "System is up to date" with thirteen lines of updates on screen.
The engine's own run() only flagged a command as missing when the shell itself was absent — which never happens. A missing tool exits 127 through the shell. So every check that gated a "could not check" skip on missing stayed silent, and a minimal image scored as if those checks had passed.

Lesson: distinguish "the tool answered" from "the tool isn't there."
Exit 127 is the tell.

3. Success messages contain the words you search for

pacman -Dk prints "No database errors have been found!" — a bare error match flagged a clean Arch system as having broken packages.
The EDAC driver's startup line is "EDAC ie31200: No ECC support" — the check matched it as an ECC problem. It was the top item in the report and cost 9 points.
mce: CPU supports N MCE banks is printed once per CPU at boot on every Intel machine. A bare mce match read it as a machine-check exception.

Lesson: a keyword match is not a diagnostic. Require an actual event, and
explicitly reject the routine preamble lines.

4. Kernel interfaces are not namespaced

Inside a 256 MB container, free -b reported the host's 15 GB.
/proc/loadavg is the host's load average while nproc reports the
container's CPU count — and the ratio between them invented an overloaded
system out of an idle container. (All five test images reported it.) lsblk
and /proc/swaps are not namespaced either, so the container was told about
the host's disks and swap.

Lesson: detect the container and say why you are skipping. That is a
limitation of the environment, not a verdict about the host.

5. Sometimes the bug is you

A fuser-based lock probe ran next to Linux Doctor's own apt-get check. So
it found the tool itself holding the dpkg lock — and told the user to wait for,
or kill, a process that was the tool. It only happened when run as root, so
almost nobody would ever have seen it.

Lesson: when you inspect a global resource, exclude your own process group.

The wrong direction of wrong

The dangerous bug is not a false alarm. It is a false all-clear.

A fresh image that never ran apt update answers "0 upgraded" with exit 0 — which read as "up to date". That is not being up to date. The check now says it cannot tell and points at apt update.
apk info -u is not a valid command at all (it exits 1 with unrecognized option 'u'), so Alpine said nothing.
Void had no update branch, so a machine with 54 pending updates was skipped and scored as current.
flatpak remote-ls --updates prints a column table whose fields contain no /, which is exactly what the count looked for → "apps are up to date" with updates pending.

Lesson: prefer "unknown" to a confident lie. A check that says "I could
not determine this, and here is the package that would let me" is worth more
than one that silently scores a broken machine as healthy.

And sometimes it answered a different question than the one asked

processes warned whenever one app used more than 20% of total RAM, ignoring what was actually free — so a 15 GB box with 9.4 GB free got a medium warning about a browser.
The same check listed a browser once per process (its memory is spread across many processes sharing one binary) and reported the largest single process as "the app".
timers called dnf-makecache.timer a broken schedule. It is enabled on an immutable system and can never run — its start condition is unmet by design, and systemctl status says so.
The KDE lock screen logs Authentication attempt too soon every time you retype a wrong password quickly. A healthy desktop got "6 recognized errors" and lost 8 points. The same string from sshd is worth seeing — only the screen locker's copy is noise.

Lesson: "the number is real" is not the same as "the number means what you
think it means."

How these are caught now

A clean-image gate runs the engine inside Fedora, Debian, Ubuntu, Alpine and Arch containers, and fails when an unexplained high or medium finding appears.
Recorded fixtures from real machines replay through the same pipeline, and every high/medium finding they produce needs a written reason.
A severity rubric and a finding-code registry so severity cannot drift silently.
No fix lands without a regression test that fails first. A wrong result is reproduced before it is changed.
The whole list is public, in both directions: docs/limitations.md names every false positive and false negative it has shipped, with the test that guards each one.

Why tell on yourself

The entire value of a diagnostic is trust — and trust is built by being
explicit about where you might be wrong, not by never being wrong. A tool that
admits "I could not check this; here is the package that fixes it" is far
more useful than one that quietly scores a minimal image as perfect.

Linux Doctor is read-only by construction: it prints the fix and never runs it.
One engine sits behind a CLI, a web dashboard and a desktop app — and the
honesty document ships with it.

→ https://github.com/7sh1d0w7x/linux-doctor

If you run it and it gets something wrong, that is the most useful bug report
the project can get — and it comes with a replayable fixture.

DEV Community: 7sh1d0w7x