DEV Community

Cover image for Your README is lying to you (and so is your Dockerfile)
Harshith M
Harshith M

Posted on

Your README is lying to you (and so is your Dockerfile)

Every file in your repository can pass its own linter while the repository as a whole lies.

Here's a project I've seen a hundred times:

  • README: "Requires Python 3.8+"
  • pyproject.toml: requires-python = ">=3.10"
  • Dockerfile: FROM python:3.9-slim
  • CI: tests on 3.9, a version users can't even install on
  • Setup guide: "run make lint", a target that was renamed months ago
  • The code: reads JWT_SECRET, which is nowhere in .env.example

Each file is fine on its own. Together they contradict each other, and nobody notices until a new contributor gets stuck on step 2, or production runs a different Python than everyone tested.

So I built truthlint: a CLI that cross-examines your repo's files against each other.

What it looks like

$ truthlint scan examples/drifting-python-api

 Truth Score  23/100 F
 Onboarding   40/100 F
 1 critical   10 high   6 medium   2 low

 HIGH  TL101-406201  Python version contradiction
   README.md says Python 3.8+ is enough, but the project requires >=3.10
   (minimum 3.10). Users following the docs on Python 3.8 will be refused by pip.
   ↳ README.md:8        claims Python 3.8+
   ↳ pyproject.toml:9   declares requires-python '>=3.10'
   Fix: State Python 3.10+ in README.md, or raise the constraint.  [auto-fix available]

 HIGH  TL202-6F78F4  Command runs an undefined make target
   `README.md` documents `make lint`, but target `lint` is not defined in
   `Makefile` (defined: format, serve, test).
   ↳ README.md:28  documents `make lint`
   ↳ Makefile      defines targets: format, serve, test
Enter fullscreen mode Exit fullscreen mode

Every finding cites the file and line on both sides of the contradiction. No vague "something seems off".

What it checks

35 rules across the files that usually drift apart:

  • Runtime versions: README, Dockerfile, CI matrix, .python-version and .nvmrc vs requires-python / engines.node
  • Documented commands: npm run X, make X, pip install ".[dev]", docker compose up worker. Does that script, target, extra or service actually exist?
  • Environment: variables your code reads but .env.example never mentions
  • Dependencies: imports you never declared (works on your machine only), and dependencies nothing imports
  • Docker: COPY of files that don't exist, lockfiles ignored during the build, CMD pointing at missing modules
  • Docs: broken links and anchors, and "project structure" trees listing files deleted long ago

It also runs an onboarding audit: it walks your setup instructions step by step, following cd, cp and python -m venv, and checks that a newcomer could actually follow them.

The part I care about most: it never runs your code

Everything is static. truthlint parses Markdown, TOML, YAML, Dockerfiles and Makefiles with line numbers, reads Python with the ast module, and simulates just enough shell state to verify commands without executing them.

It doesn't even call git. A repository's .git/config can make git run arbitrary commands, so truthlint ships its own .gitignore matcher. That makes it safe to point at a repo you just cloned from a stranger.

And when something can't be verified without running it, like git clone or pytest, it says "unverified" instead of guessing. I'd rather miss a problem than cry wolf in your CI.

Fixes you review first

truthlint fix                 # preview every patch (changes nothing)
truthlint fix --interactive   # apply them one by one, with confirmation
truthlint fix -o fixes.patch  # export a patch for git apply
Enter fullscreen mode Exit fullscreen mode

Try it in 30 seconds

pipx install truthlint
git clone https://github.com/harshith7002/truthlint
cd truthlint
truthlint scan examples/drifting-python-api    # a deliberately broken project
truthlint scan examples/healthy-library        # 100/100, zero findings
truthlint scan .                               # truthlint scores 100 on itself
Enter fullscreen mode Exit fullscreen mode

Then point it at your own project:

truthlint scan /path/to/your/repo
Enter fullscreen mode Exit fullscreen mode

I'd love to know your score. Drop it in the comments, especially if it found something embarrassing. And if it reports a contradiction that isn't real, please open an issue: false-positive reports are the feedback I want most.

⭐ GitHub: https://github.com/harshith7002/truthlint
📦 PyPI: pip install truthlint (Python 3.11+, MIT)

Top comments (1)

Collapse
 
linkbruno profile image
linkbruno.com •

Checking files against each other instead of one at a time is a smart angle. The Python 3.8 README vs requires-python vs python:3.9-slim example is painfully familiar. How does it treat documented commands with placeholders, like make test ARGS=..., and do the anchor checks follow links into other Markdown files in the repo or only headings in the same file?