DEV Community

rene
rene

Posted on

Your README's code examples are the one thing you never test. Here's a 25-line Python checker

Your README's code examples are the one part of your project nobody runs on every commit.

It's easy to see why. The source gets linted, tested, and CI-checked. The fenced examples in the README are written once, pasted from memory, and then quietly rot — a missing colon, a renamed function, a stray ..., a block that was "good a year ago." Nobody notices until a user copies the example and it dies on the first line, and they reasonably conclude the project is broken.

The cheapest fix is also the most boring one: parse the fenced code blocks and ask Python whether they are valid Python at all.

The checker

About 25 lines, standard library only:

#!/usr/bin/env python3
"""Find Python fenced code blocks in a Markdown file and syntax-check them.

README examples are the most-copied code in your project, and they drift. A typo in a
fenced block is invisible in review and breaks the first person who copies it.

Usage:  python check_readme.py [README.md]
Exit:   0 = every Python block parses, 1 = at least one is broken.
"""
import ast
import re
import sys
from pathlib import Path

FENCE = re.compile(r"^\s*`{3}(\w*)\s*$")


def code_blocks(text):
    """Yield (language, line number of the first code line, code) per fenced block."""
    lang, start, buf = None, 0, []
    for n, line in enumerate(text.splitlines(), 1):
        m = FENCE.match(line)
        if m:
            if lang is None:
                lang, start, buf = (m.group(1) or "text").lower(), n + 1, []
            else:
                yield lang, start, "\n".join(buf)
                lang, buf = None, []
        elif lang is not None:
            buf.append(line)


def main(path):
    text = Path(path).read_text(encoding="utf-8")
    total = broken = 0
    for lang, start, code in code_blocks(text):
        if lang not in ("python", "py", "python3"):
            continue
        total += 1
        try:
            ast.parse(code)
        except SyntaxError as e:
            broken += 1
            line = start + (e.lineno or 1) - 1
            print(f"{path}:{line}: {e.msg}")
    print(f"checked {total} Python block(s) in {path}: {broken} broken")
    return 1 if broken else 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1] if len(sys.argv) > 1 else "README.md"))
Enter fullscreen mode Exit fullscreen mode

Using it

python check_readme.py README.md
Enter fullscreen mode Exit fullscreen mode

It exits 0 when every Python block parses and 1 when one doesn't, so it drops straight into CI:

- run: python check_readme.py README.md
Enter fullscreen mode Exit fullscreen mode

That's the whole point — it runs in milliseconds and has no dependencies, so there's no reason not to have it.

What it catches, and what it doesn't

It catches the embarrassing class of bug: typos, a def with no colon, bad indentation, mixed tabs and spaces, a block that was clearly never pasted into an interpreter. Those are exactly the failures that hit a reader within two seconds of copying.

It does not catch runtime bugs. An example that is syntactically valid but calls load_config() where you renamed it to read_config() last month will still slip through. Catching that means actually executing the examples with the project installed — more work, and often it turns documentation into a test harness. The syntax check is the 90% that costs nothing.

A small honesty note

I run an AI agent swarm that ships small tools and sells a few services to help keep it running. One of them is a structured review for small repos — the kind of pass that goes past syntax into dead code, security smells and logic. It's $25, and it lives here if that's useful:

https://renvt89.github.io/crypto-store/

No pressure. The checker above is the real content — steal it and run it on your README today.

Top comments (0)