DEV Community

Avery Lin
Avery Lin

Posted on

Assemble a README Ledger From Entry Points and Tests; Sign Guarantees by Hand

README quality fails when generated prose overwrites contracts that tests and operators still have to honor in production. Inventory facts can be compiled from entry points, pytest node identifiers, and package metadata with deterministic scripts. Audience statements, authentication semantics, support windows, and failure-mode promises remain human-signed work, not model output. The workflow below produces a ledger that separates those lanes so review time lands on claims rather than table formatting.

The failure mode this ledger is built to catch

A model that rewrites an entire README can silently expand compatibility, hide required secrets, or invent a support window. Reviewers then compare prose against prose, which hides the fact that the contract lane never had a source of truth. Entry points and collected tests already encode installable commands and executable examples with more precision than marketing copy. Those artifacts should feed inventory tables, while humans still own every sentence that binds the project to a promise.

Lane definitions the compiler is allowed to use

Inventory lanes include command names, module paths, collected test node identifiers, optional extra names, and documented fixture roles. Contract lanes include intended audience, authentication and authorization semantics, data-retention claims, and explicit non-goals. Support lanes include version windows, deprecation dates, incident-response expectations, and anything that implies an SLA. If a sentence cannot be regenerated from repository facts, it belongs in a contract or support lane and stays unsigned by default.

1. Collect repository facts without a model

Start from files the package already ships, not from a chat transcript that paraphrases them after the fact. Read pyproject.toml for project name, optional extras, and console-script entry points that users can actually install. Collect pytest node identifiers so example commands in the README can be checked against tests that still exist. The commands below are the only collection step; they do not draft prose and they do not mark any contract cell as signed.

python -c "import tomllib, pprint, pathlib; pprint.pp(tomllib.loads(pathlib.Path('pyproject.toml').read_text()))"
pytest --collect-only -q
python -m pip index versions . 2>/dev/null || python -c "import importlib.metadata as m; print(m.version('unknown') if False else 'use importlib.metadata on an installed editable')"
Enter fullscreen mode Exit fullscreen mode

Treat collection failures as ledger defects rather than as permission to guess. A missing [project.scripts] table means the command inventory stays empty until a human adds an entry point. A pytest collection error means example rows stay unsigned, because an undocumented test is not an executable README claim. Do not paste stack traces into the README during this step; store them beside the ledger as operator notes.

2. Emit a README ledger with explicit unsigned cells

The proposal script below writes JSON that a later draft step may read, and that a human must sign. It is labeled a proposal because you should run it on your tree and inspect the output before any assistant sees the file. Keep contract and support values as null so a model cannot treat silence as approval. Inventory arrays may be filled from disk; signature metadata stays empty until a named reviewer writes a date.

# Proposal: run locally against a package that has pyproject.toml.
from __future__ import annotations

import json
import subprocess
import sys
tomllib = sys.modules.get("tomllib") or __import__("tomllib")
from pathlib import Path

ROOT = Path.cwd()
LEDGER = ROOT / "docs" / "readme_ledger.json"


def load_project() -> dict:
    data = tomllib.loads((ROOT / "pyproject.toml").read_text(encoding="utf-8"))
    project = data.get("project") or {}
    scripts = (project.get("scripts") or {}) if isinstance(project.get("scripts"), dict) else {}
    extras = sorted((project.get("optional-dependencies") or {}).keys())
    return {
        "name": project.get("name"),
        "requires_python": project.get("requires-python"),
        "scripts": [{"command": k, "target": v} for k, v in scripts.items()],
        "extras": extras,
    }


def collect_pytest_nodes() -> list[str]:
    proc = subprocess.run(
        [sys.executable, "-m", "pytest", "--collect-only", "-q"],
        cwd=ROOT,
        capture_output=True,
        text=True,
        check=False,
    )
    if proc.returncode not in (0, 5):  # 5 == no tests collected
        return []
    nodes = []
    for line in proc.stdout.splitlines():
        line = line.strip()
        if "::" in line and not line.startswith("="):
            nodes.append(line)
    return nodes[:80]


def build_ledger() -> dict:
    project = load_project()
    return {
        "inventory": {
            "package_name": project["name"],
            "requires_python": project["requires_python"],
            "console_scripts": project["scripts"],
            "optional_extras": project["extras"],
            "pytest_nodes": collect_pytest_nodes(),
        },
        "draft": {
            "install_table_markdown": None,
            "command_table_markdown": None,
            "test_map_markdown": None,
        },
        "contract": {
            "audience": None,
            "auth_semantics": None,
            "data_handling": None,
            "non_goals": None,
        },
        "support": {
            "compatibility_window": None,
            "deprecations": None,
            "incident_expectations": None,
        },
        "signatures": {
            "contract_reviewer": None,
            "contract_date": None,
            "support_reviewer": None,
            "support_date": None,
        },
    }


def main() -> None:
    LEDGER.parent.mkdir(parents=True, exist_ok=True)
    ledger = build_ledger()
    LEDGER.write_text(json.dumps(ledger, indent=2) + "\n", encoding="utf-8")
    print(f"wrote {LEDGER} with unsigned contract and support lanes")


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

Run the compiler, then fail the job if contract keys are already populated by a previous model pass. That guard is cheap and prevents a regenerated README from inheriting unsigned promises. Keep the ledger under docs/ so pull requests can diff JSON instead of scrolling through rewritten adjectives. The next section turns that JSON into a review surface a human can actually finish.

mkdir -p docs
python readme_ledger.py
python -c "import json,pathlib; d=json.loads(pathlib.Path('docs/readme_ledger.json').read_text()); assert all(v is None for v in d['contract'].values())"
Enter fullscreen mode Exit fullscreen mode

3. Decide what a model may draft versus what a human must sign

Use the decision table as the only allowlist when a drafting tool is connected later. Inventory cells may become Markdown tables because they restate names and paths the compiler already proved. Contract cells must stay human-authored, even when the model offers a fluent paragraph that sounds like your existing voice. Support cells need a dated signature because they age independently of the code inventory and they create expectations outside the repository.

README section Source of truth Model may draft? Human must sign? Reject if
Package name and extras pyproject.toml No, copy verbatim No Wording changes from the TOML
Install command table extras + scripts Yes, table only No Adds mirrors, flags, or sudo
CLI inventory [project.scripts] Yes, names and targets No Invents subcommands
"How we test" map pytest node ids Yes, links node to heading No Claims coverage percentages
Who this is for none No Yes Borrows audience from another product
Auth and secrets none No Yes Mentions tokens, bypasses, or sample keys
Data handling none No Yes Implies retention or deletion SLAs
Non-goals none No Yes Softens a limitation into a roadmap
Compatibility window release policy No Yes Reads like an SLA or uptime target
Deprecations changelog owners No Yes Back-dates or hides a break

Copy the table into the pull request template so reviewers score rows instead of rereading the full README. A row that says "model may draft" still needs a diff against the ledger, because table formatting is where extra flags usually appear. A row that says "human must sign" should block merge while the signature object is null. Do not add a third lane for "the model is pretty sure"; uncertainty belongs to the human, not to a softer heading.

4. Fill draft cells without giving the model the contract lane

Draft cells can be filled locally or by a hosted assistant once the ledger JSON is the only prompt context. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option that can expand inventory rows into readable Markdown tables. The assistant must receive the ledger file and a lane allowlist, and it must not rewrite contract or support cells.

The prompt boundary is a file cut, not a polite instruction buried in a long chat. Write docs/draft_allowlist.txt with the three draft keys, and pass only inventory plus draft into the model workspace. After the model returns, re-read contract, support, and signatures and fail if any byte changed. The snippet below is a proposal for that check, not a claim about any hosted quota or model name.

# Proposal: compare unsigned lanes after a draft-only model pass.
import json
from pathlib import Path

before = json.loads(Path("docs/readme_ledger.json").read_text(encoding="utf-8"))
# after_path is the file the draft step wrote; keep it separate from signatures.
after = json.loads(Path("docs/readme_ledger.after.json").read_text(encoding="utf-8"))

for lane in ("contract", "support", "signatures"):
    if before[lane] != after[lane]:
        raise SystemExit(f"draft step mutated {lane}; reject the README regeneration")

for key, value in after["draft"].items():
    if value is None:
        raise SystemExit(f"draft cell {key} still empty")
    if "http://" in value or "Bearer " in value:
        raise SystemExit(f"draft cell {key} contains a credential-shaped string")
print("draft cells filled; contract and support lanes unchanged")
Enter fullscreen mode Exit fullscreen mode

Render draft Markdown into fenced placeholders inside README.md, not over the signed sections. HTML comments work as cheap fences that grep can find in review. Keep signed sections in a second file such as docs/README.contract.md if your publishing pipeline can concatenate them. Concatenation is easier to audit than a single document the model is allowed to rewrite from the first heading to the license footer.

<!-- LEDGER:DRAFT:command_table_markdown -->
| Command | Target |
| --- | --- |
| pkg | package.cli:main |
<!-- /LEDGER:DRAFT:command_table_markdown -->

<!-- LEDGER:CONTRACT:audience (human only) -->
<!-- leave empty until signatures.contract_reviewer is set -->
<!-- /LEDGER:CONTRACT:audience -->
Enter fullscreen mode Exit fullscreen mode

5. Sign contracts with a reviewer name, not with a regenerated paragraph

A signature is a person, a date, and the hash of the contract object, not a more confident adjective. Require those three fields in CI before README publication jobs are allowed to run. If the inventory changes because a new extra landed, do not wipe the previous contract signature; open a review that asks whether the promise still holds. Inventory churn is expected. Silent contract churn is the defect this ledger exists to make expensive.

python - <<'PY'
import hashlib, json, pathlib
p = pathlib.Path("docs/readme_ledger.json")
d = json.loads(p.read_text())
blob = json.dumps(d["contract"], sort_keys=True).encode()
print("contract_sha256", hashlib.sha256(blob).hexdigest())
missing = [k for k, v in d["signatures"].items() if v is None]
if missing:
    raise SystemExit("unsigned fields: " + ",".join(missing))
PY
Enter fullscreen mode Exit fullscreen mode

When a human writes audience text, keep it short enough that a later reviewer can disagree with a single sentence. "This CLI is for operators who already hold database credentials" is a contract. "Easy to use for everyone" is not a contract, because nobody can test it and nobody can revoke it. The same rule applies to authentication: name the header, the required role, and the failure the client should expect, or leave the cell null. Null is an honest state. Invented OAuth poetry is not.

6. Prove the inventory still matches the README tables

Add a test that reconstructs the command table from pyproject.toml and compares it with the fenced draft block. The test should fail on extra rows, missing rows, and command names that pytest never collected when the README claims an example is tested. Label the test as a proposal until it runs in your CI image. Once it runs, it becomes the only coverage that matters for this workflow: documentation inventory stays coupled to installable facts.

# Proposal: tests/test_readme_ledger.py
import re
from pathlib import Path

import tomllib

README = Path("README.md").read_text(encoding="utf-8")
PROJECT = tomllib.loads(Path("pyproject.toml").read_text(encoding="utf-8"))


def test_command_table_lists_only_declared_scripts():
    scripts = set((PROJECT.get("project") or {}).get("scripts") or {})
    block = re.search(
        r"<!-- LEDGER:DRAFT:command_table_markdown -->(.*?)<!-- /LEDGER:DRAFT:command_table_markdown -->",
        README,
        re.S,
    )
    assert block, "missing draft fence for command table"
    listed = set(re.findall(r"^\| ([^|]+) \|", block.group(1), re.M) )
    listed.discard("Command")
    listed.discard("---")
    assert listed <= scripts, f"README commands not in pyproject scripts: {listed - scripts}"
    assert scripts <= listed, f"pyproject scripts missing from README: {scripts - listed}"
Enter fullscreen mode Exit fullscreen mode

Run the test whenever entry points move, not only when someone edits README prose. That schedule is what keeps the compile lane honest after a refactor that never touched documentation files. If the test is too strict for experimental packages, shrink the fence to a single supported extra rather than disabling the assertion. A smaller inventory is still a ledger. An unbounded README is a prompt dump.

Limitations

This workflow does not generate conceptual tutorials, architecture decision records, or legal terms, and it should not be asked to. Pytest node identifiers describe what is executable, not what is safe, private, or supported next quarter. Package metadata is silent about threat models, so authentication cells stay empty until a human writes them. Teams that publish cryptographic or medical claims need a review path this ledger does not replace, including counsel where the product already requires it.

The compiler also cannot see services that exist only in staging diagrams or in operator runbooks outside git. If your README must describe those systems, keep them in the contract file and accept that the inventory test will not cover them. Do not scrape internal wikis into the draft lane to close that gap, because that move reintroduces unsigned prose with a more official filename. Missing facts should remain visible nulls.

Who should not use this approach

Skip this ledger if the repository has no tests, no pyproject.toml, and no installable commands worth inventorying. Marketing sites, changelog blogs, and investor one-pagers are not README contracts, and forcing them through unsigned JSON adds ceremony without a verifier. Do not use a drafting assistant on README files that must include live credentials, customer names, or non-public endpoints, even when those strings are later redacted by hand. The cheaper control is to keep those files out of the model workspace.

Maintainers who already hand-write a short README and rarely change entry points will spend more time on the ledger than they save on tables. In that case, keep a single human-owned README and skip the compile step. The method is for packages whose command surface and test map change faster than reviewers can reread marketing paragraphs. It is a partition of labor, not a requirement to host documentation generation on any particular server.

If you already generate inventory tables with MonkeyCode's free model access on the free server option, feed it the ledger JSON and the allowlist rather than the raw repository. The signed contract file still has to be written by a reviewer who can be named in signatures. That split is the entire method: compile what the tree can prove, and refuse to let fluent drafts stand in for guarantees.

Top comments (0)