DEV Community

Avery Lin
Avery Lin

Posted on

Generate How-To Skeletons From Signatures, Then Gate README Promises With a Lane File

How-to documentation fails when a drafting model writes promises that no inventory can prove. Public signatures can support parameter tables and procedure skeletons, but they cannot certify install paths, support windows, or example output. This workflow builds a machine-readable inventory, lets a model draft unmarked skeletons, and requires a human to sign environment claims. The method is process control, not proof that the resulting README is correct.

The failure mode is consistent across libraries that regenerate getting-started pages from chat. A model restates function names fluently, then invents a pip extra, a Python floor, or a sample transcript that nobody executed. Readers treat those sentences as contracts. Reviewers miss them because the surrounding prose looks locally coherent. The fix is not a better prompt. The fix is a lane file that forbids the model from filling claim classes the inventory cannot support.

What an inventory can prove

An AST walk of public functions can prove names, argument lists, defaults, annotations, and docstring presence. It can also prove which symbols are exported in __all__ when that list exists. It cannot prove that a default is safe in production, that a type hint matches runtime, or that a docstring is still true. Those gaps define the human lane. Treat every unproven clause as unsigned until a person writes it beside the inventory hash.

The table below is the contract. Draft text may occupy only the left column. Anything that implies a working machine, a supported version, or a legal reading belongs on the right, even if the model could phrase it smoothly.

Claim class Model may draft Human must own
Section outline for a how-to Yes, from inventory headings No, unless the outline adds a product promise
Parameter tables Yes, copied from signatures Defaults that imply safety or units
Procedure skeletons Yes, with TODO(human) steps Any step that must succeed on a reader machine
Cross-links between symbols Yes, if both symbols are public Compatibility statements across releases
Install and extras No Exact commands, extras, and interpreter floor
Sample stdout / screenshots No, unless labeled unexecuted Any transcript presented as observed
Support, license, security No License meaning, contact path, threat claims

1. Freeze a public-symbol inventory

The first artifact is a JSON inventory keyed by qualified name. Hash that file in git so later README diffs can point at a specific symbol set. The script below is a labeled example for a small library layout. It is not a production indexer, and it ignores re-exports, C extensions, and runtime-generated APIs.

# labeled example: inventory_extract.py
from __future__ import annotations

import ast
import hashlib
import json
from pathlib import Path
from typing import Any

ROOT = Path("src")
OUT = Path("doc_inventory.json")


def public_functions(path: Path, module: str) -> list[dict[str, Any]]:
    tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
    rows: list[dict[str, Any]] = []
    for node in tree.body:
        if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
            continue
        if node.name.startswith("_"):
            continue
        args = []
        for arg in node.args.args:
            anno = ast.unparse(arg.annotation) if arg.annotation else None
            args.append({"name": arg.arg, "annotation": anno})
        rows.append(
            {
                "qualname": f"{module}.{node.name}",
                "args": args,
                "returns": ast.unparse(node.returns) if node.returns else None,
                "has_docstring": ast.get_docstring(node) is not None,
                "lineno": node.lineno,
            }
        )
    return rows


def main() -> None:
    items: list[dict[str, Any]] = []
    for path in sorted(ROOT.rglob("*.py")):
        rel = path.with_suffix("").relative_to(ROOT)
        module = ".".join(rel.parts)
        items.extend(public_functions(path, module))
    payload = {"items": items}
    text = json.dumps(payload, indent=2, sort_keys=True) + "\n"
    digest = hashlib.sha256(text.encode("utf-8")).hexdigest()[:12]
    payload["inventory_sha256_12"] = digest
    OUT.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n", encoding="utf-8")
    print(f"wrote {OUT} items={len(items)} sha={digest}")


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

Run it from the repository root after you point ROOT at the package you actually ship.

python inventory_extract.py
python -c "import json; print(json.load(open('doc_inventory.json'))['inventory_sha256_12'])"
Enter fullscreen mode Exit fullscreen mode

Store the twelve-character digest in the lane file. If the digest moves, every previously signed install sentence is stale until a human re-reads it. That rule is mechanical. It does not require judging prose quality during the inventory step.

2. Write the lane file before any draft exists

Create doc_lanes.yaml by hand. Do not ask a model to classify claim ownership, because that classification is the control plane. Keep human-owned keys empty or filled with UNSIGNED. Draft-owned keys may contain skeleton markdown that still uses placeholders.

# labeled example: doc_lanes.yaml
inventory_sha256_12: "replace-after-extract"
draft_allowed:
  outline: |
    ## Install
    TODO(human): interpreter, extras, lockfile
    ## Call the public entrypoints
    TODO(model): parameter tables from inventory only
    ## Next steps
    TODO(human): support path
human_owned:
  python_requires: UNSIGNED
  install_command: UNSIGNED
  extras: UNSIGNED
  observed_example_exit_code: UNSIGNED
  license_summary: UNSIGNED
  support_contact: UNSIGNED
forbidden_in_draft:
  - "pip install"
  - "conda install"
  - "tested on"
  - "production ready"
  - "we support"
  - "output:"
  - "SLA"
Enter fullscreen mode Exit fullscreen mode

The forbidden list is a tripwire, not a semantic parser. It catches the usual README promises that inventories cannot justify. Extend it when your corpus uses other contract verbs. Do not treat a clean scan as evidence that the how-to is true.

3. Draft only inside the allowed skeleton

Feed the model the inventory JSON, the outline, and an instruction that it may rewrite TODO(model) blocks. It must leave TODO(human) blocks untouched. It must not add install commands, version floors, or sample program output. If you need a remote drafting environment, MonkeyCode's free model access and free server option can host that narrow pass. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Keep the lane file and inventory in git either way; do not leave ownership state inside a chat transcript.

A prompt that stays inside the contract looks like the block below. It is a template, not a measured quality claim, and it names no model.

You receive doc_inventory.json and the draft_allowed.outline field.
Rewrite only sections marked TODO(model).
Use parameter names and annotations exactly as given.
Do not invent extras, Python versions, install commands, or program output.
Leave every TODO(human) line unchanged.
Return markdown for the outline only.
Enter fullscreen mode Exit fullscreen mode

If the draft inserts a command the inventory never contained, discard the draft. Regenerating is cheaper than signing a false install path. The human lane is for knowledge the repository does not encode, not for cleaning model leftovers.

4. Verify lanes before the README merge

The checker below is a labeled example. It enforces three gates: inventory digest match, no forbidden phrases in the draft file, and no remaining UNSIGNED keys when --require-signed is set. It does not score readability and it does not execute examples.

# labeled example: verify_doc_lanes.py
from __future__ import annotations

import argparse
import json
import re
import sys
from pathlib import Path

try:
    import yaml
except ImportError:
    sys.stderr.write("install pyyaml before running this checker\n")
    raise


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--inventory", default="doc_inventory.json")
    parser.add_argument("--lanes", default="doc_lanes.yaml")
    parser.add_argument("--draft", default="howto.draft.md")
    parser.add_argument("--require-signed", action="store_true")
    args = parser.parse_args()

    inventory = json.loads(Path(args.inventory).read_text(encoding="utf-8"))
    lanes = yaml.safe_load(Path(args.lanes).read_text(encoding="utf-8"))
    draft = Path(args.draft).read_text(encoding="utf-8")
    errors: list[str] = []

    expected = inventory.get("inventory_sha256_12")
    if lanes.get("inventory_sha256_12") != expected:
        errors.append("lane digest does not match inventory")

    lowered = draft.lower()
    for needle in lanes.get("forbidden_in_draft", []):
        if needle.lower() in lowered:
            errors.append(f"forbidden phrase in draft: {needle!r}")

    if re.search(r"TODO\(model\)", draft):
        errors.append("unresolved TODO(model) remains in draft")

    if args.require_signed:
        for key, value in (lanes.get("human_owned") or {}).items():
            if value == "UNSIGNED" or not str(value).strip():
                errors.append(f"unsigned human key: {key}")

    if errors:
        sys.stderr.write("\n".join(errors) + "\n")
        return 1
    print("lane check passed")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())
Enter fullscreen mode Exit fullscreen mode

Wire the cheap gates into CI without the signed-key requirement. Run --require-signed only on the release branch, after a person has replaced every UNSIGNED value with a sentence they are willing to defend. That split keeps pull requests unblocked while still stopping a tagged how-to from shipping empty promises.

pip install pyyaml
python verify_doc_lanes.py
python verify_doc_lanes.py --require-signed  # release branch only
Enter fullscreen mode Exit fullscreen mode

5. Sign the human lane with evidence, not tone

When you replace UNSIGNED, attach a pointer that another engineer can replay. An interpreter floor should cite requires-python in packaging metadata, not a remembered hallway number. An install command should cite the extra name in pyproject.toml. A sample transcript should cite a recorded session or stay labeled unexecuted. Support text should cite the actual issue tracker or mail alias. None of those pointers are decorative. They are how the next reviewer checks the claim without trusting the prose.

If packaging metadata and the how-to disagree, the how-to is wrong. Do not “fix” the conflict by asking the model to reconcile tone. Change the human lane or change the package metadata, then re-hash the inventory if public symbols moved. Documentation that cannot name its evidence is still a draft, regardless of how finished it reads.

Limitations

The inventory is incomplete for wrappers, __getattr__ exports, and compiled modules. Forbidden-phrase scanning misses paraphrases such as “drop this into a venv” that still imply a working install. Docstrings inside the tree can be false, and the extractor will still report has_docstring: true. Free drafting environments remain capable of fluent procedure text that never ran. The lane file records ownership; it does not execute the how-to on a clean machine.

This workflow also adds merge friction. Teams that publish a one-page gist for an internal spike will spend more time on YAML than on the spike. That cost is acceptable when readers will copy commands into production shells. It is wasted when the document is a scratch note with no external contract.

Who should not use this

Skip the lane file if you are writing regulated validation packages that already require executed protocols and signed protocols under a quality system. Skip it if public symbols churn daily and no digest could stay current through a review cycle. Skip it if the document’s only audience is the author, because the control plane exists to constrain claims other people will trust. In those cases, a handwritten paragraph with no model pass is the smaller honest artifact.

For libraries that already keep packaging metadata and a small public surface, the split is enough to stop the common README failure. Keep skeletons in the draft lane. Keep install paths, support, license meaning, and observed output in the human lane. Merge only when the digest matches and the unsigned keys are gone. If you draft those skeletons on free model access, export the lane file in the same commit as the inventory so review stays in git rather than in chat history.

Top comments (0)