How-to documentation fails when a drafting model writes promises that no inventory can prove. Public signatures can support parameter tables and procedure skeletons, but they cannot certify install paths, support windows, or example output. This workflow builds a machine-readable inventory, lets a model draft unmarked skeletons, and requires a human to sign environment claims. The method is process control, not proof that the resulting README is correct.
The failure mode is consistent across libraries that regenerate getting-started pages from chat. A model restates function names fluently, then invents a pip extra, a Python floor, or a sample transcript that nobody executed. Readers treat those sentences as contracts. Reviewers miss them because the surrounding prose looks locally coherent. The fix is not a better prompt. The fix is a lane file that forbids the model from filling claim classes the inventory cannot support.
What an inventory can prove
An AST walk of public functions can prove names, argument lists, defaults, annotations, and docstring presence. It can also prove which symbols are exported in __all__ when that list exists. It cannot prove that a default is safe in production, that a type hint matches runtime, or that a docstring is still true. Those gaps define the human lane. Treat every unproven clause as unsigned until a person writes it beside the inventory hash.
The table below is the contract. Draft text may occupy only the left column. Anything that implies a working machine, a supported version, or a legal reading belongs on the right, even if the model could phrase it smoothly.
| Claim class | Model may draft | Human must own |
|---|---|---|
| Section outline for a how-to | Yes, from inventory headings | No, unless the outline adds a product promise |
| Parameter tables | Yes, copied from signatures | Defaults that imply safety or units |
| Procedure skeletons | Yes, with TODO(human) steps |
Any step that must succeed on a reader machine |
| Cross-links between symbols | Yes, if both symbols are public | Compatibility statements across releases |
| Install and extras | No | Exact commands, extras, and interpreter floor |
| Sample stdout / screenshots | No, unless labeled unexecuted | Any transcript presented as observed |
| Support, license, security | No | License meaning, contact path, threat claims |
1. Freeze a public-symbol inventory
The first artifact is a JSON inventory keyed by qualified name. Hash that file in git so later README diffs can point at a specific symbol set. The script below is a labeled example for a small library layout. It is not a production indexer, and it ignores re-exports, C extensions, and runtime-generated APIs.
# labeled example: inventory_extract.py
from __future__ import annotations
import ast
import hashlib
import json
from pathlib import Path
from typing import Any
ROOT = Path("src")
OUT = Path("doc_inventory.json")
def public_functions(path: Path, module: str) -> list[dict[str, Any]]:
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
rows: list[dict[str, Any]] = []
for node in tree.body:
if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
continue
if node.name.startswith("_"):
continue
args = []
for arg in node.args.args:
anno = ast.unparse(arg.annotation) if arg.annotation else None
args.append({"name": arg.arg, "annotation": anno})
rows.append(
{
"qualname": f"{module}.{node.name}",
"args": args,
"returns": ast.unparse(node.returns) if node.returns else None,
"has_docstring": ast.get_docstring(node) is not None,
"lineno": node.lineno,
}
)
return rows
def main() -> None:
items: list[dict[str, Any]] = []
for path in sorted(ROOT.rglob("*.py")):
rel = path.with_suffix("").relative_to(ROOT)
module = ".".join(rel.parts)
items.extend(public_functions(path, module))
payload = {"items": items}
text = json.dumps(payload, indent=2, sort_keys=True) + "\n"
digest = hashlib.sha256(text.encode("utf-8")).hexdigest()[:12]
payload["inventory_sha256_12"] = digest
OUT.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n", encoding="utf-8")
print(f"wrote {OUT} items={len(items)} sha={digest}")
if __name__ == "__main__":
main()
Run it from the repository root after you point ROOT at the package you actually ship.
python inventory_extract.py
python -c "import json; print(json.load(open('doc_inventory.json'))['inventory_sha256_12'])"
Store the twelve-character digest in the lane file. If the digest moves, every previously signed install sentence is stale until a human re-reads it. That rule is mechanical. It does not require judging prose quality during the inventory step.
2. Write the lane file before any draft exists
Create doc_lanes.yaml by hand. Do not ask a model to classify claim ownership, because that classification is the control plane. Keep human-owned keys empty or filled with UNSIGNED. Draft-owned keys may contain skeleton markdown that still uses placeholders.
# labeled example: doc_lanes.yaml
inventory_sha256_12: "replace-after-extract"
draft_allowed:
outline: |
## Install
TODO(human): interpreter, extras, lockfile
## Call the public entrypoints
TODO(model): parameter tables from inventory only
## Next steps
TODO(human): support path
human_owned:
python_requires: UNSIGNED
install_command: UNSIGNED
extras: UNSIGNED
observed_example_exit_code: UNSIGNED
license_summary: UNSIGNED
support_contact: UNSIGNED
forbidden_in_draft:
- "pip install"
- "conda install"
- "tested on"
- "production ready"
- "we support"
- "output:"
- "SLA"
The forbidden list is a tripwire, not a semantic parser. It catches the usual README promises that inventories cannot justify. Extend it when your corpus uses other contract verbs. Do not treat a clean scan as evidence that the how-to is true.
3. Draft only inside the allowed skeleton
Feed the model the inventory JSON, the outline, and an instruction that it may rewrite TODO(model) blocks. It must leave TODO(human) blocks untouched. It must not add install commands, version floors, or sample program output. If you need a remote drafting environment, MonkeyCode's free model access and free server option can host that narrow pass. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Keep the lane file and inventory in git either way; do not leave ownership state inside a chat transcript.
A prompt that stays inside the contract looks like the block below. It is a template, not a measured quality claim, and it names no model.
You receive doc_inventory.json and the draft_allowed.outline field.
Rewrite only sections marked TODO(model).
Use parameter names and annotations exactly as given.
Do not invent extras, Python versions, install commands, or program output.
Leave every TODO(human) line unchanged.
Return markdown for the outline only.
If the draft inserts a command the inventory never contained, discard the draft. Regenerating is cheaper than signing a false install path. The human lane is for knowledge the repository does not encode, not for cleaning model leftovers.
4. Verify lanes before the README merge
The checker below is a labeled example. It enforces three gates: inventory digest match, no forbidden phrases in the draft file, and no remaining UNSIGNED keys when --require-signed is set. It does not score readability and it does not execute examples.
# labeled example: verify_doc_lanes.py
from __future__ import annotations
import argparse
import json
import re
import sys
from pathlib import Path
try:
import yaml
except ImportError:
sys.stderr.write("install pyyaml before running this checker\n")
raise
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--inventory", default="doc_inventory.json")
parser.add_argument("--lanes", default="doc_lanes.yaml")
parser.add_argument("--draft", default="howto.draft.md")
parser.add_argument("--require-signed", action="store_true")
args = parser.parse_args()
inventory = json.loads(Path(args.inventory).read_text(encoding="utf-8"))
lanes = yaml.safe_load(Path(args.lanes).read_text(encoding="utf-8"))
draft = Path(args.draft).read_text(encoding="utf-8")
errors: list[str] = []
expected = inventory.get("inventory_sha256_12")
if lanes.get("inventory_sha256_12") != expected:
errors.append("lane digest does not match inventory")
lowered = draft.lower()
for needle in lanes.get("forbidden_in_draft", []):
if needle.lower() in lowered:
errors.append(f"forbidden phrase in draft: {needle!r}")
if re.search(r"TODO\(model\)", draft):
errors.append("unresolved TODO(model) remains in draft")
if args.require_signed:
for key, value in (lanes.get("human_owned") or {}).items():
if value == "UNSIGNED" or not str(value).strip():
errors.append(f"unsigned human key: {key}")
if errors:
sys.stderr.write("\n".join(errors) + "\n")
return 1
print("lane check passed")
return 0
if __name__ == "__main__":
raise SystemExit(main())
Wire the cheap gates into CI without the signed-key requirement. Run --require-signed only on the release branch, after a person has replaced every UNSIGNED value with a sentence they are willing to defend. That split keeps pull requests unblocked while still stopping a tagged how-to from shipping empty promises.
pip install pyyaml
python verify_doc_lanes.py
python verify_doc_lanes.py --require-signed # release branch only
5. Sign the human lane with evidence, not tone
When you replace UNSIGNED, attach a pointer that another engineer can replay. An interpreter floor should cite requires-python in packaging metadata, not a remembered hallway number. An install command should cite the extra name in pyproject.toml. A sample transcript should cite a recorded session or stay labeled unexecuted. Support text should cite the actual issue tracker or mail alias. None of those pointers are decorative. They are how the next reviewer checks the claim without trusting the prose.
If packaging metadata and the how-to disagree, the how-to is wrong. Do not “fix” the conflict by asking the model to reconcile tone. Change the human lane or change the package metadata, then re-hash the inventory if public symbols moved. Documentation that cannot name its evidence is still a draft, regardless of how finished it reads.
Limitations
The inventory is incomplete for wrappers, __getattr__ exports, and compiled modules. Forbidden-phrase scanning misses paraphrases such as “drop this into a venv” that still imply a working install. Docstrings inside the tree can be false, and the extractor will still report has_docstring: true. Free drafting environments remain capable of fluent procedure text that never ran. The lane file records ownership; it does not execute the how-to on a clean machine.
This workflow also adds merge friction. Teams that publish a one-page gist for an internal spike will spend more time on YAML than on the spike. That cost is acceptable when readers will copy commands into production shells. It is wasted when the document is a scratch note with no external contract.
Who should not use this
Skip the lane file if you are writing regulated validation packages that already require executed protocols and signed protocols under a quality system. Skip it if public symbols churn daily and no digest could stay current through a review cycle. Skip it if the document’s only audience is the author, because the control plane exists to constrain claims other people will trust. In those cases, a handwritten paragraph with no model pass is the smaller honest artifact.
For libraries that already keep packaging metadata and a small public surface, the split is enough to stop the common README failure. Keep skeletons in the draft lane. Keep install paths, support, license meaning, and observed output in the human lane. Merge only when the digest matches and the unsigned keys are gone. If you draft those skeletons on free model access, export the lane file in the same commit as the inventory so review stays in git rather than in chat history.
Top comments (0)