DEV Community

Avery Lin
Avery Lin

Posted on

Docs Generation Needs a Handoff Contract, Not a Better Prompt

AI documentation tools can draft accurate reference prose from code, but the boundary between machine-drafted and human-owned content is a contract, not a style preference. A prompt that produces clean prose on Monday will produce confident nonsense on Tuesday, because the model has no persistent memory of your invariants or your audience. The fix is not a longer system prompt; it is a workflow that separates mechanically verifiable claims from claims that require human judgment.

Generated code fails loudly at runtime, while generated documentation fails quietly in a design review six months later. A wrong parameter table breaks a build immediately, but a wrong rationale becomes the basis for a future decision that nobody can trace. This asymmetry is why documentation generation deserves the same discipline as code review: a reproducible pipeline, a small review surface, and an explicit owner for every claim.

The Handoff Contract

The first step is to define what the model may draft and what a human must own. For content that can be checked against the source tree, the model is reliable: API signatures, parameter tables, type annotations, default values, and changelog bullets from commit messages. The model is not reliable for content that states intent: design rationale, rejected alternatives, security decisions, deprecation promises, and compatibility guarantees. Those claims have no mechanical ground truth, so they require a named human owner.

Artifact Model draft Human owns Verification
API parameter table Yes Type and default accuracy AST extraction against source
Changelog entry Yes Severity and target audience Commit message review
Migration note Draft Compatibility promise Manual sign-off
Design rationale No Yes Review meeting
Security note No Yes Threat model review

The table above is the contract that the pipeline enforces. Every generated document should carry a manifest that classifies each claim as verifiable or judgment. The pipeline should refuse to merge until each judgment claim has an approver.

A Reproducible Handoff Pipeline

The artifact below is a small extractor that turns a Python module into a documentation draft plus a claim manifest. It does not generate prose; it generates the structure that a model can fill and the checklist that a human must sign. The script uses only the standard library, so it runs in any CI environment without extra dependencies.

#!/usr/bin/env python3
"""Emit a doc draft and a claim manifest for human sign-off."""
import ast
import json
import sys
from pathlib import Path

def extract_public_api(path: Path):
    tree = ast.parse(path.read_text())
    nodes = []
    for node in ast.walk(tree):
        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
            if not node.name.startswith("_"):
                nodes.append(node)
    return nodes

def emit_draft(nodes):
    lines = ["# API Reference Draft", ""]
    for node in nodes:
        lines.append(f"## {node.name}")
        lines.append("")
        lines.append("_Draft: verify behavior before merge._")
        lines.append("")
        if isinstance(node, ast.FunctionDef):
            args = [a.arg for a in node.args.args]
            lines.append(f"Parameters: {', '.join(args) or 'none'}")
        lines.append("")
    return "\n".join(lines)

def emit_claims(nodes):
    claims = []
    for node in nodes:
        claims.append({
            "claim": f"{node.name} is public and extracted from source",
            "kind": "verifiable",
            "check": "AST extraction",
        })
        claims.append({
            "claim": f"{node.name} behavior matches the generated text",
            "kind": "judgment",
            "owner": "unassigned",
        })
    return claims

if __name__ == "__main__":
    src = Path(sys.argv[1])
    api = extract_public_api(src)
    Path("docs").mkdir(exist_ok=True)
    Path("docs/draft.md").write_text(emit_draft(api))
    Path("docs/claims.json").write_text(
        json.dumps(emit_claims(api), indent=2)
    )
Enter fullscreen mode Exit fullscreen mode

Run it on any module and you get two files: a draft with placeholders and a JSON manifest with exactly two claims per symbol. The model fills the draft while the human approves the manifest, and the pipeline then enforces the contract in CI. The manifest is the review surface, so the reviewer reads a dozen claims instead of five hundred lines of prose.

The Workflow in Five Steps

The workflow runs in five steps, and each step has a single owner. The first three steps are automated; the last two require a human decision. Keeping the steps separate prevents the model from silently owning a claim it cannot verify.

  1. Extract. Run the script on every changed module in a pull request and commit the resulting skeleton to the docs directory.
  2. Generate. Send the skeleton to a model with a constrained instruction: fill the parameter tables from the signatures, and do not add rationale or promises.
  3. Merge. The model's output replaces the placeholders in docs/draft.md, while the manifest stays untouched.
  4. Verify. A CI check fails the build when any judgment claim still has "owner": "unassigned".
  5. Approve. A human reviews only the judgment claims, which is a much smaller surface than the full prose.

The verification step is a short script that any CI runner can execute:

python docgen_handoff.py src/payments.py
python - <<'PY'
import json, sys
claims = json.load(open("docs/claims.json"))
unowned = [c for c in claims if c["kind"] == "judgment" and c["owner"] == "unassigned"]
if unowned:
    print("Unapproved judgment claims:", [c["claim"] for c in unowned])
    sys.exit(1)
PY
Enter fullscreen mode Exit fullscreen mode

The generation step needs a model endpoint, and metered APIs make that step expensive when you run it on every pull request. MonkeyCode's free model access and free server option cover the generation step without a per-token bill, which makes per-PR doc regeneration practical in the first place. The workflow itself is model-agnostic, so the contract survives any change in the underlying provider. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

What the Pipeline Cannot Verify

The extractor knows structure, not semantics, so it cannot confirm that a docstring matches runtime behavior. The manifest catches missing approvals, but it cannot catch a wrong claim that a human approves in a hurry. Free access tiers also change over time, so check the current terms before you build a pipeline that depends on them.

Who Should Not Use This Workflow

Teams without a human reviewer for judgment claims should not adopt this pipeline, because the manifest becomes a rubber stamp. Projects where documentation is the product, such as SDK guides or compliance manuals, need a voice and a provenance model that a generated draft cannot provide. Regulated environments that require audit trails for every published statement should keep generation out of the critical path entirely.

The Measure That Matters

Run the extractor on one module and count how many judgment claims survive review with edits. If the number is high, your documentation debt lives in the rationale, not the prose, and no prompt will fix it. If the number is low, the model is doing the mechanical work and your reviewers are doing the thinking. That is the division of labor this contract was designed to enforce.

Top comments (0)