Verifiability Failure Cost: A Draftability Policy for AI-Generated Docs

Deciding who may draft a documentation section matters more than deciding how to review it, because the classification happens before any token is generated. A section earns model-drafting rights only when its claims can be checked mechanically and its failure cost stays low. Everything else belongs to a human, not because models are untrustworthy, but because verification cost exceeds drafting cost. Call this the draftability policy: a tiny decision rule that turns a recurring team argument into a committed configuration file.

Why whole-document drafting collapses

When a model drafts an entire document, the review bottleneck does not disappear; it simply relocates from writing to reading the finished text. Reviewers must then inspect every claim at full attention, which is exactly the workload that generated documentation was supposed to remove. The better split happens before drafting, and it is based on two boring questions about each section. Ignoring that split produces either fully unverified prose or a human who quietly rewrites everything from memory.

Two axes, coarse values

Every documentation section can be scored on two independent axes, and coarse values are enough for a defensible decision. The first axis is verifiability: can a script, a compiler, a schema, or a live example confirm what the section claims? The second axis is failure cost: if the section is wrong, does the reader lose an hour, cause an incident, or break an audit? Neither axis requires a precise number, and debate disappears once the team agrees on the values for a section.

The three draftability tiers

Tier A — model drafts, machine checks. A Tier A section has at least one mechanical check and low failure cost, so the model may draft the full text. Runnable code examples, parameter tables derived from source, and changelog entries extracted from git log belong here. A CI pipeline can guard these sections after drafting, which shrinks the human role to verifying the verifier itself.

Tier B — model drafts, human proves. A Tier B section has a mechanical check but medium failure cost, and a model may draft it only with explicit proof markers. Troubleshooting steps, migration notes, and descriptions of edge-case behavior are the usual residents of this tier. The model writes the prose, but a human must attach evidence to each risk-bearing claim before the merge.

Tier C — human only. A Tier C section has no mechanical check or carries high failure cost, and the model contributes nothing at all. Security guidance, compliance statements, deprecation timelines, and design rationale stay in Tier C because no passing test can prove them correct. Change frequency decides when to regenerate a section, but the tier decides who is allowed to write it in the first place.

An executable policy, not a meeting

The policy stops being philosophy the moment it becomes code, and a short Python script can compute the tier for every section. Most documentation repositories already contain enough metadata to run the decision without a spreadsheet or a committee. Save the following classifier at the repository root and feed it one section descriptor per line.

# draft_policy.py — compute the draftability tier for one docs section
SECTIONS = [
    {
        "id": "api/rates-overview",
        "verifiable": ["executable-example", "schema-check"],
        "failure_cost": "low",
    },
    {
        "id": "troubleshooting/auth-timeouts",
        "verifiable": ["log-pattern"],
        "failure_cost": "medium",
    },
    {
        "id": "security/token-rotation",
        "verifiable": [],
        "failure_cost": "high",
    },
]

def decide(section):
    checks = len(section.get("verifiable", []))
    cost = section.get("failure_cost", "high")
    if checks >= 1 and cost == "low":
        return ("A", "machine")
    if checks >= 1 and cost == "medium":
        return ("B", "human-with-proof")
    return ("C", "human-only")

for section in SECTIONS:
    tier, reviewer = decide(section)
    print(f"{section['id']:<32} tier {tier}  reviewer: {reviewer}")

Run the script and the output reads like a simple table, which is exactly the level of detail a reviewer needs.

$ python draft_policy.py
api/rates-overview              tier A  reviewer: machine
troubleshooting/auth-timeouts   tier B  reviewer: human-with-proof
security/token-rotation         tier C  reviewer: human-only

A nightly drafting loop with five steps

The workflow has five steps, and each one stays intentionally boring.

1. Inventory the section tree. List every section id, its audience, and the closest mechanical check that could validate its content. Sections without an obvious check keep an empty verifiable list, and that empty list is a signal rather than a defect.
2. Assign failure cost from operational history. Use what actually happened last time a section was wrong instead of guessing in the abstract. A wrong endpoint example shows up as support tickets, while a wrong rotation interval shows up as leaked credentials.
3. Commit the policy file. Version the classifier and the section descriptors in the same repository as the documentation itself. A tracked policy file beats an oral agreement about who drafts what, and future contributors can see the reasoning in git history.
4. Run the draft job on a schedule. The nightly job regenerates Tier A content, produces Tier B drafts with proof requests, and refuses Tier C completely. A free server option and free model access, such as MonkeyCode's, keep this job running without adding a line item to the infrastructure bill.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Model access cost and server cost are the two reasons such jobs get cancelled, so a genuinely free tier moves the workflow from occasional to nightly.

1. Review only the proof surface. Merge requests should touch Tier A and Tier B content, and every Tier B proof marker must contain a concrete reference. Tier C changes appear only with a named human author, which keeps the ownership trail auditable without extra tooling.

A Tier B section keeps markers that must be resolved before the diff merges.

PROOF: <commit-sha> covers the timeout retry behavior
PROOF: <operator> manual console test on <date>, region eu-west-1

Unresolved markers are easy to hunt for because the placeholder pattern is distinctive.

$ grep -rnE '^PROOF: <' docs/

Limitations and who should not use this

Limitations matter as much as the tiers, and the first one is that verifiability is not the same as truth. A passing example proves that the system matches the example, not that the example matches the intended feature. The second limitation is that Tier B still costs real human attention, and free model calls do not reduce the price of proof. The third limitation is that the policy itself can go stale, because new sections arrive without classification and old checks stop running.

Teams in regulated domains should default every section to Tier C, because a compliance claim cannot be rescued by a log pattern. Single-maintainer projects with a handful of pages should skip the policy entirely, because classification overhead will exceed drafting time. Repositories whose examples cannot run deterministically should avoid Tier A completely, since a flaky example is worse than a cautious human.

The starting point

The next time someone proposes regenerating a whole documentation set with a model, ask a different question first. Ask which sections the machine may draft and which sections a named human can actually prove. The answer to that question, not the model choice, decides whether generated docs become an asset or another stale file. Write a three-section policy file, run the classifier once, and let the tier list drive the next writing sprint.