Assumption Tickets: A Claim Ledger for AI-Drafted Docs

A release branch merged on a Thursday. The generated changelog said the new /replay endpoint was idempotent. The handler was not. Support spent the weekend explaining duplicate charges that the docs had promised could not happen.

That failure is not a style problem. It is a claim problem. Generated documentation does not usually fail by being empty. It fails by stating something the repo never proved.

This article is a workflow, not a manifesto. It treats every sentence a model writes as an assumption ticket until a human closes it. The artifact is a markdown claim ledger, a decision table, and a checker you can run in CI.

The examples below are labeled as a proposed pipeline. They are not production telemetry from a named company.

Generated docs fail differently than generated code

Code has compilers, tests, and type checkers. Docs have readers who trust the first confident sentence. A model can invent a flag, a default, or a guarantee, and the page still renders.

Cheap generation also changes the failure mode. When drafts are free, teams publish more pages than they can reread. Stale examples survive because nobody owns the claim inside the paragraph.

Three classes show up repeatedly:

• Invented surface area: flags, env vars, and status codes that do not exist in the binary or the OpenAPI file.
• Unproven contracts: idempotency, ordering, at-least-once delivery, “safe to retry.”
• Borrowed architecture: component diagrams that describe the system the model expected, not the one in cmd/.

If you only ask a model to “write the README,” you are asking it to hide those classes in fluent prose.

Own claims, not just files

File-level ownership is too coarse. A README can contain an install snippet a model may draft and a security warning a human must write. Splitting “AI files” from “human files” still lets unverified sentences into the human file via copy-paste.

A more precise unit is the claim:

1. What is being asserted.
2. Where the evidence lives (code path, test, OpenAPI, runbook).
3. Who is allowed to mark it verified.
4. Whether the page may ship while the claim is still assumed.

Until those four fields exist, “the model drafted it” and “we published it” are the same event.

Claim states that a checker can see

Use four states. Do not invent more until the first four are enforced.

State	Meaning	May ship in public docs?
drafted	Model prose, no evidence pointer	No
assumed	Explicit guess, evidence missing	No
verified	Evidence pointer checked by a human	Yes
human-owned	Must be written or signed by a person	Only after a named owner

assumed is not a shame label. It is the honest default. The point is to keep assumed sentences out of the published tree, not to pretend the model never guesses.

Markdown convention

Keep the ledger next to the prose so diffs stay reviewable. HTML comments survive most static-site pipelines and stay invisible to readers.

<!-- claim:id=replay-idempotent type=contract status=assumed owner=unassigned evidence=none -->
`POST /replay` is idempotent for 24 hours.
<!-- /claim -->

<!-- claim:id=replay-flag type=surface status=drafted owner=unassigned evidence=cmd/replay/main.go -->
Enable replay with `--replay-window`.
<!-- /claim -->

<!-- claim:id=replay-disclosure type=policy status=human-owned owner=security evidence=SECURITY.md -->
Report replay billing bugs to [e-mail].
<!-- /claim -->

Rules that keep this from becoming decoration:

• One claim per independently false sentence. Do not wrap a whole section.
• evidence must be a repo path, test name, or schema pointer. “Looks right” is not evidence.
• human-owned claims cannot be flipped to verified by a model. A person changes the status.
• Public paths (docs/public/**, README.md) may contain only verified or closed human-owned claims.

Decision table: what a model may draft

The table is the policy. Put it in docs/CLAIM_POLICY.md and review it like an API.

Doc surface	Model may draft	Human must own	Evidence the checker should expect
Install commands	Skeleton from Makefile / package.json	Exact flags, versions, privileged steps	Script path that actually runs
API field list	Names and types from OpenAPI / proto	Meaning, nullability, error mapping	Schema file + golden response test
Examples	Happy-path sketch labeled “unverified”	Copy-paste correctness	Executable snippet or CI job
Changelog	Touched paths from git diff	User-facing impact, breaking changes	Diff + owner sign-off
Architecture	Inventory of packages and queues	Tradeoffs, rejected alternatives	ADR with a named author
SECURITY.md / threat notes	Headings only	Disclosure process, severity, contacts	Security owner
SLOs and “safe to retry”	Nothing	All retry and durability language	Load test or runbook

If a row has no evidence column, it does not belong in a generated draft. Delete the sentence.

Artifact: a claim checker

The following script is a starting checker, not a parser for every Markdown dialect. Save it as tools/check_doc_claims.py and run it against the paths you publish.

#!/usr/bin/env python3
"""Fail if public docs still contain drafted or assumed claims."""
from __future__ import annotations

import pathlib
import re
import sys

CLAIM_RE = re.compile(
    r"<!--\s*claim:(?P<attrs>.*?)-->"
    r"(?P<body>.*?)"
    r"<!--\s*/claim\s*-->",
    re.DOTALL,
)
ATTR_RE = re.compile(r"(\w+)=([^\s]+)")

PUBLIC_GLOBS = ["README.md", "docs/public/**/*.md"]
BLOCKED_IN_PUBLIC = {"drafted", "assumed"}
REQUIRED = {"id", "type", "status", "owner", "evidence"}


def iter_files(root: pathlib.Path) -> list[pathlib.Path]:
    files: list[pathlib.Path] = []
    for pattern in PUBLIC_GLOBS:
        files.extend(root.glob(pattern))
    return sorted({path for path in files if path.is_file()})


def parse_attrs(raw: str) -> dict[str, str]:
    return dict(ATTR_RE.findall(raw))


def check_file(path: pathlib.Path) -> list[str]:
    text = path.read_text(encoding="utf-8")
    errors: list[str] = []
    found = list(CLAIM_RE.finditer(text))
    if not found and path.suffix == ".md":
        errors.append(f"{path}: no claim markers in a public doc")
        return errors

    seen_ids: set[str] = set()
    for match in found:
        attrs = parse_attrs(match.group("attrs"))
        missing = REQUIRED - attrs.keys()
        loc = f"{path}#{attrs.get('id', 'missing-id')}"
        if missing:
            errors.append(f"{loc}: missing fields {sorted(missing)}")
            continue
        if attrs["id"] in seen_ids:
            errors.append(f"{loc}: duplicate id")
        seen_ids.add(attrs["id"])
        if attrs["status"] in BLOCKED_IN_PUBLIC:
            errors.append(
                f"{loc}: status={attrs['status']} cannot ship; "
                f"evidence={attrs['evidence']}"
            )
        if attrs["status"] == "verified" and attrs["evidence"] in {"none", "n/a", "-"}:
            errors.append(f"{loc}: verified claims need a real evidence pointer")
        if attrs["status"] == "human-owned" and attrs["owner"] in {"unassigned", "model"}:
            errors.append(f"{loc}: human-owned claim has no person")
        body = match.group("body").strip()
        if not body:
            errors.append(f"{loc}: empty claim body")
    return errors


def main() -> int:
    root = pathlib.Path(".")
    errors: list[str] = []
    for path in iter_files(root):
        errors.extend(check_file(path))
    if errors:
        print("claim ledger failed:")
        for item in errors:
            print(f"  - {item}")
        return 1
    print(f"claim ledger ok: {len(iter_files(root))} public file(s)")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Wire it next to the tests you already trust:

python3 tools/check_doc_claims.py

# .github/workflows/doc-claims.yml
name: doc-claims
on:
  pull_request:
    paths: ["README.md", "docs/**", "tools/check_doc_claims.py"]
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: python3 tools/check_doc_claims.py

The checker is intentionally strict about missing markers on public pages. A page with no claims is how invented sentences sneak back in.

Drafting step without pretending the model is a reviewer

Keep generation on a branch. Feed the model only evidence files: the diff, the OpenAPI snippet, the test names. Ask for claim-wrapped markdown, not a finished README.

A prompt shape that stays honest:

Draft claim-wrapped markdown for docs/public/replay.md.
Use only these inputs: openapi.yaml#/paths/~1replay, cmd/replay/main.go,
and testdata/replay/*.json.
Every sentence must be a claim block.
If the inputs do not prove a contract, set status=assumed and evidence=none.
Never set status=verified.
Never draft SECURITY.md body copy.

That last line matters. Models will fill policy pages if you leave a hole.

If you need a drafting workspace that is not your production cluster, MonkeyCode’s free model access and free server option can host that isolated generate-and-wrap step. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The checker still runs in your CI. Do not treat the drafting box as an evidence source.

Do not paste secrets into any drafting workspace. Pass paths, schemas, and redacted diffs.

Human close-out, in order

A reviewer should not start with tone. Start with tickets.

1. List every assumed and drafted claim in the PR.
2. For each one, open the evidence path or delete the sentence.
3. Flip to verified only after running the command or reading the test.
4. Assign human-owned rows to a named person. team is not a person.
5. Re-run tools/check_doc_claims.py before merge.

A useful review comment is short: claim replay-idempotent is assumed; handler has no dedupe key. That is cheaper than debating whether the prose “sounds accurate.”

Test plan for the workflow itself

Treat the ledger like production code. A minimal plan:

• Fixture A: README with one assumed contract. Checker must exit 1.
• Fixture B: same README with verified and evidence=internal/replay/idempotency_test.go. Checker must exit 0 only if that file exists.
• Fixture C: SECURITY.md body drafted by the model with status=drafted. Checker must exit 1 even if the prose is careful.
• Fixture D: duplicate id values. Checker must exit 1.
• Fixture E: public page with zero claim markers. Checker must exit 1.

If you skip Fixture E, authors will “forget” to wrap new paragraphs.

Limitations

This workflow does not prove the system. It only prevents unmarked guesses from looking like documentation.

Known gaps:

• HTML comments can be stripped by some MDX pipelines. If your renderer drops comments, move the ledger to sidecar YAML and keep stable anchors in the prose.
• The regex parser will break on nested comments or claim tags inside code fences. Keep claims outside fences.
• verified is still a human judgment. A wrong evidence path will satisfy the checker.
• Generated diagrams and screenshots are out of scope. They need a different hash-and-compare job.
• Multi-language docs multiply tickets. Translate after verification, not before.

The method also adds latency. That is the cost of not shipping an invented --replay-window.

Who should not use this

Skip the ledger if you are writing a personal gist, a spike, or an internal scratchpad that will never be linked from support. The ceremony is for pages other people will treat as contracts.

Skip it if nobody will run the checker. Unenforced markers become folklore faster than unmarked prose.

Skip it if legal or security docs must come from counsel templates. A model should not draft those bodies at all, even as assumed.

What changes after a week of using it

Public docs get shorter. That is a signal, not a regression. Sentences without evidence disappear instead of being polished.

Changelogs stop claiming behavior. They list files, then a human writes the user-facing line.

The model still drafts. It just stops being allowed to close its own tickets. That single restriction is the difference between a documentation generator and a documentation liability.