Stamp Generated Doc Sentences as Extracted, Inferred, or Promised

Generated library documentation fails most often when drafted sentences quietly become product promises without a named human owner. A practical control is to stamp every sentence with one provenance token before any model output is merged. Extracted claims may be drafted by a model because they map to files, signatures, tests, or packaging metadata. Promised claims must stay human-authored, because they create support obligations that no extractor can verify.

Why provenance belongs in the generation loop

Most documentation pipelines treat generation as a single pass that emits finished Markdown from a repository snapshot. That design hides a category error: extraction, inference, and commitment are not the same kind of sentence. Extraction can be checked against the tree; inference is a hypothesis; a promise is a social contract with users. If those three classes share one unmarked file, reviewers cannot tell which lines are evidence and which lines are commitments.

Three tokens with hard boundaries

Extracted claims

An extracted sentence restates a fact that already exists in a machine-readable source inside the repository. Valid sources include function signatures, CLI parsers, test names, package metadata, declared configuration keys, and export tables. The sentence should remain true if a reader opens the cited path and reads the cited symbol. If the path or symbol cannot be named, the sentence is not extracted and must not carry that token.

Inferred claims

An inferred sentence is a grouping, ordering, or likely-usage remark that is not directly present in source. Models produce these lines easily, which is why they need a review queue rather than a merge default. Inference may draft a tutorial outline or cluster related commands, but it may not invent behavior or versions. Treat every inferred line as a hypothesis that expires when the cited files change in a later commit.

Promised claims

A promised sentence tells a user what the project will do, support, guarantee, or continue to provide. Typical promised material includes compatibility windows, supported runtimes, stability labels, and samples presented as working contracts. No model should author these lines, even when the surrounding inventory was extracted cleanly from the tree. A named maintainer must write or explicitly adopt each promised sentence before the document is published.

Artifact: a provenance map and a local checker

The following artifact combines a JSON ownership map, a heading-marker convention, and a standard-library Python checker. Generation tools may fill files listed as extracted; they must refuse to write files listed as promised. Continuous integration should fail the build when a promised heading lacks a signer or when banned verbs leak. The checker is a local proposal you can run against fixtures; it does not replace reading the promised section.

Ownership map (docs/provenance.json)

{
  "version": 1,
  "files": {
    "docs/extracted.md": "extracted",
    "docs/inferred.md": "inferred",
    "docs/promised.md": "promised"
  },
  "banned_phrases": {
    "extracted": [
      "guarantee",
      "always works",
      "never fails",
      "we support",
      "production-ready",
      "sla",
      "will continue to"
    ],
    "inferred": [
      "guarantee",
      "supported until",
      "production-ready",
      "sla"
    ],
    "promised": []
  }
}

Keep extracted inventory, inferred hypotheses, and promised commitments in three files so overwrite rules stay obvious. A path filter on the extract job should list docs/promised.md as read-only, matching a CODEOWNERS entry for maintainers. If an extract job can write the promised file, the tokens in headings become decoration rather than control. The map is the contract; the Markdown comments are only the per-section evidence for that contract.

Heading markers

User-facing drafts should mark every second-level heading with a provenance comment that the checker can parse. Extracted headings need a source path; inferred headings need a reviewer state; promised headings need a signer. Paragraphs inherit the heading token until the next second-level heading, which keeps the grammar easy to lint. Do not mix tokens inside one section, because mixed sections are how commitments hide under inventory language.

## Flags <!-- provenance:extracted source:src/cli.py -->

`pack` accepts `--out` and `--force` as declared on `parse_args` in `src/cli.py`.

## Suggested reading order <!-- provenance:inferred reviewer:pending -->

New users often read flags first and then the pack example, based on adjacent heading names.

## Compatibility <!-- provenance:promised signer:pending -->

(human text only; do not generate this body)

On the default branch, reviewer=pending and signer=pending should fail the checker so unmarked hypotheses cannot publish. Working branches may keep pending until a person replaces it with a reviewer or signer identity. A model identifier is never a valid signer, including placeholders such as model, bot, or none. Forged signers are worse than a hard fail, because they look reviewed while remaining machine-authored.

Checker (scripts/check_doc_provenance.py)

The script below is a proposal. Execute it on the fixtures in this article before enabling it as a required merge job.

#!/usr/bin/env python3
"""Fail generated docs that omit provenance or let inventory files own promises."""
from __future__ import annotations

import argparse
import json
import re
import sys
from pathlib import Path

HEADING_RE = re.compile(r"^##\s+(.+?)\s*(<!--\s*(.*?)\s*-->)?\s*$")
TOKEN_RE = re.compile(
    r"provenance:(extracted|inferred|promised)"
    r"(?:\s+source:(?P<source>\S+))?"
    r"(?:\s+reviewer:(?P<reviewer>\S+))?"
    r"(?:\s+signer:(?P<signer>\S+))?"
)
FORBIDDEN_SIGNERS = {"pending", "model", "bot", "none", ""}


def iter_sections(text: str):
    current = None
    body: list[str] = []
    for line in text.splitlines():
        match = HEADING_RE.match(line)
        if match:
            if current is not None:
                yield current, "\n".join(body)
            current = {
                "title": match.group(1).strip(),
                "comment": (match.group(3) or "").strip(),
            }
            body = []
        elif current is not None:
            body.append(line)
    if current is not None:
        yield current, "\n".join(body)


def check_file(
    path: Path, expected: str, banned: list[str], repo: Path
) -> list[str]:
    errors: list[str] = []
    sections = list(iter_sections(path.read_text(encoding="utf-8")))
    if not sections:
        return [f"{path}: no second-level headings with provenance markers"]
    for section, body in sections:
        token_match = TOKEN_RE.search(section["comment"])
        if not token_match:
            errors.append(
                f"{path}: heading {section['title']!r} lacks a parseable provenance comment"
            )
            continue
        token = token_match.group(1)
        if token != expected:
            errors.append(
                f"{path}: heading {section['title']!r} is {token}, expected {expected}"
            )
        if token == "extracted":
            source = token_match.group("source")
            if not source:
                errors.append(
                    f"{path}: extracted heading {section['title']!r} missing source"
                )
            else:
                source_path = source.split(":", 1)[0]
                if not (repo / source_path).exists():
                    errors.append(
                        f"{path}: extracted source {source_path!r} does not exist"
                    )
        if token == "inferred":
            reviewer = token_match.group("reviewer") or ""
            if reviewer in FORBIDDEN_SIGNERS:
                errors.append(
                    f"{path}: inferred heading {section['title']!r} reviewer={reviewer!r}"
                )
        if token == "promised":
            signer = token_match.group("signer") or ""
            if signer in FORBIDDEN_SIGNERS:
                errors.append(
                    f"{path}: promised heading {section['title']!r} missing a human signer"
                )
        lowered = body.lower()
        for phrase in banned:
            if phrase.lower() in lowered:
                errors.append(
                    f"{path}: banned phrase {phrase!r} under {section['title']!r} ({token})"
                )
    return errors


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--map", type=Path, default=Path("docs/provenance.json"))
    parser.add_argument("--repo", type=Path, default=Path("."))
    args = parser.parse_args()
    spec = json.loads(args.map.read_text(encoding="utf-8"))
    errors: list[str] = []
    files = spec.get("files", {})
    banned_all = spec.get("banned_phrases", {})
    for rel, token in files.items():
        path = args.repo / rel
        if not path.is_file():
            errors.append(f"missing mapped file: {rel}")
            continue
        errors.extend(
            check_file(path, token, banned_all.get(token, []), args.repo)
        )
    if errors:
        print("provenance check failed:")
        for item in errors:
            print(f"  - {item}")
        return 1
    print(f"provenance check passed for {len(files)} file(s)")
    return 0


if __name__ == "__main__":
    sys.exit(main())

Decision table

User-facing claim	Token	Allowed drafter	Merge rule
Restates a flag, type, or path	extracted	model after a deterministic dump	source path must exist
Groups commands into a learning order	inferred	model into a review file	named reviewer, not pending
Names supported runtimes or stability	promised	human only	signer required
Pasteable sample implied to work	promised	human only	signer required
Restated test node id	extracted	model after collecting tests	source path must exist

When a row is ambiguous, default to promised rather than extracted, because false support promises cost more than missing inventory. A pasteable example is promised whenever the surrounding text implies that the sample will work for a user. A restated function signature is extracted only when the module path in the marker still exists in HEAD. These three rules catch the common failure mode in which a model writes “always works” beside a real flag list.

Fixture commands

The commands below create a passing fixture and then a failing fixture so the checker can be evaluated without a real library. They are a proposal, not a recorded run from a production repository, and they use only the Python standard library. If the failing run does not mention the banned phrase and the missing signer, the script was not executed as shown. Fix the script before you wire it into any merge job, because a silent pass is more dangerous than a noisy fail.

mkdir -p docs scripts src
cat > src/cli.py << 'EOF'
def parse_args():
    return {"out": str, "force": bool}
EOF

cat > docs/extracted.md << 'EOF'
## Flags <!-- provenance:extracted source:src/cli.py -->

`pack` accepts `--out` and `--force` as declared on `parse_args` in `src/cli.py`.
EOF

cat > docs/inferred.md << 'EOF'
## Suggested reading order <!-- provenance:inferred reviewer:docs-oncall -->

Readers who already know the flag names can skip to the pack example next.
EOF

cat > docs/promised.md << 'EOF'
## Compatibility <!-- provenance:promised signer:maintainer -->

Python 3.11 and 3.12 are the runtimes this package tests in CI on the default branch.
EOF

python3 scripts/check_doc_provenance.py --map docs/provenance.json --repo .
# expected labeled output if the map and files match:
# provenance check passed for 3 file(s)

printf '\nwe support every runtime\n' >> docs/extracted.md
sed -i.bak 's/signer:maintainer/signer:pending/' docs/promised.md
python3 scripts/check_doc_provenance.py --map docs/provenance.json --repo .
# expected labeled output:
# provenance check failed:
#   - docs/extracted.md: banned phrase 'we support' under 'Flags' (extracted)
#   - docs/promised.md: promised heading 'Compatibility' missing a human signer

Protect the promised file after the fixture is understood, using path ownership rather than a prompt instruction.

# CODEOWNERS
docs/promised.md @your-org/maintainers

A six-step generation protocol

Use the sequence below as a generation protocol rather than as a one-shot prompt against the whole README.

Step 1 — List sources that can prove a claim

List the files that can prove a claim: parsers, public modules, tests, package manifests, and generated JSON schemas. Record each source as a glob in the provenance map so later extract jobs cannot wander into narrative files. Do not ask a model to discover sources by crawling the entire tree, because that crawl produces inferred paths. If a candidate source is not checked in, treat every sentence drawn from it as inferred until the file lands.

Step 2 — Freeze promises as empty human stubs

Create a promised Markdown file that contains headings and signer fields, but no model-generated body text at all. Protect that file with a CODEOWNERS rule or a path filter in the extract job so generation cannot overwrite it. Humans fill compatibility, support windows, and contract examples in a later step that is not part of generation. Until a signer field is non-empty, continuous integration should treat the whole document set as still unpublished.

Step 3 — Dump inventory, then draft extracted sentences

Run a deterministic extractor first: dump command names, flags, return types, and test node ids into YAML or JSON. Only after that dump exists should a model turn inventory rows into sentences, each citing a source path. Every generated sentence must start under an extracted heading and must include a source citation in the marker. Delete any sentence the extractor cannot cite; do not demote it silently into the promised file.

Step 4 — Park inferences in a review queue

If the model wants to add sequencing, clustering, or typical-usage language, write those lines into inferred.md only. Require a human to accept or delete each inferred paragraph before it can be copied into the user-facing document. Rejected inferences are useful failure data: they show where the model tried to invent a workflow you do not own. Do not auto-merge inferred text, because a later extract pass will not know that the hypothesis has gone stale.

Step 5 — Human-adopt promised sentences, then assemble the view

A maintainer writes promised sentences in the protected file, including any example that users will treat as a contract. Copy-paste samples that imply the command works for users are promises, even when the flags themselves were extracted. Assemble the published README from the three files only after the checker reports zero unmarked blocks and zero leaks. The published file may omit the tokens if you keep the three sources as the canonical tree and generate the view.

Step 6 — Invalidate extracts without rewriting commitments

When public signatures, CLI flags, or packaging metadata change, invalidate extracted sentences that cited the old paths. Inferred paragraphs that referred to those paths return to the review queue rather than remaining in the published view. Promised paragraphs stay until a human edits them; extraction must never refresh a support promise because a flag was renamed. This split is the entire point of the protocol: inventory churn must not rewrite support language by accident.

Where a hosted extract pass fits

Some teams run the extract-to-prose pass on a hosted drafting service instead of keeping a local inference process. MonkeyCode offers free model access and a free server option that can host that extract-only drafting pass. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Use that hosted pass only for extracted inventory; keep the promised file on a machine the model cannot write.

The ownership map still applies if you never use a hosted service and run extraction as ordinary repository scripts. Hosted drafting does not change which sentences are promises, and it does not make a signer field optional. If the extract job can see docs/promised.md as writable, move that job’s output path before you tune prompts. Prompt text is a weak control compared with file permissions and the checker.

What this protocol does not prove

The checker does not prove that an extracted sentence is a correct reading of the cited source file. It only proves that a token exists, that promised headings name a signer, and that banned verbs stayed out of inventory. Syntax dumps can still be wrong, tests can name behavior that public APIs do not expose, and markers can be forged. Reviewers still need to sample extracted paragraphs against the tree, especially after a large rename or packaging change.

Who should not use this approach

Skip this protocol if the document is a legal notice, a security advisory, or a marketing page with slogan constraints. Skip it if no human will read inferred.md, because the queue becomes a second unmarked README with extra comments. Skip it for throwaway notes that will never be published, where the cost of markers exceeds the cost of a bad sentence. Do not use a model to populate signer fields; a forged signer is worse than an unsigned promised heading that fails CI.

Start with the checker on one library README and expand the map only after the first promised section has a signer. The useful outcome is not more generated prose; it is a repository where every user-facing claim has a named owner. Inventory can move at the speed of extraction; promises should move at the speed of a maintainer who will still be around.