DEV Community

Avery Lin
Avery Lin

Posted on

Obligation Stays Human: A Four-Class Consequence Ledger for Generated Docs

Generated documentation fails most often when a model writes an obligation that no human reviewed in context. Observational material can be drafted from public sources and then checked by tests, diffs, and allowlisted citations. Obligation text, paging runbooks, and irreversible procedures change what operators will do under time pressure. A consequence ledger records that split per heading so generation cannot silently cross the ownership line.

Heading outlines and citation rules still leave a gap after a page structure has been approved. A section titled overview can smuggle a must-not rule into a bullet that looks like harmless background. A section titled runbook can receive a generated rollback command that was never executed against staging. The missing control is a per-heading record of the harm that follows if a wrong sentence is obeyed.

Four consequence classes, one ownership rule

Teams can assign every documentation heading to one of four consequence classes before any model call. Observation class covers descriptions of current behavior that a reader can verify against code or logs. Reference class covers extracted tables, field lists, and flags that a generator may copy from an allowlisted schema. Obligation class covers promises, prohibitions, service objectives, and policy language that bind the shipping organization.

Action class covers commands, paging, rollback steps, and any procedure that mutates shared production state. A model may draft Observation and Reference text, while a human must own Obligation and Action text. Any mixed heading is split into two headings or upgraded to human-owned before generation starts. The ledger stores that decision so continuous integration can reject a draft that landed in the wrong class.

Class Reader harm if wrong Model may draft? Human must own?
observe Confusion, wasted time Yes, with review Optional review
reference Wrong parameter used Yes, from allowlisted sources Schema owner signs
obligate Broken promise or policy No Named owner
act Production mutation or pages No On-call owner

The table is a decision aid, not a writing style guide, and it should be versioned beside the docs tree. Class assignment depends on consequence, not on heading depth, voice, or how easy the section looks to generate. A short overview that states a paging threshold is still obligate. A long architecture narrative that never binds operators can remain observe.

Ledger format the validator can load

Keep the ledger in version control next to the documentation root rather than inside prompt text. Each row names a heading path, a class, a human owner, and the glob of files a model is allowed to touch. Generated files live under a dedicated prefix so the check can ignore handwritten narrative without hashing every paragraph.

# docs/consequence_ledger.yaml
version: 1
generated_prefix: docs/generated/
human_prefix: docs/owned/
headings:
  - path: "Services / Checkout / What it does"
    class: observe
    owner: docs-oncall
    allow_generate: true
  - path: "Services / Checkout / Environment variables"
    class: reference
    owner: checkout-api
    allow_generate: true
    source: openapi/checkout.yaml
  - path: "Services / Checkout / Availability objective"
    class: obligate
    owner: sre-checkout
    allow_generate: false
  - path: "Runbooks / Checkout / Rollback payments"
    class: act
    owner: sre-checkout
    allow_generate: false
Enter fullscreen mode Exit fullscreen mode

Heading paths must match the rendered outline, including parent titles, so renamed sections fail closed. allow_generate: false is stronger than a comment in a prompt because the validator never reads the prompt. Owners are roster identifiers, not model names, and an empty owner on obligate or act is a hard error.

Numbered workflow from outline to merge

  1. Freeze the outline as heading paths and refuse to generate until every path has a ledger row.
  2. Classify each path using the consequence table, splitting mixed headings instead of averaging their risk.
  3. Route observe and reference paths into docs/generated/ with the allowlisted source attached for reference.
  4. Leave obligate and act paths as stubs under docs/owned/ that only a named human may fill.
  5. Run the ownership validator on the merge ref and fail if generated files contain owned heading paths.
  6. Require the listed owner to acknowledge diffs that touch obligate or act files, even when a model produced nearby text.
  7. Reclassify a heading upward after any incident where operators followed generated text that should have been owned.

The sequence is deliberately boring because the failure is usually a silent class change, not a missing metaphor. Reclassification only moves toward human ownership unless a written review shows the heading no longer binds anyone. Downclassing from act to observe is a product decision and should not happen inside a generation job.

A small validator you can run locally

The script below is a proposed local check, not a claim about production incident rates. It loads the ledger, walks Markdown headings, and fails when generated files include paths classified as obligate or act. It also fails when a ledger path is missing from the tree, which catches outline drift after a rename.

#!/usr/bin/env python3
"""check_ownership.py — fail if generated docs cover human-owned headings."""
from __future__ import annotations

import pathlib
import re
import sys

import yaml

HEADING = re.compile(r"^(#{1,6})\s+(.*)\s*$")


def heading_paths(markdown: str) -> list[str]:
    stack: list[str] = []
    found: list[str] = []
    for raw in markdown.splitlines():
        match = HEADING.match(raw)
        if not match:
            continue
        level = len(match.group(1))
        title = match.group(2).strip()
        stack = stack[: level - 1] + [title]
        found.append(" / ".join(stack))
    return found


def load_ledger(path: pathlib.Path) -> dict:
    data = yaml.safe_load(path.read_text())
    if not data or "headings" not in data:
        raise SystemExit("ledger is missing headings")
    return data


def main(repo: pathlib.Path) -> int:
    ledger = load_ledger(repo / "docs/consequence_ledger.yaml")
    generated_root = repo / ledger["generated_prefix"]
    owned_paths = {
        row["path"]: row
        for row in ledger["headings"]
        if row["class"] in {"obligate", "act"} or not row.get("allow_generate", False)
    }
    errors: list[str] = []

    for row in ledger["headings"]:
        if row["class"] in {"obligate", "act"} and not row.get("owner"):
            errors.append(f"owned path lacks owner: {row['path']}")

    if generated_root.exists():
        for md in generated_root.rglob("*.md"):
            for path in heading_paths(md.read_text()):
                if path in owned_paths:
                    errors.append(f"{md}: generated text under owned path {path}")

    documented = set()
    for md in (repo / "docs").rglob("*.md"):
        documented.update(heading_paths(md.read_text()))
    for path in owned_paths:
        if path not in documented:
            errors.append(f"ledger path missing from docs tree: {path}")

    for item in errors:
        print(item, file=sys.stderr)
    return 1 if errors else 0


if __name__ == "__main__":
    sys.exit(main(pathlib.Path(".").resolve()))
Enter fullscreen mode Exit fullscreen mode

Wire the check into the same job that already lints Markdown so ownership does not depend on a chat transcript. A one-line target keeps the contract visible to people who never open the Python file.

.PHONY: docs-ownership
docs-ownership:
    python3 scripts/check_ownership.py
Enter fullscreen mode Exit fullscreen mode

Label generated files at the top with the class they are allowed to contain, which helps reviewers scan diffs without opening the ledger. The comment is documentation for humans; the YAML file remains the only source the validator trusts.

<!-- consequence-class: reference -->
# Environment variables

| Name | Source | Required |
| --- | --- | --- |
| `CHECKOUT_TIMEOUT_MS` | openapi/checkout.yaml | yes |
Enter fullscreen mode Exit fullscreen mode

Where free model access belongs in this workflow

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access is relevant only for observe and reference drafts that the ledger already marks as generatable. MonkeyCode's free server option is relevant as a place to run check_ownership.py beside existing documentation tests, without moving obligation text into the model prompt. Do not send docs/owned/ bodies to a model job, even when the job is unpaid, because ownership is a consequence rule rather than a billing rule.

A practical split looks like a generate step that receives only allowlisted sources for reference headings, then a verify step that runs the script above. If the verify step fails, the generate step is not retried against owned paths; a human edits the stub instead. That constraint is the entire method. The product does not need extra claims about unnamed models, quotas, hardware, or lasting availability for the ledger to work.

What this check does not prove

The validator does not prove that observational drafts are factually correct, only that they did not land under human-owned heading paths. It does not replace legal review, security review, or an executable example harness for command blocks that appear in owned runbooks. It will not detect an obligation written as casual prose under an observe heading if the outline was classified too loosely. Garbage classification in the ledger produces garbage enforcement, and the first incident should force an upward reclassify.

Teams should not use this approach when every sentence is already human-written and generation is not on the table. They should not use it for medical, safety, or similarly regulated procedures where even observational restatements need a specialist, because a green ownership check would look like clearance. They should not use it when no named human will accept act stubs, since an empty owner is worse than an honest undocumented runbook. Solo README files with no paging consequence gain little from a four-class ledger and should skip the ceremony.

Outline drift remains the usual operational failure. A renamed parent heading breaks path equality and fails the build, which is intended, but a copied subsection under a new parent can bypass the ledger until someone adds a row. Review the ledger in the same pull request that changes the outline, not in a later cleanup. If a generated reference table can still page someone, the heading was misclassified and belongs in act.

The core conclusion does not change after those limits. Models may draft what a reader can verify or extract; humans keep the sentences that bind the organization or change production. A four-class ledger plus a closed-failing check makes that split visible in the repository instead of hoping a prompt will remember it. If observational sections are already drafted with hosted models, the same ownership check can run on MonkeyCode's free server option beside those tests.

Top comments (0)