DEV Community

Avery Lin
Avery Lin

Posted on

Reject Model Diffs That Cross Human-Owned Documentation Paths

Core conclusion: a reviewed ownership manifest must list every documentation path as model-draft or human-own. Continuous integration should reject every model-driven pull request that edits any path marked human-own. Human-owned files cover deprecation calendars, support windows, incident language, and contractual limits the spec cannot prove. The sections below give a JSON manifest, a Python gate, and a decision table for review.

Path permits fail closed when the generator selects the wrong file

A paragraph-level classifier still misses any model that quietly rewrites an entire support policy file overnight. Path permits fail closed when the generator touches the wrong file, which is cheaper than debating every sentence. Mixed files remain a hazard, so keep model-draft content inside dedicated generated directories and nowhere else. Treat this gate as a filesystem ACL for documentation, not as a substitute for later claim review.

Decide rewrite permission before the first prompt is sent

Teams often prompt a model against the whole docs/ tree and then hope reviewers will notice overreach. That sequence inverts the control flow, because the dangerous write already exists in the branch before anyone reads it. The safer order is inventory, classify, generate into allowed paths, then verify the diff against the manifest. The classification step is a human review of paths, not a model self-assessment of its own restraint.

The list below is a proposal for public HTTP APIs that already keep a reviewed OpenAPI document in git. It is not permission to invent product claims that the specification and fixtures do not contain. Each allowed file still needs a later mechanical check against the OpenAPI Specification. Path permission only answers which files a generator may touch.

What a model may draft inside model-draft paths

  1. Operation indexes. A model may list operationId values and HTTP methods that already exist on reviewed paths objects. It must not add endpoints that appear only in training data or in an uncommitted sketch.
  2. Parameter tables. A model may render names, locations, and required flags copied from parameters and requestBody schemas. Default values belong here only when the reviewed document actually declares those defaults.
  3. Documented status codes. A model may list status codes that already appear under responses for the same operation. It must not invent 429 or 503 behavior that the spec and tests do not describe.
  4. Fixture-backed examples. A model may copy request or response bodies that already live as fixtures beside contract tests. If a fixture is absent, the example slot stays empty rather than becoming plausible JSON.
  5. Resolvable $ref maps. A model may emit intra-spec links whose targets resolve in the current commit. Broken or cross-product links stay out of the generated file until a human adds them.

What a human must own on human-own paths

  1. Sunset language. Deprecation dates, migration deadlines, and “will be removed on” sentences need a named owner and a dated source. A generator has no evidence for a calendar the repository does not already store.
  2. Support and availability. Regional hours, severity definitions, and channel lists are operational facts, not schema facts. Putting them in docs/generated/ invites a model to smooth over an outage window.
  3. Contractual limits. Quotas, latency promises, and indemnifying adjectives such as “guaranteed” or “always” stay in human-own files. OpenAPI can describe a field; it cannot sign a service level.
  4. Launch status. Words such as GA, stable, beta, and unsupported are release-process outputs. The model may repeat a status only after a human file already records that status for the same operation.
  5. Incident and security narrative. Postmortem links, threat-model summaries, and “why this auth scheme” essays remain human-authored. Generated reference pages may point to those essays without rewriting them.

Unknown paths default to deny. A new markdown file that is missing from the manifest is not an implied model-draft file. Reviewers add a rule first, then allow generation, which keeps the ACL from rotting into a suggestion list.

Artifact: a path-ownership manifest the checker can parse

The following JSON is a proposal you can adapt; it is not an audit of a named company or a measured production rollout. Commit it as docs/ownership.json beside the documentation tree so the permits version with the files they govern. Owners are roles, not marketing claims, and source is a repository path the generator is allowed to read.

{
  "version": 1,
  "model_pipeline_label": "model",
  "deny_unknown_paths": true,
  "generated_root": "docs/generated/",
  "human_root": "docs/human/",
  "rules": [
    {
      "path": "docs/generated/openapi-reference.md",
      "permit": "model-draft",
      "source": "openapi.yaml"
    },
    {
      "path": "docs/generated/parameter-tables.md",
      "permit": "model-draft",
      "source": "openapi.yaml"
    },
    {
      "path": "docs/generated/status-index.md",
      "permit": "model-draft",
      "source": "openapi.yaml"
    },
    {
      "path": "docs/human/deprecations.md",
      "permit": "human-own",
      "owner": "api-steward"
    },
    {
      "path": "docs/human/support-windows.md",
      "permit": "human-own",
      "owner": "api-steward"
    },
    {
      "path": "docs/human/sla.md",
      "permit": "human-own",
      "owner": "legal-review"
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Keep generated files and human files in separate directories so a single markdown document cannot mix permits. If a page needs both a compiled table and a sunset date, split it into two files and link them. Directory split is boring on purpose: reviewers can see overreach from the path list alone.

Numbered workflow

  1. Inventory the tree. From a clean checkout, list every documentation path the generator could reach, including includes and generated stubs. Record the command output in the pull request that introduces the manifest, so missing files are visible.
git ls-files 'docs/**/*.md' 'docs/**/*.mdx' | sort > /tmp/doc-paths.txt
wc -l /tmp/doc-paths.txt
Enter fullscreen mode Exit fullscreen mode
  1. Classify each path once. For every line in that inventory, choose model-draft or human-own using the lists above, not using model output. If a file is mixed, split it before the first generation run rather than tagging it as both.

  2. Bind sources for model-draft files. Each model-draft rule names a repository source such as openapi.yaml or tests/fixtures/. A prompt that needs a file outside that source list is out of scope for the pipeline.

  3. Draft only into allowed paths. Run generation in a working tree that already contains the manifest, and pass the allowed path list into the job as an explicit argument. A scratch environment can draft only the paths your manifest marks as model-draft, without touching human-own files. MonkeyCode's free model access and free server option can host that isolated drafting job when you want one. Disclosure: This article was prepared as part of MonkeyCode's product outreach, and the product is mentioned only as an optional drafting host. The ownership gate still runs in ordinary CI and does not depend on any particular drafting host.

# Proposal: pass an allow-list file; do not glob all of docs/
cut -d'"' -f4 docs/ownership.json | grep '^docs/generated/' > /tmp/allow.txt
# generator_cmd is a placeholder for your local or hosted drafter
# generator_cmd --allow-file /tmp/allow.txt --source openapi.yaml
Enter fullscreen mode Exit fullscreen mode
  1. Label the pipeline on the merge request. Set DOC_PIPELINE=model in CI for generator branches, and leave it unset for steward branches. The checker below reads that label; without it, even model-draft files require a human path-by-path ack.

  2. Fail on crossing diffs, then review remainder. Run the checker against git diff --name-only before reviewers read prose. Human-own edits continue in a separate pull request that cannot carry the DOC_PIPELINE=model label.

Artifact: a standard-library checker for the merge diff

The script is a proposal. It uses only the Python standard library so a CI image without extra packages can still fail closed. Save it as scripts/check_doc_ownership.py and invoke it after the generator step, not instead of human review of remaining claims.

#!/usr/bin/env python3
"""Fail when a model pipeline edits human-own or unknown doc paths."""
from __future__ import annotations

import json
import os
import subprocess
import sys
from pathlib import Path

MANIFEST = Path("docs/ownership.json")


def git_changed_files(base: str) -> list[str]:
    out = subprocess.check_output(
        ["git", "diff", "--name-only", f"{base}...HEAD"],
        text=True,
    )
    return [line.strip() for line in out.splitlines() if line.strip()]


def load_rules() -> dict[str, dict]:
    data = json.loads(MANIFEST.read_text())
    rules = {}
    for row in data["rules"]:
        rules[row["path"]] = row
    return data, rules


def main() -> int:
    base = os.environ.get("DOC_OWNERSHIP_BASE", "origin/main")
    pipeline = os.environ.get("DOC_PIPELINE", "")
    data, rules = load_rules()
    changed = git_changed_files(base)
    doc_changed = [
        path for path in changed if path.startswith("docs/")
    ]
    errors = []
    for path in doc_changed:
        rule = rules.get(path)
        if rule is None and data.get("deny_unknown_paths", True):
            errors.append(f"unknown doc path (deny): {path}")
            continue
        if rule is None:
            continue
        if pipeline == data.get("model_pipeline_label") and rule["permit"] == "human-own":
            errors.append(
                f"model pipeline touched human-own path {path} "
                f"(owner={rule.get('owner', 'unassigned')})"
            )
        if rule["permit"] not in {"model-draft", "human-own"}:
            errors.append(f"invalid permit on {path}: {rule['permit']}")
    if pipeline == data.get("model_pipeline_label"):
        generated_root = data.get("generated_root", "docs/generated/")
        human_root = data.get("human_root", "docs/human/")
        for path in doc_changed:
            if path.startswith(human_root):
                errors.append(f"model pipeline wrote under {human_root}: {path}")
            if path.startswith("docs/") and not path.startswith(generated_root) and path != str(MANIFEST):
                if path not in rules or rules[path]["permit"] != "model-draft":
                    errors.append(f"model pipeline wrote outside generated_root: {path}")
    if errors:
        print("doc ownership check failed:")
        for item in errors:
            print(f"  - {item}")
        return 1
    print(f"doc ownership check passed for {len(doc_changed)} doc path(s)")
    return 0


if __name__ == "__main__":
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode

Wire it with a short recipe that also prints the crossing files for reviewers. Keep the base ref explicit so forked workflows do not silently compare against an empty tree.

export DOC_PIPELINE=model
export DOC_OWNERSHIP_BASE=origin/main
python3 scripts/check_doc_ownership.py
git diff --name-only origin/main...HEAD -- docs/
Enter fullscreen mode Exit fullscreen mode

Decision table for review comments

Use the table during review instead of arguing about tone. The CI action column is the only automated output; the evidence column is what a human still owes for model-draft files that passed the path gate.

Fragment Permit Required evidence in-repo Model pipeline CI
Operation index from paths model-draft Reviewed openapi.yaml on the same commit Allow if path listed
Parameter table from schemas model-draft parameters / requestBody objects Allow if path listed
Example body model-draft Fixture file that tests already load Allow if path listed
$ref link list model-draft Targets that resolve in this commit Allow if path listed
Deprecation or sunset date human-own Product calendar or ticket id in the human file Fail
Support hours or regions human-own Steward-signed human file Fail
Quota, latency, or “guaranteed” human-own Legal or SRE-signed human file Fail
GA / beta / unsupported human-own Launch record in the human file Fail
Any docs/ path missing from the manifest deny New rule merged first Fail

Path permission is necessary and not sufficient. A model can still invent a sentence inside an allowed generated file, which is why fixture equality and spec compilation remain separate jobs. Those jobs are out of scope here; this article only stops the generator from rewriting the wrong files.

Limitations

The gate cannot see intent inside a permitted file, so a model-draft page can still over-claim unless another checker compares it to OpenAPI and fixtures. Path granularity fails when teams insist on one long markdown book with mixed guarantees and tables. Manifest rot is real: a renamed human file that nobody reclassifies becomes an unknown path and fails CI, which is safer than a silent allow. The DOC_PIPELINE label is an honor system unless your CI platform injects it from a protected workflow, so do not let feature branches set the label locally and skip the remote job. This proposal does not measure generator quality, latency, or cost, and it does not assign permanence to any drafting host.

Who should not use this approach

Do not use path permits as the only control when counsel must approve every sentence, including compiled tables, because a filesystem ACL will not satisfy that review. Skip it for essay-only sites that have no OpenAPI or fixture source of truth, since model-draft would be an empty set. Avoid it when the documentation tree cannot be split, and a single generated file must keep contractual language beside parameter tables. Small libraries with one maintainer and ten lines of README gain little, because the inventory step costs more than reading the diff. If your process forbids hosted drafting of any kind, keep the manifest and checker and run generation only on locked internal runners.

If you adapt the manifest, commit the checker beside it so the permit file cannot drift without a failing test.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Top comments (0)