Configuration reference pages fail when generated prose is treated as the source of production defaults, secret classes, or deprecation windows. A parser can list every flag and environment identifier that the current commit actually references in code. A drafting model can rewrite help-text into clearer purpose sentences without inventing runtime behavior or cluster values. A human still must sign defaults, secret classification, required-in-production status, and breakage policy before publish.
This article describes a documentation-generation workflow built around that split rather than around a vendor feature list. The original artifact is a small extractor, a signed-cell grid, and a publish gate that fails on leftover UNSIGNED markers. The method remains useful if the drafting environment is replaced with a local editor and a reviewer checklist.
The failure mode is mixed authority, not missing fluency
Teams often ask a chat model to write the configuration section from a README plus a handful of nearby files. The resulting table looks complete, yet the default column frequently describes a laptop checkout rather than a production deploy. Secret-bearing names get copied into example blocks because the prompt never distinguished public flags from credentials. Deprecated switches remain documented because the model never saw the removal commit that deleted the parser option.
The failure is mixed authority. Identifiers, flag strings, and in-code help text are recoverable from the tree with a deterministic walk. Production defaults, rotation policy, log-safety, and the calendar window for a breaking rename are not recoverable from that walk. Treating both classes as one generated blob produces documentation that reads well and ships false operational claims.
Adjacent writing about cognitive atrophy in software work is easy to misread as a ban on drafting tools. The narrower engineering problem is quieter and more testable than that essay genre. Unsigned operational cells are a review defect, and they can be refused in CI without arguing about whether models should exist.
Ownership matrix for a configuration page
Use the following matrix as the contract for every configuration reference that this workflow emits. Cells marked compile come from parsers and literal references. Cells marked draft may be filled by a model and still require a human pass. Cells marked signed must stay empty or UNSIGNED until a named reviewer writes them.
| Cell | Source of truth | Lane |
|---|---|---|
| Flag names, env names, config keys | Parsers and literal references in the tree | Compile |
| Short purpose prose | Help strings plus a constrained model draft | Draft |
| Non-secret value shapes | Types, choices, and validators in code | Compile, then review |
| Production default | Deploy manifests and operator runbooks | Signed |
| Secret class | Security review: public, confidential, or prohibited-in-logs | Signed |
| Required in production | Launch checklist and capacity assumptions | Signed |
| Deprecation and breakage window | Release policy, not commit-message folklore | Signed |
Do not let the drafting lane write signed cells even as "probably" or "typically" statements. Those hedges become production folklore the first time an on-call engineer copies them into an incident note. Empty signed cells are honest; hedged signed cells are a defect.
Workflow
1. Freeze identifiers from parsers, not from chat output
Start from the same commit you intend to document, because configuration names drift across branches faster than narrative README sections. Extract identifiers with a script that CI can rerun, and treat chat output as non-facts until a reviewer merges selected sentences into the draft column. The extractor below is a labeled worked example for a tiny CLI, not a claim about any production codebase.
# extract_config_ids.py — labeled example, not a production inventory
from __future__ import annotations
import argparse
import ast
import json
import sys
from pathlib import Path
ENV_FUNCS = {"getenv", "get"}
class ConfigVisitor(ast.NodeVisitor):
def __init__(self) -> None:
self.rows: list[dict[str, str]] = []
def visit_Call(self, node: ast.Call) -> None:
func = node.func
name = ""
if isinstance(func, ast.Attribute):
name = func.attr
elif isinstance(func, ast.Name):
name = func.id
if name in {"add_argument"} and node.args:
flag = ast.literal_eval(node.args[0]) if isinstance(node.args[0], ast.Constant) else None
help_text = ""
for kw in node.keywords:
if kw.arg == "help" and isinstance(kw.value, ast.Constant):
help_text = str(kw.value.value)
if isinstance(flag, str) and flag.startswith("-"):
self.rows.append(
{
"id": flag,
"kind": "flag",
"help": help_text,
}
)
if name in ENV_FUNCS and node.args:
key = ast.literal_eval(node.args[0]) if isinstance(node.args[0], ast.Constant) else None
if isinstance(key, str) and key.isupper():
self.rows.append({"id": key, "kind": "env", "help": ""})
self.generic_visit(node)
def extract(path: Path) -> list[dict[str, str]]:
tree = ast.parse(path.read_text(encoding="utf-8"))
visitor = ConfigVisitor()
visitor.visit(tree)
seen: set[tuple[str, str]] = set()
unique: list[dict[str, str]] = []
for row in visitor.rows:
key = (row["kind"], row["id"])
if key in seen:
continue
seen.add(key)
unique.append(row)
return unique
def main() -> int:
parser = argparse.ArgumentParser(description="Compile config identifiers from one module.")
parser.add_argument("source", type=Path)
parser.add_argument("--out", type=Path, required=True)
args = parser.parse_args()
payload = extract(args.source)
args.out.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")
return 0
if __name__ == "__main__":
sys.exit(main())
Run it against a single module first so the identifier list stays reviewable in one pull request.
python extract_config_ids.py ./examples/widgetctl.py --out ./docs/_generated/config_ids.json
2. Emit a reference grid with signed columns already empty
The second script should write Markdown that already contains UNSIGNED in every operational cell. Empty operational cells are better than a confident wrong default, because later review can see the gap without diffing against a hallucinated number. Purpose prose may be copied from in-code help strings at this stage, which keeps the compile lane deterministic before any model runs.
# emit_config_grid.py — labeled example
from __future__ import annotations
import json
import sys
from pathlib import Path
HEADER = """# Configuration reference (unsigned operational cells)\n\nCompile lane: identifier and help text. Draft lane: purpose. Signed lane: still UNSIGNED.\n\n| ID | Kind | Purpose (draft) | Prod default | Secret class | Required in prod | Breakage window |\n| --- | --- | --- | --- | --- | --- | --- |\n"""
def main() -> int:
src = Path(sys.argv[1])
dest = Path(sys.argv[2])
rows = json.loads(src.read_text(encoding="utf-8"))
lines = [HEADER]
for row in rows:
purpose = row.get("help") or "DRAFT_NEEDED"
lines.append(
f"| `{row['id']}` | {row['kind']} | {purpose} | UNSIGNED | UNSIGNED | UNSIGNED | UNSIGNED |\n"
)
dest.write_text("".join(lines), encoding="utf-8")
return 0
if __name__ == "__main__":
raise SystemExit(main())
python emit_config_grid.py ./docs/_generated/config_ids.json ./docs/config-reference.md
3. Allow a model to draft purpose prose only
If a drafting environment is available, constrain the prompt to compile columns plus the purpose column. Paste no production secrets, no live cluster values, and no private incident notes into that prompt. Ask for sentence-level rewrites of existing help text, not for defaults, sample credentials, or "typical production" values.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode currently offers free model access and a free server option, which can host this drafting step without becoming the publisher of record. Keep that pair on purpose prose and on non-secret value shapes that already appear in help strings or validators. Do not ask the model to invent production defaults, to classify secrets, or to guess whether a flag is required after launch.
A prompt that respects the matrix looks like the block below. It is a template for operators to adapt, not a measured quality claim.
You may rewrite the Purpose column using only the ID, Kind, and existing help text.
Do not fill Prod default, Secret class, Required in prod, or Breakage window.
Do not invent example values that look like credentials, tokens, hostnames, or ports.
Leave any purpose you cannot support from help text as DRAFT_NEEDED.
4. Human-sign the four operational cells
A reviewer fills production default, secret class, required-in-production, and breakage window in a named commit. Signing is an edit with a person on the byline, not an emoji reaction on the pull request that added the grid. Secret class should use a closed vocabulary such as public, confidential, or prohibited-in-logs, because free-text labels drift across authors.
Production default should cite a manifest path or a runbook heading rather than a remembered number. Breakage window should cite a release policy document or an issue identifier, not a model paraphrase of recent commit subjects. If the reviewer does not know the value, the cell stays UNSIGNED and the page stays unpublished.
5. Gate publish on unsigned operational cells
CI should fail the documentation job when any signed column still contains UNSIGNED, DRAFT_NEEDED, or a hedge token that belongs in the draft lane. The checker below is intentionally strict on a small token list so it can run without a language model.
# check_signed_config_grid.py — labeled example
from __future__ import annotations
import re
import sys
from pathlib import Path
BANNED = {"UNSIGNED", "DRAFT_NEEDED", "TODO", "TBD", "probably", "typically"}
SIGNED_INDEXES = {3, 4, 5, 6} # prod default through breakage window
def main() -> int:
text = Path(sys.argv[1]).read_text(encoding="utf-8")
failures: list[str] = []
for line_no, line in enumerate(text.splitlines(), start=1):
if not line.startswith("|") or line.startswith("| ID") or re.match(r"\|\s*---", line):
continue
cells = [c.strip() for c in line.strip("|").split("|")]
if len(cells) < 7:
continue
for idx in SIGNED_INDEXES:
value = cells[idx]
if any(token.lower() in value.lower() for token in BANNED):
failures.append(f"L{line_no} col{idx + 1}: {value}")
if failures:
print("unsigned or hedged operational cells:")
print("\n".join(failures))
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main())
python check_signed_config_grid.py ./docs/config-reference.md
A Makefile target keeps the three commands in one place without implying that Make is the only legal orchestrator.
.PHONY: config-docs
config-docs:
python extract_config_ids.py ./examples/widgetctl.py --out ./docs/_generated/config_ids.json
python emit_config_grid.py ./docs/_generated/config_ids.json ./docs/config-reference.md
@echo "Draft purpose prose, then human-sign operational cells before check."
.PHONY: config-docs-check
config-docs-check:
python check_signed_config_grid.py ./docs/config-reference.md
Worked grid after a human pass
The following fragment is a labeled example after signing, not telemetry from a live service. Purpose prose may originate in a drafting pass. Operational cells must not.
| ID | Kind | Purpose (draft) | Prod default | Secret class | Required in prod | Breakage window |
| --- | --- | --- | --- | --- | --- | --- |
| `--region` | flag | Selects the regional API endpoint group. | `us-east-1` from deploy/prod.yaml | public | yes | none in 1.x |
| `WIDGET_API_TOKEN` | env | Authenticates machine callers to the public API. | injected by vault, never defaulted | prohibited-in-logs | yes | rename requires 2.0 |
Notice that the token row has no example value. The grid documents classification and required status without turning the page into a credential leak. A model that volunteers sk-example or a paste-shaped string has left the draft lane and the checker should not be weakened to allow it.
Limitations
The extractor understands a narrow slice of argparse and os.environ call shapes, and it will miss dynamically built names. Teams that generate flags from YAML schemas, Cobra command trees, or reflection-heavy frameworks need an extractor that walks those sources instead of this example visitor. The publish gate only proves that signed cells are non-empty and free of listed hedges; it does not prove that a default matches production.
Re-running the emitter will clobber human-signed cells unless you store signatures in a side file and merge by identifier. That merge step is part of the workflow debt, and skipping it will train reviewers to stop signing because their work disappears. The drafting environment also cannot classify secrets from identifier spelling alone, which is why secret class remains a signed cell even when a name contains TOKEN.
Who should not use this approach
Do not use this workflow if no human owns production defaults for the service being documented. A grid of UNSIGNED cells is safer than a generated table, but it is not a substitute for an owner. Do not send live credentials, customer identifiers, or incident timelines into a drafting prompt in order to "improve" purpose prose.
Regulated releases that require signed operational values before any draft exists should start from the signed columns and skip the model entirely. This method is also a poor fit for wikis where CI cannot refuse publication, because the whole point is a mechanical gate rather than a style guide paragraph.
The useful outcome is a configuration page whose compileable names stay in lockstep with parsers while operational claims stay attributable. If a drafting host with free model access is already in the toolchain, keep it on the purpose column and let humans remain the only signers of defaults and secret classes.
Top comments (0)