DEV Community

Avery Lin
Avery Lin

Posted on

Turn Parser-Visible Flags Into a Config Reference Grid; Sign Defaults and Secret Classes

Configuration reference pages fail when generated prose is treated as the source of production defaults, secret classes, or deprecation windows. A parser can list every flag and environment identifier that the current commit actually references in code. A drafting model can rewrite help-text into clearer purpose sentences without inventing runtime behavior or cluster values. A human still must sign defaults, secret classification, required-in-production status, and breakage policy before publish.

This article describes a documentation-generation workflow built around that split rather than around a vendor feature list. The original artifact is a small extractor, a signed-cell grid, and a publish gate that fails on leftover UNSIGNED markers. The method remains useful if the drafting environment is replaced with a local editor and a reviewer checklist.

The failure mode is mixed authority, not missing fluency

Teams often ask a chat model to write the configuration section from a README plus a handful of nearby files. The resulting table looks complete, yet the default column frequently describes a laptop checkout rather than a production deploy. Secret-bearing names get copied into example blocks because the prompt never distinguished public flags from credentials. Deprecated switches remain documented because the model never saw the removal commit that deleted the parser option.

The failure is mixed authority. Identifiers, flag strings, and in-code help text are recoverable from the tree with a deterministic walk. Production defaults, rotation policy, log-safety, and the calendar window for a breaking rename are not recoverable from that walk. Treating both classes as one generated blob produces documentation that reads well and ships false operational claims.

Adjacent writing about cognitive atrophy in software work is easy to misread as a ban on drafting tools. The narrower engineering problem is quieter and more testable than that essay genre. Unsigned operational cells are a review defect, and they can be refused in CI without arguing about whether models should exist.

Ownership matrix for a configuration page

Use the following matrix as the contract for every configuration reference that this workflow emits. Cells marked compile come from parsers and literal references. Cells marked draft may be filled by a model and still require a human pass. Cells marked signed must stay empty or UNSIGNED until a named reviewer writes them.

Cell Source of truth Lane
Flag names, env names, config keys Parsers and literal references in the tree Compile
Short purpose prose Help strings plus a constrained model draft Draft
Non-secret value shapes Types, choices, and validators in code Compile, then review
Production default Deploy manifests and operator runbooks Signed
Secret class Security review: public, confidential, or prohibited-in-logs Signed
Required in production Launch checklist and capacity assumptions Signed
Deprecation and breakage window Release policy, not commit-message folklore Signed

Do not let the drafting lane write signed cells even as "probably" or "typically" statements. Those hedges become production folklore the first time an on-call engineer copies them into an incident note. Empty signed cells are honest; hedged signed cells are a defect.

Workflow

1. Freeze identifiers from parsers, not from chat output

Start from the same commit you intend to document, because configuration names drift across branches faster than narrative README sections. Extract identifiers with a script that CI can rerun, and treat chat output as non-facts until a reviewer merges selected sentences into the draft column. The extractor below is a labeled worked example for a tiny CLI, not a claim about any production codebase.

# extract_config_ids.py — labeled example, not a production inventory
from __future__ import annotations

import argparse
import ast
import json
import sys
from pathlib import Path

ENV_FUNCS = {"getenv", "get"}


class ConfigVisitor(ast.NodeVisitor):
    def __init__(self) -> None:
        self.rows: list[dict[str, str]] = []

    def visit_Call(self, node: ast.Call) -> None:
        func = node.func
        name = ""
        if isinstance(func, ast.Attribute):
            name = func.attr
        elif isinstance(func, ast.Name):
            name = func.id

        if name in {"add_argument"} and node.args:
            flag = ast.literal_eval(node.args[0]) if isinstance(node.args[0], ast.Constant) else None
            help_text = ""
            for kw in node.keywords:
                if kw.arg == "help" and isinstance(kw.value, ast.Constant):
                    help_text = str(kw.value.value)
            if isinstance(flag, str) and flag.startswith("-"):
                self.rows.append(
                    {
                        "id": flag,
                        "kind": "flag",
                        "help": help_text,
                    }
                )

        if name in ENV_FUNCS and node.args:
            key = ast.literal_eval(node.args[0]) if isinstance(node.args[0], ast.Constant) else None
            if isinstance(key, str) and key.isupper():
                self.rows.append({"id": key, "kind": "env", "help": ""})

        self.generic_visit(node)


def extract(path: Path) -> list[dict[str, str]]:
    tree = ast.parse(path.read_text(encoding="utf-8"))
    visitor = ConfigVisitor()
    visitor.visit(tree)
    seen: set[tuple[str, str]] = set()
    unique: list[dict[str, str]] = []
    for row in visitor.rows:
        key = (row["kind"], row["id"])
        if key in seen:
            continue
        seen.add(key)
        unique.append(row)
    return unique


def main() -> int:
    parser = argparse.ArgumentParser(description="Compile config identifiers from one module.")
    parser.add_argument("source", type=Path)
    parser.add_argument("--out", type=Path, required=True)
    args = parser.parse_args()
    payload = extract(args.source)
    args.out.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")
    return 0


if __name__ == "__main__":
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode

Run it against a single module first so the identifier list stays reviewable in one pull request.

python extract_config_ids.py ./examples/widgetctl.py --out ./docs/_generated/config_ids.json
Enter fullscreen mode Exit fullscreen mode

2. Emit a reference grid with signed columns already empty

The second script should write Markdown that already contains UNSIGNED in every operational cell. Empty operational cells are better than a confident wrong default, because later review can see the gap without diffing against a hallucinated number. Purpose prose may be copied from in-code help strings at this stage, which keeps the compile lane deterministic before any model runs.

# emit_config_grid.py — labeled example
from __future__ import annotations

import json
import sys
from pathlib import Path

HEADER = """# Configuration reference (unsigned operational cells)\n\nCompile lane: identifier and help text. Draft lane: purpose. Signed lane: still UNSIGNED.\n\n| ID | Kind | Purpose (draft) | Prod default | Secret class | Required in prod | Breakage window |\n| --- | --- | --- | --- | --- | --- | --- |\n"""


def main() -> int:
    src = Path(sys.argv[1])
    dest = Path(sys.argv[2])
    rows = json.loads(src.read_text(encoding="utf-8"))
    lines = [HEADER]
    for row in rows:
        purpose = row.get("help") or "DRAFT_NEEDED"
        lines.append(
            f"| `{row['id']}` | {row['kind']} | {purpose} | UNSIGNED | UNSIGNED | UNSIGNED | UNSIGNED |\n"
        )
    dest.write_text("".join(lines), encoding="utf-8")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())
Enter fullscreen mode Exit fullscreen mode
python emit_config_grid.py ./docs/_generated/config_ids.json ./docs/config-reference.md
Enter fullscreen mode Exit fullscreen mode

3. Allow a model to draft purpose prose only

If a drafting environment is available, constrain the prompt to compile columns plus the purpose column. Paste no production secrets, no live cluster values, and no private incident notes into that prompt. Ask for sentence-level rewrites of existing help text, not for defaults, sample credentials, or "typical production" values.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode currently offers free model access and a free server option, which can host this drafting step without becoming the publisher of record. Keep that pair on purpose prose and on non-secret value shapes that already appear in help strings or validators. Do not ask the model to invent production defaults, to classify secrets, or to guess whether a flag is required after launch.

A prompt that respects the matrix looks like the block below. It is a template for operators to adapt, not a measured quality claim.

You may rewrite the Purpose column using only the ID, Kind, and existing help text.
Do not fill Prod default, Secret class, Required in prod, or Breakage window.
Do not invent example values that look like credentials, tokens, hostnames, or ports.
Leave any purpose you cannot support from help text as DRAFT_NEEDED.
Enter fullscreen mode Exit fullscreen mode

4. Human-sign the four operational cells

A reviewer fills production default, secret class, required-in-production, and breakage window in a named commit. Signing is an edit with a person on the byline, not an emoji reaction on the pull request that added the grid. Secret class should use a closed vocabulary such as public, confidential, or prohibited-in-logs, because free-text labels drift across authors.

Production default should cite a manifest path or a runbook heading rather than a remembered number. Breakage window should cite a release policy document or an issue identifier, not a model paraphrase of recent commit subjects. If the reviewer does not know the value, the cell stays UNSIGNED and the page stays unpublished.

5. Gate publish on unsigned operational cells

CI should fail the documentation job when any signed column still contains UNSIGNED, DRAFT_NEEDED, or a hedge token that belongs in the draft lane. The checker below is intentionally strict on a small token list so it can run without a language model.

# check_signed_config_grid.py — labeled example
from __future__ import annotations

import re
import sys
from pathlib import Path

BANNED = {"UNSIGNED", "DRAFT_NEEDED", "TODO", "TBD", "probably", "typically"}
SIGNED_INDEXES = {3, 4, 5, 6}  # prod default through breakage window


def main() -> int:
    text = Path(sys.argv[1]).read_text(encoding="utf-8")
    failures: list[str] = []
    for line_no, line in enumerate(text.splitlines(), start=1):
        if not line.startswith("|") or line.startswith("| ID") or re.match(r"\|\s*---", line):
            continue
        cells = [c.strip() for c in line.strip("|").split("|")]
        if len(cells) < 7:
            continue
        for idx in SIGNED_INDEXES:
            value = cells[idx]
            if any(token.lower() in value.lower() for token in BANNED):
                failures.append(f"L{line_no} col{idx + 1}: {value}")
    if failures:
        print("unsigned or hedged operational cells:")
        print("\n".join(failures))
        return 1
    return 0


if __name__ == "__main__":
    raise SystemExit(main())
Enter fullscreen mode Exit fullscreen mode
python check_signed_config_grid.py ./docs/config-reference.md
Enter fullscreen mode Exit fullscreen mode

A Makefile target keeps the three commands in one place without implying that Make is the only legal orchestrator.

.PHONY: config-docs
config-docs:
    python extract_config_ids.py ./examples/widgetctl.py --out ./docs/_generated/config_ids.json
    python emit_config_grid.py ./docs/_generated/config_ids.json ./docs/config-reference.md
    @echo "Draft purpose prose, then human-sign operational cells before check."

.PHONY: config-docs-check
config-docs-check:
    python check_signed_config_grid.py ./docs/config-reference.md
Enter fullscreen mode Exit fullscreen mode

Worked grid after a human pass

The following fragment is a labeled example after signing, not telemetry from a live service. Purpose prose may originate in a drafting pass. Operational cells must not.

| ID | Kind | Purpose (draft) | Prod default | Secret class | Required in prod | Breakage window |
| --- | --- | --- | --- | --- | --- | --- |
| `--region` | flag | Selects the regional API endpoint group. | `us-east-1` from deploy/prod.yaml | public | yes | none in 1.x |
| `WIDGET_API_TOKEN` | env | Authenticates machine callers to the public API. | injected by vault, never defaulted | prohibited-in-logs | yes | rename requires 2.0 |
Enter fullscreen mode Exit fullscreen mode

Notice that the token row has no example value. The grid documents classification and required status without turning the page into a credential leak. A model that volunteers sk-example or a paste-shaped string has left the draft lane and the checker should not be weakened to allow it.

Limitations

The extractor understands a narrow slice of argparse and os.environ call shapes, and it will miss dynamically built names. Teams that generate flags from YAML schemas, Cobra command trees, or reflection-heavy frameworks need an extractor that walks those sources instead of this example visitor. The publish gate only proves that signed cells are non-empty and free of listed hedges; it does not prove that a default matches production.

Re-running the emitter will clobber human-signed cells unless you store signatures in a side file and merge by identifier. That merge step is part of the workflow debt, and skipping it will train reviewers to stop signing because their work disappears. The drafting environment also cannot classify secrets from identifier spelling alone, which is why secret class remains a signed cell even when a name contains TOKEN.

Who should not use this approach

Do not use this workflow if no human owns production defaults for the service being documented. A grid of UNSIGNED cells is safer than a generated table, but it is not a substitute for an owner. Do not send live credentials, customer identifiers, or incident timelines into a drafting prompt in order to "improve" purpose prose.

Regulated releases that require signed operational values before any draft exists should start from the signed columns and skip the model entirely. This method is also a poor fit for wikis where CI cannot refuse publication, because the whole point is a mechanical gate rather than a style guide paragraph.

The useful outcome is a configuration page whose compileable names stay in lockstep with parsers while operational claims stay attributable. If a drafting host with free model access is already in the toolchain, keep it on the purpose column and let humans remain the only signers of defaults and secret classes.

Top comments (0)