Extract a CLI Command Tree in CI, Then Sign Destructive Examples Outside the Model

CLI documentation fails in review when generated prose and operator judgment share a single markdown file. Generated flag tables stay current across releases only when a build step compiles them from the parser. Humans still own destructive examples, exit-code contracts, and environment assumptions that no parser can emit. The practical split is an extracted command tree plus a signed overlay that the documentation build refuses to skip.

This workflow targets command-line tools whose public surface is an argparse (or equivalent) tree, not HTTP schemas or config catalogs. The compiled artifact holds command paths, flags, defaults, and help strings taken from the parser at import time. The overlay holds destructiveness, exit-code meaning, required environment, and worked examples that can mutate remote state. A model may draft low-risk synopsis stubs; it does not become the owner of paste-ready commands.

Why mixed CLI pages drift

Most CLI pages mix three kinds of statements that rot at different speeds after a release. Parser-backed statements include subcommand names, flag spellings, default values, required arguments, and help strings shipped in the binary. Human-backed statements include whether a command deletes data, which non-zero exits are expected, and which environment variables must exist. Model-backed drafts can propose a synopsis paragraph, but they cannot certify that a sample invocation is safe to paste.

When those three statement classes live in one hand-edited page, reviewers argue about tone while the flag list silently lags. When a model rewrites the whole page, invented examples look fluent and the missing --force flag disappears from the table. Treating the parser as a compiler input, and the overlay as a signature file, makes that failure visible in CI instead of in customer tickets.

Decision table: what may be drafted

Label the following table as a proposed contract, not as a claim about any particular vendor parser. Adjust the “must sign” column if your binary exposes plugin commands that the freeze file cannot enumerate.

Cell in the published page	Source of truth	Model may draft a stub?	Human must sign before publish?
Command path (widgets delete)	Parser tree	No	No, compile only
Flag names, types, defaults	Parser actions	No	No, compile only
Required positional arguments	Parser actions	No	No, compile only
One-line synopsis	Overlay, optional model stub	Yes, labeled draft	Yes, if the command is public
Destructive classification	Overlay only	No	Yes
Exit-code contract	Overlay only	No	Yes
Required environment variables	Overlay only	No	Yes
Example that talks to a network	Overlay only	No	Yes
Owner / escalation alias	Overlay only	No	Yes

The rule is mechanical rather than stylistic. If the parser can prove the cell, compile it. If the parser cannot prove the cell, refuse to publish until a human signature exists. Models sit in the middle only for synopsis text that the join step still marks draft until signed.

Workflow

Follow the six steps in order. Skipping the freeze file or the coverage test returns you to a mixed markdown page with no gate.

1. Freeze the command tree from the parser, not from README headings or chat output.
2. Emit flag tables as a build artifact that the docs renderer consumes without editing.
3. Allow a model to draft synopsis stubs only for cells the table marks draftable.
4. Require human signatures for destructive class, exit codes, environment, and networked examples.
5. Fail the docs job when a public command lacks a required overlay signature.
6. Join tree plus overlay into the published page, preserving provenance comments.

The rest of this article implements that sequence with a small Python CLI, an extractor, an overlay schema, and a coverage test. Treat the code as a reproducible example, not as production telemetry from a live fleet.

Step 1 — Keep a parser of record

The example binary below is intentionally small so the freeze file stays reviewable. delete is destructive; list is not. That distinction does not appear in argparse metadata, which is exactly why the overlay exists.

# example: widgets_cli.py
import argparse
import json
import sys


def build_parser() -> argparse.ArgumentParser:
    parser = argparse.ArgumentParser(prog="widgets", add_help=True)
    sub = parser.add_subparsers(dest="command", required=True)

    p_list = sub.add_parser("list", help="List widgets in the current project.")
    p_list.add_argument("--format", choices=("table", "json"), default="table")
    p_list.add_argument("--limit", type=int, default=50)

    p_del = sub.add_parser("delete", help="Delete a widget by identifier.")
    p_del.add_argument("widget_id")
    p_del.add_argument("--force", action="store_true", help="Skip the confirmation prompt.")
    p_del.add_argument("--dry-run", action="store_true")
    return parser


def main(argv=None) -> int:
    args = build_parser().parse_args(argv)
    if args.command == "list":
        print(json.dumps({"items": [], "limit": args.limit}))
        return 0
    if args.command == "delete" and not args.force and not args.dry_run:
        print("refusing to delete without --force or --dry-run", file=sys.stderr)
        return 2
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Importing build_parser() from the docs job avoids scraping --help text, which is lossy for types and defaults. If your CLI is not Python, emit the same freeze file from Cobra, Clap, or a hand-maintained JSON Schema; the overlay contract does not care how the tree was produced.

Step 2 — Compile the command tree

The extractor walks parser._subparsers and writes one object per command path. It records flags the parser can prove and leaves judgment cells empty. Empty is correct: filling them here would launder guesses into the compiled lane.

# example: tools/extract_cli_tree.py
import json
from argparse import _StoreAction, _StoreTrueAction
from widgets_cli import build_parser


def flag_record(action) -> dict | None:
    if not action.option_strings:
        return None
    return {
        "flags": list(action.option_strings),
        "required": bool(getattr(action, "required", False)),
        "default": None if isinstance(action, _StoreTrueAction) else action.default,
        "choices": list(action.choices) if action.choices else None,
        "help": action.help or "",
        "type": "bool" if isinstance(action, _StoreTrueAction) else (
            getattr(action.type, "__name__", "str") if isinstance(action, _StoreAction) else "str"
        ),
    }


def extract() -> dict:
    parser = build_parser()
    commands = []
    for action in parser._subparsers._group_actions:
        for name, sub in action.choices.items():
            flags = [flag_record(a) for a in sub._actions]
            commands.append({
                "path": f"widgets {name}",
                "help": sub.description or sub.format_help().splitlines()[0],
                "positionals": [
                    {"name": a.dest, "help": a.help or ""}
                    for a in sub._actions if not a.option_strings and a.dest != "help"
                ],
                "flags": [f for f in flags if f],
            })
    return {"prog": parser.prog, "commands": commands}


if __name__ == "__main__":
    print(json.dumps(extract(), indent=2, sort_keys=True))

Run the freeze as a regular compile step so pull requests cannot “update docs later”:

python tools/extract_cli_tree.py > docs/cli/command_tree.json
git diff --exit-code docs/cli/command_tree.json

--exit-code fails the job when the parser changed and the committed freeze file did not. That is the entire point of compiling documentation from source constants rather than from a model transcript.

Step 3 — Own the overlay the parser cannot emit

Store human judgment beside the freeze file, keyed by command path. The example overlay below is YAML for review, but JSON Schema validation is the actual gate. signed_by is an operator identity your review process already understands; do not invent a cryptographic story the repository does not implement.

# example: docs/cli/overlay.yaml
commands:
  "widgets list":
    destructive: false
    exit_codes:
      0: "Printed the page; an empty list is still success."
      2: "Usage error, including an unknown --format value."
    env: []
    example:
      argv: ["widgets", "list", "--format", "json", "--limit", "10"]
      network: false
    synopsis: "Read-only listing of widgets for the current project."
    owner: "platform-oncall"
    signed_by: "docs-owner"
    signed: true
  "widgets delete":
    destructive: true
    exit_codes:
      0: "Delete accepted, or --dry-run printed the planned request."
      2: "Refused without --force or --dry-run; also usage errors."
    env: ["WIDGET_API_TOKEN"]
    example:
      argv: ["widgets", "delete", "wg_123", "--dry-run"]
      network: true
      note: "Never publish a --force example that targets a shared project."
    synopsis: "Deletes a widget; confirmation is skipped only with --force."
    owner: "platform-oncall"
    signed_by: "docs-owner"
    signed: true

Notice the published example for delete uses --dry-run rather than --force. That choice is not recoverable from argparse. A model that completes “show a typical invocation” will often emit --force because the flag exists and the help string mentions confirmation. The overlay exists so that completion never reaches the published page unsigned.

Step 4 — Coverage test that fails the docs build

The test loads both files and applies the decision table. Public commands must exist in the overlay. Destructive commands must ship a non-networked or explicitly dry-run example. Unsigned rows fail even if the synopsis looks complete.

# example: tests/test_cli_overlay_coverage.py
import json
from pathlib import Path
import yaml  # PyYAML in the docs job only

TREE = json.loads(Path("docs/cli/command_tree.json").read_text())
OVERLAY = yaml.safe_load(Path("docs/cli/overlay.yaml").read_text())["commands"]

REQUIRED = ("destructive", "exit_codes", "env", "example", "owner", "signed")


def test_every_parser_command_has_overlay_row():
    paths = {row["path"] for row in TREE["commands"]}
    missing = sorted(paths - set(OVERLAY))
    extra = sorted(set(OVERLAY) - paths)
    assert missing == [], f"unsigned commands: {missing}"
    assert extra == [], f"overlay paths not in parser freeze: {extra}"


def test_required_cells_are_signed():
    for path, row in OVERLAY.items():
        for key in REQUIRED:
            assert key in row, f"{path} missing {key}"
        assert row["signed"] is True, f"{path} is not signed"
        assert row["signed_by"], f"{path} missing signer"
        assert isinstance(row["exit_codes"], dict) and 0 in row["exit_codes"]
        assert isinstance(row["env"], list)
        assert "argv" in row["example"]
        if row["destructive"]:
            argv = " ".join(row["example"]["argv"])
            assert "--dry-run" in argv or row["example"].get("network") is False, (
                f"{path} publishes a destructive example without a dry-run"
            )

Run it in the same job that builds the site:

python tools/extract_cli_tree.py > docs/cli/command_tree.json
pytest tests/test_cli_overlay_coverage.py -q
python tools/join_cli_docs.py > docs/cli/generated.md

If you render with MkDocs or Sphinx, treat generated.md as an input, not as a file authors edit. Hand edits belong in overlay.yaml so the next freeze cannot clobber signed cells.

Step 5 — Join without letting drafts escape

The join script is deliberately boring. It prints a provenance banner, the compiled flag table, then overlay cells, and it refuses to interpolate unsigned synopses. Proposed implementation:

# example: tools/join_cli_docs.py
import json
from pathlib import Path
import yaml

tree = json.loads(Path("docs/cli/command_tree.json").read_text())
overlay = yaml.safe_load(Path("docs/cli/overlay.yaml").read_text())["commands"]

print("<!-- generated: do not edit; source is command_tree.json + overlay.yaml -->")
print("# widgets CLI reference\n")
for cmd in tree["commands"]:
    path = cmd["path"]
    owned = overlay[path]
    if not owned.get("signed"):
        raise SystemExit(f"refusing to join unsigned command: {path}")
    print(f"## `{path}`\n")
    print(owned["synopsis"] + "\n")
    print("| Flag | Type | Default | Help |")
    print("| --- | --- | --- | --- |")
    for flag in cmd["flags"]:
        names = ", ".join(f"`{n}`" for n in flag["flags"])
        default = "" if flag["default"] is None else f"`{flag['default']}`"
        print(f"| {names} | {flag['type']} | {default} | {flag['help']} |")
    print("\n**Exit codes**")
    for code, meaning in owned["exit_codes"].items():
        print(f"- `{code}`: {meaning}")
    print("\n**Example (signed)**")
    print("```

bash")
    print(" ".join(owned["example"]["argv"]))
    print("

```")
    if owned["env"]:
        print("\nRequires environment: " + ", ".join(f"`{e}`" for e in owned["env"]))
    print(f"\n_Owner: {owned['owner']}. Signed by {owned['signed_by']}. _\n")

The banner comment is part of the artifact. Reviewers who see a docs diff that edits generated.md directly should reject the change and point at the overlay.

Where a model pass is allowed

A model is useful for first-pass synopsis text after a new subcommand lands, provided the prompt cannot write destructive, exit_codes, env, or example. Feed only the freeze-file object for that command, ask for a single sentence synopsis, and write the result into a separate stubs.yaml that the coverage test does not treat as signed. A human copies a stub into overlay.yaml after checking the binary.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access can draft those synopsis stubs from the freeze file, and the free server option can run the extractor, coverage test, and join script as an ordinary docs job. Neither the model pass nor the server run fills signature cells; the overlay remains a human-owned file in the repository.

Keep the model output labeled. A suggested filename is docs/cli/stubs.yaml with status: draft on every row. The join script should not read that file. If a stub is accidentally merged into overlay.yaml without signed: true, the coverage test already fails.

Limitations and who should skip this

This approach assumes a finite command tree that can be frozen at compile time. Plugin CLIs that load subcommands from $PATH at runtime need an additional allow-list; otherwise the freeze file will flap between developer laptops. GUIs, REPL tools, and wizards with branching prompts are a poor fit because the parser does not contain the interaction graph.

Do not use the model stub step for regulated binaries whose every example requires legal review, or for tools where a wrong example can destroy customer data without a dry-run flag. The coverage test checks that a dry-run token is present; it cannot prove the token is implemented. That proof still lives in the test suite for the binary itself.

Exit-code contracts in the overlay are documentation, not runtime enforcement. If widgets delete begins returning 3 for “not found,” the overlay will stay wrong until a human edits it. Pair this pipeline with a small characterization test that asserts documented codes still occur, or accept that the overlay can lag behavior the same way any signed document can lag.

Teams that already generate HTTP catalogs or config atlases should still keep CLI overlays separate. Flag names collide with config keys, and destructiveness is a CLI-specific cell. Reusing one overlay schema across surfaces hides missing signatures behind optional fields.

What to measure in review

Track three counts on the docs job summary: parser commands, signed overlay rows, and unsigned stubs. The first two must match. The third may be non-zero during a release branch, but it must not be an input to join_cli_docs.py. If the unsigned stub count grows for several releases, the bottleneck is review capacity, not model quality, and adding another draft pass will not close the gate.

The conclusion does not depend on any vendor remaining free. Compile what the parser can prove, sign what a paste can break, and fail the build when those files disagree. If you already run catalog jobs in CI, the same coverage pattern extends to CLI pages without folding judgment into generated prose.