Build a CLI Command Ledger Before You Generate Help Text

Public CLI help fails in a predictable place: the tree of subcommands is mechanical, while harm is not. A parser definition already lists flags, types, and defaults with enough structure to compile a ledger. A language model can turn that ledger into readable usage paragraphs without opening the repository. It cannot classify which flags delete data, which options accept secrets, or which examples are safe to paste into production shells.

This article describes a two-pass documentation workflow for Python CLIs built with argparse or Click. Pass one compiles a command ledger from the abstract syntax tree and never asks a model to discover the tree. Pass two may draft synopsis prose from that ledger, while a reviewer signs three human-owned fields before any help page is published.

The failure mode the workflow is built around

Generated help text usually drifts in three ways that grep-driven README updates do not catch. Subcommands appear in marketing copy after they were renamed in code, while retired aliases remain in tutorials for months. Default values are restated in prose that no longer matches default= in the parser. Worst, copy-paste examples invoke destructive flags against real clusters because the generator treated every option as equally safe.

Those three failures have different owners. Tree shape, flag names, types, and defaults are compileable facts. Destructive classification, secret-bearing options, and runnable examples are judgments. Mixing them in one chat transcript is how stale help pages get a confident tone.

What the model may draft versus what a human must own

Treat every help page as two files that share a command id. The ledger holds facts extracted from parser construction. The signature sheet holds judgments that must remain outside the model, including when the draft prose looks polished.

Field	Source of truth	Who may draft	Who must own
Command path and aliases	AST of add_parser / @click.command	Compiler only	Reviewer confirms no extra commands
Flag names, types, required, defaults	AST of add_argument / @click.option	Compiler only	Reviewer confirms defaults are non-secret
Short synopsis paragraph	Ledger JSON	Model, optional	Reviewer may edit tone
Long usage narrative	Ledger JSON	Model, optional	Reviewer may edit tone
destructive (none / reversible / irreversible)	Not in AST	Never	Human
secret_in (none / flag / env / file)	Heuristic at most	Never as final	Human
example_cmd that is safe to paste	Not in AST	Never as final	Human
Exit-code contract	Tests or sys.exit sites, not chat	Compiler may list candidates	Human

The rule is narrower than “do not use a model.” The model may write sentences. It may not invent commands, change defaults, or certify that an example is harmless.

Pass one: compile the command ledger

The compiler should walk parser construction, not runtime --help output. Runtime help can be locale-dependent, and it hides flags that are added only on some branches. AST extraction is incomplete for dynamically built parsers; that limitation is recorded later rather than papered over by a model.

1. Scope the files the compiler is allowed to read

Limit the walk to modules that actually construct the CLI, typically cli.py, commands/*.py, and the package __main__.py. Do not send application business logic to a drafting model later. The ledger is the only artifact that should leave the checkout during pass two.

# Proposal: collect parser modules without packing the whole repo.
find src -name '*.py' | rg 'argparse|click|typer' > /tmp/cli-inputs.txt

2. Extract subcommands, flags, and defaults

The following script is a starting extractor for argparse.add_parser and add_argument keyword facts. It is a proposal you should run against your own tree; it does not execute Click decorators or resolve variables that are not string literals.

# extract_cli_ledger.py — proposal; run locally, do not pipe source to a model.
from __future__ import annotations

import ast
import json
import sys
from pathlib import Path


def lit(node):
    if isinstance(node, ast.Constant):
        return node.value
    return None


class ArgparseLedger(ast.NodeVisitor):
    def __init__(self, path: str):
        self.path = path
        self.commands: list[dict] = []

    def visit_Call(self, node: ast.Call):
        func = node.func
        name = func.attr if isinstance(func, ast.Attribute) else None
        if name == "add_parser":
            cmd = lit(node.args[0]) if node.args else None
            help_ = next((lit(k.value) for k in node.keywords if k.arg == "help"), None)
            self.commands.append({
                "id": f"{self.path}::{cmd}",
                "kind": "argparse.subparser",
                "path": self.path,
                "command": cmd,
                "help_kw": help_,
                "flags": [],
            })
        if name == "add_argument":
            flags = [lit(a) for a in node.args if isinstance(a, ast.Constant)]
            opts = {k.arg: lit(k.value) for k in node.keywords if k.arg}
            target = self.commands[-1] if self.commands else {
                "id": f"{self.path}::root",
                "kind": "argparse.root",
                "path": self.path,
                "command": None,
                "help_kw": None,
                "flags": [],
            }
            if not self.commands:
                self.commands.append(target)
            target["flags"].append({
                "flags": flags,
                "dest": opts.get("dest"),
                "type_name": opts.get("type") if isinstance(opts.get("type"), str) else None,
                "required": opts.get("required"),
                "default": opts.get("default"),
                "help_kw": opts.get("help"),
                "action": opts.get("action"),
            })
        self.generic_visit(node)


def extract(path: Path) -> list[dict]:
    tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
    visitor = ArgparseLedger(str(path))
    visitor.visit(tree)
    return visitor.commands


def main(argv: list[str]) -> int:
    rows: list[dict] = []
    for raw in argv[1:]:
        rows.extend(extract(Path(raw)))
    json.dump({"commands": rows}, sys.stdout, indent=2, default=str)
    return 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv))

python extract_cli_ledger.py src/myapp/cli.py src/myapp/commands/*.py > cli_ledger.json

3. Normalize ids so drafts cannot invent commands

Every later paragraph must cite id. If a draft mentions a command path that is absent from cli_ledger.json, the publish step rejects the file. That check is cheaper than asking a model whether it hallucinated a subcommand, and it does not depend on the drafting backend.

# reject_unknown_commands.py — proposal
import json, re, sys
from pathlib import Path

ledger = json.loads(Path("cli_ledger.json").read_text())
known = {row["id"] for row in ledger["commands"]}
body = Path(sys.argv[1]).read_text()
for cited in re.findall(r"command-id:\s*(\S+)", body):
    if cited not in known:
        raise SystemExit(f"unpublished command id: {cited}")

4. Keep secrets out of the ledger

Defaults that look like connection strings, tokens, or file paths to credentials should be replaced with a placeholder before any remote draft. The compiler can flag default values that match a conservative pattern; it cannot prove absence of secrets. A human still reviews the JSON.

SECRETISH = ("password", "token", "secret", "api_key", "dsn")

def scrub(flag: dict) -> dict:
    dest = (flag.get("dest") or "").lower()
    help_kw = (flag.get("help_kw") or "").lower()
    if any(s in dest or s in help_kw for s in SECRETISH):
        return {**flag, "default": "[redacted-before-draft]"}
    return flag

Pass two: optional prose, required signatures

Once the ledger is compiled and scrubbed, a model may draft synopsis text keyed by id. The prompt should receive JSON only, not repository files. If you want that draft pass without standing up your own inference host, MonkeyCode's free model access and free server option can consume the ledger and return usage paragraphs. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The compile step still runs locally, and the signature sheet still stays outside the model.

5. Emit a signature sheet with empty human lanes

Do not let the drafting step create these keys. Generate a stub from the ledger so missing signatures fail CI.

# cli_signatures.yaml — generated stub; humans fill the own_* keys
commands:
  - id: src/myapp/cli.py::reset-db
    own_destructive: null   # none | reversible | irreversible
    own_secret_in: null     # none | flag | env | file
    own_example_cmd: null   # must not target shared prod
    own_exit_codes: null    # map of int -> meaning
    own_reviewer: null
    own_reviewed_at: null

6. Reject publish when human lanes are still null

# check_signatures.py — proposal
import sys, yaml
from pathlib import Path

REQUIRED = (
    "own_destructive",
    "own_secret_in",
    "own_example_cmd",
    "own_exit_codes",
    "own_reviewer",
    "own_reviewed_at",
)

data = yaml.safe_load(Path("cli_signatures.yaml").read_text())
missing = []
for row in data["commands"]:
    for key in REQUIRED:
        if row.get(key) in (None, "", []):
            missing.append(f"{row['id']}.{key}")
if missing:
    print("unsigned CLI docs:\n" + "\n".join(missing))
    sys.exit(1)

7. Gate examples that the compiler marked as high-churn flags

A useful local heuristic is not a signature. Flags named delete, drop, reset, purge, force, or yes should block publish until own_destructive is not none. The heuristic exists to fail closed, not to classify harm for the reviewer.

DANGER_TOKENS = ("delete", "drop", "reset", "purge", "force", "yes", "force-yes")

def danger_flags(command: dict) -> list[str]:
    hits = []
    for flag in command.get("flags", []):
        joined = " ".join(str(x).lower() for x in (flag.get("flags") or []))
        if any(tok in joined for tok in DANGER_TOKENS):
            hits.extend(flag.get("flags") or [])
    return hits

A reproducible review checklist

Use this as a merge checklist rather than as a claim that the pipeline was timed in production. Each item maps to a file you can diff.

1. cli_ledger.json is regenerated from the same commit as the help pages.
2. Every command-id in drafted Markdown exists in the ledger.
3. No ledger default contains a live secret; redaction happened before any remote call.
4. cli_signatures.yaml has non-null own_destructive, own_secret_in, and own_example_cmd.
5. Examples for irreversible commands use fake hosts, disposable databases, or explicitly local paths.
6. Exit-code notes cite tests or sys.exit sites, not a model paraphrase.
7. Click or dynamically built parsers that the AST missed are listed in a gaps.md file by hand.

Limitations

The extractor above does not evaluate Python. Flags assembled from variables, config files, or plugin entry points will be absent from the ledger. Click and Typer decorator trees need a separate visitor; mixing them into one script without tests will silently drop commands. Defaults that are function calls will show as null, which is safer than executing those calls during documentation builds.

Heuristic danger tokens both over-flag (--force-color) and under-flag (--apply on a mutate path). That is why own_destructive is not optional. Models also compress exit-code tables into “nonzero means error,” which is wrong for CLIs that use 2 for usage errors and 3 for partial apply. Keep those maps human-owned.

Remote drafting, including a free hosted option, still means the ledger leaves your machine. Scrub defaults and never upload parser modules that embed internal hostnames you would not put in a public README.

Who should not use this approach

Do not use this workflow if the CLI is a thin wrapper whose real interface is an HTTP API; document the API contract instead. Do not use it as a substitute for integration tests that prove examples run. Do not use it when parsers are generated at runtime from a remote schema, because the AST ledger will be emptier than --help and will look authoritative anyway.

Teams that already generate man pages from the parser at build time can keep that generator for flag lists. They still need the signature sheet for examples and destructive class. The compile step is not a reason to skip that ownership.

The durable output is not the synopsis paragraph. It is a ledger that can be regenerated, a signature file that cannot, and a publish gate that refuses help text when those two disagree. If you run the ledger through a free remote draft pass, keep the signed lanes in version control beside the generated prose rather than inside the chat that produced it.