Config Catalogs as a Build Artifact: Extract Schema Keys, Sign Every Risk Cell

A config catalog stays honest only when keys come from schema and risk language stays human-signed. Models may rephrase type, required flags, and descriptions that already exist on each JSON Schema property. They must not add keys, guess defaults, classify secrets, or write rollout sentences for production. This article proposes a compile-and-sign pipeline you can run against a fixture, not a production memoir.

The pipeline freezes a facts file, optionally drafts compile-lane Markdown, then refuses merge when human cells are empty. A reviewer signs secret class, default risk, and rollout text in a sidecar that CI can diff. The gate also rejects any key that does not appear in the schema-derived facts file. That equality check is the reason to treat the catalog as a build output rather than a wiki page.

Split compile cells from signed cells

Every catalog row has four compile cells and three signed cells with different authors. Compile cells are mechanical projections, so a script or a model may format them. Signed cells change incident response, so they remain empty until a named reviewer writes them. Mixing those authors in one Markdown table is how invented keys and unsigned risk claims reach production.

Use this decision table as the contract for both the generator and the merge gate. Keep it in the repository beside the extractor so review comments can point at a row instead of a vibe.

Column	Source of truth	Allowed author	Fail the build when
key	JSON Schema property name	extractor only	missing from schema or invented in prose
type	JSON Schema type	extractor or formatter	contradicts schema
required	JSON Schema required array	extractor or formatter	contradicts schema
schema_description	JSON Schema description	extractor or formatter	text not present in schema
secret_class	operator policy	human	empty, or written inside a generated file
default_risk	operator incident review	human	empty, or claims a default absent from schema
rollout	operator change calendar	human	empty, or names an undocumented owner

Numbered ownership rules keep the table from drifting during review.

1. Extract keys, types, required flags, and descriptions before any Markdown exists.
2. Allow a formatter or model to touch those four cells only, never to insert a fifth key.
3. Leave secret_class, default_risk, and rollout blank in every generated file.
4. Merge only when the sidecar contains a human value for each signed cell on every key.

Working tree for the fixture

Keep schema, facts, draft, sidecar, and gate as separate files so diffs stay reviewable. The names below are a proposal for a small repository, not a required house style. Stable paths matter more than the brand of documentation generator you wrap around them.

docs-catalog/
  schema/app-config.schema.json
  facts/app-config.facts.json
  draft/app-config.compile.md
  draft/app-config.compile.json
  signed/app-config.human.json
  policy/secret-class.json
  tools/extract_facts.py
  tools/render_compile.py
  tools/gate_catalog.py
  Makefile

The extractor reads schema only. The renderer reads facts only. The gate reads facts, compile JSON, sidecar, and policy together, then prints a machine-readable failure list. Humans never paste a live .env into that path, because live values are not documentation facts.

1. Freeze schema facts before prose exists

Start with a JSON Schema that already lists properties, types, required keys, and descriptions. Do not ask a model to invent that schema from a chat transcript. The fixture below is small enough to review in one sitting and large enough to exercise the gate, including one credential-shaped key with no schema default.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "AppConfig",
  "type": "object",
  "required": ["LOG_LEVEL", "API_BASE_URL"],
  "properties": {
    "LOG_LEVEL": {
      "type": "string",
      "enum": ["debug", "info", "warn", "error"],
      "description": "Process log verbosity."
    },
    "API_BASE_URL": {
      "type": "string",
      "description": "Base URL for the public API client."
    },
    "SESSION_SIGNING_KEY": {
      "type": "string",
      "description": "HMAC key for session cookies."
    },
    "FEATURE_EXPORT_CSV": {
      "type": "boolean",
      "description": "Enables the CSV export handler."
    }
  }
}

The extractor walks properties and required, then writes a facts file with a stable key order. Stable order keeps later Markdown diffs small when only one description changes. Defaults are recorded only when the schema actually contains a default keyword, so later risk text cannot pretend a shipping default exists.

# tools/extract_facts.py
from __future__ import annotations

import json
import sys
from pathlib import Path


def extract(schema: dict) -> dict:
    required = set(schema.get("required", []))
    properties = schema.get("properties") or {}
    keys = []
    for name in sorted(properties):
        node = properties[name] or {}
        keys.append(
            {
                "key": name,
                "type": node.get("type", "unknown"),
                "required": name in required,
                "enum": node.get("enum"),
                "schema_description": node.get("description") or "",
                "has_default": "default" in node,
                "schema_default": node.get("default") if "default" in node else None,
            }
        )
    return {"title": schema.get("title") or "Config", "keys": keys}


def main() -> None:
    src = Path(sys.argv[1])
    dest = Path(sys.argv[2])
    schema = json.loads(src.read_text(encoding="utf-8"))
    dest.parent.mkdir(parents=True, exist_ok=True)
    dest.write_text(json.dumps(extract(schema), indent=2) + "\n", encoding="utf-8")


if __name__ == "__main__":
    main()

Run the extractor from Make so reviewers do not paste schema into a chat window. The facts file is the only input a remote draft should ever see, because it contains no live values and no production hostnames beyond what the schema already published.

facts/app-config.facts.json: schema/app-config.schema.json tools/extract_facts.py
    python tools/extract_facts.py $< $@

draft/app-config.compile.json: facts/app-config.facts.json tools/render_compile.py
    python tools/render_compile.py $< draft/app-config.compile.json draft/app-config.compile.md

gate:
    python tools/gate_catalog.py \
      facts/app-config.facts.json \
      draft/app-config.compile.json \
      signed/app-config.human.json \
      policy/secret-class.json

2. Render compile-lane Markdown from facts only

A local renderer should remain the default path, because table layout does not require a model. An optional remote draft is useful only when descriptions need light copy-editing for column width, and only when the input is the facts file. If you already keep schema and prose apart, an optional draft pass through MonkeyCode can format the compile lane on free model access and the free server option without receiving a live environment file. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

This fixture keeps the stricter rule: compile cells must match facts, including description text. If a later experiment stores a paraphrased draft_description, keep schema_description beside it so the gate can still prove provenance. Do not send .env, cookie material, or unsigned risk notes to any remote endpoint.

# tools/render_compile.py
from __future__ import annotations

import json
import sys
from pathlib import Path


def render(facts: dict) -> tuple[dict, str]:
    rows = []
    lines = [
        f"# {facts['title']} compile lane",
        "",
        "| key | type | required | schema_description |",
        "| --- | --- | --- | --- |",
    ]
    for item in facts["keys"]:
        rows.append(
            {
                "key": item["key"],
                "type": item["type"],
                "required": item["required"],
                "schema_description": item["schema_description"],
            }
        )
        desc = item["schema_description"].replace("|", "\\|")
        lines.append(
            f"| {item['key']} | {item['type']} | {str(item['required']).lower()} | {desc} |"
        )
    lines.append("")
    lines.append("Human-owned columns live in signed/app-config.human.json.")
    lines.append("")
    return {"title": facts["title"], "keys": rows}, "\n".join(lines)


def main() -> None:
    facts = json.loads(Path(sys.argv[1]).read_text(encoding="utf-8"))
    compile_json, compile_md = render(facts)
    json_path = Path(sys.argv[2])
    md_path = Path(sys.argv[3])
    json_path.parent.mkdir(parents=True, exist_ok=True)
    md_path.parent.mkdir(parents=True, exist_ok=True)
    json_path.write_text(json.dumps(compile_json, indent=2) + "\n", encoding="utf-8")
    md_path.write_text(compile_md, encoding="utf-8")


if __name__ == "__main__":
    main()

Notice the rendered table has no secret class, no default-risk paragraph, and no rollout owner. Those sentences do not exist until a reviewer writes them. That absence is a feature of the compile lane, not an unfinished draft.

3. Fill the three human-owned cells in a sidecar

The sidecar is JSON so the gate can test emptiness without parsing Markdown. Humans edit it during review, not inside a generated file. Allowed secret_class values stay tiny on purpose: none, restricted, and credential. Broader taxonomies belong in policy documents, not in a merge gate that must stay deterministic.

{
  "reviewer": "docs-oncall",
  "keys": {
    "LOG_LEVEL": {
      "secret_class": "none",
      "default_risk": "Unset verbosity falls back to process defaults; debug can leak request paths in shared logs.",
      "rollout": "Ship with info in production. Change requires a 24h freeze and log-pipeline review."
    },
    "API_BASE_URL": {
      "secret_class": "none",
      "default_risk": "There is no schema default. A wrong base URL silently sends client traffic to the wrong origin.",
      "rollout": "Change behind a feature flag. Rollback owner is the API client maintainers."
    },
    "SESSION_SIGNING_KEY": {
      "secret_class": "credential",
      "default_risk": "There is no schema default and no sample value. An empty key would mint unverifiable cookies.",
      "rollout": "Rotate out of band. Never paste values into tickets, catalogs, or model prompts."
    },
    "FEATURE_EXPORT_CSV": {
      "secret_class": "restricted",
      "default_risk": "Enabling export can enlarge bulk data leaving the app. There is no schema default.",
      "rollout": "Enable per tenant after a data-review checklist. Rollback is a config flip, not a migrate."
    }
  }
}

Policy sits in a fourth file so the gate can reject a credential-shaped key classified as none without hard-coding product names in Python. The fixture policy only lists keys that must not be none. Teams can extend that list when a new signing secret appears in schema.

{
  "allowed_secret_class": ["none", "restricted", "credential"],
  "forbid_secret_class_none": ["SESSION_SIGNING_KEY"]
}

4. Gate merge with key-set equality and empty-cell checks

The gate is the actual documentation test. It does not score prose quality, and it does not call a model. It proves three mechanical properties: the compile JSON and facts file name the same keys, every facts key has three non-empty signed cells, and policy forbids none on listed credential keys. It also rejects a default-risk sentence that claims a schema default when has_default is false.

# tools/gate_catalog.py
from __future__ import annotations

import json
import sys
from pathlib import Path


def load(path: str) -> dict:
    return json.loads(Path(path).read_text(encoding="utf-8"))


def main() -> int:
    facts, compiled, human, policy = (load(p) for p in sys.argv[1:5])
    failures: list[str] = []
    fact_keys = [item["key"] for item in facts["keys"]]
    compile_keys = [item["key"] for item in compiled["keys"]]
    if set(fact_keys) != set(compile_keys) or fact_keys != compile_keys:
        failures.append("compile keys must equal sorted facts keys")
    human_keys = set((human.get("keys") or {}).keys())
    if human_keys != set(fact_keys):
        failures.append("sidecar keys must equal facts keys")
    allowed = set(policy.get("allowed_secret_class") or [])
    forbid_none = set(policy.get("forbid_secret_class_none") or [])
    facts_by_key = {item["key"]: item for item in facts["keys"]}
    for key in fact_keys:
        row = (human.get("keys") or {}).get(key) or {}
        for cell in ("secret_class", "default_risk", "rollout"):
            value = (row.get(cell) or "").strip()
            if not value or value.upper() == "TODO":
                failures.append(f"{key}.{cell} is empty")
        secret_class = (row.get("secret_class") or "").strip()
        if secret_class and secret_class not in allowed:
            failures.append(f"{key}.secret_class is not in policy")
        if key in forbid_none and secret_class == "none":
            failures.append(f"{key} cannot be secret_class=none")
        risk = (row.get("default_risk") or "").lower()
        if not facts_by_key[key]["has_default"] and "schema default is" in risk:
            failures.append(f"{key}.default_risk claims a schema default that does not exist")
    for item in compiled["keys"]:
        blob = json.dumps(item)
        if "secret_class" in blob or "default_risk" in item or "rollout" in item:
            failures.append(f"{item['key']} compile row contains human-owned fields")
    for line in failures:
        print(f"FAIL {line}")
    if failures:
        print(f"{len(failures)} catalog gate failure(s)")
        return 1
    print("catalog gate passed")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Reproducible test plan

Label the following as an unexecuted fixture plan unless you run it on your own checkout. Each step names the input mutation and the expected exit code. That is the artifact reviewers can copy into CI without adopting any product narrative.

1. Run extract_facts.py on the fixture schema and expect four keys in sorted order: API_BASE_URL, FEATURE_EXPORT_CSV, LOG_LEVEL, SESSION_SIGNING_KEY.
2. Run render_compile.py and expect a four-column table with no secret_class column and compile JSON keys identical to facts keys.
3. Delete SESSION_SIGNING_KEY.rollout from the sidecar, run the gate, and expect exit code 1 with SESSION_SIGNING_KEY.rollout is empty.
4. Add REDIS_URL to compile JSON only, run the gate, and expect exit code 1 because compile keys no longer equal facts keys.
5. Set SESSION_SIGNING_KEY.secret_class to none, run the gate, and expect exit code 1 from the policy list.
6. Restore the sidecar shown above, run the gate, and expect exit code 0 and the line catalog gate passed.

These checks are data-driven in the narrow sense: they count keys, compare sets, and reject empty strings. They do not claim a reduction in incidents, and they do not publish latency or quality scores for any model.

Limitations

JSON Schema descriptions go stale when code changes and the schema file does not. The extractor as written does not resolve $ref, allOf, or vendor extensions, so composed schemas will under-count keys until you add a ref walker. The gate does not prove that runtime configuration matches the schema, and it cannot see secrets that operators inject only in a cloud console.

Optional remote drafting can still paraphrase a description if you relax the verbatim rule. Free model access and a free server option do not change that limitation, and this article does not assert quotas, named models, hardware, duration, or benchmark numbers. Catalog key names can themselves be sensitive in some threat models; if publishing the key list is forbidden, this workflow is the wrong control.

Who should not use this approach

Skip the pipeline if the team has no schema, struct, or equivalent machine source of truth for configuration. Skip it for narrative architecture essays that are not catalogs. Skip it if the operating rule is to upload .env files or session material to a remote model for “better docs.” Skip it when rollout language is legally privileged and must not live next to public schema text in the same git tree.

Teams that already generate OpenAPI field tables should not paste this catalog beside those tables without a join key. Two generated inventories of the same process environment will drift, and the gate above only protects one facts file. In that situation, pick a single extract path and point both renderers at it.

The core conclusion does not depend on any vendor lane remaining available. Extract keys from schema, keep risk sentences in a signed sidecar, and fail the build when those sets disagree. That is the catalog test; prose generators are optional formatters sitting beside it.