DEV Community

Avery Lin
Avery Lin

Posted on

Compile a Config Atlas From Schema; Hand-Write Secret Class and Blast Radius

Configuration reference pages rot when humans copy keys by hand and when models invent defaults that never existed in schema. The durable split is mechanical rather than cultural: compile every key, type, and constraint from the schema file. A human-owned semantics file must then supply secret class, blast radius, and rollback notes for each key. A small atlas compiler plus a stale-key test makes that split enforceable in CI.

Why review cannot own this split

Pull-request review fails at config documentation for a structural reason, not a motivation problem among writers. Schemas gain keys across many commits, while narrative files lag until an incident forces an expensive rewrite. Models asked to refresh the docs will fill those gaps with plausible defaults, example hostnames, and production advice that no schema field actually supports.

The failure mode is mixed authority sitting on a single Markdown page. Generated type columns sit beside unsigned claims such as safe-for-production with no file boundary between them. Reviewers then argue about tone while the real defect is a key that never received a secret class. Treat the atlas as a compile unit and the semantics tree as a signed unit, then let CI reject drift between those trees.

What a model may draft versus what a human must own

Use the decision table below as the contract for any documentation-generation pass on configuration. Rows are content classes, not writing-style preferences for the same paragraph. If a cell says human-owned, a model may propose a TODO scaffold only, never a publishable sentence.

Content class Source of truth Model may draft Human must own
Key path, JSON type, enum, min, max Schema Table rows and column layout Nothing; do not hand-edit generated rows
Default value Schema default only Render that default or emit an em dash Never invent a typical production default
Example value Human file Placeholder tokens such as <host> Any value that could reach a real system
Secret class (public / restricted / secret) Human file Empty enum plus a comment The classification itself
Blast radius (process / cluster / customer) Human file Section headings The radius claim
Rollback and never-set-in-prod orders Human file Checklist skeleton Every imperative claim
Cross-links by key prefix Derived paths Suggested link lists Whether a link is operationally valid

This table is the article's contract rather than a style guide. Later sections implement it as files and a failing test so the boundary does not depend on reviewer memory.

Artifact: three files and one gate

The proposed layout keeps generated bytes out of the semantics tree on purpose. Operators should treat atlas/ as build output and semantics/ as the only review surface for claims.

config/
  schema.json
  atlas/
    keys.json          # generated; do not edit
    reference.md       # generated tables only
  semantics/
    OWNERSHIP.yml      # human: secret_class, blast_radius, rollback
    examples.yml       # human: example values
  tools/
    compile_atlas.py
    check_stale_keys.py
Enter fullscreen mode Exit fullscreen mode

1. Start from a reviewed schema, not from chat

The schema below is a fixture for the compiler, not a production contract copied from any live service. Nested objects flatten to dotted paths so the stale-key test can perform exact string matches. Do not ask a model to author this file from a README, because chat output is not an input fact set.

{
  "$id": "https://example.invalid/config.schema.json",
  "type": "object",
  "additionalProperties": false,
  "properties": {
    "http": {
      "type": "object",
      "properties": {
        "bind_addr": { "type": "string", "default": "127.0.0.1:8080" },
        "timeout_ms": {
          "type": "integer",
          "minimum": 1,
          "maximum": 60000,
          "default": 5000
        }
      },
      "required": ["bind_addr"]
    },
    "auth": {
      "type": "object",
      "properties": {
        "api_token": { "type": "string", "minLength": 16 },
        "issuer": { "type": "string" }
      },
      "required": ["api_token"]
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Required flags stay in the schema object rather than in prose. The compiler records paths and constraints; it does not infer which keys are secrets from English names.

2. Compile the atlas as JSON, then render Markdown tables

The compiler walks properties, records constraints, and refuses to emit a default unless the schema defined one. The Python below is a proposed example, not a published package, and it has not been executed against a private corpus.

# tools/compile_atlas.py — proposed example, not a published package
from pathlib import Path
import json


def walk(node, prefix=""):
    rows = []
    if node.get("type") == "object":
        for name, child in (node.get("properties") or {}).items():
            path = f"{prefix}.{name}" if prefix else name
            rows.extend(walk(child, path))
        return rows
    return [{
        "key": prefix,
        "type": node.get("type", "unknown"),
        "default": node["default"] if "default" in node else None,
        "minimum": node.get("minimum"),
        "maximum": node.get("maximum"),
        "minLength": node.get("minLength"),
    }]


def main():
    schema = json.loads(Path("config/schema.json").read_text())
    rows = walk(schema)
    Path("config/atlas").mkdir(parents=True, exist_ok=True)
    Path("config/atlas/keys.json").write_text(json.dumps(rows, indent=2) + "\n")
    lines = [
        "# Configuration atlas",
        "",
        "| Key | Type | Default | Constraints |",
        "| --- | --- | --- | --- |",
    ]
    for r in rows:
        default = "`" + json.dumps(r["default"]) + "`" if r["default"] is not None else ""
        constraints = []
        if r["minimum"] is not None:
            constraints.append(f"min {r['minimum']}")
        if r["maximum"] is not None:
            constraints.append(f"max {r['maximum']}")
        if r["minLength"] is not None:
            constraints.append(f"minLength {r['minLength']}")
        lines.append(
            f"| `{r['key']}` | {r['type']} | {default} | {', '.join(constraints) or ''} |"
        )
    Path("config/atlas/reference.md").write_text("\n".join(lines) + "\n")


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

The generated Markdown contains keys, types, defaults, and constraints only. It must not contain the words production, secret, safe, or rollback, because those words are claims rather than schema facts.

3. Keep human claims in a typed ownership file

OWNERSHIP.yml is the signed unit, and every atlas key must appear in it exactly once. Values are closed enumerations so a model cannot invent a fourth secret class during a draft pass. Example values live in a second human file so a generator never copies a live token into reference.md.

# config/semantics/OWNERSHIP.yml — human-owned; review this file
keys:
  http.bind_addr:
    secret_class: public
    blast_radius: process
    rollback: "Restore the previous bind_addr and restart the process."
  http.timeout_ms:
    secret_class: public
    blast_radius: process
    rollback: "Lower timeout_ms only after checking upstream p99, then restart."
  auth.api_token:
    secret_class: secret
    blast_radius: customer
    rollback: "Rotate the token in the secret store; do not paste it into tickets."
  auth.issuer:
    secret_class: restricted
    blast_radius: cluster
    rollback: "Point issuer back to the prior URL; flush provider caches."
Enter fullscreen mode Exit fullscreen mode
# config/semantics/examples.yml — human-owned
keys:
  http.bind_addr: "127.0.0.1:8080"
  http.timeout_ms: "5000"
  auth.api_token: "<rotate-in-secret-store>"
  auth.issuer: "https://issuer.example.invalid"
Enter fullscreen mode Exit fullscreen mode

Rollback strings are reviewed as operations, not as documentation flavor. If a key has no rollback sentence, the gate fails even when the generated table looks complete.

4. Fail the build on stale, missing, or invented keys

The gate is a set comparison, which keeps the test boring and reviewable. Proposed checker, also unexecuted as a package:

# tools/check_stale_keys.py — proposed example
from pathlib import Path
import json, sys, yaml

atlas = {r["key"] for r in json.loads(Path("config/atlas/keys.json").read_text())}
own = yaml.safe_load(Path("config/semantics/OWNERSHIP.yml").read_text())["keys"]
ex = yaml.safe_load(Path("config/semantics/examples.yml").read_text())["keys"]
allowed_secret = {"public", "restricted", "secret"}
allowed_blast = {"process", "cluster", "customer"}
errors = []

if set(own) != atlas:
    errors.append(
        f"OWNERSHIP keys != atlas: extra={set(own) - atlas} missing={atlas - set(own)}"
    )
if set(ex) != atlas:
    errors.append(
        f"examples keys != atlas: extra={set(ex) - atlas} missing={atlas - set(ex)}"
    )

for key, rec in own.items():
    if rec.get("secret_class") not in allowed_secret:
        errors.append(f"{key}: invalid secret_class")
    if rec.get("blast_radius") not in allowed_blast:
        errors.append(f"{key}: invalid blast_radius")
    if not str(rec.get("rollback", "")).strip():
        errors.append(f"{key}: empty rollback")

ref = Path("config/atlas/reference.md").read_text().lower()
for banned in ("production", "secret", "safe", "rollback"):
    if banned in ref:
        errors.append(f"generated reference.md contains banned claim word: {banned}")

if errors:
    print("\n".join(errors))
    sys.exit(1)
print(f"atlas gate ok: {len(atlas)} keys")
Enter fullscreen mode Exit fullscreen mode

Wire both tools as a single CI step so a schema change without a semantics update is a red build. The optional git diff --exit-code step rejects uncommitted atlas drift after compile.

# ci/config-atlas.yml — proposed local or hosted job
steps:
  - run: pip install pyyaml
  - run: python tools/compile_atlas.py
  - run: python tools/check_stale_keys.py
  - run: git diff --exit-code -- config/atlas
Enter fullscreen mode Exit fullscreen mode

Reviewers then read OWNERSHIP.yml instead of scanning a mixed page for silent default changes. The generated table is evidence of schema shape, not a second place to negotiate production policy.

Numbered workflow for a documentation-generation pass

Follow this sequence on every config-key change. Skipping the local gate and hoping review will notice a missing secret class is how the mixed-authority page returns.

  1. Change schema.json in the same commit that introduces or removes a configuration key.
  2. Run python tools/compile_atlas.py and inspect keys.json for type and default fidelity only.
  3. If the stale-key test reports missing keys, add rows to OWNERSHIP.yml and examples.yml by hand.
  4. Optionally ask a model to draft TODO scaffolds for new rows, then replace every TODO before merge.
  5. Run python tools/check_stale_keys.py and the atlas git diff --exit-code check locally.
  6. Merge only when generated tables and human claims refer to the identical key set.

Step 4 is the only model-shaped step in the sequence. The model may list new key paths and emit empty YAML stubs with secret_class: TODO. Publishing those stubs is a gate failure, which is the reason the enumerations are closed.

Where a free model and a free server fit

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A local compiler should remain the source of generated tables, because schema walking does not need a model at all. MonkeyCode's free model access is relevant only for step 4: drafting YAML stubs and section headings for new keys, still behind the stale-key gate. MonkeyCode's free server option is relevant when the team wants that gate on a shared runner without first provisioning a permanent box.

Keep schema files, ownership files, and the compiler in the application repository either way. Remote drafting does not become a second source of truth for defaults or secret class. If the stubber invents a key that the schema does not contain, the set comparison fails and the draft is discarded.

Limitations

JSON Schema features such as oneOf, if/then, and pattern properties are not covered by the walker above. Operators who rely on those keywords need a richer pointer language than dotted paths, plus tests for each alternative branch. Defaults that vary by environment must not be written into schema.json as global defaults; they belong in overlays that the atlas should list as unset.

Banned-word scanning on reference.md is a coarse heuristic rather than a linguistic analyzer. A later author who adds a safety column to the generator will trip the gate until the renderer is fixed. The check exists to keep claims out of generated bytes, not to score writing quality. Closed enumerations will also annoy teams that need a fourth blast-radius value; extend the enum in the checker and in review policy together, never in a model prompt alone.

Key names that contain a banned substring would fail a naive scan even when the generated cell is only a path. Replace the substring test with a column-level allowlist before using this gate on a schema that already uses those tokens as identifiers.

Who should not use this approach

Do not use this workflow for legal terms, privacy notices, or security advisories, because those documents are not schema-derived compile units. Do not use it when the product has no typed configuration schema and keys exist only in tribal runbooks. Do not feed production secret values into examples.yml or into a model prompt in order to make the docs look realistic.

Teams that already generate a full configuration manual from a single model session, including defaults and production guidance, should not add this compiler as a formatter. The compiler is a boundary, not a prettier for mixed pages. If that boundary is unacceptable, leave the docs fully handwritten and stop asking models to refresh them on every schema change.

The publishable result is a config reference whose tables cannot drift from schema and whose operational claims cannot appear without a named owner file. That surface is smaller than a generated narrative, and it is the surface a set-comparison test can actually defend.

Top comments (0)