Build an Assumption Ledger Before You Trust Agent-Written Config

Agents do not only invent CLI flags and package names. They invent places. A hostname that never resolved. A port nothing listens on. An env key your process never reads.

Here is the conclusion I keep coming back to: if a model wrote the config, every host, port, path, and environment variable in that patch is a claim. Prove the claim. Then apply. Not the other way around.

You already glance at the diff, right? That is not enough. A twenty-line Compose change can hide three services that do not exist in your repo. Did you grep for them, or did you trust the comment that said "standard Redis sidecar"?

This is a from-zero tutorial. Each stage ends with a verification command. If a stage fails, stop. Do not ask the agent to "fix" a failed check by writing more fiction.

What we are going to build

A tiny assumption ledger: JSON extracted from a unified diff, then checked against the real tree and the real machine.

The ledger is boring on purpose. Boring is auditable.

1. Capture a patch as proposed.diff.
2. Extract hosts, ports, env keys, and filesystem paths.
3. Verify each class of claim with local commands.
4. Only then allow git apply.

I will optionally send the extraction step to a free hosted model later. The gate itself stays deterministic. That split matters.

Stage 0 — Plant a sample lie

Create a throwaway directory. Do not point this at production config. Ever.

mkdir -p /tmp/assumption-ledger/app
cd /tmp/assumption-ledger
git init -q
printf 'from flask import Flask\napp = Flask(__name__)\n' > app/server.py
cat > app/.env.example <<'EOF'
APP_PORT=8080
DATABASE_URL=postgres://localhost:5432/app
EOF
git add app && git commit -qm 'seed'

Verify stage 0:

test -f app/server.py && git log -1 --oneline

You should see one commit. No Compose file. No Redis. That absence is the point.

Now save the agent's proposed patch. This is a labeled example, not a transcript from a production run.

cat > proposed.diff <<'EOF'
diff --git a/docker-compose.yml b/docker-compose.yml
new file mode 100644
--- /dev/null
+++ b/docker-compose.yml
@@ -0,0 +1,16 @@
+services:
+  web:
+    build: .
+    ports:
+      - "8080:8080"
+    environment:
+      CACHE_URL: redis://cache-worker:6379/0
+      BILLING_API: https://billing.internal.svc
+    volumes:
+      - ./certs/prod.pem:/run/certs/prod.pem:ro
+  cache-worker:
+    image: redis:7
+    ports:
+      - "6379:6379"
EOF

Three claims just appeared. cache-worker as a hostname. billing.internal.svc as if you run Kubernetes. ./certs/prod.pem as if the file exists. Would you apply this because Compose parsed?

Stage 1 — Lock a ledger schema

Do not start with a model. Start with a shape you can grep.

cat > ledger.schema.json <<'EOF'
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "required": ["hosts", "ports", "env_keys", "paths"],
  "properties": {
    "hosts": { "type": "array", "items": { "type": "string" } },
    "ports": { "type": "array", "items": { "type": "integer" } },
    "env_keys": { "type": "array", "items": { "type": "string" } },
    "paths": { "type": "array", "items": { "type": "string" } }
  }
}
EOF

Verify stage 1:

python3 - <<'PY'
import json, pathlib
json.loads(pathlib.Path('ledger.schema.json').read_text())
print('schema ok')
PY

If that prints schema ok, you have a contract. The extractor must fill it. Nothing else.

Stage 2 — Extract without a model

Regex will miss poetry in comments. It will not miss redis://cache-worker:6379. Start there. Models are for leftovers.

cat > extract_ledger.py <<'PY'
#!/usr/bin/env python3
"""Proposed extractor: deterministic claims from a unified diff."""
import json, re, sys
from pathlib import Path

HOST_RE = re.compile(
    r"(?:https?|redis|postgres|mysql|amqp)://([A-Za-z0-9._-]+)", re.I
)
HOSTNAME_RE = re.compile(
    r"\b([A-Za-z0-9-]+\.(?:internal|svc|local|lan))\b", re.I
)
PORT_RE = re.compile(r"(?<![\d.])(\d{2,5})(?![\d.])")
ENV_RE = re.compile(r"\b([A-Z][A-Z0-9_]{2,})=|environment:\s*\n(?:\s+[A-Z0-9_]+:.+")+"")
ENV_KEY_RE = re.compile(r"^\+\s*(?:-\s*)?([A-Z][A-Z0-9_]{2,}):\s", re.M)
ENV_ASSIGN_RE = re.compile(r"^\+\s*([A-Z][A-Z0-9_]{2,})=", re.M)
PATH_RE = re.compile(r"(?:^|\s)(\./[A-Za-z0-9._/-]+|/run/[A-Za-z0-9._/-]+)")

# Ports that show up in every YAML indent. Drop noise.
SKIP_PORTS = {0, 1, 7, 16}

def added_lines(diff: str) -> str:
    return "\n".join(
        line[1:] for line in diff.splitlines()
        if line.startswith("+") and not line.startswith("+++")
    )

def extract(diff: str) -> dict:
    added = added_lines(diff)
    hosts = set(HOST_RE.findall(added)) | set(HOSTNAME_RE.findall(added))
    ports = set()
    for m in re.finditer(r":(\d{2,5})\b", added):
        ports.add(int(m.group(1)))
    for m in re.finditer(r'"(\d{2,5}):(\d{2,5})"', added):
        ports.add(int(m.group(1)))
        ports.add(int(m.group(2)))
    env_keys = set(ENV_KEY_RE.findall(diff)) | set(ENV_ASSIGN_RE.findall(diff))
    paths = set(PATH_RE.findall(added))
    return {
        "hosts": sorted(h for h in hosts if h.lower() not in {"localhost"}),
        "ports": sorted(p for p in ports if p not in SKIP_PORTS),
        "env_keys": sorted(env_keys),
        "paths": sorted(paths),
    }

def main() -> int:
    diff = Path(sys.argv[1]).read_text()
    ledger = extract(diff)
    Path(sys.argv[2]).write_text(json.dumps(ledger, indent=2) + "\n")
    print(json.dumps(ledger, indent=2))
    return 0

if __name__ == "__main__":
    raise SystemExit(main())
PY
chmod +x extract_ledger.py
python3 extract_ledger.py proposed.diff ledger.json

Verify stage 2:

python3 - <<'PY'
import json
from pathlib import Path
led = json.loads(Path('ledger.json').read_text())
assert 'cache-worker' in led['hosts'] or 'billing.internal.svc' in led['hosts']
assert 6379 in led['ports'] or 8080 in led['ports']
assert 'CACHE_URL' in led['env_keys'] or 'BILLING_API' in led['env_keys']
assert any('prod.pem' in p for p in led['paths'])
print('extractor caught the planted lies')
PY

If that assertion fails, fix the extractor. Do not shrug and "let the LLM handle it." A model that missed prod.pem will also miss the next secret path.

Stage 3 — Verify each class of claim

Extraction without checks is journaling. We need a fail-closed verifier.

cat > verify_ledger.py <<'PY'
#!/usr/bin/env python3
"""Proposed verifier: every ledger entry must be justified by the repo or the operator."""
import json, os, re, socket, sys
from pathlib import Path

ROOT = Path(sys.argv[1]).resolve()
ledger = json.loads(Path(sys.argv[2]).read_text())
allow_hosts = {
    h.strip() for h in Path('allow.hosts').read_text().splitlines()
    if h.strip() and not h.startswith('#')
} if Path('allow.hosts').exists() else set()

errors = []

# Hosts must be in the repo as a service name, or on an allowlist.
compose = ''
for p in ROOT.rglob('docker-compose*.yml'):
    compose += p.read_text() + '\n'
for p in ROOT.rglob('compose*.yaml'):
    compose += p.read_text() + '\n'

# We verify against the *current tree*, not the patch. That is the point.
repo_text = []
for p in ROOT.rglob('*'):
    if p.is_file() and p.suffix in {'.py', '.env', '.example', '.yml', '.yaml', '.md', '.toml'}:
        try:
            repo_text.append(p.read_text())
        except UnicodeDecodeError:
            pass
blob = '\n'.join(repo_text)

for host in ledger['hosts']:
    named_in_compose = bool(re.search(rf'^\s{{2}}{re.escape(host)}:', compose, re.M)) if compose else False
    mentioned = host in blob
    if host in allow_hosts:
        continue
    if not named_in_compose and not mentioned:
        errors.append(f'host {host!r} is not in the current tree or allow.hosts')

for key in ledger['env_keys']:
    if key not in blob and not any(key in line for line in blob.splitlines()):
        errors.append(f'env key {key!r} is not declared in the current tree')

for path in ledger['paths']:
    rel = path[2:] if path.startswith('./') else path
    local = ROOT / rel if not path.startswith('/') else Path(path)
    # Absolute container paths are claims about the image, not the laptop.
    if path.startswith('/'):
        errors.append(f'container path {path!r} has no matching file in the repo')
        continue
    if not (ROOT / rel).exists():
        errors.append(f'path {path!r} does not exist in the repo')

for port in ledger['ports']:
    # Bound ports on loopback are informational. Unmentioned ports in a
    # brand-new file are still claims; require them in .env.example or docs.
    if str(port) not in blob:
        errors.append(f'port {port} is not mentioned in the current tree')

if errors:
    print('FAIL')
    print('\n'.join(f'- {e}' for e in errors))
    sys.exit(1)
print('PASS')
PY

Write an empty allowlist so the rule is visible.

printf '# one hostname per line, only after a human signed off\n' > allow.hosts
python3 verify_ledger.py app ledger.json; echo exit:$?

Verify stage 3: you want a non-zero exit. cache-worker is only in the patch, not in app/. billing.internal.svc never existed. ./certs/prod.pem is missing. The verifier should refuse.

If it prints PASS, your checks are too weak. Tighten them before you wire this into git apply.

Stage 4 — Optional LLM pass for comments, not for truth

Deterministic extractors skip sentences like "spin up the usual billing sidecar on the internal mesh." That is the only gap I will give a model.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I treat MonkeyCode here as a place to run that extraction prompt against free model access, and to park the job on the free server option so the laptop stays out of the loop. The verifier never leaves the repo. If the model adds a host the regex missed, it still has to survive verify_ledger.py.

Labeled prompt, not a production transcript:

You extract assumptions from a unified diff. Return JSON only with keys
hosts, ports, env_keys, paths. Do not invent services that are not named
or clearly implied. Do not include localhost. Never suggest applying the patch.

Merge rule: union of regex ledger and model ledger, then verify the union. The model does not get a veto. It gets extra candidates to fail.

Verify stage 4:

# After you write model-ledger.json from the prompt above:
python3 - <<'PY'
import json
from pathlib import Path
base = json.loads(Path('ledger.json').read_text())
# If you skipped the model, copy the regex ledger so the merge is still testable.
extra = json.loads(Path('model-ledger.json').read_text()) if Path('model-ledger.json').exists() else {"hosts":[],"ports":[],"env_keys":[],"paths":[]}
merged = {k: sorted(set(base[k]) | set(extra.get(k, []))) for k in base}
Path('ledger.merged.json').write_text(json.dumps(merged, indent=2) + '\n')
print('merged keys', {k: len(merged[k]) for k in merged})
PY
python3 verify_ledger.py app ledger.merged.json; echo exit:$?

Still fail-closed. Good.

Stage 5 — Apply only after PASS

Wrap the last mile so "the agent said it was fine" is not a merge path.

cat > apply_if_clean.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
diff_file=${1:?usage: apply_if_clean.sh proposed.diff}
python3 extract_ledger.py "$diff_file" ledger.json
python3 verify_ledger.py app ledger.json
git apply --check "$diff_file"
echo "ledger passed; git apply --check passed. Human still has to read the diff."
# Intentionally no git apply here.
EOF
chmod +x apply_if_clean.sh
./apply_if_clean.sh proposed.diff; echo exit:$?

Verify stage 5: the script must stop before git apply --check if the ledger failed. Read the output. If you see a successful apply check on this sample, you removed the verifier. Put it back.

When you want Redis, add it to the tree first. Commit the service. Then the same patch's cache-worker host becomes a current-tree fact, not a hallucination. Order of operations is the whole design.

What this will not catch

A hostname in allow.hosts is a human decision. If you rubber-stamp billing.internal.svc, the ledger becomes theater.

Ports that already appear in docs will pass even if nothing listens. This gate answers "did the agent invent a token?" not "is Redis healthy?"

Container paths like /run/certs/prod.pem cannot be proven from a laptop. I fail them on purpose. If your image really ships that file, say so in the repo with a Dockerfile COPY the extractor can see.

Do not send real .env values to any hosted model. Diffs that contain secrets should stay on the regex path only.

Who should not use this

Skip it if you already require generated Compose to come from a checked-in template with no free-form agent edits. You do not need a second gate.

Skip it if the "agent" is allowed to create greenfield infrastructure from an empty directory. Every host will be new. The ledger will fail everything, and you will disable it.

Skip it for production incident hotfixes where a human is already pairing on the exact service list. This workflow is for unattended or semi-attended patches, the kind that land at 11 p.m. because the model sounded sure.

The habit I want

Config is a set of promises about the world. Models generate promises cheaply. Checks are cheaper than outages.

Run the extractor. Read the ledger out loud. If you cannot explain why billing.internal.svc is allowed, it is not allowed. Apply the patch after the world matches the claims — not because the YAML indented cleanly.