Turn Event Constants Into a Facts File; Keep Replay and PII Off the Model

Event portals go stale when names are compiled from memory while delivery promises are left to a chat-shaped draft. A durable workflow extracts webhook and bus names from source, then refuses to publish until replay, ordering, and PII are signed. Language models may rewrite adjacent comments into short summaries and may scaffold redacted examples from fixtures you already trust. They must not invent broker semantics, customer blast radius, or privacy class, because those cells page humans during incidents.

Why event catalogs drift faster than route tables

Route tables usually sit beside HTTP handlers, so a missing path is visible during ordinary code review. Event names often hide in scattered string constants, which lets producers ship new payloads while the portal still lists last quarter. Operators then replay traffic using docs that never stated idempotency keys, and finance jobs double-post because the catalog stayed silent. The corrective control is a hashed facts file plus an overlay that CI treats like a required type, not like optional prose.

Incident write-ups cluster around four cells that look like documentation but behave like production contracts under load. Two services may share one event name while required fields diverge after a change that review labeled compatible. Portal text may claim at-least-once delivery while the broker remains best-effort, which turns a retry button into duplicate side effects. Example payloads may include email or PAN-shaped numbers, and unsigned blast-radius cells hide which jobs will charge twice during replay.

Decision table: compile-time facts versus signature-time promises

The table below splits compile-time facts from signature-time promises so reviewers can see which lane a cell belongs in. Extraction should stay deterministic and should hash in CI, because a model rewrite of names would hide producer drift. Prose drafts may consume comments and field labels only; production payloads and secrets stay out of that optional step.

Cell	Source of truth	Model may draft?	Human must sign?	Build fails if empty?
Event name and version	Source constants	No; extract only	No	Yes
Producer path	AST or emit-site index	No; extract only	Confirm unknown paths	Yes if unknown
Payload field list	Struct or schema tags	Yes, from schema	Confirm types	Yes
Comment summary	Adjacent comments	Yes	Edit	No
Delivery guarantee	Ops and broker contract	No	Yes	Yes
Ordering / partition key	Architecture owners	No	Yes	Yes
Replay policy	Incident owners	No	Yes	Yes
PII class	Privacy review	No	Yes	Yes
Blast radius	Service owners	No	Yes	Yes
Example payload	Checked fixtures	Yes, redacted	Approve	Yes if unredacted

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Teams that already extract constants in CI can send comment blocks to MonkeyCode's free model access on the free server option. The draft then merges into an overlay that humans still sign, which keeps generated sentences from becoming delivery contracts. That split also keeps model-shaped work off the production docs runner without implying quota, latency, hardware, or model-identity claims. Pin your own image, fail closed on unsigned cells, and treat free access as availability rather than a benchmark.

Numbered workflow

The numbered steps below are a compiler pipeline for event docs, not a writing style guide for portal tone. Each stage either extracts a fact, requires a human cell, or fails the job with a list of missing signatures. Skip any stage that would let a model fill signed_by, because a predicted owner is not an on-call rotation.

1. Freeze a facts file from event constants

Begin with a module that already lists event names as uppercase string constants, rather than with a transcript of remembered topics. The extractor walks a Python syntax tree, captures those assignments, and writes a JSON facts file that CI can hash every merge. Mark the JSON as generated output, and refuse hand edits after merge so drift always shows up as a source change.

# src/billing/events.py — source of truth for names, not for delivery prose
INVOICE_FINALIZED_V1 = "invoice.finalized.v1"
INVOICE_VOIDED_V1 = "invoice.voided.v1"

# extract_events.py — deterministic facts only; do not send this output through a model
import ast
import json
import pathlib
import sys

def string_constants(path: pathlib.Path) -> list[dict]:
    tree = ast.parse(path.read_text(encoding="utf-8"))
    rows = []
    for node in tree.body:
        if not isinstance(node, ast.Assign):
            continue
        if not isinstance(node.value, ast.Constant) or not isinstance(node.value.value, str):
            continue
        for target in node.targets:
            if isinstance(target, ast.Name) and target.id.isupper():
                rows.append({
                    "symbol": target.id,
                    "name": node.value.value,
                    "file": str(path),
                    "lineno": node.lineno,
                })
    return sorted(rows, key=lambda r: r["name"])

if __name__ == "__main__":
    src = pathlib.Path(sys.argv[1])
    out = pathlib.Path(sys.argv[2])
    payload = {"events": string_constants(src)}
    out.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n", encoding="utf-8")

python extract_events.py src/billing/events.py docs/events.generated.json
sha256sum docs/events.generated.json

Run the extractor in the docs job before render, and store the SHA next to the overlay so reviewers can see both artifacts. If constants live in several packages, invoke the script once per path and concatenate events under a stable sort key. Do not ask a model to invent missing names, because an invented name cannot be traced to a producer path.

2. Require an owned overlay with matching keys

The overlay is a human-edited YAML map whose keys must match generated event names exactly, including version suffixes. Required cells cover delivery, ordering, replay, PII class, blast radius, and a signed_by value that names a rotation, not a model. Empty cells fail the build on purpose, because an unsigned catalog is more dangerous than a missing page in the portal.

# docs/events.overlay.yaml — human-owned; never generate this file
events:
  invoice.finalized.v1:
    delivery: at_least_once
    ordering: per_account_id
    replay: "manual, finance on-call only, after idempotency check"
    pii: "account_id internal; never email, pan, or name"
    blast_radius: "ledger projector, dunning mailer, tax export"
    signed_by: "billing-oncall"
    signed_at: "2026-09-17"
  invoice.voided.v1:
    delivery: at_least_once
    ordering: per_account_id
    replay: "forbidden after tax export window"
    pii: "account_id internal; never email"
    blast_radius: "ledger projector, audit archive"
    signed_by: "billing-oncall"
    signed_at: "2026-09-17"

Date the signature on the day a person actually reviewed the broker contract, and refresh it when delivery language changes. A stale signed_at is still better than a generated claim, but owners should rotate signatures when the bus topology changes. Keep comments in the overlay for humans; never paste production payload samples into this file without a redaction review.

3. Optionally draft summaries, then discard unsigned prose

When constants have adjacent comments, a model may rewrite those comments into one-sentence summaries stored under a drafts directory. Copy nothing from drafts into the overlay until a human fills signed_by and the required operational cells listed above. The prompt boundary below is a proposal and remains unexecuted until you pin a runner image and a deny-list for payloads.

You rewrite adjacent source comments into one sentence.
Do not invent delivery, ordering, replay, PII class, or blast radius.
If the comment is missing, return UNREVIEWED rather than guessing.

# proposed local gate — label as a proposal until your runner exists
pip install pyyaml pytest
python extract_events.py src/billing/events.py docs/events.generated.json
python join_overlay.py docs/events.generated.json docs/events.overlay.yaml

Treat any draft that mentions delivery or replay as contaminated, and delete it rather than editing the invented contract language. Comment text is the only allowed model input in this step, which keeps customer data out of the optional rewrite. If a comment is missing, write UNREVIEWED into the draft and leave the portal summary blank until an owner writes one.

4. Join, render, and fail closed

The join program is the docs compiler: it renders Markdown and returns non-zero when any required overlay cell is empty. Unsigned values print as UNSIGNED in the portal source so a leaked preview cannot look complete during an incident. Wire this exit code to the same merge gate you use for tests, because docs that lie are a production defect.

# join_overlay.py — fail closed on missing human cells
import json
import pathlib
import sys
import yaml

REQUIRED = ("delivery", "ordering", "replay", "pii", "blast_radius", "signed_by")

def main(gen_path: str, overlay_path: str) -> None:
    generated = json.loads(pathlib.Path(gen_path).read_text(encoding="utf-8"))
    overlay = yaml.safe_load(pathlib.Path(overlay_path).read_text(encoding="utf-8"))["events"]
    missing: list[str] = []
    lines = ["# Event catalog", ""]
    for row in generated["events"]:
        name = row["name"]
        cell = overlay.get(name) or {}
        holes = [k for k in REQUIRED if not cell.get(k)]
        if holes:
            missing.append(f"{name}: {', '.join(holes)}")
        lines.append(f"## `{name}`")
        lines.append(f"- symbol: `{row['symbol']}` ({row['file']}:{row['lineno']})")
        lines.append(f"- delivery: {cell.get('delivery', 'UNSIGNED')}")
        lines.append(f"- ordering: {cell.get('ordering', 'UNSIGNED')}")
        lines.append(f"- replay: {cell.get('replay', 'UNSIGNED')}")
        lines.append(f"- pii: {cell.get('pii', 'UNSIGNED')}")
        lines.append(f"- blast radius: {cell.get('blast_radius', 'UNSIGNED')}")
        lines.append(f"- signed by: {cell.get('signed_by', 'UNSIGNED')}")
        lines.append("")
    pathlib.Path("docs/events.md").write_text("\n".join(lines), encoding="utf-8")
    if missing:
        print("unsigned overlay cells:")
        print("\n".join(missing))
        sys.exit(1)

if __name__ == "__main__":
    main(sys.argv[1], sys.argv[2])

Extend REQUIRED when owners add a new contract cell, and version that list in git so the compiler cannot quietly drop a gate. Do not let the model propose removing a required key to make the job green; that is an ownership change, not a style edit. Keep the miss list in CI logs so reviewers can assign overlay rows without reading the entire generated JSON file.

5. Add a redaction test, not a vibe check

Example payloads leak more often than narrative docs, so add a redaction test that fails on email marks and long digit runs. The test is a mechanical gate rather than a privacy program; legal classification still lives in the signed pii overlay cell. Skip the suite only when no example directory exists, and never skip it because a draft model promised the fixtures were clean.

# test_event_examples.py
import pathlib
import re
import pytest

FORBIDDEN = [
    re.compile(r"@"),
    re.compile(r"\b\d{13,19}\b"),
    re.compile(r"\b(?:ssn|pan)\b", re.I),
]

def test_examples_are_redacted():
    root = pathlib.Path("docs/examples")
    if not root.exists():
        pytest.skip("no examples yet")
    for path in root.glob("*.json"):
        text = path.read_text(encoding="utf-8")
        for pat in FORBIDDEN:
            assert not pat.search(text), f"possible PII in {path}"

pytest -q test_event_examples.py
python join_overlay.py docs/events.generated.json docs/events.overlay.yaml

What the model may draft, restated

Restated, the model may draft comment summaries, section order, and redacted scaffolding from fixtures that owners already accepted. It may propose a table layout when owners introduce a new required overlay key and need a render template for the portal. It must not classify PII, invent at-least-once language, or fill signed_by from a predicted team name in the repository graph. Those cells are operational contracts, and contracts need a person who can be paged when replay goes wrong at two in the morning.

Limitations and who should skip this

This approach assumes event names exist as stable constants in a small set of files that the extractor can parse. Teams that emit ad-hoc interpolated strings will generate an incomplete catalog and a false sense of coverage in the portal. The join script does not prove the broker honors delivery or ordering; it only proves a human typed those words into YAML. Do not send production payload captures into the draft step, because those captures are data subject to retention rules, not comments.

Skip this workflow when legal requires a records-retention review that no overlay cell can represent without counsel on the signature. Skip it when the bus is still unnamed and every service publishes a generic update object with no versioned event name. Skip it when the docs job cannot fail the merge, because unsigned catalogs become wallpaper that incidents will quote as policy. A free model on a free server does not relax those constraints; it only drafts sentences that owners still have to reject.

If continuous integration already hashes a generated event facts file, the useful next step is a signed overlay rather than a longer prompt. If the extractor is already green, a free-model pass on a free server can draft comment summaries for overlay review. Names stay in the compiler; promises stay with the rotation that will be paged when a replay is issued. Nothing in that optional pass should be allowed to write delivery, replay, PII class, or signed_by on its own.