Compile a Webhook Event Catalog From Code; Hand-Sign Delivery Guarantees

Webhook documentation fails when generated prose asserts delivery, ordering, or authentication that no schema file can prove. The practical split is a compiled event catalog plus a human-owned file of guarantees the compiler cannot observe. Generated Markdown may list event names, payload fields, fixture paths, and sample bodies without inventing production behavior. Only a reviewer should assert retry policy, ordering, authentication, PII handling, or public availability of an event.

Most outbound-event pages mix three different kinds of statement into one tutorial-shaped Markdown blob. Registration data is mechanical and should be compiled from the worker or publisher module on every pull request. Narrative glue is optional and can be drafted from that extract if a reviewer still edits the result. Delivery semantics, security promises, and customer-visible status are not properties of JSON Schema, and they must remain signed.

Separate the catalog from the warranty

Treat webhook documentation as two files that a renderer may merge, never as a single chat transcript. The extracted catalog answers what the process can emit today, including names, required fields, and fixture-derived examples. The owned warranty answers what the organization will stand behind after a customer builds a consumer. If those files disagree, continuous integration should fail before the documentation site publishes webhooks.md.

A compact decision table keeps the boundary explicit during both human review and later lint implementation. Mechanical columns come from registries and fixtures, while warranty columns require a named human signer. Draftable columns are limited to outlines and field descriptions that a forbidden-verb check can still reject.

Claim class	Source of truth	Model may draft?	Publish rule
Event name, payload keys, JSON types	Publisher registry and schema	No, compile only	Must match extract
Example body	Committed fixture	No, copy only	Hash the fixture
Section outline, field descriptions	Extract plus style guide	Yes, labeled DRAFT	Reviewer edit required
Public versus internal	Product owner	No	signed_by required
Delivery, retry, ordering	Platform owner	No	signed_by required
Auth and signature scheme	Security owner	No	signed_by required
PII and retention class	Privacy owner	No	signed_by required

The table is the method, not decoration, because it tells the linter which keys are legal in each file. Compile-only rows should never pass through a model, because names and types are already in source. Draft rows may use a model only after the extract exists and only with a DRAFT status marker. Warranty rows fail the build when signed_by is empty, even if the prose sounds confident.

Step 1: Register events in one module the extractor can read

Keep a single in-repo registry rather than scraping Markdown pages that a model already wrote. The following example is a local illustration, not a measured production corpus, and reviewers should adapt the loader to their publisher. An extractor that reads the docs/ directory as input will echo stale claims and hide missing registrations.

# events_registry.py — local illustration, not a production dump
EVENTS = {
    "invoice.finalized": {
        "schema": {
            "type": "object",
            "required": ["invoice_id", "currency", "total_cents"],
            "properties": {
                "invoice_id": {"type": "string"},
                "currency": {"type": "string"},
                "total_cents": {"type": "integer"},
                "customer_id": {"type": "string"},
            },
        },
        "fixture": "fixtures/invoice_finalized.json",
        "publisher": "billing/webhooks.py",
    },
    "invoice.voided": {
        "schema": {
            "type": "object",
            "required": ["invoice_id", "reason"],
            "properties": {
                "invoice_id": {"type": "string"},
                "reason": {"type": "string"},
            },
        },
        "fixture": "fixtures/invoice_voided.json",
        "publisher": "billing/webhooks.py",
    },
}

The extractor should compile JSON that records event names, required keys, fixture hashes, and source paths only. Fixture hashes make example drift visible when someone edits a sample without updating the published page. Source paths give reviewers a jump to the publisher, which is faster than trusting a generated summary.

# extract_events.py — local illustration
import hashlib
import json
import pathlib
import importlib


def sha256(path: str) -> str:
    data = pathlib.Path(path).read_bytes()
    return hashlib.sha256(data).hexdigest()


def extract(module_name: str, out: str) -> None:
    mod = importlib.import_module(module_name)
    catalog = []
    for name, spec in sorted(mod.EVENTS.items()):
        catalog.append({
            "name": name,
            "required": spec["schema"]["required"],
            "properties": sorted(spec["schema"]["properties"]),
            "fixture": spec["fixture"],
            "fixture_sha256": sha256(spec["fixture"]),
            "publisher": spec["publisher"],
        })
    pathlib.Path(out).write_text(json.dumps({"events": catalog}, indent=2) + "\n")


if __name__ == "__main__":
    extract("events_registry", "events.extracted.json")

Run the extractor in CI so a newly registered event cannot ship without a documentation row. A one-line git diff check keeps the catalog reviewed like generated code instead of like a blog paragraph. If the extract changes, the pull request must show it, even when no human edited documentation files.

python extract_events.py
git diff --exit-code events.extracted.json

Step 2: Store warranties in YAML that the model does not write

Create events.owned.yaml with keys the extractor is forbidden to emit under any circumstance. Every event in the extract must have a matching owned record before the renderer writes customer Markdown. Missing rows are build failures rather than TODOs, because unsigned public events become accidental contracts.

# events.owned.yaml — human-edited illustration
events:
  invoice.finalized:
    public: true
    delivery: at_least_once
    ordering: none
    retry: "1m, 10m, 1h; abandoned after 24h"
    auth: hmac_sha256_header
    pii: customer_id_is_internal_identifier
    compatibility: "additive fields only through 2026-12-31"
    owner: billing-platform
    signed_by: "[e-mail]"
    signed_at: "2026-09-21"
  invoice.voided:
    public: false
    delivery: unknown
    ordering: unknown
    retry: unspecified
    auth: internal_only
    pii: unreviewed
    compatibility: internal
    owner: billing-platform
    signed_by: "[e-mail]"
    signed_at: "2026-09-21"

Unknown is a legal value and is often more honest than a generated exactly-once sentence in customer docs. Internal events should remain in the catalog so reviewers see them during deletion and rename work. The renderer then omits public: false rows from the customer site while CI still demands signatures. Compatibility strings are owned claims with dates, and they need a later review when signed_at ages.

Step 3: Lint unsigned events and leaked warranty verbs

The linter is the actual documentation test because it grades signatures and leaked verbs rather than style. It checks whether owned records exist, whether names match the extract, and whether draft text smuggles warranty language. Prose quality remains a human problem, and that limit should stay visible in the job output.

# lint_webhook_docs.py — local illustration
import json
import pathlib
import re
import sys
import yaml

FORBIDDEN_IN_DRAFT = re.compile(
    r"\b(exactly-once|at-least-once|guarantees|never retries|PCI|public API)\b",
    re.I,
)
REQUIRED_OWNED = {
    "public", "delivery", "ordering", "retry", "auth",
    "pii", "compatibility", "owner", "signed_by", "signed_at",
}


def load():
    extracted = json.loads(pathlib.Path("events.extracted.json").read_text())
    owned = yaml.safe_load(pathlib.Path("events.owned.yaml").read_text())
    return extracted, owned


def lint(extracted, owned) -> list[str]:
    errors = []
    extracted_names = {row["name"] for row in extracted["events"]}
    owned_names = set(owned["events"])
    for name in sorted(extracted_names - owned_names):
        errors.append(f"unsigned event: {name}")
    for name in sorted(owned_names - extracted_names):
        errors.append(f"owned event not in extract: {name}")
    for name, rec in owned["events"].items():
        missing = REQUIRED_OWNED - set(rec)
        if missing:
            errors.append(f"{name} missing keys: {sorted(missing)}")
        if not rec.get("signed_by") or not rec.get("signed_at"):
            errors.append(f"{name} lacks signature metadata")
    draft_path = pathlib.Path("events.draft.md")
    draft = draft_path.read_text() if draft_path.exists() else ""
    if draft and FORBIDDEN_IN_DRAFT.search(draft):
        errors.append("draft prose contains warranty verbs; move them to events.owned.yaml")
    return errors


if __name__ == "__main__":
    errs = lint(*load())
    if errs:
        print("\n".join(errs))
        sys.exit(1)
    print("webhook doc lint ok")

Label the draft file clearly so nobody pastes it into the customer portal without a second review. A header such as <!-- STATUS: DRAFT-FROM-EXTRACT --> makes the next reviewer faster and grep-friendly. Forbidden-verb lists are repository policy, and teams should extend them when a new over-claim appears in review.

Step 4: Draft outlines from the extract, not from a product chat log

Optional prose is useful for field descriptions and for a signature-verification skeleton that still cannot assert production retry behavior. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option can sit behind this optional drafting step without touching warranties. Prompt inputs should be the extracted JSON and a style guide, never the owned YAML file.

Feed the model the extract and ask for outlines labeled DRAFT so reviewers can see untrusted sentences. Do not send events.owned.yaml into the prompt if the goal is to prevent warranty invention. After the draft returns, run the forbidden-verb linter before a human copies any sentence into webhooks.md.

You receive events.extracted.json only.
Write Markdown section stubs: heading, field table, and a placeholder for examples.
Do not assert delivery, retries, ordering, authentication, PII, or public status.
Mark every paragraph with STATUS: DRAFT-FROM-EXTRACT.

The free server option matters as a place to run extract, lint, and draft jobs without mixing those roles. Signing keys and signed_by edits belong on a reviewed pull request, not in the drafting workspace. No model name, quota, or hardware claim is required for the split to work, and none is asserted here.

Step 5: Render customer docs from both files after lint passes

The renderer should copy mechanical fields from the extract and copy warranties only from owned YAML. Unsigned events and internal events must not appear on the customer page even if draft stubs exist. Example bodies should come from hashed fixtures rather than from JSON regenerated during the render job.

# render_webhooks.py — local illustration
import json
import pathlib
import yaml


def render() -> None:
    extracted = json.loads(pathlib.Path("events.extracted.json").read_text())
    owned = yaml.safe_load(pathlib.Path("events.owned.yaml").read_text())
    lines = ["# Webhooks", ""]
    by_name = {row["name"]: row for row in extracted["events"]}
    for name, warranty in owned["events"].items():
        if not warranty.get("public"):
            continue
        row = by_name[name]
        lines += [
            f"## `{name}`",
            "",
            f"- Delivery: {warranty['delivery']}",
            f"- Ordering: {warranty['ordering']}",
            f"- Retry: {warranty['retry']}",
            f"- Auth: {warranty['auth']}",
            f"- Compatibility: {warranty['compatibility']}",
            f"- Required fields: {', '.join(row['required'])}",
            f"- Fixture: `{row['fixture']}` (sha256 `{row['fixture_sha256'][:12]}…`)",
            "",
        ]
    pathlib.Path("webhooks.md").write_text("\n".join(lines))


if __name__ == "__main__":
    render()

Wire the three programs in a single target so nobody runs render before lint.

# illustration
docs-webhooks:
    python extract_events.py
    python lint_webhook_docs.py
    python render_webhooks.py

Continuous integration should run extract, lint, and render in that order before any documentation deploy job. When the site is generated, a git diff --exit-code webhooks.md check keeps unpublished drift off the default branch. Reviewers still read the owned YAML, because that is where a wrong at-least-once claim would appear. A passing lint run is not a load test of the webhook pipeline and should not be described as one.

What this workflow does not prove

The pipeline does not prove that production publishers match the registry without a runtime assertion on emit. It does not prove HMAC verification works, and it does not replace contract tests against a staging bus. Fixture hashes detect silent example drift, but they do not prove a fixture remains valid after schema changes. A JSON Schema validator should still run against each fixture during the same extract job.

Drafted field descriptions can still invert meaning even when warranty verbs are absent from the file. The linter will not catch ordinary English errors, which remain a reviewer responsibility before any merge. Compatibility dates in the owned file are claims rather than clocks, and they expire unless someone rereads signed_at. Teams should calendar a review of signatures older than their compatibility window rather than trusting stale YAML.

Who should skip this approach

Teams with a single internal script and no external consumers do not need a public warranty file. Publishers that already generate docs from a signed OpenAPI overlay may already have this split under different filenames. Organizations that cannot name an owner for signed_by will produce empty ceremony, which is worse than a short human-written page. Do not use a drafting model on the warranty file, and do not treat free drafting capacity as evidence that delivery semantics were reviewed.

If event names already land in CI as JSON, add the unsigned-event lint before any drafting job joins the pipeline. The cheapest correct page is an extract plus a sparse warranty file, not a fluent chapter with unsigned retries. Keep the customer Markdown generated, keep the signatures human, and keep the model on the extract-only side of the fence.