Harvest Structured Log Keys Into a Catalog, Then Refuse Unsigned Retention Claims

Structured log documentation fails when field names are compiled from source while retention and PII sentences are still invented in prose. A harvest script can list keys, types, and call sites with high recall, but it cannot certify how long events live or whether a value is personal data. Treat the catalog as a compile output and treat every privacy promise as a signed claim that a named owner must attach. The workflow below keeps those two artifacts separate so generated drafts cannot smuggle compliance language into customer-facing pages.

This article proposes an unexecuted, local workflow rather than a measured production study. No latency numbers, retention defaults, or audit outcomes are claimed. Teams should run the extractor against their own repository and discard any drafted sentence that the facts file cannot support.

Why log tables rot while privacy sentences stay dangerous

Application code gains and drops log fields every sprint, especially around payments, sessions, and background workers. Markdown tables that once matched extra= dictionaries then drift, so on-call readers trust a document that no longer describes the stream. Assistants trained on mixed handbooks often complete the table with storage talk, then guess a retention window because nearby paragraphs already mention disks. That guess is the failure mode this lint gate exists to block.

Customer docs that mention GDPR, SOC 2, or “no PII in logs” are not documentation flavor text. They are commitments that survive the next refactor of a logger helper. A model that never queried the data platform, legal register, or deletion job has no evidence for those commitments. Compiling keys from abstract syntax trees is therefore allowed, while compiling legal duration is not.

What a model may draft versus what a human must own

Use the following decision table as a review checklist before any publish step. Rows marked draftable may be written from harvested facts alone. Rows marked owned require a human signature block and a dated source other than model output.

Claim class	Example sentence	Lane	Allowed evidence
Field inventory	request_id is emitted as a string on http.request	Draftable	AST harvest, tests, fixtures
Call-site map	order.paid is logged in payments/worker.py	Draftable	Path and line from harvest
Example shape	Sample JSON with redacted tokens	Draftable	Fixture files checked into git
Cardinality hint	request_id is high cardinality	Draftable only if tests show uniqueness	Test names in the facts file
Retention days	Events are stored for thirty days	Owned	Platform policy plus named owner
PII status	Logs contain no email addresses	Owned	Data map plus named owner
Vendor sharing	Streams are forwarded to a vendor	Owned	Contract and architecture review
Deletion SLA	User erasure completes within forty-eight hours	Owned	Runbook owner, not a model

The table is the article’s core artifact together with the scripts below. If a drafted paragraph cannot be traced to the draftable rows, it does not belong in the generated body. Reviewers should reject the whole page when an owned row appears without a signature fence.

Proposed harvest format

Keep a facts file that machines may regenerate and humans may diff. YAML is enough, and it should record only what source and tests can prove. The following snippet is a proposed shape, not an observed production export.

# facts/log_fields.yaml  — regenerate; do not hand-edit keys
version: 1
source_commit: "REPLACE_WITH_GIT_SHA"
fields:
  - key: request_id
    python_type: str
    event: http.request
    files:
      - path: app/http/middleware.py
        line: 84
    test_nodes:
      - tests/http/test_middleware.py::test_request_id_present
  - key: order_id
    python_type: str
    event: order.paid
    files:
      - path: payments/worker.py
        line: 191
    test_nodes: []
owned_claims: []   # humans append signed blocks in docs, not here

Empty test_nodes is a signal, not a license to invent coverage. Drafted prose may say the field exists at a path. Drafted prose may not say the field is verified in production or safe for a warehouse export.

Workflow

Follow the numbered steps in order. Skipping the lint step reintroduces the original failure, because chat output will complete privacy sentences by pattern.

1. Harvest logger keys from Python ASTs

Run a small extractor over the application package and write facts/log_fields.yaml. The script below is labeled proposed; adapt the event-name heuristic to your logging helper before trusting the output. It understands logger.info("event", extra={...}) and structlog-style logger.info("event", key=value) keyword arguments.

# harvest_log_fields.py — proposed local helper, not a packaged product
from __future__ import annotations

import ast
import sys
from pathlib import Path

EVENT_ATTRS = {"info", "warning", "error", "debug", "exception"}

class Harvest(ast.NodeVisitor):
    def __init__(self, rel: str) -> None:
        self.rel = rel
        self.rows: list[dict] = []

    def visit_Call(self, node: ast.Call) -> None:
        func = node.func
        name = None
        if isinstance(func, ast.Attribute) and func.attr in EVENT_ATTRS:
            name = func.attr
        if name is None:
            self.generic_visit(node)
            return
        event = None
        if node.args and isinstance(node.args[0], ast.Constant) and isinstance(
            node.args[0].value, str
        ):
            event = node.args[0].value
        keys: list[tuple[str, str]] = []
        for kw in node.keywords:
            if kw.arg == "extra" and isinstance(kw.value, ast.Dict):
                for k in kw.value.keys:
                    if isinstance(k, ast.Constant) and isinstance(k.value, str):
                        keys.append((k.value, "unknown"))
            elif kw.arg and kw.arg not in {"exc_info"}:
                keys.append((kw.arg, type(kw.value).__name__))
        for key, typ in keys:
            self.rows.append(
                {
                    "key": key,
                    "python_type": typ,
                    "event": event or "",
                    "path": self.rel,
                    "line": node.lineno,
                }
            )
        self.generic_visit(node)


def harvest(root: Path) -> list[dict]:
    rows: list[dict] = []
    for path in root.rglob("*.py"):
        if "tests" in path.parts or path.name.startswith("test_"):
            continue
        tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
        visitor = Harvest(str(path.relative_to(root)))
        visitor.visit(tree)
        rows.extend(visitor.rows)
    return rows


if __name__ == "__main__":
    root = Path(sys.argv[1] if len(sys.argv) > 1 else ".")
    for row in harvest(root):
        print(f"{row['path']}:{row['line']}\t{row['event']}\t{row['key']}")

Command to emit a sortable inventory before you wrap it as YAML:

python harvest_log_fields.py app | sort -u > /tmp/log-keys.tsv
git rev-parse HEAD > facts/SOURCE_COMMIT

The tab-separated file is the compile input. It is not a privacy policy, and it should never be copied into a public handbook without the later lint.

2. Draft only field prose from the facts file

Prompting a model with the live repository invites it to quote comments, tickets, and old runbooks that already contain retention language. Feed the facts file only, and constrain the output to a markdown table with key, event, path, and example. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Free model access is enough for this constrained draft, and the free server option can run harvest_log_fields.py when a laptop checkout is inconvenient, provided the facts file still lands in git for review.

A proposed system prompt, labeled as unexecuted instruction text, looks like the following block.

You receive facts/log_fields.yaml.
Write a markdown table with columns key, event, source path, and notes.
Notes may restate types and call sites from the file.
Do not mention retention, PII, GDPR, SOC 2, vendors, deletion, or warehouses.
If a field lacks test_nodes, write "unverified by tests" in notes.
Do not invent keys that are absent from the file.

If the model adds a column named retention, discard the draft and rerun with a shorter prompt. The failure is expected; the process must not negotiate with that column.

3. Attach human-owned claims in a signature fence

Owned sentences live in a separate markdown fence that the linter parses. The fence records the owner, the date, and a non-model source. Proposed shape:

<!-- SIGNED_LOG_CLAIM
owner: sre-oncall
date: 2026-09-21
source: data-platform retention ticket DP-1842
claims:
  - field: request_id
    retention_days: 14
    pii: false
  - field: order_id
    retention_days: 14
    pii: false
-->

The surrounding handbook may render a human-written paragraph that cites those claims. The paragraph is still owned text. Regenerating the catalog must not rewrite the fence. If a harvested key disappears, the linter should fail until the fence is edited by the same owner class.

4. Lint the handbook before publish

The linter has three jobs. It proves every table key exists in the facts file, it proves no draftable section uses banned privacy tokens, and it proves every owned token sits inside a signed fence whose keys still exist. Proposed checker:

# lint_log_docs.py — proposed CI gate
from __future__ import annotations

import re
import sys
from pathlib import Path

BANNED = re.compile(
    r"\b(retention|pii|gdpr|soc\s*2|personal data|right to be forgotten|"
    r"warehouse|vendor)\b",
    re.I,
)
FENCE = re.compile(
    r"<!-- SIGNED_LOG_CLAIM\b(.*?)-->",
    re.S,
)
KEY_ROW = re.compile(r"^\|\s*`?([a-zA-Z_][a-zA-Z0-9_]*)`?\s\|", re.M)


def load_keys(facts: str) -> set[str]:
    return set(re.findall(r"^\s+- key: ([a-zA-Z_][a-zA-Z0-9_]*)", facts, re.M))


def fence_keys(md: str) -> set[str]:
    found: set[str] = set()
    for block in FENCE.findall(md):
        found.update(re.findall(r"field:\s*([a-zA-Z_][a-zA-Z0-9_]*)", block))
    return found


def main(doc: Path, facts: Path) -> int:
    text = doc.read_text(encoding="utf-8")
    keys = load_keys(facts.read_text(encoding="utf-8"))
    errors: list[str] = []
    for m in KEY_ROW.finditer(text):
        key = m.group(1)
        if key in {"key", "event", "Claim"}:
            continue
        if key not in keys and key not in {"field", "python_type"}:
            # table headers are skipped by the allow-list above; unknown keys fail
            if key not in {"Example", "notes", "Notes", "Source"}:
                if key.islower() and key not in keys:
                    errors.append(f"undocumented key in table: {key}")
    body_without_fences = FENCE.sub("", text)
    if BANNED.search(body_without_fences):
        errors.append("privacy token outside SIGNED_LOG_CLAIM fence")
    for key in fence_keys(text):
        if key not in keys:
            errors.append(f"signed claim for missing field: {key}")
    for err in errors:
        print(err, file=sys.stderr)
    return 1 if errors else 0


if __name__ == "__main__":
    raise SystemExit(main(Path(sys.argv[1]), Path(sys.argv[2])))

CI invocation stays boring on purpose:

python harvest_log_fields.py app > /tmp/log-keys.tsv
python lint_log_docs.py docs/observability/log-fields.md facts/log_fields.yaml

A non-zero exit is the publish blocker. Do not “fix” a banned-token failure by asking the model to rephrase GDPR as “regional rules.” The token class is the point of the gate.

Review protocol for the signed fence

Require two human checks that no compiler can replace. First, the owner named in the fence must be a real rotation, not a model persona, and the source ticket must still be open or accepted. Second, a reviewer who did not write the draft must confirm that example payloads are redacted and that high-cardinality keys are not described as warehouse dimensions. Record both checks on the pull request rather than inside generated prose, because pull request metadata is harder to regenerate by accident.

If legal counsel already maintains a record of processing, copy identifiers into the source: field instead of summarizing the memo. Summaries drift. Identifiers can be grepped when the harvest drops a key that the memo still names.

Limitations

The AST harvest misses dynamically computed key names, wrapper helpers, and logs emitted from other languages. Regex on extra= will also miss fields injected by middleware after the call site the document describes. Cardinality, sampling, and sink routing are invisible to this script, so notes about uniqueness remain speculative unless a test node is listed. The banned-token list is English-centric and will not catch euphemisms, which is why a human reviewer still reads the page.

This workflow does not implement deletion, encryption, or access control. Passing the linter means the handbook did not invent a privacy sentence in the draftable section. It does not mean the platform meets a regulation. Treat counsel, data-platform owners, and security review as upstream of any public sentence about PII.

Who should not use this approach

Do not use harvest-and-draft on a public privacy notice if no named human will sign the fence on every release. Do not use it to backfill SOC 2 language for a customer questionnaire. Do not point the model at production log samples that may already contain personal data. Do not auto-merge catalog pull requests on services that emit billing identifiers until the owner class is staffed. Teams that only need internal dashboards can keep Grafana labels undocumented; the cost of this gate is justified when external readers will treat the page as a promise.

The compile lane remains useful even when the owned lane is empty. Publish the field table, mark every row unverified by tests, and omit retention entirely. Silence is a more accurate privacy document than a fluent paragraph the facts file cannot support.

When the harvest, the table, and the signature fence stay on separate lifecycles, log documentation stops pretending that a draft pass completed the compliance work. Keep the lint gate in CI, and keep owned claims in the fence that only a human may edit.