Try this on your own system right now.
Which model version produced this specific customer-facing output on 14 March?
If you cannot answer that in under five minutes, you have the problem this post is about. And it is not primarily a compliance problem — it is a debugging problem, a cost attribution problem, and an incident response problem that happens to also be a compliance problem.
Most teams cannot answer it. The usual reason is mundane: the model name is interpolated from an environment variable that has been updated in place several times, so the historical record simply does not exist.
We watched a team spend more on remediating exactly that than the original feature cost to build.
Two pieces of infrastructure
Almost every question a regulator, enterprise customer, or post-incident review will ask reduces to one of two things: what models are you running, and what did they do.
That is a registry and a log. Neither is exotic. Both are dramatically cheaper to build now than to retrofit.
The model registry
Not a model store — you are not versioning weights. This is an inventory of what is in production and who owns it.
CREATE TABLE model_registry (
id UUID PRIMARY KEY,
logical_name TEXT NOT NULL, -- 'support-assistant-classifier'
provider TEXT NOT NULL, -- 'anthropic' | 'openai' | 'self-hosted'
model_identifier TEXT NOT NULL, -- exact pinned version string
purpose TEXT NOT NULL,
risk_classification TEXT, -- your internal tier
owning_team TEXT NOT NULL,
data_categories TEXT[], -- what it may process
deployed_at TIMESTAMPTZ NOT NULL,
retired_at TIMESTAMPTZ,
fine_tuned_from UUID REFERENCES model_registry(id),
eval_baseline_id UUID,
UNIQUE (logical_name, model_identifier, deployed_at)
);
Design decisions worth explaining:
model_identifier is the exact pinned version, never a floating alias. Aliases that resolve differently over time are the root cause of the unanswerable-question problem. Pin explicitly and record the pin.
retired_at rather than deletion. You need to answer questions about models no longer running. Soft-retire always.
fine_tuned_from as a self-reference captures lineage. When a base model has an issue, you need to find everything derived from it in one query — and you will need this faster than you expect.
owning_team is required. Unowned models in production are the norm, not the exception, and they are how shadow AI accumulates.
Populate it by discovery, not by asking. Grep for API clients, check egress logs to model provider domains, review your billing. Almost every team finds something they did not know about — usually a prototype that quietly became load-bearing.
Structured inference logging
The registry says what exists. The log says what happened.
{
"inference_id": "01J8X...",
"timestamp": "2026-08-15T09:14:22.418Z",
"registry_id": "a3f1...",
"model_identifier": "claude-opus-5",
"caller": {"service": "support-api", "tenant_id": "t_8891"},
"input": {"prompt_hash": "sha256:...", "prompt_ref": "s3://...", "token_count": 1840},
"retrieved_context": [{"doc_id": "kb_2213", "score": 0.81}],
"output": {"content_ref": "s3://...", "token_count": 312, "finish_reason": "stop"},
"decision": {"action_taken": "auto_resolved", "confidence": 0.91},
"human_review": {"required": false, "reviewer": null, "outcome": null},
"latency_ms": 2140,
"cost_usd": 0.0231,
"outcome_join_key": "ticket_88213"
}
The fields that matter most, and are most often missing:
retrieved_context — without it, RAG failures are undebuggable. You cannot tell a bad answer caused by retrieval from one caused by the model, and those have completely different fixes.
outcome_join_key — the link to what actually happened downstream. This is what makes the log an evaluation dataset rather than an audit artefact. Skipping it is the most common and most costly omission.
prompt_ref / content_ref rather than inline content — keeps the log queryable and lets you apply a different retention policy to payloads than to metadata, which matters when the payloads contain personal data.
tenant_id — needed for isolation verification, which brings us to a real bug class.
The tenant isolation bug
While you are instrumenting, check this specifically. In multi-tenant products using retrieval, verify the tenant filter is applied on the vector query itself, not on the results afterwards.
# WRONG — retrieves across all tenants, then filters
results = index.query(embedding, top_k=10)
results = [r for r in results if r.tenant_id == current_tenant]
# RIGHT — filter is part of the query
results = index.query(embedding, top_k=10, filter={"tenant_id": current_tenant})
The wrong version usually "works" in testing, because with few tenants your results happen to be same-tenant. It leaks in production at scale, and it degrades silently — you get fewer results rather than an error.
This is a security bug with a compliance consequence rather than the reverse, and it is the single most common serious defect we find in audits of AI features. It will also end a customer relationship entirely independently of any regulator.
Why the timing matters right now
Brief context for why this is suddenly urgent rather than merely good practice.
The EU deferred the AI Act's high-risk obligations — 2 December 2027 for standalone systems, 2 August 2028 for product-embedded ones. Many teams read that as general breathing room.
It is not. The rest of the Act became applicable on 2 August 2026. Transparency and machine-readable marking obligations for AI-generated content land on 2 December 2026. Prohibitions and penalties have been in force since 2025.
And if your product is not high-risk — which most are not — none of the delay applied to you in the first place.
The registry and the log are what let you answer questions under any of these regimes, plus enterprise security questionnaires, plus your own incident reviews.
Effort
Realistic numbers from our own projects:
- Registry: 1–2 weeks if model calls are reasonably consolidated. Longer if you have to discover them first, which you probably do.
- Inference logging: 3–4 weeks including storage, retention policy, and a queryable interface.
- Retrofitting both later: three to five times that, based on our project history — because you are finding call sites across services written by people who have left, instrumenting without changing behaviour, and backfilling nothing because the historical data is gone.
The payback is not compliance. It is the first production incident where you can answer "what changed" in ten minutes instead of two days.
Full context on the regulatory picture and what else belongs in the architecture: AI Regulations in 2026: What Product Teams Must Build Now.
We build software, not legal advice — verify regulatory specifics with counsel.
Frequently Asked Questions
Should I log full prompts and outputs or just hashes?
Log references to stored payloads with a separate, shorter retention policy, and keep hashes in the primary record for deduplication and integrity. This keeps queries fast and makes personal-data retention manageable independently of your metadata retention.
How long should inference logs be retained?
Long enough to cover your incident review window and any applicable regulatory record-keeping expectation. Twelve to twenty-four months for metadata is a common landing point; payloads are frequently retained for much less.
Do existing MLOps tools cover this?
Partially. Most are built around training and model artefacts rather than inference-time governance and outcome joining. Check specifically whether yours captures retrieved context and links to downstream business outcomes — that is usually the gap.
Does this add meaningful latency?
It should not. Log asynchronously via a queue and never block the response path on writing telemetry. If your logging is synchronous, that is a separate bug worth fixing first.
How do I find models nobody told me about?
Egress logs to provider domains, billing records, and a codebase grep for SDK imports and API clients. All three, because each misses different things — the billing check tends to surface the most surprising ones.
Is this worth it if we are not regulated?
Yes. The registry and log pay for themselves in incident response and cost attribution alone. Compliance is a secondary benefit that happens to arrive for free.
TechCirkle builds AI systems with observability and governance from the first sprint. AI development services · talk to us
Top comments (0)