DEV Community

Cover image for Designing an Audit Trail You Can Actually Query Two Years Later
James Sanderson
James Sanderson

Posted on

Designing an Audit Trail You Can Actually Query Two Years Later

Compliance and audit dashboard interface

Most systems that claim to have an audit trail have an activity log. The distinction only becomes visible when someone asks a question the log cannot answer, and in regulated systems that question arrives with a legal deadline attached.

The question is: why did you close this alert?

An activity log answers what happened: alert 88214 closed by user 412 at 14:32 with reason code NO_SAR. That is not the question. The question is what the firm knew at 14:32 on that day, what rule version produced the alert, what the customer's risk rating was before three subsequent updates, and what reasoning the analyst was shown.

Those are reconstruction requirements, and you cannot add them later because the data was never captured.

What "point-in-time" actually demands

Four properties, and each one has teeth.

1. Append-only decisions. A decision record is never updated. A correction is a new record referencing the old one. This sounds obvious and is routinely violated by UPDATE cases SET status = ..., which destroys the only copy of what the previous state was.

2. Versioned reference data. Customer risk ratings, counterparty classifications, country risk scores, PEP lists — all of these change, and all of them influence decisions. If your customer table holds only current state, every historical decision becomes unexplainable the first time a field changes.

The workable pattern is bitemporal: each record carries both a valid-time range (when the fact was true in the world) and a transaction-time range (when the system believed it). Regulators care about the second one, because the question is what you knew, not what was actually true.

customer_risk_rating
  customer_id
  rating
  valid_from, valid_to        -- when this was true
  recorded_from, recorded_to  -- when we believed it
Enter fullscreen mode Exit fullscreen mode

Reconstruction is then a query with recorded_from <= T AND recorded_to > T.

3. Versioned controls. Every rule, threshold, and policy is an addressable, versioned object with an approval record. A decision references the control version that produced it, not the control.

This is the one teams skip most often. Thresholds end up in a config table that an operations user edits through a UI with no approval workflow and no history. When the examiner asks what the threshold was last March and who approved it, the honest answer is that nobody knows.

4. Persisted model calls. If a model contributed to a recommendation, the prompt, the retrieved context, the model version identifier, the parameters, and the raw output all persist alongside the decision. Treating inference as ephemeral is the modern equivalent of not logging the rule version.

A workable schema shape

decision
  id
  subject_type, subject_id     -- alert / customer / report
  decided_at
  decided_by                   -- user or system principal
  outcome
  control_version_id           -- FK, immutable
  policy_version_id            -- FK, immutable
  inputs_snapshot_ref          -- pointer to materialised input state
  reasoning_ref                -- what the human was shown
  model_invocation_id          -- nullable FK
  supersedes_decision_id       -- nullable, for corrections
Enter fullscreen mode Exit fullscreen mode

Two implementation notes that save pain:

Materialise the input snapshot. You can reconstruct inputs by querying bitemporal tables, and you should be able to. But also write a materialised snapshot of exactly what was fed into the decision. Reconstruction queries are correct in theory and fragile in practice across two years of schema evolution. The snapshot is your ground truth; the queries are the check.

Store reasoning as presented, not as generated. If the analyst saw a summarised brief, persist the brief they saw. Persisting only the model's raw output leaves you unable to demonstrate what actually informed the human.

The replay requirement

A second capability, less discussed and equally load-bearing: you must be able to replay a control change against history before deploying it.

A threshold change looks harmless. In production it can produce a forty percent increase in alert volume on a Monday morning, which the operations team absorbs by closing faster, which is its own problem.

Replay needs the same bitemporal substrate. To evaluate control version 7 against last year's transactions, you need last year's reference data as it stood, not as it is now. Teams that build reconstruction for audit get replay almost for free; teams that build replay first usually end up with reconstruction too.

Analyst working across monitoring dashboards

Storage, because someone will ask

The usual objection is cost. Some honest numbers on shape rather than dollars.

Decision records are small and bounded by decision volume, which is bounded by alert volume. Input snapshots are the large item, and they compress well because consecutive snapshots for the same subject are highly similar — content-addressed storage with deduplication cuts this dramatically. Model invocations are the fast-growing item if you persist full retrieved context, so store the retrieval result by reference to an immutable document store rather than inline.

The growth curve is predictable and linear. The cost of not having it is not.

Where this sits in the build

Practical sequencing, if you are starting: build the bitemporal data layer and the decision log before the detection logic. It feels backwards — detection is the product, evidence is the paperwork — but detection can be replaced and evidence cannot be reconstructed.

Retrofitting this onto a running compliance system is, honestly, a rebuild. The data you need was thrown away every time a row was updated in place.

Full architecture guide, including the detection layer, model risk governance under SR 11-7 and the EU AI Act, and a twelve-month build sequence: Financial Compliance Software: Architecture for AI-Native Controls. More on our custom software development work.

Frequently Asked Questions

What is the difference between an audit log and an audit trail?

An activity log records what happened — who did what, when. An audit trail supports reconstruction: what the system knew at the moment of a decision, which control version applied, and what reasoning was presented. Reconstruction requires versioned inputs and immutable decisions; a log of events cannot produce it retroactively.

What is bitemporal modelling and why does compliance need it?

Bitemporal records carry both valid time (when a fact was true in the world) and transaction time (when the system believed it). Regulators ask what you knew at the time of a decision, which is transaction time. Systems tracking only valid time cannot answer that question after a correction.

Should model prompts and outputs be persisted?

Yes — prompt, retrieved context, model version identifier, parameters and raw output, linked to the decision. Treating inference as ephemeral is equivalent to not recording which rule version fired. Store large retrieved context by reference into an immutable document store rather than inline.

How do you test a control change safely?

Replay it against historical transactions using reference data as it stood at that time, not as it stands now. This produces the actual delta in alert volume and detection coverage before deployment. The bitemporal substrate that supports audit reconstruction also supports replay.

Can point-in-time reconstruction be added to an existing system?

Rarely without a rebuild of the data layer. The required history was destroyed each time a record was updated in place. You can start capturing from today forward, but decisions made before the change remain unexplainable.

How large does this storage get?

Decision records scale with alert volume and stay small. Input snapshots dominate but deduplicate well, since consecutive snapshots for the same subject are near-identical — content-addressed storage handles this. Model invocation context grows fastest and should be stored by reference.

Top comments (0)