DEV Community: Ethan Beirne

Bitemporal AI Memory: How to Preserve What an Agent Knew Then

Ethan Beirne — Sun, 19 Jul 2026 04:58:47 +0000

Bitemporal AI memory solves a problem ordinary RAG cannot see

Most agent memory systems optimize for a simple question: which stored chunks are relevant now?

Production systems eventually need a harder question:

What information was available to the agent when it made a specific decision?

Those questions are not equivalent. If a fact, policy, permission, or architecture decision is corrected later, a latest-state memory system can silently rewrite the context behind an earlier action.

Two timelines, not one

Bitemporal memory records two independent clocks for every fact:

Event time: when the fact was valid in the real world.
System time: when the AI system learned, revised, or retired the fact.

Consider a contract amendment that became effective on January 1 but entered the agent's memory on January 12. A decision made on January 5 must not use that amendment, even though its effective date is earlier.

A single timestamp cannot represent both truths.

Why overwriting breaks historical replay

Suppose a customer risk rating changes from low to high. If the memory layer overwrites the original row, a later audit sees only the high rating. It may appear that the agent ignored information that it did not actually possess.

A bitemporal record keeps:

the original value
the correction
when each value applied
when the system received each version
which version was eligible for each retrieval

That supports three different queries:

Current truth: what is valid now?
Known at decision time: what could the agent have used then?
Revised historical truth: what do we now believe was true on that date?

A practical test for any memory system

You can test this without a large benchmark:

Store an architecture decision and its source.
Run an agent task that retrieves it.
Add a correction later with an older effective date.
Replay the original task using its historical cutoff.
Verify that the later correction does not leak backward.
Confirm that a current query does use the correction.

If both queries return the same memory version, the system is storing history but not reconstructing historical knowledge.

What a production implementation should preserve

A useful bitemporal memory layer should retain:

immutable version identities
event-time and system-time validity windows
source provenance and checksums
retrieval filters and selected memory versions
policy and permission results
links from decisions back to supporting evidence
deterministic as-of queries

The goal is not to store private model reasoning. The goal is to preserve enough evidence to explain which state produced a material decision.

Why this matters beyond compliance

Temporal correctness improves more than audits. It also helps with:

reproducible evaluations
backtests without lookahead bias
debugging behavior after memory corrections
incident reconstruction
policy-version migration
multi-agent workflows using changing shared state

Founder disclosure: I am building Lians, an open-source bitemporal memory and record layer for AI agents. The complete guide, including query models and evaluation checks, is available at Bitemporal AI Memory: Event Time, System Time, and Historical Recall.

I am especially interested in how other teams handle late-arriving facts. Do you preserve historical versions, or does your current memory stack mostly optimize for the latest state?

Try to Break Our AI Memory Benchmark

Ethan Beirne — Sat, 18 Jul 2026 07:10:33 +0000

Facts change. An earnings forecast is revised. A policy is amended. A medication dose is corrected. An entity record is updated.

Many AI memory systems retain both the old and new versions. When retrieval ranks both highly, stale information can silently enter the model context.

We built Lians to prevent that failure while preserving the historical record. That is a meaningful technical claim, especially for regulated AI, so it should be independently tested rather than accepted as marketing.

The challenge

We have opened a public challenge asking developers to try to break Lians temporal recall.

The included benchmark currently reports:

0 stale facts in top-5 recall
100 percent supersession accuracy on 22 fact pairs
Point-in-time reconstruction through recall_at and snapshot

We want adversarial tests, not applause.

Run the benchmark

git clone https://github.com/Lians-ai/Lians.git
cd Lians/agentmem
pip install -e ".[dev]"
pytest tests/test_supersession_benchmark.py tests/test_recall_quality.py -v

No API key is required.

Try to break it

The most useful tests will go beyond the included examples:

Add a fact with an event timestamp.
Add a corrected or superseding fact.
Confirm present-time recall excludes the stale version.
Reconstruct what was known before the correction.
Change the wording while keeping the same underlying entity and metric.
Test ambiguous names, partial corrections, conflicting sources, or delayed updates.

We are specifically looking for:

Stale facts appearing in present-time recall
A valid update failing to supersede the old version
Two distinct facts being incorrectly merged
Historical reconstruction returning information that was not yet known
Differences across environments or embedding configurations

Report evidence

The canonical challenge is GitHub issue #60.

Please include the fact pair or dataset, your environment, the command or script, the expected result, and the actual result.

If a failure is reproducible, we will turn it into a regression test and credit the contributor. People who find meaningful edge cases will also be invited to a technical pairing session with the maintainers.

Why this matters

A current answer and a historical reconstruction are different products.

A system reviewing a past financial decision, clinical recommendation, legal analysis, or policy action must preserve what was knowable at the time. Later corrections should improve current answers without rewriting the original decision context.

That is the standard we want Lians to meet. The fastest way to improve the product is to expose the benchmark, make the claims falsifiable, and welcome critical results.

If your team is deploying an agent that depends on changing facts, you can also request a free temporal-memory audit at lians.ai. We will examine one sanitized workflow and identify where stale facts or missing evidence could affect reliability.

Why AI audit logs break when the facts change

Ethan Beirne — Thu, 16 Jul 2026 14:35:13 +0000

Most AI agent audit logs can tell you what prompt ran, what model answered, and which tools were called.

That is useful until someone asks a harder question six months later:

Why did the agent make that decision with the information available at the time?

Imagine an agent reviewing a company, customer, or transaction. It retrieves a policy, a filing, an entity record, and a risk rule. Later, the policy is updated, the filing is amended, and the entity record is corrected.

If you replay the workflow using today's sources, you are not reproducing the original decision. You are running a new decision against a different world.

You have an activity log, not decision reconstruction.

The hidden problem is time

Agent systems need to distinguish at least three clocks:

Event time: when the agent performed an action.
Valid time: when an underlying fact was true in the real world.
System time: when your platform learned or stored that fact.

A record can be valid in March, corrected in June, and audited in September. The September review may need both the March version that influenced the original decision and the June correction that changed the current truth.

Without those distinctions, old facts can resurface as if they are current, or current facts can silently rewrite the past.

What evidence must be preserved

For a meaningful reconstruction, I would preserve:

The exact retrieved source content or an immutable source version
The prompt, policy, and instruction versions
Model and tool configurations
The permissions and identity active during the run
Intermediate facts, citations, and transformations
The order and timestamp of each state transition

Logging only an ID is not enough if the document, policy, or record behind that ID can change later.

Content hashes help prove integrity, but the original content or a recoverable immutable version still needs to exist.

Corrections, changes, and disagreements are different

A new fact should not always overwrite an old fact.

Correction: The previous record was wrong. Preserve it for audit, but make the correction authoritative for current decisions.

Real-world change: Both facts were true at different times. Give each fact a validity interval.

Disagreement: Two sources conflict and neither one clearly supersedes the other. Preserve both with provenance and surface the conflict.

This structure lets current retrieval avoid stale information while point-in-time retrieval can still reproduce what the system knew earlier.

Why this matters beyond compliance

Decision reconstruction is not only an audit requirement. It improves:

Debugging when an agent behaves unexpectedly
Evaluations across prompt, model, and policy versions
Incident response after a source or rule changes
Human review of high-impact decisions
Trust between operators, customers, and regulators

If an agent cannot explain what evidence it used at decision time, it becomes harder to improve safely.

What we are building

At Lians, we are building a system of record for AI in regulated workflows. The goal is to reconstruct what an AI system knew, did, and why at the moment a decision was made, even after the underlying facts change.

We are three weeks old, pre-1.0, and working with a functioning product. We are looking for 5 to 7 design partners in financial research, risk, compliance, and other evidence-heavy agent workflows.

If you operate an agent where policies, records, filings, or external sources change over time, I would genuinely value hearing how you handle reconstruction today.

What breaks first in your current audit trail when the facts change?