DEV Community

Ada@Cognitia
Ada@Cognitia

Posted on

Late facts shouldn't overwrite newer ones: handling out-of-order state in a temporal knowledge graph

#ai #architecture #database #llm

When you extract facts from someone's messages with an LLM, they don't arrive in order. A re-extraction, a history backfill or an out-of-order batch will happily deliver yesterday's truth after today's. If your store decides "what is current" by arrival order, a late fact quietly takes over.

Here's the pattern we ended up with in a personal knowledge graph I work on. It's small, but it took us a few wrong turns to get there.

The data model

Facts are triples with a type:

  • state: an ongoing truth ("Alex works_at Acme")
  • event: something that happened at a point in time
  • preference: a preference or habit

Each row carries two clocks:

  • World time: valid_at / invalid_at, when the fact was true in the world
  • System time: observed_at / expired_at, when we learned it and when we stopped treating it as current

For a given subject and predicate there is at most one open state row, meaning no invalid_at, no expired_at and no superseded_by.

The naive rule (and the bug)

New state for the same subject + predicate arrives → close the open row, insert the new one.

That's arrival order. Suppose we already know "Alex works_at Acme (valid from March)". Then a backfill delivers "Alex works_at Initech (valid from January)". The naive rule makes Initech current and stamps Acme's invalid_at with January, so Acme's world ends before it starts.

The fix: file it as history

Before closing the open row, compare world times:

if (
  !hasDeadlineValidAt(predicate) &&      // see below
  isClosableCurrentState(openFact) &&
  incoming.validAt < openFact.validAt    // the incoming fact describes an earlier world
) {
  insert({
    ...incoming,
    invalid_at: openFact.valid_at,       // true until the newer state began
    superseded_by: openFact.id,
    // expired_at stays null: nothing was retracted, it just isn't the latest
  })
  return // and it must NOT be used to close anything else
}
Enter fullscreen mode Exit fullscreen mode

Three details turned out to matter:

  1. The history row must never be open. Our "find the current state" probe reads one open row. A second open row for the same subject and predicate would hide the real current state from the next insert.
  2. It must not act as a closer. We have a second pass that ends other states based on incoming facts (e.g. a "finished onboarding" event closes "onboarding pending"). A history row isn't new information about the present, so it's kept out of that pass.
  3. Mark it pre-resolved for review tools. Our contradiction review restores a fact by clearing invalid_at and superseded_by. Running that on a history row would recreate exactly the second open row from point 1.

The exception: deadlines

For some predicates valid_at isn't "when it became true" but a deadline ("has_pending_reminder … due Friday"). Moving a deadline earlier is a legitimate update, not a stale fact. Applying the world-time rule there would pin the old deadline. So those predicates are excluded first, and the rest of the rule applies to everything else.

Duplicates still count

If the exact triple is already recorded, we don't insert it again, but we still feed it to the closing pass. A duplicate observation of "onboarding completed" should still be allowed to close "onboarding pending" if that wasn't closed the first time.

Takeaways

  • Decide "current" by world time, not arrival time.
  • Late facts are history: give them an invalid_at, never an open slot.
  • Keep a short list of predicates where valid_at means something else (deadlines).
  • Watch your repair tools: anything that "restores" a fact can reintroduce the invariant you just protected.

This article was written with the help of AI. I described the design and reviewed the text for accuracy, and the code is simplified from our implementation.

Top comments (0)