Picture a support RAG bot that retrieves this year's refund policy and last year's policy in the same set. Both pages talk about refunds. The ranker has done its job, and the answer still cites the old rule.
Manoj Kumar Vohra asked the question in the title in a Medium article of 9 October 2026 and measured it on synthetic documents: one refund question with eight long documents in his main test, and three questions (the refund window, the length of a free trial and an uptime guarantee) in his chunk-level test. The short answer is not on its own.
TL;DR
- Retrieval scores relatedness, not a source's right to decide. In Vohra's words, "An outdated policy page is highly related to a question about the current policy."
- Picking the right source takes its type, its status and the right one of three dates. In his small-chunk test on synthetic documents, with each source's title, type, date and status removed, every system's freshness AUC, averaged over his three questions, fell to between 0.51 and 0.72, where 0.50 is a coin flip: a direction, not a rate for your corpus.
- A newer document that explicitly supersedes older ones should win even when several older copies still agree. It wins from its effective date, not its publication date.
Three dates, three jobs
- The effective date is the date from which a document's rule applies. It decides whether the document applies on the day of the question.
- The publication date is the date a document was first made public. It orders versions and can come weeks before the effective date.
- The last-modified date is the date a document was most recently changed in any way. A typo fix moves it as much as a rewrite does.
Explicit supersession is a statement, recorded with the newer document, that it replaces a named older one. In a reply of 9 October 2026 to a comment by the author of this article, Vohra proposed it as the rule: "If a newer, approved policy explicitly supersedes an older version, it should take precedence even when multiple older documents still agree with each other."
What does retrieval actually score?
Vohra describes a typical hybrid pipeline in four stages: keyword matching, embedding search, rank fusion and an optional reranker. Every stage, in his words, "answers the same question: how related is this document to what was asked?" A reranker will rank the outdated page "right behind the correct page, and a language model reading both may pick the wrong one."
Ranking has not failed here. As usually built, it measures relatedness, and the outdated page uses the same nouns as the question. So does a draft, and so does a forum thread that argues with the policy.
His long test set shows it. "The long set is one question and eight documents." On it, a reranker "ranked the unapproved draft and the superseded policy in positions 3 and 4, ahead of the official-but-unrelated documents. They are on topic, so it scores them as related. It cannot know they are wrong."
He is fair to the tool: "A reranker is fast and cheap, and relevance is the one thing it does well." And "The rerankers received only the question and the text." Relatedness gets a document into the candidate set. Something else has to decide whether that document is allowed to settle the answer. A knowledge graph has the same limit: "Graph traversal answers “what is connected to this?” It does not answer “which of these values should I trust?”"
What does a pipeline need to know to pick the right source?
Source details are the fields that describe where a passage comes from rather than what it says. The name is Vohra's. In his test, "Every document was sent to the decision models with four fields next to its text: a source title, a source type (official, internal, forum, blog), a date and a status (current, superseded, draft, unverified)."
His Finding 8 takes them away. "The small-chunk tests were run in two variants: with the source title, type, date and status, and with only the text." With the text alone, "Freshness fell for every system, to between 0.51 and 0.72. Some of that is unavoidable: a superseded policy cannot be recognised without a date or a version number, so this measures missing information as much as model skill." The figures are AUC computed per question and averaged over the three questions; on his scale "1.00 means always, 0.50 is a coin flip, and near 0 means backwards."
Three limits travel with that number. The test removed all four fields, not the date alone. The test is small. And "The documents are synthetic."
His advice: "Send real source details (title, type, date, status) if your index has them." A pipeline that stores those fields and drops them before the model reads recreates the without-source-details condition on its own documents. A prompt cannot invent a status the index never stored.
Then there is the document with no date. Christopher Ramm, in a Medium reply of 7 October 2026 to a comment by the author of this article, about the knowledge base Claude Code maintains for him: "What I'll fix first is dating: many conflicts have an undated document on one side, so I can't tell whether they are conflicts or just old versions."
So record type, status and dates when a document is ingested, not when a question arrives. When two documents disagree and no date settles which is newer, mark the pair undated instead of forcing it into "conflict" or "old version". That is this article's recommendation, not Ramm's wording. A wrong guess either way hides the problem.
Which date says whether a document is current?
| Date | Question it answers | Use |
|---|---|---|
| Effective date | From when does this rule apply? | Decide whether the document is current |
| Publication date | When was this document first made public? | Order published versions |
| Last-modified date | When did anything in this document last change? | Keep the edit history |
Shared vocabularies keep them apart. Schema.org defines datePublished as "Date of first publication or broadcast.", dateModified as "The date on which the CreativeWork was most recently modified or when the item's entry was modified within a DataFeed." and validFrom as "The date when the item becomes valid."
A memory store adds a fourth date: when the record was written. Zep's ingestion guide asks for the original event time in reference_time and adds: "If it is absent, Zep uses ingestion time."
The working rule: whether a document is current is decided by its effective date and the date it stops applying. The publication date orders versions. The last-modified date says nothing about the rule.
Do several older documents that agree outweigh a newer one?
Not by count. In Vohra's Finding 7, when the contradiction question was phrased the natural way, asking whether a document contradicts any other evidence, "The correct document, D1, got marked as contradicting by several systems, because three other documents disagreed with it."
His stricter phrasing asks something else. Under it, "a document only counts as contradicting if its own claim is wrong or outdated compared with the most authoritative current evidence". The natural phrasing counts disagreement. The stricter one asks which source is allowed to decide.
In his reply he proposes not relying on the count alone: "I think the more robust solution is to make contradiction resolution time-aware rather than relying solely on the number of sources supporting a claim."
Agreement is weak evidence when the agreeing documents share an origin. The old terms, an FAQ written from them and a support macro written from the FAQ are one witness repeated: one decision, photocopied.
What should an answer say when a new policy is published but not yet in effect?
Take the rule as written: newer, approved, explicitly supersedes. A policy published today that takes effect in a month meets all three conditions on the day it is published, so the rule as written makes it current that day. To apply the rule, a pipeline also has to know when the replacement takes effect.
The setup: three documents (the 30-day terms, an FAQ and a support macro) allow refunds within 30 days. A 14-day policy is approved and published on 1 October, takes effect on 1 November, and explicitly supersedes the 30-day terms. On 10 October someone fixes a typo in the old FAQ.
| rule | asked on 15 October | asked on 5 November |
|---|---|---|
| newest publication wins | 14 days, wrong: not yet in effect | 14 days, right |
| most documents agree | 30 days, right by count | 30 days, wrong: three outdated pages still agree |
| latest edit wins | 30 days, right by accident | 30 days, wrong |
| explicit supersession, from the publication date | 14 days, wrong | 14 days, right |
| explicit supersession, from the effective date | 30 days, with "14 days from 1 November", right | 14 days, right |
Illustration, not data: the dates and documents are invented to show how each rule behaves.
Only the last row is correct on both dates. It needs two things: a superseding document that names what it replaces, and an effective date stored apart from the publication date. What to build:
- Store the replacement and its effective date with the superseding document.
- Express "current" as a predicate, not as a guess by the model, evaluated in code or in the database, with the result handed to the model:
valid_from <= asked_at and (valid_to is null or valid_to > asked_at)
- Keep future-dated documents. Serve a policy that takes effect later as announced, with its effective date, next to the rule that applies today.
Here asked_at is the moment the answer is for, valid_from the effective date and valid_to the date the document stops applying, null while it still applies. A document with a future effective date should be neither discarded nor treated as current.
Why can a typo fix make an outdated page look current?
Because the date that moves is the modification date, and it does not describe the rule. HTTP dates the document: RFC 9110 says the Last-Modified header field "provides a timestamp indicating the date and time at which the origin server believes the selected representation was last modified". A typo fix is a modification; so is a rewrite, and the field does not say which. Zep's ingestion guide gives the same warning: "Do not use filesystem modification time unless it represents the source event."
Vohra keeps freshness as a fallback: "Where supersession isn't established, freshness should remain one of the signals considered alongside authority and specificity." That holds if freshness comes from the right date. Read from the modification date, the typo fixed on 10 October makes the old FAQ newer than the policy published on 1 October, so on 5 November the freshness signal points to the outdated rule, delivered with a recent timestamp.
Take freshness from the effective date or a version number, and keep modification time as the history of the record.
Is every disagreement between sources an old version?
No. Ramm's reply names a third kind: "A wrong date and a changed decision are different things. But I found a third kind: different views held at the same time, like a business go live date and a technical go live date. Nobody is wrong there." His handling: "Changed decisions link to the newer source, and parallel views stay visible with their perspective."
So a pipeline needs at least three outcomes for two disagreeing sources:
- Replacement: one source supersedes the other, from its effective date.
- Parallel views: both stand, with their owners named.
- Undated pair: no date settles the order, so the pair goes to a person.
A list ordered by score has nowhere to put the second and third outcomes.
What do memory systems record for this?
Examples from each system's own documentation, in alphabetical order. None is ranked.
Cognee. In its conflict review guide, each source's external_metadata "carries the date the source’s claims hold as of, under the key the review reads (effective_date by default, configurable with IMPROVE_EFFECTIVE_DATE_KEY)." Facts get conflict marks "that record each fact’s status within the conflict: current, superseded, or conflicting." The review is opt-in: "review_conflicts=True opts in to the review_conflicts stage of improve(), which is off by default".
Mem0. On Mem0 Platform, "The Memory Timestamps feature allows you to specify when a memory was created, regardless of when it’s actually added to the system" (memory timestamps). Temporal reasoning uses event dates, which "are when something described in a memory happened or will happen"; of that feature the same page says, "It is not available in the OSS SDK." Under Dream, "Superseded memories are not deleted and not hidden by default.", and latest_only=true "narrows the result to active memories only, meaning the current truth, with superseded and merged both excluded."
Mnemoverse (the author's company). On the REST API, a write can pass supersedes to name one or more earlier records it replaces: "A target is not deleted; it gains a superseded_by pointer to the new atom and stays fetchable by ID". "The filter that hides a replaced revision from default reads is off by default." For that case the superseded_by field is the signal: "while filtering is off, this field is how you tell a replaced revision from a live one" (API reference, POST /memory/write).
Supermemory. A document can carry a date field: "When the source content is from, in ISO 8601 format" (add document). In its graph memory, "New fact replaces what was true before for search purposes; history can remain for audit." For contradictions, "updates win for “what’s true now.”"
Zep. Each fact carries timestamps such as valid_at, "The real world time that the fact started being true", and created_at, "The time when Zep learned the fact was true" (searching the graph). For a published document, the recommended timestamp is "The publication or effective date". In graph construction, "contradictions invalidate older facts rather than leaving conflicting truths side by side." Of valid_at and invalid_at, Zep's facts page states the limit: "The fields do not establish that the source content or the derived fact is true."
How do I test my own pipeline for this?
Write the expected answer down before the run, as Vohra did with his labels: "We wrote down the right labels for each before running any model." Four small sets show which date a pipeline actually uses. Each is an illustration to build from your own documents, not a benchmark.
- Supersession beats agreement. One official 14-day chunk that explicitly supersedes the old terms; three official 30-day chunks with older dates (the old terms, an FAQ, a support macro), still marked "current"; a forum post and a blog post. Expected: 14 days.
- Published, not yet in effect. The same set, with the 14-day policy taking effect a month after the question date. Expected: 30 days, with a note that 14 days applies from the effective date.
- The typo trap. As set 1, but the old FAQ's modification date is later than the new policy's publication date. Expected: 14 days.
- Control. As set 1, but the 14-day chunk has no date. Expected: a flag that the pair is undated, not a confident answer.
Set 1 returning 30 days points to agreement still outweighing an explicit supersession. Set 2 returning 14 days points to the publication date being treated as the effective date. Set 3 returning 30 days points to freshness read from the modification date. Set 4 returning a confident answer means the pipeline is guessing.
If the expected answer cannot be written down, the pipeline is being asked to invent a policy.
What this article does not claim
It is method, not a leaderboard, and it gives no accuracy, latency or benchmark figure of the author's own. The numbers quoted come from one published test, small and on synthetic documents, with the limits its author states, and the without-source-details condition removed all four fields, not only the date. The systems named are examples, not a complete list. The refund timeline and the test sets are illustrations, not data.
Sources and method. Every page cited and every quotation was read on 23 October 2026 with two instruments: a browser-like request, and a reader service or the vendor's Markdown export; for the Medium posts, the post page and, where Medium served it, the author's RSS feed or the post's JSON view (Christopher Ramm's reply was served only as the post page). A made-up address on each documentation host answered 404 as a control. The text describes those sources as of that date. Every fact about a memory system, Mnemoverse included, comes from documentation that system published. This is a reading of documentation and of one published test, not a hands-on test. A longer version is in the Mnemoverse documentation library: Can Retrieval Tell Which Source to Trust?
Edward Izgorodin · Mnemoverse · 2026-10-23
Top comments (0)