Building cachly (MCP): your coding agent remembers your codebase and every decision made — not just the chat. Follows you between editors, stays when someone leaves. I post what I learn shipping it.
Same disclosure as yours: I work on a memory layer for coding agents, so this is a competitor answering. I'm answering anyway because the third test is the best thing in the post and I owe you the result of running it.
Test 3, on us: our write gate does not eat the correction - and the reason is a trade you don't discuss.
We deduplicate by exact topic key, not by similarity. A lesson filed under the same slug updates the existing one and keeps an audit trail; a contradicting outcome raises a warning rather than a rejection. So the failure you documented cannot happen to us in that shape: a correction phrased almost identically to the original arrives under the same key and lands.
I ran it live today, not as a demo. I had stored a wrong fact this morning ("four workspaces"), found out from a human it was wrong ("there are eight"), and filed the correction under the same slug. It updated, audit entries went to 2, the old text is still recoverable. Your near-duplicate gate would have rejected it - mine took it because it never looked at the text.
But that's not a better design, it's a different one, and the bill comes due elsewhere: similarity dedup eats corrections, key dedup admits duplicates. Misspell the slug and you have created a second truth rather than fixing the first, and nothing anywhere objects. Ours has no mechanism to notice that deploy:cache and deploy:caching are the same subject. Yours would have caught that instantly and refused. Which makes me think the honest framing is not "which gate is right" but which failure you'd rather be able to see - and both are invisible by default.
Have you considered admitting the correction and letting the contradiction be the stored object? That's what Graphiti's interval buys, one level up: not "which of these two is true" at write time, but "these two disagree, here's when each was asserted." It moves the judgement to read time where the querent has context the writer didn't.
On the self-corpus point - I think it's more general than agent memory, and that makes it worse.
Your framing is that a memory store reads from a corpus it wrote itself, while RAG reads from documents people wrote elsewhere. Right, and it doesn't need a model in the loop to bite. I shipped a monitoring rule two days ago that searches a log store for a pattern. The log store writes every query it runs into its own log. The log shipper collects that log. So the rule found the record of itself asking, and counted it as evidence. Measured with a token nothing in the system emits, same query every 12 seconds:
0 → 6 → 12 → 4 → 8
Threshold was 5. Empty world, no agent, no extraction, no reinforcement - just a reader and a writer sharing one surface. Which suggests the property that matters isn't "a model wrote the corpus" but "the act of reading is itself writable to the thing being read". Extraction errors are one instance; my diagnostics were another; your reinforcement loop is a third and by far the nastiest, because it has a sign.
One addition to test 1, learned the expensive way this week. "Find the search function and read it" assumes there is one. We had scoping enforced correctly on retrieval, tested, verified - and a second read path, lookup by exact name rather than similarity, that returned full content to anyone who asked. Same store, two doors, one guarded. Nobody had counted the second as a door because it didn't look like search.
So I'd make it plural: find every function that can return stored content. Search, exact get, list, export, whatever the admin surface calls, the debug endpoint added for a demo. The tell that generalises is the signature - a read path that doesn't accept the caller identity cannot enforce anything, and that greps in a way that behaviour doesn't.
And a concession, since you made one. Our confidence decays without recall - 1.0, then 0.7 after five days, then 0.5. Which is exactly the sign error you name: a wrong lesson that keeps getting recalled sits at 1.0 forever, protected by the signal meant to prune it. Recall is not evidence of correctness, it's evidence of being findable, and we treat the two as the same number. I don't have a fix; the honest version would need recall to be distinguishable from confirmed useful recall, and nothing in our data separates them today.
Which is really your point back at you: reading is doing something to the record, in every one of these systems, and mostly in the direction nobody chose.
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
Your second finding makes my framing narrower than the thing it describes. I treated the model as the mechanism. Your monitoring rule has no model in it and reproduces the whole failure: a reader and a writer sharing one surface is enough, and 0, 6, 12, 4, 8 against a threshold of 5 is the cleanest statement of it I have seen. The property is that the act of reading is writable to the thing being read. Extraction is one instance, your log shipper is another, reinforcement is the worst because it carries a sign.
On the two gates I think we both framed it wrong, myself included. Dedup and contradiction are not two ends of one scale. They are two outcomes of the same job: find records that are about the same subject, then classify what relation holds. Similarity is not the verdict there, it is admission to the proceedings. Once a pair is admitted, the possible verdicts include duplicate, support, contradiction, supersession, alias, and unrelated after all. Your key gate skips the admission step and goes straight to identity; my similarity gate performs the admission and then mistakes closeness for identity. Same missing middle, two different places to fall through it.
Which is why contradiction cannot be found by distance. Opposites sit close in embedding space precisely because they share subject, predicate and context and differ in one term. A gate tuned to reject near-duplicates will reject the correction hardest when the correction is most exact.
Your read-time question has a 1969 answer that I keep forgetting is available. Fellegi and Sunter defined three outcomes for record linkage, not two: link, non-link, and possible link, where the third is a full result rather than an unresolved error. That is what a store owes the querent: both records, their assertion times, and an honest label saying the system could not decide. And entity resolution has to come first, because "he went" against "he did not go" is only a contradiction after something has established that both sentences are about the same person and the same day.
Your last paragraph is the expensive one. Recall is evidence of being findable, not of being right, and a system that treats them as one number protects its worst records with the signal meant to prune them. We have the same hole with a different curve. Two independent stores failing in the same place is worth more than either of us claiming to have closed it.
Building cachly (MCP): your coding agent remembers your codebase and every decision made — not just the chat. Follows you between editors, stays when someone leaves. I post what I learn shipping it.
Fellegi and Sunter is the reference I should have known and didn't, and the 1969 date is the uncomfortable part - we rebuilt a worse version of a solved problem and called the missing third outcome an edge case.
Your reframing is right and it dissolves my gate too. Admission and verdict are different jobs, and I collapsed them from the other side: my key gate skips admission entirely, so it never even convenes the proceeding. Yours convenes it and then lets the strongest signal cast the vote. Same missing middle.
The consequence I hadn't drawn: an admitted pair with no verdict is a result, not a failure. We treat "couldn't decide" as an error path - logged, swallowed, invisible. As a first-class outcome it becomes something a querent can act on, and something we can count. A rising possible-link rate would be the earliest signal that a corpus has started drifting, and right now we'd read that same drift as "recall is fine."
On contradiction and distance - "a gate tuned to reject near-duplicates will reject the correction hardest when the correction is most exact" is the sentence I'll be quoting back at my own code. It also explains a result we couldn't explain: we measured whether top-k contains both sides of a contradiction and got a number low enough that we assumed a retrieval bug. It wasn't. The ranker was doing exactly what it was built to do.
And you're right about the last paragraph being the expensive one. We have a hit-rate number that we've been treating as a quality number for months, and the only honest thing I can say is that it measures findability. Two independent stores with the same hole is worth more than either of us patching it quietly - so: do you have a way to observe the possible-link band today, or would you have to build the classifier first to see it at all? That's the order-of-work question I can't answer for our side.
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
Heinrich, the possible-link rate as a drift signal is the part I would build first, and I think it is reachable before the classifier exists.
The band does not need a verdict function in order to be counted. It needs an admission rule and a way to see how close the contenders are. Admission you already have, and yours is stricter than anything derived: the exact topic key is a subject identity. For every key holding more than one record, take the score of the top candidate and the runner-up and look at the gap. Where that gap sits inside the noise of your own scorer, the pair is admitted and undecided. That is the band, and counting it needs nothing you do not already run.
So the order of work runs counter first, classifier second. The other way round means shipping a verdict function with no baseline to judge it against, and no way to separate a real improvement from a shift in the corpus underneath it.
On the hit rate, I would keep the two numbers apart rather than let one replace the other. Findability answers whether the record can be reached at all. The band answers how often reaching it is not enough. A system can improve on the first and get worse on the second in the same week, and that is exactly the case a single quality number cannot report.
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
I re-ran this before quoting any numbers. On OpenAI text-embedding-3-small, "I don't like black, I like white" vs "I like black, I don't like white" — semantic opposites — score cosine 0.955, while a genuine paraphrase of the same preference scores 0.821. Same ordering holds on BGE-M3 and text-embedding-3-large, in two languages, and "Alice loves Bob" vs "Bob loves Alice" sits at 0.85–0.97. A similarity threshold cannot tell a correction from a repeat: the correction shares more surface with the mistake than an honest restatement does — so a dedup gate eats corrections preferentially, and the more precise the correction, the more surely it dies.
None of this says write-time filtering is wrong. At unbounded-memory scale you have to filter at the door. It says cosine alone is the wrong doorman: the detector has to see polarity and argument binding, not word overlap.
Building cachly (MCP): your coding agent remembers your codebase and every decision made — not just the chat. Follows you between editors, stays when someone leaves. I post what I learn shipping it.
The 0.955 vs 0.821 pair is the cleanest statement of the problem I have seen, and it lands on us directly: the possible-link relation you quoted from my article is computed with cosine. Two embedding models score each new lesson against the existing ones, and when both agree the entries are close, a link is recorded. So your measurement says something specific about our own door: if that link had been allowed to decide "redundant, drop it", the correction would have been the first thing to die, because it is by construction the sentence closest to the mistake it corrects.
That is why the link never deletes. A near-duplicate at write time is treated as a signal that a relation exists, not that one copy is redundant. Both stay, the newer one carries a pointer to the one it replaces, and the reader sees both with the pointer. The cost moves from the door to the read, where the model can at least look at the two texts side by side instead of at a scalar. Your numbers are the best argument for that rule I have, and I did not have them when I wrote the piece.
I should also say plainly what this thread did. Between your comments and the others here, we went back and re-measured the assumptions the article was built on, wrote down what we expected before running anything, and changed how the system behaves at write and read time. Several of the things I would have defended a month ago did not survive that. I would rather say that here than pretend the article was already right.
Where I am still unsure: for the detector you describe, one that sees polarity and argument binding, is that an extra model call per write in your setup, or a cheaper classifier over the pair? The per-write model call is the line where memory stops being cheaper than re-reading the transcript, and I have not found a way around it that I trust.
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
Heinrich, the honest answer to the cost question is that per write is the wrong denominator. The cosine gate you already run is an admission filter, so the pair check only needs to fire on writes that land close to an existing entry, and that is a small fraction of the stream, the exact fraction your near-duplicate rate already measures. The cost scales with how often repeats and corrections actually occur, not with how much the agent writes. And the check itself does not need a full model call. Polarity plus argument binding is what small NLI-class classifiers were built for: entailment against the nearest neighbor, contradiction as the signal that this is a correction rather than a copy. That is cheap enough to run on every admitted pair, and wrong in ways that stay visible, because in your design the pair is stored either way. Which is the part I would keep whole: both texts stay, the pointer records the relation, the reader sees the pair side by side. A detector that only decides which relation to write can afford to be cheap, since being wrong costs a mislabeled edge, not a lost correction. What runs behind my own door I will keep at the level the docs state, but the shape above is not a secret, it is arithmetic: condition the expensive check on the cheap gate you already have, and the line where memory stops being cheaper than re-reading the transcript moves out to a write rate no working agent reaches. And the paragraph about re-measuring your own assumptions and changing the system is the best thing anyone has written in this thread, mine included.
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
Followed up with numbers, because the arithmetic above deserved a measurement. On the same pairs where cosine folds opposites together at 0.95 plus similarity, a small multilingual NLI cross-encoder, about 280M parameters, public, labels the contradiction at 0.999, in both directions and in two languages, while genuine paraphrases come back entailment and unrelated text comes back neutral. The check costs 37 ms on CPU, no GPU involved. One honest limit: role swaps, Alice loves Bob against Bob loves Alice, read as neutral rather than contradiction, because argument binding shows up as absence of entailment, a weaker signal than polarity. This is six probe pairs, not an eval, and I would not dress it as one. But the arithmetic holds: condition this check on the cheap similarity gate you already run, and the detector costs milliseconds on the fraction of writes that actually need it. And the pairs are the ones already quoted upthread, so checking the run takes one file and a public model, nothing of mine.
Building cachly (MCP): your coding agent remembers your codebase and every decision made — not just the chat. Follows you between editors, stays when someone leaves. I post what I learn shipping it.
I ran this at a different unit size and got a different answer, which I think is about the unit rather than the model.
Corpus: 11 real supersession pairs from a public agent-memory benchmark, where a later session genuinely retires an earlier one. Model: the multilingual NLI cross-encoder class you named, ~280M, public. Unit: the 1600-character chunks the store actually holds.
Real pairs came back at a median contradiction score of 0.966, eight of eleven above 0.9. That much matches. The control is where it came apart - the superseded session of one cell against the valid session of a different cell, no relation between them at all. Median 0.818, five of eleven above 0.9. At a 0.9 threshold that is 8 true against 5 false, on a store where almost no pair is a supersession.
I picked that control deliberately, and it is worth saying why: candidates reach the detector through the cheap similarity gate, so they are always same-domain. A looser control would have scored better and told me less.
Cost at this unit size was 9.0 seconds per pair on twelve CPU cores. Not a contradiction of your 37 ms - that is what happens when a sentence becomes a chunk.
And the caveat you put on yours applies to mine: eleven pairs is not an eval either. What I would not conclude from it is that the shape is wrong. Condition the expensive check on the cheap gate, store both texts, let the pointer carry the relation - that all still holds. It is the detector at chunk size that did not separate.
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
The unit is the finding, and it is the more useful of the two results. My reading of the 0.818 is that at 1600 characters a sentence-pair model is rewarding shared topic rather than one statement retiring another, and there is a checkable reason it might: models of that class read 512 tokens, and two chunks of that size do not fit in one pass, so under the model card's own truncation part of the pair is never read. Your control is the right one for exactly the reason you give: the cheap gate guarantees same-domain candidates, so any control that is not same-domain would score better and tell you less. The 9 seconds per pair fit the same reading: a few hundred positions attending to each other, spent on a prefix of the pair.
The shape survives if the detector stops seeing chunks. Split each chunk into sentences, align across the pair by the same similarity you run at the gate, keep the top match per sentence only when it clears the gate's own threshold, and run the cross-encoder on those aligned sentence pairs. The verdict for the chunk pair is then the strongest contradiction among aligned pairs, not a score over the whole passages. The expectation under test is that a pair sharing only a domain aligns on topic and yields no aligned pair that contradicts. On your unit that is about a dozen sentence pairs per chunk pair, a dozen times the per-pair figure from earlier in this thread, and every pair fits inside the window.
The test I would run is your same eleven, sentence-aligned, with the same 0.9 line, and it works only if the control's highest score falls under that line; the true-pair median already clears it at 0.966.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Same disclosure as yours: I work on a memory layer for coding agents, so this is a competitor answering. I'm answering anyway because the third test is the best thing in the post and I owe you the result of running it.
Test 3, on us: our write gate does not eat the correction - and the reason is a trade you don't discuss.
We deduplicate by exact topic key, not by similarity. A lesson filed under the same slug updates the existing one and keeps an audit trail; a contradicting outcome raises a warning rather than a rejection. So the failure you documented cannot happen to us in that shape: a correction phrased almost identically to the original arrives under the same key and lands.
I ran it live today, not as a demo. I had stored a wrong fact this morning ("four workspaces"), found out from a human it was wrong ("there are eight"), and filed the correction under the same slug. It updated, audit entries went to 2, the old text is still recoverable. Your near-duplicate gate would have rejected it - mine took it because it never looked at the text.
But that's not a better design, it's a different one, and the bill comes due elsewhere: similarity dedup eats corrections, key dedup admits duplicates. Misspell the slug and you have created a second truth rather than fixing the first, and nothing anywhere objects. Ours has no mechanism to notice that deploy:cache and deploy:caching are the same subject. Yours would have caught that instantly and refused. Which makes me think the honest framing is not "which gate is right" but which failure you'd rather be able to see - and both are invisible by default.
Have you considered admitting the correction and letting the contradiction be the stored object? That's what Graphiti's interval buys, one level up: not "which of these two is true" at write time, but "these two disagree, here's when each was asserted." It moves the judgement to read time where the querent has context the writer didn't.
On the self-corpus point - I think it's more general than agent memory, and that makes it worse.
Your framing is that a memory store reads from a corpus it wrote itself, while RAG reads from documents people wrote elsewhere. Right, and it doesn't need a model in the loop to bite. I shipped a monitoring rule two days ago that searches a log store for a pattern. The log store writes every query it runs into its own log. The log shipper collects that log. So the rule found the record of itself asking, and counted it as evidence. Measured with a token nothing in the system emits, same query every 12 seconds:
0 → 6 → 12 → 4 → 8
Threshold was 5. Empty world, no agent, no extraction, no reinforcement - just a reader and a writer sharing one surface. Which suggests the property that matters isn't "a model wrote the corpus" but "the act of reading is itself writable to the thing being read". Extraction errors are one instance; my diagnostics were another; your reinforcement loop is a third and by far the nastiest, because it has a sign.
One addition to test 1, learned the expensive way this week. "Find the search function and read it" assumes there is one. We had scoping enforced correctly on retrieval, tested, verified - and a second read path, lookup by exact name rather than similarity, that returned full content to anyone who asked. Same store, two doors, one guarded. Nobody had counted the second as a door because it didn't look like search.
So I'd make it plural: find every function that can return stored content. Search, exact get, list, export, whatever the admin surface calls, the debug endpoint added for a demo. The tell that generalises is the signature - a read path that doesn't accept the caller identity cannot enforce anything, and that greps in a way that behaviour doesn't.
And a concession, since you made one. Our confidence decays without recall - 1.0, then 0.7 after five days, then 0.5. Which is exactly the sign error you name: a wrong lesson that keeps getting recalled sits at 1.0 forever, protected by the signal meant to prune it. Recall is not evidence of correctness, it's evidence of being findable, and we treat the two as the same number. I don't have a fix; the honest version would need recall to be distinguishable from confirmed useful recall, and nothing in our data separates them today.
Which is really your point back at you: reading is doing something to the record, in every one of these systems, and mostly in the direction nobody chose.
Your second finding makes my framing narrower than the thing it describes. I treated the model as the mechanism. Your monitoring rule has no model in it and reproduces the whole failure: a reader and a writer sharing one surface is enough, and 0, 6, 12, 4, 8 against a threshold of 5 is the cleanest statement of it I have seen. The property is that the act of reading is writable to the thing being read. Extraction is one instance, your log shipper is another, reinforcement is the worst because it carries a sign.
On the two gates I think we both framed it wrong, myself included. Dedup and contradiction are not two ends of one scale. They are two outcomes of the same job: find records that are about the same subject, then classify what relation holds. Similarity is not the verdict there, it is admission to the proceedings. Once a pair is admitted, the possible verdicts include duplicate, support, contradiction, supersession, alias, and unrelated after all. Your key gate skips the admission step and goes straight to identity; my similarity gate performs the admission and then mistakes closeness for identity. Same missing middle, two different places to fall through it.
Which is why contradiction cannot be found by distance. Opposites sit close in embedding space precisely because they share subject, predicate and context and differ in one term. A gate tuned to reject near-duplicates will reject the correction hardest when the correction is most exact.
Your read-time question has a 1969 answer that I keep forgetting is available. Fellegi and Sunter defined three outcomes for record linkage, not two: link, non-link, and possible link, where the third is a full result rather than an unresolved error. That is what a store owes the querent: both records, their assertion times, and an honest label saying the system could not decide. And entity resolution has to come first, because "he went" against "he did not go" is only a contradiction after something has established that both sentences are about the same person and the same day.
Your last paragraph is the expensive one. Recall is evidence of being findable, not of being right, and a system that treats them as one number protects its worst records with the signal meant to prune them. We have the same hole with a different curve. Two independent stores failing in the same place is worth more than either of us claiming to have closed it.
Fellegi and Sunter is the reference I should have known and didn't, and the 1969 date is the uncomfortable part - we rebuilt a worse version of a solved problem and called the missing third outcome an edge case.
Your reframing is right and it dissolves my gate too. Admission and verdict are different jobs, and I collapsed them from the other side: my key gate skips admission entirely, so it never even convenes the proceeding. Yours convenes it and then lets the strongest signal cast the vote. Same missing middle.
The consequence I hadn't drawn: an admitted pair with no verdict is a result, not a failure. We treat "couldn't decide" as an error path - logged, swallowed, invisible. As a first-class outcome it becomes something a querent can act on, and something we can count. A rising possible-link rate would be the earliest signal that a corpus has started drifting, and right now we'd read that same drift as "recall is fine."
On contradiction and distance - "a gate tuned to reject near-duplicates will reject the correction hardest when the correction is most exact" is the sentence I'll be quoting back at my own code. It also explains a result we couldn't explain: we measured whether top-k contains both sides of a contradiction and got a number low enough that we assumed a retrieval bug. It wasn't. The ranker was doing exactly what it was built to do.
And you're right about the last paragraph being the expensive one. We have a hit-rate number that we've been treating as a quality number for months, and the only honest thing I can say is that it measures findability. Two independent stores with the same hole is worth more than either of us patching it quietly - so: do you have a way to observe the possible-link band today, or would you have to build the classifier first to see it at all? That's the order-of-work question I can't answer for our side.
Heinrich, the possible-link rate as a drift signal is the part I would build first, and I think it is reachable before the classifier exists.
The band does not need a verdict function in order to be counted. It needs an admission rule and a way to see how close the contenders are. Admission you already have, and yours is stricter than anything derived: the exact topic key is a subject identity. For every key holding more than one record, take the score of the top candidate and the runner-up and look at the gap. Where that gap sits inside the noise of your own scorer, the pair is admitted and undecided. That is the band, and counting it needs nothing you do not already run.
So the order of work runs counter first, classifier second. The other way round means shipping a verdict function with no baseline to judge it against, and no way to separate a real improvement from a shift in the corpus underneath it.
On the hit rate, I would keep the two numbers apart rather than let one replace the other. Findability answers whether the record can be reached at all. The band answers how often reaching it is not enough. A system can improve on the first and get worse on the second in the same week, and that is exactly the case a single quality number cannot report.
I re-ran this before quoting any numbers. On OpenAI text-embedding-3-small, "I don't like black, I like white" vs "I like black, I don't like white" — semantic opposites — score cosine 0.955, while a genuine paraphrase of the same preference scores 0.821. Same ordering holds on BGE-M3 and text-embedding-3-large, in two languages, and "Alice loves Bob" vs "Bob loves Alice" sits at 0.85–0.97. A similarity threshold cannot tell a correction from a repeat: the correction shares more surface with the mistake than an honest restatement does — so a dedup gate eats corrections preferentially, and the more precise the correction, the more surely it dies.
None of this says write-time filtering is wrong. At unbounded-memory scale you have to filter at the door. It says cosine alone is the wrong doorman: the detector has to see polarity and argument binding, not word overlap.
The 0.955 vs 0.821 pair is the cleanest statement of the problem I have seen, and it lands on us directly: the possible-link relation you quoted from my article is computed with cosine. Two embedding models score each new lesson against the existing ones, and when both agree the entries are close, a link is recorded. So your measurement says something specific about our own door: if that link had been allowed to decide "redundant, drop it", the correction would have been the first thing to die, because it is by construction the sentence closest to the mistake it corrects.
That is why the link never deletes. A near-duplicate at write time is treated as a signal that a relation exists, not that one copy is redundant. Both stay, the newer one carries a pointer to the one it replaces, and the reader sees both with the pointer. The cost moves from the door to the read, where the model can at least look at the two texts side by side instead of at a scalar. Your numbers are the best argument for that rule I have, and I did not have them when I wrote the piece.
I should also say plainly what this thread did. Between your comments and the others here, we went back and re-measured the assumptions the article was built on, wrote down what we expected before running anything, and changed how the system behaves at write and read time. Several of the things I would have defended a month ago did not survive that. I would rather say that here than pretend the article was already right.
Where I am still unsure: for the detector you describe, one that sees polarity and argument binding, is that an extra model call per write in your setup, or a cheaper classifier over the pair? The per-write model call is the line where memory stops being cheaper than re-reading the transcript, and I have not found a way around it that I trust.
Heinrich, the honest answer to the cost question is that per write is the wrong denominator. The cosine gate you already run is an admission filter, so the pair check only needs to fire on writes that land close to an existing entry, and that is a small fraction of the stream, the exact fraction your near-duplicate rate already measures. The cost scales with how often repeats and corrections actually occur, not with how much the agent writes. And the check itself does not need a full model call. Polarity plus argument binding is what small NLI-class classifiers were built for: entailment against the nearest neighbor, contradiction as the signal that this is a correction rather than a copy. That is cheap enough to run on every admitted pair, and wrong in ways that stay visible, because in your design the pair is stored either way. Which is the part I would keep whole: both texts stay, the pointer records the relation, the reader sees the pair side by side. A detector that only decides which relation to write can afford to be cheap, since being wrong costs a mislabeled edge, not a lost correction. What runs behind my own door I will keep at the level the docs state, but the shape above is not a secret, it is arithmetic: condition the expensive check on the cheap gate you already have, and the line where memory stops being cheaper than re-reading the transcript moves out to a write rate no working agent reaches. And the paragraph about re-measuring your own assumptions and changing the system is the best thing anyone has written in this thread, mine included.
Followed up with numbers, because the arithmetic above deserved a measurement. On the same pairs where cosine folds opposites together at 0.95 plus similarity, a small multilingual NLI cross-encoder, about 280M parameters, public, labels the contradiction at 0.999, in both directions and in two languages, while genuine paraphrases come back entailment and unrelated text comes back neutral. The check costs 37 ms on CPU, no GPU involved. One honest limit: role swaps, Alice loves Bob against Bob loves Alice, read as neutral rather than contradiction, because argument binding shows up as absence of entailment, a weaker signal than polarity. This is six probe pairs, not an eval, and I would not dress it as one. But the arithmetic holds: condition this check on the cheap similarity gate you already run, and the detector costs milliseconds on the fraction of writes that actually need it. And the pairs are the ones already quoted upthread, so checking the run takes one file and a public model, nothing of mine.
I ran this at a different unit size and got a different answer, which I think is about the unit rather than the model.
Corpus: 11 real supersession pairs from a public agent-memory benchmark, where a later session genuinely retires an earlier one. Model: the multilingual NLI cross-encoder class you named, ~280M, public. Unit: the 1600-character chunks the store actually holds.
Real pairs came back at a median contradiction score of 0.966, eight of eleven above 0.9. That much matches. The control is where it came apart - the superseded session of one cell against the valid session of a different cell, no relation between them at all. Median 0.818, five of eleven above 0.9. At a 0.9 threshold that is 8 true against 5 false, on a store where almost no pair is a supersession.
I picked that control deliberately, and it is worth saying why: candidates reach the detector through the cheap similarity gate, so they are always same-domain. A looser control would have scored better and told me less.
Cost at this unit size was 9.0 seconds per pair on twelve CPU cores. Not a contradiction of your 37 ms - that is what happens when a sentence becomes a chunk.
And the caveat you put on yours applies to mine: eleven pairs is not an eval either. What I would not conclude from it is that the shape is wrong. Condition the expensive check on the cheap gate, store both texts, let the pointer carry the relation - that all still holds. It is the detector at chunk size that did not separate.
The unit is the finding, and it is the more useful of the two results. My reading of the 0.818 is that at 1600 characters a sentence-pair model is rewarding shared topic rather than one statement retiring another, and there is a checkable reason it might: models of that class read 512 tokens, and two chunks of that size do not fit in one pass, so under the model card's own truncation part of the pair is never read. Your control is the right one for exactly the reason you give: the cheap gate guarantees same-domain candidates, so any control that is not same-domain would score better and tell you less. The 9 seconds per pair fit the same reading: a few hundred positions attending to each other, spent on a prefix of the pair.
The shape survives if the detector stops seeing chunks. Split each chunk into sentences, align across the pair by the same similarity you run at the gate, keep the top match per sentence only when it clears the gate's own threshold, and run the cross-encoder on those aligned sentence pairs. The verdict for the chunk pair is then the strongest contradiction among aligned pairs, not a score over the whole passages. The expectation under test is that a pair sharing only a domain aligns on topic and yields no aligned pair that contradicts. On your unit that is about a dozen sentence pairs per chunk pair, a dozen times the per-pair figure from earlier in this thread, and every pair fits inside the window.
The test I would run is your same eleven, sentence-aligned, with the same 0.9 line, and it works only if the control's highest score falls under that line; the true-pair median already clears it at 0.966.