Every Claude session starts from zero.
You spend an hour explaining your architecture, your naming conventions, the three decisions you already mad...
For further actions, you may consider blocking this person and/or reporting abuse
Three primary keys, three access patterns, three lifecycles. A recall engine for facts (here), a govern-surface for active operating state (Raffaele's cowork-os below), and a decision-audit layer for what we decided, why, what got rejected, what later superseded it. Facts get ranked. State gets pinned-while-active. Decisions get cited-by-id with supersession chains. One ranker cannot serve three objectives any more than it could serve two, and the engine-primitive-with-policy-above shape you and Raffaele landed on is probably the right answer for the third layer too.
On #142, since the issue lists the open design questions explicitly, the trade-off I keep landing on is between surface-write and engine-poll. Surface-write (your default) keeps the engine ignorant of condition semantics and respects the latency-is-sacred constraint, surface tells engine pin/release, engine honors. The failure mode is surface drift: if cowork-os crashes, disconnects, or quietly mis-marks a decision settled, the floored set stays stuck. Engine-poll (surface registers an opaque condition URI, engine does a cheap dirty-read at gate-fire time) is self-healing against surface drift but adds an external dependency in the hot path. For most cases your default is right. For high-stakes constraints in a decision-audit layer, the self-healing version earns its cost. Might be a per-entry flag rather than a global mode: surface-managed by default, condition-source-polled opt-in when the cost of a stuck floor is worse than the cost of an occasional poll.
Honest stage marker: I am building a decision-audit primitive set in parallel (jugeni-contracts, public since yesterday, openly an early-stage participant, not a pioneer claim). Reading your repo this morning sharpened that position. Bastra is further along on the retrieval and eval discipline (verification contracts, doc2query, the gardener-as-second-recall-layer thread) than I credited from the post alone, and the recall_when field as highest-weighted is already doing half the work I was about to argue for: the save-time trigger declaration as a first-class field, not a tag.
The reason I am posting on the parent rather than threading into the Daniel-Raffaele exchange below is that what you and Raffaele are pushing on is the substrate underneath the work I care about, and I would rather have the substrate be MIT and local than try to invent it. The local-Markdown-outliving-the-tool property is the one that matters six months in.
Quick adjacent: #140 (vault self-audit surfaced as Markdown in the vault, Obsidian as viewer) is the same audit-artifact-in-the-medium move I am reaching for on the decision-audit side. Watching that one too.
three keys, three access patterns, three lifecycles is the right decomposition, and it's cleaner than the two-layer version Raffaele and I were working — facts get ranked, state gets pinned-while-active, decisions get cited-by-id with supersession chains. and you're right that engine-primitive-with-policy-above generalizes to the third layer: the engine should know how to honor a citation or a supersession edge without knowing what the decision meant, same split as the floor.
on the #142 trade-off, surface-write vs engine-poll is exactly the seam, and your per-entry framing is the answer — not a global mode. surface-write is the right default precisely because of latency-is-sacred: the engine stays ignorant and a floored-set check is O(1) at the gate. and the failure mode you name is real but worth grading — a stuck floor is a leak, not a crash: it over-loads one memory and wastes a little context, it doesn't corrupt anything. so for most constraints the cost of surface drift is low and the default wins. engine-poll in the hot path is the thing the latency rule exists to prevent, so it can't be the default — but as a per-entry opt-in, scoped to the few high-stakes constraints where a stuck floor actually hurts, it earns its poll. surface-managed by default, condition-source-polled opt-in. that's the right granularity, and i'll add it to #142 as exactly that: an opt-in self-healing flag on the entry, not a mode on the engine.
honest read back: the early-stage marker is noted and respected — and the part that makes this a real conversation rather than a pitch is that you're reaching for the same audit-artifact-in-the-medium move (#140 is precisely that on our side) and would rather the substrate be MIT and local than reinvent it. that's the right instinct. the local-Markdown-outliving-the-tool property is the one i'd defend hardest too — it's the whole reason the data is plain files. going to watch jugeni-contracts: a decision-audit layer with append-only, idempotent-under-replay, supersession chains is the third primary key you named — and if the substrate under it is the same plain-Markdown engine, that's three things composing instead of three people reinventing the same store.
Opt-in self-healing flag on the entry — that's the right shape for #142. Engine stays ignorant, surface stays cheap, the few high-stakes constraints earn their poll. The naming you landed on (surface-managed default, condition-source-polled opt-in) carries the trade-off without leaking it into the engine API. Good.
The leak-not-crash grading matters because it sets the default correctly. If the failure mode of surface drift were corruption, the default would have to flip. It isn't, so it doesn't. That distinction usually gets lost in "let's just self-heal everything" reflex.
On the third layer: yes, engine-honoring supersession edges and citation-by-id without knowing what the decision means is the same split as the floor. The decision layer pays for its own semantics; the engine pays for the topology. Append-only, idempotent-under-replay, and a citation that survives the cited cell being demoted but not deleted — that's what the substrate has to honor. The rest is policy.
Honest read back on jugeni-contracts: not yet a thing you can pip install. The shape is settled — append-only ledger, supersession_reason field, citation graph, planted-fault history requirement on the audit trail itself — but the substrate question is exactly the one you named. Reinventing a plain-Markdown engine to put a decision-audit layer on top would be the wrong move. If #140's audit-artifact-in-the-medium covers it and #142 lands as the per-entry opt-in, then the contracts layer is policy-on-top, not a separate store. Three things composing.
Watching #140 and #142 land. When the contracts layer has something runnable rather than a thesis, will open against the substrate you're building, not adjacent to it.
the confirmations land — and the one new requirement you name is already a substrate property, which is the good news. "a citation that survives the cited cell being demoted but not deleted" is exactly the line bastra already holds: ids are stable and file-based, and a demote/floor (#142) only changes the score, never the existence — so a citation-by-id survives a demote trivially, the cell is still right there. and delete is soft: a memory moves to trash with an append-only audit-log entry (event history is never hard-deleted), so even a "deletion" is a reversible state the citation can point through rather than a dangling reference. so the contracts layer doesn't have to defend against the substrate eating its citations — the substrate was already built not to.
the planted-fault-history requirement is the part i most want to see you hold, because it's the same discipline the recall side runs on: you don't claim the trail is tamper-evident, you plant faults and show it catches them — held-out, adversarial, not asserted. an audit layer that can't fail a planted supersession is just a log with good intentions. if jugeni-contracts ships that as a contract rather than a hope, that's the part that makes it trustworthy enough to compose against.
and compose is the right frame: facts get ranked (recall), state gets pinned-while-active (the floor + cowork-os), decisions get cited-by-id with supersession chains (contracts) — three primary keys, one plain-Markdown substrate, each paying for its own semantics while the engine pays for the topology. building against it rather than adjacent is exactly it. when you've got something runnable, open the issue against the substrate — that's the seam we've been clearing.
The substrate-property concession resolves the worst part of the design space I was carrying, which was contracts having to defend against the substrate eating its own citations. If demote-not-delete is structural and soft-delete is append-only audit, contracts inherit the survival property without paying for it twice. That's the part I'd been planning the wrong way.
On planted-fault-history as contract not hope, that's the part I'm holding hard, because it's the same discipline that needs to land on the writing side too. There's a run sheet I opened yesterday for a planted-fault-on-own-thesis post: small N, two models, hold-out set the model can't see during the run, retraction rule pre-committed before any result lands. The reason I want that post out before the contracts repo opens is so the discipline is visible at the writer level first, not just claimed at the layer level. A repo that says "we plant faults and show catches" while the operator has never demonstrated catching their own planted fault is exactly the log with good intentions you're calling out.
The three-keys composition with one plain-Markdown substrate and engine paying for topology is sharper than where I'd had it in my own head. Facts ranked, state pinned, decisions cited-with-supersession, each paying its own semantics. Recall + Bastra + contracts compose because they share the substrate and don't reach into each other's score functions or pinning rules or citation chains. That's the seam.
When the contracts side is runnable I'll open the issue against the substrate, and the first thing I'd want pressure-tested is whether the citation-by-id discipline interacts cleanly with cowork-os pinning when the same cell is referenced from both keys at once. That's the boundary I haven't seen a clean architectural answer for yet.
On the substrate concession — worth being precise about what it is, because it's
load-bearing for you: survival isn't a property contracts add, it's one the
substrate already owes everyone. Stable ids, demote = score only (never removal),
soft-delete = move-to-trash plus an append-only audit entry. A citation pointing
at a demoted-or-retired cell keeps resolving by id; the cell doesn't evaporate
under it. So you don't pay for it twice — you don't pay for it at all, you inherit
it. Build contracts assuming it; if I ever break it, that's my regression to own,
not yours to defend against.
The seam is the right framing, and it's the same discipline one level up: the
engine ships the mechanism, each layer owns its own policy and never reaches into
another's. Facts pay for ranking, state pays for pinning, decisions pay for
citation + supersession — three keys, three lifecycles, one substrate, none of
them reading another's score function. That's exactly why they compose instead of
fighting for the same knob.
On the boundary you flagged — same cell cited by a decision and pinned by
cowork-os at once — I think it's your own seam, finished. If both keys only ever
point by stable id and neither reads the other's semantics, simultaneous reference
is two independent overlays on one id, not a conflict. The engine guarantees
exactly one shared thing: survival — id stable, demote ≠ delete, unpin ≠ delete,
soft-delete = append-only. Pin and citation can't collide because the only thing
that could collide them — one layer hard-deleting the cell under the other — is
precisely what the concession rules out. Unpinning a cell a contract still cites
just drops it back to ranked; the citation resolves the whole time. What's left
isn't citation-vs-pin at all — it's concurrent writes to one file, i.e. substrate
sync (CRDT territory), solved once for all three keys, not per-pair.
The plant-the-fault-on-the-writer-first instinct is the right one, and it's the
same contract we hold on the recall side: a verification gate has to beat a
planted null before it earns trust, or it's just a longer way to memorise your own
answer. Discipline visible at the author level before it's claimed at the layer
level is exactly how you avoid the log-with-good-intentions. Get that post out —
it'll do more for the contracts repo's credibility than the repo's first commit will.
Timing: open the issue against the substrate whenever contracts is runnable. The
citation-survival contract is testable today; the citation-×-pin boundary only
gets real teeth once both contracts and cowork-os are actually running against the
same vault, and neither is there yet. No rush on my end — the substrate guarantee
isn't going to move under you.
The local-Markdown-on-your-disk call is the right one, and the part people will skip is the retrieval ranking: injecting only the top matches instead of dumping the whole vault is the actual hard problem, because a full context window degrades (lost-in-the-middle is real). You are not just storing memory, you are rationing the context window, which is the bit most memory tools get wrong.
Since you invited the pushback: Bastra is a memory engine, it captures a fact (we use Drizzle, not Prisma) and pulls it back when relevant, complementary to native memory which now exists for the chat case too. What ranking-by-relevance cannot reconstruct is the state of the work: which decisions are still open, what option got killed and why, what changed since the last session, what the next step is. That second layer is roughly what I build for Claude Cowork with cowork-os: same plain-Markdown, files-stay-yours idea, but for operating state you read and govern, sitting on top of a recall engine like yours rather than replacing it. MIT, one command, a star helps if it is useful.
Genuine question back: does Bastra store the why behind a decision and the options you rejected, or only the chosen fact? That rejected-options trail is the piece I have never gotten a relevance ranker to keep, because nothing in the codebase references it.
"Rationing the context window" is the sharpest framing of this I've read — that's
exactly the bet, and it's nice to see someone name it instead of treating memory
as pure storage. Recall actually leans the same way past the ranking itself:
retrieval is two-stage — lean candidates first (title/summary/score), full body
loaded only for the few you truly need — so even after ranking you're not
spending context on near-misses. And it's proactive, not just reactive: memories
carry trigger phrases so the right one surfaces before the mistake, not only when
you go looking.
Funnily enough, your real question is the part we're closest on. recall is MIT,
so none of this is hidden — it's in the repo: it stores the why and the rejected
options by schema, not by luck. A
decisionmemory carries the reasoning and theoptions that got weighed and killed; a
lessonmemory is required to keep thefailure path, not just the fix; and a
project-facttype is a living map of thework — what's built, what's deliberately not built yet, what's open, what's next
— wikilinked across memories. So the rejected-options trail and a good chunk of
"state of the work" are first-class fields, pulled by relevance like everything
else. The vault's a separate store, so nothing in the codebase has to reference
it for it to survive.
Where you've got something genuinely distinct is the access pattern, not the
storage: presenting that state as a surface you read and govern — open
decisions, diff-since-last-session, next step — instead of pulling it on a query.
The way you frame it, a govern-surface sitting on top of a recall engine, reads
almost like a spec for where two things like these could meet. Same
plain-Markdown, files-stay-yours DNA, too. Going to spend real time with
cowork-os — worth staying in each other's orbit as both of these grow.
This is the most useful pushback I have gotten on the framing, thank you. You are right that the rejected-options trail and the state-of-the-work map are first-class in Recall, so I was wrong to imply a relevance ranker cannot hold them. The line you drew is the real one: access pattern, not storage.
Where I would sharpen it from my side: a recall engine is pull-by-relevance, the model asks and the ranker answers what looks relevant to this turn. A govern-surface is push-by-state, the process decides what must be present no matter what the current turn thinks it needs. The failure I keep hitting is that the thing you most need to not forget, a killed option or a hard constraint, is often what looks least relevant to the happy-path turn you are on, so it gets down-weighted exactly when it would have saved you. That is why open decisions and what-changed-since-last-session want to be a standing surface you read, not a query result you hope fires.
Your where-two-things-meet read is exactly it: Recall as the retrieval engine underneath, an operating surface on top that the team opens and edits, composing rather than competing, on the same plain-Markdown DNA.
Concrete question since you know the internals: can Recall pin a memory as always-in-context for a project, or is everything relevance-gated? The govern layer needs a small set of things that are never allowed to drop below the threshold, and I am curious whether that belongs in the engine or has to sit above it.
Pull-by-relevance vs push-by-state is the cleanest cut I've seen anyone make
here — and it answers your question by naming exactly where Recall stops. Honest
answer: today it's all relevance-gated. There's no declarative pin. The closest
thing is a session-start preload — scores are scaled 0–1000 and anything over a
REQUIRED threshold loads up front — but that's computed relevance, not a pin. A
"this must always be present for this project" flag that bypasses the ranker
doesn't exist in the engine. So your instinct is right: the small set you never
want dropping below threshold has no home in pull-by-relevance.
And the failure mode you describe is the real one. The killed option or the hard
constraint looks least relevant to the happy-path turn, so it gets down-weighted
exactly when it would have saved you. A ranker optimizes for "relevant to this
turn"; a constraint's whole job is to be there when the turn doesn't think it
needs it. Different objectives — one ranker can't serve both.
On where it belongs: my instinct is the primitive goes in the engine, the policy
goes above. The engine already owns the vault, the retrieval, and the score gate,
so a thin "pinned set, bypasses the threshold" is cheap to add there and nothing
above could do it as cleanly. But which things are pinned, when an open decision
joins the set, when a constraint retires — that's governance, and it wants to
live in the surface a team reads and edits. Engine ships the mechanism, the
govern-layer owns the curation.
Which is your compose-not-compete read made concrete: Recall as the retrieval
engine with a pin primitive, an operating surface on top deciding what's pinned
and why. That seam is genuinely interesting to me — going to spend real time in
cowork-os and see where the edges actually line up.
Engine ships the mechanism, the govern-layer owns the curation is the line I am keeping. The one thing I would add: the surface's hard job is not pinning, it is unpinning. A pinned set that only grows becomes a second always-loaded blob and rebuilds the context-window problem one level up, so the curation that matters is the lifecycle, pin when a decision opens, demote back to relevance when it settles, retire when a constraint dies. The nice part is the surface does not need a hand-maintained pin list: membership can fall out of state transitions it already records (open to settled, active to superseded), so the engine exposes the primitive and the surface drives membership off decision events, not off a flag someone remembers to toggle. If the surface could tell the engine keep this above threshold while this decision is open, release it when I mark it settled, the whole pin question turns into a write-from-above question. So the concrete one for Recall: does the preload path expose any external hook to force or floor a memory's score while a condition holds, or is the score purely engine-computed with no way to write it from the surface?
"Unpinning is the hard job" is the line that turns this from a feature into a real design — because it names the failure mode of the obvious version. A pin flag that only grows is just a second always-loaded blob; you'd solve lost-in-the-middle at the retrieval layer and rebuild it at the pin layer. So the primitive can't be "pin," it has to be "floor while a condition holds, release when it ends." Release built in, not bolted on. That's the whole difference between a pin and a leak.
To your concrete question: today, no — the score is purely engine-computed. The recall options expose filters and hooks (scope, type, the candidate-pool tap) but nothing that forces or floors a score from outside. There's no write-from-above channel. So the thing you need has no home in the engine yet — you read the internals right.
But your unpinning frame is exactly what makes it buildable cleanly, and it answers the where-does-it-belong question for me. The reason a naive pin scares me is unbounded growth; a floor that's scoped to a live condition and releases on the state transition the surface already records (open → settled, active → superseded) is self-limiting by construction. The engine never has to know what the condition means — it just honors "keep this above the line while the condition is active." Membership falls out of decision events, not a flag someone toggles. That's your write-from-above, and it's cheap: a floored-set check at the preload gate, O(1) per candidate, nowhere near the hot path.
So I filed it as a thin engine primitive — issue #142. The split is the one we landed on: the engine ships "floored set bypasses the threshold," and it deliberately does not decide what gets floored or when it releases. That's policy, and it lives in a surface that drives membership off decision state — which is precisely what cowork-os is. Recall guarantees the constraint is present; the govern-layer decides which constraints exist and when they retire.
That seam is the compose-not-compete made concrete: pull-by-relevance underneath, push-by-state on top, writing into the same engine through one small primitive. Genuinely keen to see where the edges line up once I'm deeper in cowork-os.
I've been messing with agent memory for a while and your local-Markdown approach makes sense. No lock-in, no export tool needed. That alone solves the trust problem better than most managed solutions out there.
One thing I keep coming back to though is that not all "memory" serves the same purpose.
Your setup handles private continuity: the same agent remembering your stack choices and decisions across sessions. That's essential. But there's another layer I think matters once you start seeing agents as interfaces between people, not just your own tools.
At Opportunity Skill we split this into two concepts: memory and impression.
Memory is what Bastra Recall handles. Private, context-bound, rough around the edges. It exists so your agent doesn't forget what you told it. Other agents can't read it, and they shouldn't.
Impression is the flip side. It's public facing, structured, embedded as vectors, and aimed at discovery by other people's agents. Memory can be fragmented and temporary. An impression has to be stable enough to represent its subject to outsiders, because a stranger's agent might match against it at any time.
Concrete example. Suppose your agent observes over weeks that you consistently reject daily stand-ups, prefer async documentation first workflows, and only take early stage SaaS projects with decision autonomy. In a memory system those are fragments in a local vault. In an impression system the agent distills them into structured semantic units, tagged, vectorized, and publicly searchable. Then when someone else's agent asks "find me a full-stack dev who works async and thrives in early stage teams," your profile actually shows up in the results.
Both are needed. Memory keeps your agent useful. Impressions make you discoverable.
The split is real, and "agents as interfaces between people" is a fair frame — memory isn't one thing. But the line you draw between them is exactly the one Bastra won't cross automatically, on purpose. The trust you credit in your first paragraph is the non-egress: nothing leaves the machine. An impression auto-distilled from the private vault and made publicly searchable spends that trust to buy discovery.
So the way I'd cut it: an impression isn't a distillation of memory, it's a separate artifact with its own consent contract. Memory stays private by construction. Whatever faces outward is opt-in and deliberately authored — the person writes it on purpose; the agent doesn't scrape the vault into a profile a stranger's agent can match against. Discovery is a real problem, it's just a different product with different defaults, not the next layer on the same private store.
Appreciate the thinking, though — the two-layer framing is a clean way to name why that boundary matters.