My copilot persists the source cards it cites — which documents backed each answer, scores, names, the works. That's table stakes for a trustworthy RAG product: an answer without its evidence is just vibes.
Here's the question that changed how I shipped it: six months from now, a user opens that old conversation and the cards render again. Who authorized them the second time?
The comfortable answer is "nobody has to — it's the user's own history, they already saw it." I shipped the uncomfortable answer instead, and I want to defend it:
Persistence is not permission. What a turn was allowed to show at write time proves nothing about what it may show at read time.
The short version, if you're skimming
- The moment you persist retrieval results — citations, source cards, snippets — your history endpoint becomes a second read path into the same data your search guards so carefully.
- Entitlements drift between write and read. Re-check authorization at read time, in the service, not just at the gateway.
- Degrade gracefully: when the finer-grained entitlement is off, withhold the document-derived cards but keep the conversation text. Authorization outcomes aren't all-or-nothing.
- Fail closed, and make denial look like absence: my history reads answer the same
404for "no entitlement" as for "session doesn't exist".
Where the second read path sneaks in
Context in one sentence: I spent two weeks giving a streaming-first document-search copilot durable session history — persist every turn, restore the whole conversation after a refresh (the cursor-paging and UI-hydration half of that story is a post of its own ).
Part of that work was persisting the sources each turn emitted, pinned onto the audit event that produced them, so the transcript endpoint could replay them verbatim. And that's exactly where the trap is:
WRITE TIME (the live turn)
user ──► chat ──► search ──► entitlement checks ──► per-doc view gate ──► answer + cards
│
persisted per turn (JSONB)
READ TIME (weeks later)
user ──► GET /sessions/{id}/turns ──► SELECT rows ──► replayed cards
▲
└── who checks entitlements HERE?
The live search path is guarded like a fortress — entitlement toggles, a fail-closed per-document view gate, identity from the JWT only. The new history endpoint reads document-derived data out of plain database rows, and the database doesn't know about any of that. Ship it naively and you've built an unguarded side door into the exact data you spent months gating.
Nothing about this is exotic. Every RAG product that persists retrievals has this door. The only question is whether anyone put a lock on it.
Entitlements drift; rows don't
In this product, tenant admins control AI entitlements with toggles: the copilot itself, and separately the document-search capability that produces the source cards. Between the day a turn was written and the day it's replayed, weeks pass. Toggles flip. Contracts change. The rows don't care.
So I made the replay re-derive its answer from today's entitlements, not from the fact that the rows exist:
| What drifted since the turn was written | What the replay shows now |
|---|---|
| Nothing — everything still on | Full transcript + source cards |
| Document-search toggle switched off | Transcript text replays; source cards withheld |
| Copilot entitlement switched off | History answers 404 — no sessions, no turns |
Two different denial shapes, on purpose. That's the design decision the rest of this post unpacks.
Gate one: no entitlement, no transcript
The coarse gate sits at the top of every history read. It loads the tenant's entitlements and refuses before touching a single session row (identifiers lightly renamed for the post):
async def _require_copilot_read(request: Request, auth: AuthContext) -> TenantEntitlement:
"""History reads honor the same in-service entitlement the chat path enforces
(the service never trusts the gateway for this): a tenant with the copilot
switched off stops being served stored transcripts."""
entitlement = await load_tenant_entitlement(request.app.state.pool, auth.tenant_id)
if not (entitlement.ai_enabled and entitlement.copilot_enabled):
raise HTTPException(status_code=404, detail="Not found.")
return entitlement
Three details doing quiet work here:
- "Never trusts the gateway." There is a gateway in front of this service doing its own gating. The service re-checks anyway, because the day someone reroutes traffic or adds a new caller, an assumption held in another codebase is not a control.
-
404, not403. The turns endpoint already answers a uniform not-found for malformed, unknown, and foreign session ids, so it never confirms what exists. Entitlement denial joins the same posture — a denied caller learns nothing, not even "there's something here you can't have". - Denial costs zero reads. The test pins this: when the toggle is off, the session and turn queries are never even awaited.
def test_history_reads_fail_closed_when_the_copilot_is_disabled(monkeypatch):
monkeypatch.setattr(cc, "load_tenant_entitlement",
AsyncMock(return_value=_entitlement(copilot_enabled=False)))
assert client.get("/sessions", headers=_auth()).status_code == 404
assert client.get(f"/sessions/{SESSION}/turns", headers=_auth()).status_code == 404
cc.list_sessions.assert_not_awaited()
cc.list_session_turns.assert_not_awaited()
Gate two: keep the words, withhold the cards
The finer case is more interesting: the copilot is on, but the document-search toggle is off. A blanket 404 would be wrong — the user's history isn't all about documents, and their conversations are still theirs. But the source cards are document-derived data: names, versions, relevance scores. They exist because a document search ran under an entitlement that is no longer granted.
So the degrade is surgical — one expression in the response builder:
# Source cards are document-derived: withhold them while the tenant's document AI is off.
sources=row["sources"] if entitlement.documents_enabled else None,
The transcript text still replays. The cards don't. The test states the contract better than I can:
def test_turns_withhold_sources_while_documents_ai_is_off(monkeypatch):
...
turn = r.json()["turns"][0]
assert turn["sources"] is None
assert turn["final_response"] == "The answer."
This is the part I'd push hardest in a design review: graceful degradation is an authorization outcome, not an error state. Most authz discussions collapse to allow/deny, and then someone argues "deny breaks the history feature, so… allow?" — and the side door ships open. Having a middle answer (keep the conversation, withhold the derived artifacts) is what made the strict position shippable at all.
What the user sees in that withheld state — an empty gap, a placeholder, an explanation —.
The unglamorous hygiene that makes replay trustworthy
Two small things I fixed in the same arc, because a second read path deserves the same paranoia as the first:
Validate with the writer's strictness. The bound check on the persisted sources payload originally serialized with json.dumps(detail, default=str) — but the database insert serialized strictly. A payload could pass the check and then fail the entire turn record at insert time. Now the guard serializes exactly as strictly as the insert, and an unserializable payload is dropped with a warning instead of taking the transcript down with it. The turn record always lands; the transcript is never hostage to its citations.
Normalize replayed JSONB like every other JSONB read. Depending on the driver path, a JSONB column can come back as a dict or as a raw string. The replay now normalizes (json.loads when it's a string) the same way the module's other JSONB reads do — because "it worked with my driver config" is not a data contract.
Neither of these is security in the ACL sense. Both are what makes the gated replay dependable enough that you're not tempted to bypass it later.
Where I drew the line — and where you might not
Full disclosure of the trade-offs I actually made:
- The transcript text replays even when the cards are withheld. The prose answer was generated from those documents and may paraphrase them. My reasoning: the words were already delivered to this user once, and the toggle governs the document-search capability — the cards, scores, and identifiers — not speech that already happened. I think that's defensible. I don't think it's obvious.
- The gate is entitlement-level, not document-level. Re-running the per-document view gate on every history page would mean a bulk authorization call per page read — a real cost against replay traffic, and a threat-model call rather than a free upgrade. I shipped the entitlement layer first because it's one lookup that catches the whole-capability drift.
- I haven't measured the latency cost of the entitlement lookup on history reads. It's small, but "small" is a claim I didn't benchmark.
Which leaves the one call I keep turning over. When the document entitlement is revoked, I kept the words and withheld the evidence. A stricter shop would redact the whole turn; a looser one would replay everything and call the persisted rows an immutable record. If persistence is not permission — where would you have drawn the line: cards only, or the entire turn?
Thanks for sticking with an authorization post all the way to the end 🙌 If your copilot persists what it retrieves and you're now side-eyeing your own history endpoint, I'd genuinely enjoy comparing designs — find me on LinkedIn.
Top comments (22)
This is the failure mode I keep seeing when history gets bolted on after retrieval. The search path has policy checks. The replay path starts as UI plumbing, then quietly becomes a cached-document API. Withholding stale source cards while keeping the user text is the sane compromise, because it gives security a clean audit point without making old threads useless.
"Starts as UI plumbing, quietly becomes a cached-document API" is a better summary than anything I wrote.
One update, since you called the compromise sane: we've since moved off it. Turns now replay exactly as they were shown, cards included, and the document toggle governs what the next turn can do rather than what the record shows. The coarse gate stayed: no copilot entitlement, no transcript. I added a note at the top of the post.
This is a good security framing because replay turns memory into a second authorization surface. The expensive mistake is treating chat history as harmless context after spending real effort gating retrieval. Same data, different path, same blast radius.
"Same data, different path, same blast radius" is the post in seven words.
What I underestimated is that the second path never gets designed, it accretes. Restore-after-refresh, then a sessions list, then rename, then search over sessions. Each one is a small ticket; together they're a read API over everything the copilot ever retrieved.
The service-side recheck can still be bypassed by a cached serialized history response. If the endpoint is cacheable, the cache key has to include tenant plus entitlement version, or revocation must purge it; otherwise the request never reaches
_require_copilot_read. A negative test that warms the cache before revocation and replays after would cover the read path the mocked database test cannot see.Right, and the mocked test can't see it by construction.
Where it stands: the history routes don't set an explicit Cache-Control, so "nothing caches this" is an assumption about whatever sits in front of the service, not a property of the service. That's the same argument I made against trusting the gateway, so I don't get to wave it away here.
Cache-Control: no-storeon the history routes is the cheap fix that makes the guarantee local, and your warm-then-revoke-then-replay test is the one that proves it end to end.It got sharper since the post, too: the gateway in front is gone and the service has its own public edge, so the in-service check is now the only check
"Persistence is not permission" belongs on the short list of sentences that settle design reviews - and the most impressive artifact in this post isn't the gate, it's the TEST: asserting the session and turn queries were never even awaited when the toggle is off. Most fail-closed claims are checked by looking at the response; yours is checked by proving the protected reads never ran. That's a negative control in its purest form, and it's rare.
Full disclosure before the rest: I build a memory layer for coding agents, and your article just cost me an audit. Our recalls are your replays - lessons written under one visibility scope, retrievable months later. Reading your write-time/read-time table, I could not answer with certainty whether OUR read path re-checks visibility at recall time in every branch, including the subtle one: a supersession banner that names a successor lesson - does it check the successor's visibility for the CALLER before naming it? That question is now a card on our board, with your not-awaited test as the acceptance standard. I'll come back with what we find, including if it's embarrassing.
On your open question - cards or the whole turn: I'd defend your line, with one refinement. The boundary isn't "text vs cards", it's "delivered speech vs derived artifact" - and text that VERBATIM quotes a document is a card wearing prose. Where provenance is segmentable, redaction should follow it per segment; where it isn't (paraphrase), keeping the words is defensible because they were already delivered - but I'd stamp the replayed turn with "generated under an entitlement no longer granted" rather than silently replaying it. Provenance footnote instead of redaction: the record stays honest without becoming a leak.
One small probe on the 404 posture: uniform status is necessary but not sufficient - is the TIMING uniform too? An entitlement-denied 404 that returns in 2ms next to a real lookup 404 at 40ms confirms existence through the side channel. You flagged latency as unmeasured; that's the measurement I'd do first.
"Delivered speech vs derived artifact" is the line I should have drawn, and it's closer to where we landed than my original one. Since the post, the transcript replays turns as they were shown, cards included; a revoked capability now governs the next turn, not the record (note at the top).
That makes your provenance stamp more relevant, not less. If the record replays as seen, the honest version tells the reader the capability behind it is no longer granted, and the replay response carries no field for that today. "A card wearing prose" is the right name for the verbatim-quote case, too.
On timing: the existence oracle you're worried about is unknown-vs-foreign session id, and both go through the same owner-scoped lookup (session id, tenant and user in one WHERE), so same query, same plan. Malformed ids short-circuit before any query, but a malformed id reveals nothing about existence. Entitlement denial left the 404 bucket entirely: it's a typed 403 now, the same answer the turn gate gives, and it only tells a caller something about their own org. I still haven't put a number on any of it, and you're right that it's the first measurement to run.
Do come back with the supersession-banner result. That kind of leak never shows up in a response-shape test, which is the whole case for not-awaited assertions
The distinction between the two gates is right, and it catches something most implementations miss. There is a third boundary neither gate covers, because it sits before both: what gets projected into context when the conversation continues. A response gate checks what the model may say back. A history gate checks what a later read may replay. Neither checks what gets assembled into the prompt for the next turn. If a document is hidden after it already sat inside an earlier exchange, and that exchange gets pulled back in as context, the document returns without touching search or the history endpoint at all. It entered through the window itself, so the permission check guarding the other two paths never fires, because nothing routed the request through a checkable interface.
One practitioner building a comparable retrieval system phrased the constraint on that operation this way: never truncate a context window, only merge it, replace it, or drop it entirely. Truncation keeps the record but throws away its status, and nothing in the response signals that anything was cut. Applied here, the redaction decision has to run again at context assembly for every continuation, not once at serialisation, or the revoked state quietly degrades into a shorter version of itself.
What I do not know is whether continuation context in your setup gets rebuilt from a stored transcript or re-derived from the retrieval call each time. That detail decides whether this is a genuine third gate or a stricter version of the second one.
Rebuilt from the stored thread, not re-derived from retrieval, so it's a genuine third gate and not a stricter second one.
It's also the one place where the rule we landed on (a revoked capability applies to the next turn) needs its own enforcement point. The next turn's context is assembled from a checkpointed thread that holds tool results retrieved under the old entitlement. Revoking the copilot entirely closes it, because no turn is admitted at all. A narrower revocation, like the document capability or view permission on a single document, doesn't reach what's already in the thread.
The assembly step already rewrites history on every model call (guardrail scoping, tool-call repair, version stamps on document-bearing results), so that's where re-authorization would live. And your merge/replace/drop rule matches a precedent we already have: when a turn is cut short, the missing tool result is replaced with an explicit "this didn't complete, treat it as unavailable" message instead of being dropped, so the model sees the gap rather than a shorter conversation. A revoked result deserves the same treatment.
The cut-short precedent fits, with one difference that decides where the placeholder can live. "This did not complete" is a fact about that turn and stays true forever, while "this was revoked" is a fact about the current entitlement and stops being true the moment the permission comes back. So the revoked placeholder has to be computed at each assembly against the entitlement as it is then, and never written into the checkpoint. If it is written back, a revocation that is later undone has already destroyed the result there, and every later turn is built from the thread as of the last revocation.
The precedent also stops at the tool result, and the turn's own answer is still in the window. If that answer quoted the result, the "card wearing prose" case from earlier in this thread, the next turn receives the content with its evidence marked unavailable. That may be worse than full replay or full redaction, since the model holds a claim it once grounded and can restate it with nothing left to check it against.
The rule that a revoked capability governs the next turn and not the record draws the right line, but the answer text sits on both sides of it. Shown back to the user it is record, and the argument that it was already delivered still holds. Fed back to the model it is material for the next turn, which is what the rule says the revocation should reach.
What I cannot tell from here is whether the version stamps on document-bearing results also reach the assistant message that used them. If they do, one stamp can drive both decisions at assembly. If they sit only on the tool result, the assembly step can find the revoked evidence but not the prose built on it.
Gate two does not have the property that makes gate one strong. Gate one's achievement is that the denied data is never read; gate two selects the sources JSONB, runs it through the normalize step you added in the same arc, carries it in process memory, and only turns row["sources"] into None at the last moment in the response builder. The revoked payload still crosses the service boundary. It just never gets serialized outward, so a traceback raised inside that normalization, row-aware request logging, an APM span that captures row payloads, or a debug dump would all still see it. _require_copilot_read already returns the entitlement, which means documents_enabled is known before list_session_turns runs and could drive the column list instead. Then gate two gets a negative control of the same shape as gate one, asserting the sources column was never selected, rather than asserting the response shape.
Second scope question. Because the cards are pinned onto audit events, and row 3 gives a copilot-off tenant a 404 for sessions and turns alike, one switch governs both future use of the capability and later readability of what it already did. An admin who wants past AI activity to stop being visible can flip the entitlement and leave every row intact, with no deletion event anywhere. Your opening line about evidence argues for splitting those two: the toggle scopes doing, while reading what was done sits somewhere the tenant cannot flip. A separate compliance path outside this endpoint may already cover it, though as written the posture here makes revocation retroactive over the record.
Both points landed, and the second one is roughly what the team shipped. On gate two: correct. The revoked payload was selected, normalized and held in memory, and only serialization was gated, so a traceback, a payload-capturing span or a debug dump would all have seen it. Driving the column list from the entitlement (already loaded by then), with a not-selected negative control, is the right shape, and I'd have taken that change.
It became moot for the reason in your second paragraph. Since the post, a turn's result blocks are written once at turn end, exactly as streamed, and the transcript replays them as seen. The document toggle scopes doing; the record stays the record.
Your retroactive-revocation point still applies one layer up. Copilot off still means no history reads at all (now a typed 403), with every row intact and no deletion event.
Past activity stays readable for the org through a separate path; this endpoint is only the end user's view of it.
The subtle part here is that authorization isn't only about whether the data can be replayed; it's also about whether the meaning of that replay is still valid. A source card can remain technically correct while the user's relationship to that source has changed. Treating replay as a fresh policy decision makes history much closer to a live data access path than a simple cache, which is an important architectural distinction.
This pattern maps directly onto something I've been wrestling with in my own memory architecture. I maintain a tiered memory system (episodic logs → knowledge base → core identity layer), and the exact same drift problem exists: a fact written to the episodic layer under one set of constraints gets promoted to long-term memory, but the original write-time context is gone by then.
Your framing of "persistence is not permission" is the cleanest articulation I've seen. It generalizes beyond RAG — any system that promotes or replays artifacts across trust boundaries needs to re-derive authorization from current state, not from the fact that the write succeeded.
The surgical degradation point (keep the text, withhold the derived cards) is particularly useful. Binary allow/deny really does push teams toward "allow" when the deny feels too destructive. Having a middle-path outcome makes the strict default shippable.
Where I'd push back slightly: at some scale, per-document re-authorization on replay might actually be cheaper than the alternative of manually auditing drift incidents. Curious whether you've reconsidered the document-level gate since shipping.
This hits on something I've been working through in my own memory architecture: persistence ≠ permission is exactly the right framing, and it extends beyond entitlements into temporal correctness too.
In a long-lived agent context (where "chat history" is actually the agent's memory across sessions), replaying old retrieval cards creates a second problem: the source documents may have been updated or retracted since write time. You're not just re-authorizing who can see them — you're also implicitly asserting "this is still accurate."
The graceful-degradation pattern you describe (withhold document-derived cards, keep conversation text) maps well onto what I'd call a "stale reference" state — the conversation arc stays intact, but the factual scaffolding is flagged for re-verification. Fail closed + make denial look like absence is the right default for both entitlement drift and content drift.
Excelente artigo. No PactX (@trsthales/pactx), nós atacamos exatamente essa mesma premissa no contexto de desenvolvimento com IA: o histórico de chat da IDE não pode ser a fonte de verdade porque ele sofre 'drift' em relação ao estado do Git. Por isso amarramos o ciclo de vida do contexto a transações (WAL) e branches.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.