Definition: In multi-agent systems, an empty memory recall often means this agent is not allowed to see that fact — not the fact does not exist. Collapsing both into [] is how agents invent over permission holes.
This is one of the core problems we design for at Cartha — an ops/governance layer for AI agent fleets (scoped memory, traces, costs, hard budgets). Docs: How to Use Cartha.
The failure mode
Most agent stacks treat memory like a vector DB with vibes:
retrieve top-k → stuff into the prompt → hope the model behaves.
That works for a single demo agent.
It fails when you have more than one agent, more than one user, or more than one team sharing infrastructure.
Classic setup:
- Support agent
- Finance / billing agent
- Shared user id or “company memory”
Support asks: “What’s this user’s plan and last payment?”
If finance-scoped memory is out of scope for support, a naive stack returns nothing.
The support agent then:
- assumes nothing is known
- invents a plan
- or calls tools it shouldn’t
You notice when a customer is wronged — or when someone asks: what did the agent know when it said that?
Silent empty vs explicit withhold
Two denial modes:
| Mode | What the agent sees | Risk |
|---|---|---|
| silent | Empty hits | Agent invents over a permission hole |
| denied_hint | Explicit “matched but withheld” (no content) | Agent can refuse to invent; may leak existence |
For agent fleets, explicit withhold is the better default.
Not perfect privacy (existence can leak).
But silent empty trains models to hallucinate authority they don’t have.
A good denial sounds like:
A finance-scoped memory matched, but this agent cannot read that scope. Do not invent a substitute — treat this as withheld context.
That’s a permission event, not missing data.
Scopes that mean something
| Scope | Who can recall |
|---|---|
user |
Agents serving that user |
agent |
Only this agent (private) |
team |
Agents sharing a team id |
org |
Whole organization |
The critical part isn’t the enum names.
It’s enforcement on the server — not “please only request allowed scopes” in a system prompt.
If the agent can query a raw index with broader credentials, your “scopes” are cosplay.
At Cartha, memory scopes are enforced server-side; org default denial mode is denied_hint (silent is opt-in). See the full walkthrough: cartha.in/how-to-use.
Minimal shape (any stack)
python
# Store with an explicit scope
await remember(user_id=uid, content="Prefers email", scope="user")
# Recall — requested scopes still get server-clamped
hits = await recall(
user_id=uid,
context="contact preference",
scope=["user", "team"],
)
# Prefer structured denials, not only empty lists
# so the agent can branch: "no access" vs "no data"
Top comments (0)