DEV Community

pranav-afk
pranav-afk

Posted on • Originally published at cartha.in

Empty Memory Isn’t Missing Data — It’s a Permission Hole

Definition: In multi-agent AI systems, an empty memory recall often means this agent is not allowed to see that fact — not the fact does not exist. If your stack turns both cases into [], the model will invent over a permission hole.

That problem is one reason we built Cartha — a governance-first agent operations platform (scoped memory, traces, hard budgets, cost attribution). Full setup guide: How to Use Cartha.


The multi-agent failure mode

Single-agent demos treat memory like a vector DB:

embed → top-k → stuff into the prompt → hope.

That breaks as soon as you have:

  • more than one agent
  • more than one user
  • more than one team

Example

  • Support agent
  • Finance agent
  • Shared user_id

Support asks: “What’s this customer’s plan and last payment?”

If finance memory is out of scope for support, a naive stack returns nothing.

The support agent then:

  1. assumes nothing is known
  2. invents a plan
  3. or calls tools it shouldn’t

You only notice when the customer is wronged — or when someone asks:
“What did the agent know when it said that?”


Silent empty vs explicit withhold

Mode Agent sees Risk
silent Empty hits Hallucination over a permission hole
denied_hint Explicit “matched but withheld” (no content) Agent can refuse to invent

For fleets, explicit withhold is the better default.

Not perfect privacy (existence can leak).
But silent empty trains agents to fake authority.

A good denial sounds like:

A finance-scoped memory matched, but this agent cannot read that scope. Do not invent a substitute — treat this as withheld context.

That’s a permission event, not missing data.

On Cartha, org default denial mode is denied_hint (silent is opt-in in Settings). Details: cartha.in/how-to-use.


Scopes that actually mean something

Scope Who can recall
user Agents serving that user
agent Only this agent
team Agents sharing a team id
org Whole organization

The enum is not the product.

Enforcement is. If the agent can query a raw index with broader credentials, “scopes” are cosplay.

Cartha enforces scopes server-side on store/recall — not only in a system prompt.


What to log (for humans and for ops)

When recall is denied, log:

  • agent id
  • requested scopes
  • denial reason
  • that content was withheld (not the secret content)

Otherwise you only get “the model made something up” with no path to why.

This pairs with multi-agent debugging: wrongness often starts at a boundary (handoff or memory boundary), not at the final sentence.


Minimal pattern (any framework)


python
# Store with an explicit scope
await remember(user_id=uid, content="Prefers email", scope="user")

# Recall — still server-clamped
hits = await recall(
    user_id=uid,
    context="contact preference",
    scope=["user", "team"],
)

# Prefer structured denials, not only empty lists
# so the agent branches: "no access" vs "no data"
Enter fullscreen mode Exit fullscreen mode

Top comments (0)