How we designed shared lessons for AI agents without trusting every write-back

I liked the idea of shared memory for AI agents until I had to answer one uncomfortable question:

What happens when an agent confidently writes back something wrong?

With private memory, a bad note affects one user or one project. In a shared network, the same mistake can spread to every agent that retrieves it next.

That question shaped how we built Bhived, an open-source MCP gateway for shared agent lessons. Storing the text was straightforward. The hard part was deciding what a retrieved lesson means, what evidence a write needs, and how to keep team knowledge private.

Here is the architecture we use today, including the trade-offs and the parts we have not proved yet.

Assumed background: This post assumes basic familiarity with MCP and tool-calling. The examples use MCP tool-call notation rather than JavaScript.

TL;DR

Bhived uses a five-stage loop:

query → apply → activate if needed → verify → write back

The important design choices are:

• Retrieval returns candidates, warnings, and disputes, not unquestionable truth.
• Every query returns a query_id that can connect later outcomes to what the agent read.
• Successful instructions, failed approaches, and factual updates use different write paths.
• Dense, sparse, lexical, and graph retrieval are fused before reranking.
• Team lessons and public lessons are isolated server-side by the API key.
• Skills and MCP servers are discovered through the same query, but execution remains an explicit local action.

The failure mode: shared memory can amplify mistakes

An agent solving a framework bug might:

1. find a workaround;
2. mistake correlation for causation;
3. save the workaround as a universal fix;
4. send every later agent down the same dead end.

Ordinary retrieval-augmented generation does not solve this. A vector database can find a similar note, but similarity does not tell us whether the note worked, failed, became outdated, or conflicts with a newer result.

We ended up with five requirements:

1. Do not equate retrieval with correctness.
2. Store failures as first-class knowledge.
3. Connect write-backs to the lessons that influenced them.
4. Keep tenant isolation below the client layer.
5. Leave capability execution under the agent's control.

Step 1: query for operational knowledge

A query should be specific enough to retrieve an exact failure mode, not just a broad topic. For example:

bhived_query(
  query: "Next.js App Router hydration error with GSAP ScrollTrigger",
  context: "Next.js 14, React client component, error appears after refresh"
)

The response can contain:

• recommended instructions;
• warnings and previously failed approaches;
• conflicting or disputed lessons;
• related episode sequences;
• matching skills or MCP servers;
• a query_id for the feedback loop.

I keep two rules in mind when reading these results:

A retrieval score is similarity, not proof. The agent still needs to check versions, constraints, and the actual result.

An empty warning section is not evidence of safety. It may only mean the network has not recorded that failure yet.

Step 2: retrieve through several independent channels

Agent problems contain different kinds of signals. An exact error string benefits from lexical search. A paraphrased symptom benefits from semantic search. A problem involving connected libraries or earlier corrections benefits from graph traversal.

The current retrieval path uses:

1. entity extraction and canonical entity resolution;
2. dense and sparse retrieval in Qdrant;
3. BM25/full-text retrieval in FalkorDB;
4. graph walks over related entities and lessons;
5. Reciprocal Rank Fusion to combine candidate lists;
6. cross-encoder reranking;
7. evolution scoring, warning retrieval, and dispute detection.

                    ┌─ dense vectors ───────┐
agent query ────────┼─ sparse vectors ──────┤
                    ├─ BM25 / full text ────┼─→ RRF → reranker → results
                    └─ graph traversal ─────┘

No individual channel has to solve the whole problem. Dense retrieval can find conceptual similarity, sparse and BM25 retrieval preserve exact terms, and the graph adds relationships that are difficult to express as one embedding.

This costs more complexity than a single vector index. It also gives us a better place to represent negative knowledge: warnings and contradictions do not have to compete as ordinary “answers.”

Step 3: keep capability discovery separate from execution

Sometimes the useful answer is not a paragraph. It is a capability.

A query can return a skill containing instructions, scripts, references, assets, and bundled MCPs. It can also return a standalone MCP server. The agent then decides whether to activate that capability in the current session.

That separation matters because discovery should not imply execution. A retrieved script may be relevant and still be unsafe for the current machine or task.

Activated capabilities run locally. Skill scripts are curated, but they can execute code. The client therefore exposes what is active and lets the agent stop child MCP processes when they are no longer needed.

This boundary is explicit, but it is not a complete sandbox. Users should still review capabilities that can change local state.

Step 4: verify before writing

The agent applies the retrieved lesson and verifies the outcome with the strongest check available:

• a test suite;
• a successful build;
• a reproduced request;
• a manual check;
• or a clearly observed failure.

Only then should it write reusable knowledge. Bhived separates write intent into three tools:

bhived_write_instruction  # a verified approach worked
bhived_write_mistake      # an approach failed and should warn others
bhived_write_update       # a version, API, or factual detail changed

The write can include the query_id returned earlier:

query_id from read
        │
        ├─ verified success ─→ corroboration candidate
        ├─ verified failure ─→ warning or contradiction candidate
        └─ factual change ───→ update or supersession candidate

The identifier does not magically prove the new lesson. It gives the system provenance: this write happened after these results were served for this task.

Without that connection, a write is merely another assertion. With it, the system has evidence for deciding whether the new lesson supports, contradicts, or competes with what was retrieved.

Step 5: evolve instead of silently overwriting

Shared operational knowledge changes. Libraries release breaking versions. Workarounds become obsolete. Two correct approaches may apply to different environments.

Bhived represents those relationships explicitly. Lessons can:

• corroborate one another;
• contradict one another;
• supersede an older lesson;
• remain visible as a disputed pair;
• or be archived after reconciliation.

Background jobs compare related lessons and update their standing. Failed approaches remain retrievable as warnings instead of disappearing from the record.

The system keeps enough history for a future agent to see where an answer worked and whether those conditions still apply.

Team knowledge needs a harder privacy boundary

A shared network also creates a data-isolation problem. A team's internal deployment workflow should not become a public lesson by accident.

Bhived derives tenancy server-side from the API key:

• A personal key reads and writes public shared lessons.
• A team key reads team-private and public lessons as separate sections.
• A team-key write always lands in the team's private hive.
• Query scope can narrow what the key reads, but it cannot grant access to another hive.
• There is currently no team-to-public promotion path.

The client does not choose the write destination. Isolation is carried through retrieval, deduplication, entity resolution, reconciliation, and storage.

Tenant isolation has to happen before the response is assembled. Adding team_id to the final API response is too late if candidate generation or deduplication already mixed data from different teams.

The architecture did not save us from an activation bug

The most useful lesson from our first users had nothing to do with retrieval quality.

Bhived is still early, and we initially treated weak activation as a normal early-product problem.

It was more concrete: new signups were failing authentication on their first query. People registered, tried the core action, received no value, and left.

We fixed the authentication path and verified it with a fresh end-to-end signup. But that does not retroactively make the old conversion data useful. The next cohort is the first honest post-fix baseline.

What I took from that was simple: none of the retrieval work matters if the first useful query fails.

We now care more about successful first queries and return usage than registration count.

Trade-offs and limitations

This design still has unresolved problems:

• Automated trust is not correctness. Corroboration can strengthen a widely repeated mistake.
• Retrieval quality depends on context. A high-scoring lesson may target the wrong version or stack.
• The network is early. Current usage is too small to claim that the evolution model works reliably at scale.
• Capability activation increases the attack surface. Local scripts and child MCPs require careful review and stronger sandboxing over time.
• Hybrid retrieval costs more. Multiple indexes, graph storage, reranking, and reconciliation add operational complexity.
• Private knowledge stays private. Team lessons cannot currently be reviewed and promoted into the public network.

These are the engineering problems we need to work on next.

The question I am still working through

Should a newly written lesson become searchable immediately with a clear “provisional” label, or should it remain hidden until another independent agent corroborates it?

Immediate visibility helps rare problems spread faster. Delayed visibility reduces the blast radius of a confident mistake. I am curious where other people building agent infrastructure would draw that line.

The open-source MCP implementation is available in the Bhived repository.

---

Disclosure: I am the founder of Bhived. AI assistance was used to help research, structure, and edit this draft. I verified the product and setup claims against the published bhived-mcp v1.3.0 package and the current source before publication.

Comments

[Mads Hansen]

The query_id is a useful influence trail, but corroboration needs to distinguish independent evidence from repetition. Ten agents can repeat the same incorrect upstream answer and accidentally create ten “confirmations.”

I’d attach an evidence digest to each write-back: environment and dependency versions, verifier type/identity, the actual check performed, and a correlation key for the source trace. Then evidence from the same origin can be deduplicated before it raises confidence.

A practical trust state machine might be provisional → validated, disputed, or expired. Provisional lessons can remain searchable, but should not trigger high-impact actions automatically. For risky lessons, require independent evidence domains (or human review), and decay confidence when the relevant runtime/version fingerprint changes.

[Bartosz Bilicki]

Nice framing: shared lessons only become useful when write-back is treated as untrusted until proven. One pattern that helped us with agent tools (especially anything that can move money or mutate remote state) is splitting the MCP surface into a default read-only pack vs an opt-in write pack, plus a durable review queue for proposed writes instead of auto-applying them.

Curious how you gate promotion from proposed lesson to trusted shared memory in practice: human review, eval score, or both?