Why we stopped passing JSON between AI Agents (and built a Concurrent Trie instead)

OckhamNode — Sun, 19 Apr 2026 06:38:51 +0000

If you’ve spent any time building multi-agent AI systems recently (using frameworks like AutoGen, CrewAI, or LangGraph), you’ve probably hit the exact same wall we did.

It starts great. You chain three agents together: a Researcher, a Writer, and a Reviewer. But as the context grows, the architecture begins to collapse under its own weight.

We realized the standard way of handling state in AI swarms is fundamentally broken. Here is why, and how we engineered our way out of it by building a tier-1 concurrent state broker in Go called Hyperloom.

The Bottleneck: The "Token Tax"

Right now, the industry standard for multi-agent communication is passing massive JSON objects around.
If your swarm has generated 50,000 tokens of context, Agent 4 receives all 50,000 tokens, parses them, appends its own 2,000 tokens, and passes 52,000 tokens to Agent 5.

This is an architectural anti-pattern. It’s the equivalent of sending an entire Postgres database over a REST API just to update a single column. It results in massive API bills and horrific latency.

The Breaking Point: Cascading Hallucinations

The breaking point for us wasn't just the cost; it was the brittleness.

If Agent 4 in a 5-step pipeline hallucinates a corrupted JSON schema, the workflow crashes. Because the state is passed linearly, a failure at Step 4 destroys the compute and API costs spent on Steps 1, 2, and 3. You have to restart the entire loop.

Using traditional databases (Redis/Postgres) to store this state didn't work either. If you have 20 agents trying to read and write to a shared memory pool simultaneously, locking an entire JSON row in a database creates a massive swarm-wide bottleneck.

The Shift: Treating AI State like Tier-1 Infrastructure

We decided to stop treating LLM context as a massive string and start treating it as a highly concurrent distributed system.

We built Hyperloom a Memory Graph and Context Broker that uses a Concurrent Trie Forest.

Instead of agents passing state directly to each other, they act as decoupled microservices. They subscribe to the broker. When an agent finishes a thought, it doesn't pass the whole history; it publishes a localized context_diff (e.g., just the new paragraph it wrote).

Fine-Grained Locking in Go

Instead of locking the entire memory tree during a write, we implemented sync.RWMutex locks at the node level of the Trie.

This means Agent A can update the /session_1/memory path while Agent B simultaneously appends to /session_1/intent without ever blocking each other. The swarm can scale to thousands of concurrent reads/writes.

The Rollback Engine ("Ghost Branches")

To solve the hallucination problem, we implemented Speculative Execution.
When an agent submits an action, it writes to a "Ghost Branch" (a shadow state). If your verification logic flags a 400 error (e.g., bad JSON, failed tool call), the broker calls Revert().

The broker instantly drops the corrupted pointer in $< 1ms$. The bad memory is cleanly severed from the graph, and the agent is forced to retry from the last valid checkpoint. No pipeline crashes. No wasted downstream compute.

The Result: The Time-Travel Debugger

Because the backend is essentially an append-only event stream of state diffs, we realized we had accidentally built a version control system for AI thoughts.

To make this usable, we built a Next.js/React Flow frontend that hooks into Hyperloom's WebSocket stream.

You can visually watch the tree of agent thoughts grow in real-time. If an agent hallucinates, the node flashes red and is pruned. Even better you can grab a timeline slider, scrub backward through the swarm's execution, and click on any node to see exactly what context an agent was looking at before it made a mistake.

Try it out (We Open-Sourced it)

We kept the broker entirely in-memory for now to hit ~2,000 req/s, but we are actively exploring the best way to implement a zero-blocking Write-Ahead Log (WAL) for durability.

We also shipped it with a native MCP (Model Context Protocol) bridge, so Claude Desktop can read/write to the graph directly.

You can spin up the entire broker locally with one command:
docker run hyperloom

Check out the repo here: OckhamNode/hyperloom

I’d love for the Go engineers and AI builders here to tear the architecture apart. Are we crazy for using sync.RWMutex at the node level instead of a purely lock-free data structure? Let me know in the comments.

DEV Community: OckhamNode