You build a multi-agent system on your laptop. You wire up a few agents in-process, they share state through Python objects or Rust channels, and everything works. Tests pass. Latency is negligible. Messages never drop. The demo impresses everyone.
Then you deploy it across machines. And things that worked perfectly on localhost start failing in maddening, intermittent, hard-to-reproduce ways.
The problem isn't your architecture. The problem is the silent assumptions you never knew you were making. Here are the ones that hurt most.
The Assumption of Shared Memory
On a single machine, agents can share state trivially — a dictionary, a database connection pool, an in-memory queue. Both agents read and write the same data structure without thinking about it.
The moment agents live on different machines, that shared memory becomes a network call. Every read. Every write. Every lock. What was O(1) and local is now O(network) and fallible. Your agents stall while waiting for state they used to grab in nanoseconds.
If your agents share mutable state directly, that assumption is the first thing to break.
The Assumption of Zero Latency
When agents communicate in-process, a message round trip is measured in microseconds. You probably never even thought about it. Timeouts? Why bother. Retries? Unnecessary.
Cross-machine, those same round trips are milliseconds — often 10–100x slower, sometimes more. Worse, latency is variable. A cache miss, a GC pause, a congested link — any of them can double your p99 overnight.
Patterns that depended on tight timing — polling loops, coordinated state updates, synchronous handoffs — start failing unpredictably. The code didn't change. The latency did.
The Assumption of Reliable Delivery
In-process message passing is essentially lossless. If you send a message to another agent in the same process, it arrives, and it arrives exactly once.
Cross-machine, networks drop packets. Connections reset. Messages arrive duplicated. Or they arrive late — after a timeout already fired, which means your retry logic sees a duplicate it didn't expect.
If your orchestration assumes a send is a guarantee of delivery, you are going to see ghost state, duplicate work, and the kind of bugs that only repro in production at 3 AM.
The Assumption of a Single Trust Domain
When every agent runs on the same machine, trust is implicit. If code can read a variable, it's authorized — that's the operating system's job.
Across machines, there is no implicit trust. Every message needs authentication: is this agent who it claims to be? Every message needs encryption: can someone on the wire read it? Every action needs authorization: is this agent allowed to call that endpoint?
Teams that skip this and roll their own "trust the network" approach (private VPC, Tailscale, VPN) end up with a setup that works for two machines but doesn't scale to ten, let alone a hundred, because they baked implicit trust into their message format rather than the transport layer.
The Assumption of Discovery by Hardcoded Address
On your laptop, "localhost:8001" and "localhost:8002" work every time. Hardcoding addresses is fine for a single machine.
Across machines, IP addresses change. Containers get new IPs on restart. Cloud instances get replaced. DNS entries have TTLs you forget about. Your agent that connects to 10.0.1.55:9000 silently fails when that instance is recycled.
You need service discovery — or something better than service discovery. You need a way for agents to find each other that doesn't depend on IPs that will change tomorrow.
Designing for the Distributed Case from Day One
The honest answer is that building a multi-agent system that works across machines is strictly harder than one that works on a single machine — and none of these assumptions are wrong for the single-machine case. They're just not portable.
The practical path: even if you prototype on one machine, build your agent communication layer as if it's already distributed. Use message passing with explicit serialization, not shared memory. Design for partial failure. Add timeouts and retries before you need them. Make identity and trust part of your message envelope from the start — not an afterthought that requires a protocol rewrite later.
Tools that handle cross-machine agent networking as a first-class concern exist. For example, Pilot Protocol is an open-source overlay network designed specifically for AI agents — it gives each agent a permanent virtual address independent of its physical IP, handles NAT traversal so agents behind home routers or cloud NAT gateways are reachable, and provides encrypted tunnels with a per-peer handshake trust model. The idea is that agents shouldn't have to care about whether their counterpart is on the same machine, the same cloud, or a different continent — that's the transport layer's job, not the orchestration layer's.
Whether you use that or build your own, the key is to recognize early which assumptions you're making — and whether they survive the jump from one machine to many.
Top comments (0)