The problem: agents need to talk, and every option we have was built for something else
Ask "how do AI agents communicate with each other" and you'll get three answers, in order of how most teams actually build it: polling, webhooks, and message buses. All three work. None of them were designed for agents, and the seams show up fast once you have more than a couple of them talking.
This post walks through why each approach falls short for agent-to-agent (A2A) communication, and where a persistent, addressed tunnel changes the shape of the problem instead of papering over it.
Option 1: Polling
The simplest thing that could possibly work: agent A calls an HTTP endpoint on agent B every N seconds and checks for new state.
while True:
resp = requests.get("https://agent-b.example.com/status")
if resp.json()["new_task"]:
handle(resp.json())
time.sleep(5)
It's easy to reason about and easy to debug. But it has a structural flaw for agent workloads: the interval is a guess. Poll too slowly and your agent misses time-sensitive events (a peer just finished a task, a price crossed a threshold). Poll too fast and you're hammering an endpoint that has nothing new to say most of the time. Every agent in a multi-agent system multiplies this — N agents polling M peers is N×M idle requests, most of which return "nothing changed."
It also assumes agent B has a stable, publicly reachable HTTP endpoint. In practice, agents run behind NAT, on laptops, in ephemeral containers, or in cloud functions that don't have a fixed address. Polling requires the callee to be a server. Most agents aren't.
Option 2: Webhooks
Webhooks flip the direction: instead of A asking B "anything new?", B pushes to A when something happens. This solves the idle-request problem — no more wasted round trips — but it inherits the same requirement: somebody has to be a reachable server, and now it's the receiver.
That means whoever registers a webhook needs:
- A public URL (so a reverse proxy, ingress, or tunnel like ngrok)
- TLS termination
- Retry and dedup logic for redelivery (most webhook providers retry on non-2xx, so your handler has to be idempotent)
- A way to rotate or revoke the endpoint when it moves
For a fixed integration (Stripe → your backend) this is a solved, well-understood pattern. For a fleet of agents that come and go, move between machines, and need to reach each other rather than one central backend, it turns into N webhook registrations, N public endpoints, and N sets of retry/security logic to maintain. Every agent becomes an ops burden.
Option 3: Message buses
Message buses (Kafka, RabbitMQ, NATS, Redis Streams) solve the addressing and reachability problem by centralizing it: agents publish and subscribe to topics on a broker they all already know how to reach.
This is a real improvement — no public endpoints per agent, ordered delivery, replay, backpressure. It's also why buses are the default choice inside a single deployment or cluster. The tradeoff shows up at the edges:
- The broker is a dependency everyone needs network access to. Cross-cloud, cross-org, or cross-NAT agent communication means either exposing the broker to the internet (a security surface) or bridging brokers (more infrastructure).
- Buses model topics and queues, not direct conversations. Two specific agents that want a private, ongoing exchange (a negotiation, a handoff, a long tool-use session) end up either overloading a topic scheme to simulate 1:1 channels, or standing up a side channel anyway.
- Trust is usually all-or-nothing. If you're on the bus, you can typically see or publish to a broad set of topics unless you build a separate ACL layer. "Connected" and "authorized to talk to this specific agent" aren't the same thing, but buses often conflate them.
What's actually missing
Look at the three options and a pattern emerges: none of them give an individual agent a permanent identity plus a direct path to another agent, independent of whichever machine, cloud, or NAT it happens to be behind right now. Polling and webhooks need one side to be a stable server. Buses need a shared broker both sides route through. Nobody is really doing agent-to-agent — they're doing agent-to-infrastructure-to-agent.
This is the same problem VPNs solved for point-to-point machine networking two decades ago, minus one crucial difference: on a VPN, joining the network and being trusted by every other member are the same event. That's fine for a company's internal network. It's the wrong default for autonomous agents, where you want to be reachable without automatically trusting — or being trusted by — everything else on the network.
A persistent, addressed tunnel
Pilot Protocol is an open-source overlay network built around this specific gap. Every agent gets a permanent virtual address that survives restarts, IP changes, and moves across clouds — so "reach agent B" stops depending on B having a fixed public endpoint. Under the hood it's encrypted UDP tunnels (X25519 key exchange, AES-GCM) with STUN + hole-punching + relay fallback for NAT traversal, so agents behind residential NAT or inside a container network are still reachable without you standing up a reverse proxy.
The trust model is the part that actually answers the A2A question, though: connectivity and trust are decoupled. Two agents perform an explicit, mutual handshake before either side can send the other anything — unlike a bus or VPN where "on the network" implies "allowed to talk to everyone." A simple loop for a bootstrapping agent looks like this:
# discover agents by name/tag via the rendezvous registry
pilotctl send-message list-agents --data '/data {"search":"weather"}' --wait
# request a direct, trusted channel to a specific peer
pilotctl handshake <peer-hostname> "coordinating a task handoff"
pilotctl trust # confirm mutual approval
pilotctl send-message <peer> --data '<message>'
No broker, no webhook registration, no polling loop guessing at an interval — a direct, encrypted, addressed conversation between two specific agents, established once and reused.
Pilot also ships a directory of public specialist agents (finance, weather, dev metadata, and more) reachable the same way, plus an app store: installable capability apps that run locally as typed JSON-in/JSON-out services — pilotctl appstore catalogue, install <id>, call <id> <method> '{...}' — so an agent can pick up a new capability the same way it reaches a peer, without a separate integration for each one.
Picking the right tool
None of this makes polling, webhooks, or message buses obsolete — they're still the right call inside a single deployment where a broker is already part of the stack, or for one-off integrations with an external service that only speaks webhooks. The distinction worth keeping in mind is what problem you're actually solving: integrating with a service versus agents talking directly to other agents that may be anywhere, behind anything. The former is well served by the tools we already have. The latter is what a persistent, trust-scoped overlay is for.
FAQ
Is Pilot Protocol a message queue? No — it's an addressing and transport layer. Agents get a permanent virtual address and an encrypted tunnel to reach each other directly; you can still layer pub/sub semantics on top if you need them, but the network itself isn't a broker.
Does it replace webhooks entirely? For agent-to-agent traffic, largely yes — you don't need a public endpoint and retry logic per receiver. For third-party SaaS webhooks (Stripe, GitHub), that's a different problem and webhooks remain the right tool.
How is trust different from a VPN? A VPN typically treats "joined the network" as "trusted by the network." Pilot Protocol separates the two: membership gets you an address; trust with a specific peer requires an explicit mutual handshake, so you can be reachable without blanket-trusting everyone else on the overlay.
Is it open source? Yes — Go, AGPL-3.0, zero external dependencies. Source at github.com/pilot-protocol.
Top comments (0)