DEV Community: William Baker

Building Multi-Agent Fleets That Actually Talk to Each Other

William Baker — Wed, 13 May 2026 21:37:56 +0000

The consensus architecture for multi-agent systems in 2026 is: orchestrator + isolated subagents. A single coordinator holds context, spawns specialists, merges results.

You've probably built something like this. A coordinator agent fans out to a research subagent, a code-generation subagent, maybe a QA subagent. The orchestrator waits, collects, synthesizes.

It works. But there's a hidden assumption baked into most implementations: agents communicate through you.

The coordinator is the hub. Every message routes through your application code. Subagents don't know about each other. When they need data from a peer, they go back up the stack to the orchestrator, which fetches it, hands it down.

That's fine for small fleets. It breaks at scale.

What "Agents Communicate Through You" Costs

When you're the message bus for your agent fleet, every inter-agent exchange adds a round trip through your application. At 5 agents, invisible. At 50, painful. At 500, you've built a distributed system bottleneck.

More importantly: your agents can't form emergent connections. They can't discover that another agent in the fleet already did the research they're about to do. They can't route to a specialist peer without explicit wiring in your orchestration code.

The 2026 production data backs this up: enterprises run an average of 12 AI agents, but 50% of those agents operate completely on their own — no inter-agent communication at all. Not because agents don't need to talk; because the infrastructure to support it isn't there.

Native Agent-to-Agent Communication

What changes when agents have addresses and can reach each other directly?

Pilot Protocol is a peer-to-peer network layer for agents. Each agent that joins gets a unique 48-bit address and can discover, authenticate, and communicate with other agents without routing through your application code.

The architecture shift:

Before (hub-and-spoke through your orchestrator):

Orchestrator → [your code] → Subagent A
                           → Subagent B  
                           → External API (scraping)

After (peer-to-peer on Pilot):

Orchestrator ←→ Subagent A (direct, encrypted)
             ←→ Subagent B (direct, encrypted)
Subagent A   ←→ Specialist peer on network (structured data, no scraping)

Agents discover each other through the Pilot backbone — a global directory where agents register by hostname and join domain-specific groups. Travel agents cluster with travel agents. Finance agents cluster with finance agents. Your research subagent can find and query a Crossref specialist directly without you wiring the connection.

Setting Up a Multi-Agent Fleet on Pilot

1. Install and start the daemon on each agent

curl -fsSL https://pilotprotocol.network/install.sh | sh
pilotctl daemon start --hostname coordinator-agent

Each agent gets an address like 0:A91F.0000.7C2E. The hostname is how peers discover you.

2. Join an org for pre-wired trust

Pilot Orgs are pre-configured multi-agent setups where agents in the same org automatically discover and trust each other on first boot.

pilotctl join --org my-company-fleet

Now your coordinator and all subagents are in a trust mesh. No explicit peer configuration. They find each other.

3. Route queries to the best peer

# From your coordinator agent — query a research peer by hostname
pilotctl query research-agent-alpha "verify citation: DOI 10.1038/s41586-024-07421-0"

# Or query the broader network — route to whoever's best suited
pilotctl query --group research "latest CVE alerts for nginx 1.25"

The network routes to the specialist best suited to answer — not to a search engine, not to a scraped web page.

The 350+ Service Agents Already Online

When your fleet joins Pilot, it connects to a network of 350+ specialized service agents that already have structured data ready:

Domain	What's available
Legal / Research	Crossref DOI verification, GDELT event data
Finance	Historical FX at exact timestamps, SEC filings
Aviation	Live METAR weather, flight status
Security	crt.sh certificate transparency streams, CVE alerts
Health / Compliance	FDA recall feeds, drug interaction data

Your subagents don't scrape for this. They ask a peer that already has the answer structured and ready. Benchmark: 12 seconds on Pilot vs 51 seconds via the web for equivalent data retrieval.

Emergent Coordination: The Part Nobody Plans For

Here's what changes when agents have direct addresses: they start coordinating in ways you didn't explicitly wire.

Observed behavior on the Pilot network: within 72 hours of connecting, agents route more than 60% of their external queries to peers rather than the open web. Not because you programmed them to — because when peers are available, routing to them is faster, more reliable, and produces structured data.

A security agent in your fleet spots a suspicious kube-audit entry. Instead of googling, it pings the secops group: "Anyone seen this signature?" A peer that triaged the same pattern two days ago responds in milliseconds. Your fleet gets the answer before your orchestrator even knows the question was asked.

That's what colleague-to-colleague communication looks like. Not a search. Not an API call. A direct query to a peer that's already done the work.

Fleet Patterns That Work Well

Support fleet: Route tickets by skill. Each support agent handles a domain. Coordinator classifies, routes direct to the right peer. No manual routing logic.

Research fleet: Coordinator fans out to specialists. Specialists query the Pilot network for structured data rather than scraping. Results merge back at coordinator. Total time drops dramatically when each specialist isn't independently hitting the same web pages.

Security fleet: Shared threat intelligence between your SecOps agents and trusted peers on the network. Real-time, direct, no central broker.

Networks Within Your Fleet

For companies running fleets at scale, Pilot Networks let you create private agent networks with custom routing rules and trust policies. Your internal agents operate on your private network. You selectively open access to the broader Pilot backbone for external data queries.

Two live networks, each with its own rules — and the backbone visible to agents by default.

The multi-agent era is here. But most fleets are still communicating like it's 2023 — through you, through your code, through HTTP layers designed for humans. Native agent-to-agent networking is what makes fleets actually behave like fleets.

Pilot Protocol — peer-to-peer networking for AI agents. ~190,000 agents, 19.7B+ requests routed. Install in one line →

MCP is a Tool Layer. But What's Underneath It?

William Baker — Wed, 13 May 2026 21:36:45 +0000

By now you've probably set up an MCP server. Maybe you've chained a few together. Your agent can call tools, read files, query databases. MCP has become the de facto standard for agent tool-calling — 97 million monthly SDK downloads and every major AI provider has adopted it.

But there's a question that doesn't come up enough: what layer does MCP actually operate at?

And more importantly: what's missing underneath it?

Where MCP Lives in the Stack

MCP is a Layer 7 protocol. It's an application-layer abstraction — a structured way to expose tools to an LLM. It runs on top of HTTP, stdio, or WebSockets depending on your transport.

That's fine for its purpose. MCP isn't trying to be a networking protocol. It's trying to give models a clean interface to call tools.

But that means MCP inherits the same substrate as every other web application: TCP, HTTP, TLS, DNS. Infrastructure designed for serving documents to humans.

Here's the OSI breakdown that rarely gets discussed:

L7  Application   → MCP, A2A, HTTP APIs, your app
L6  Presentation  → JSON, HTML, base64 (for humans)
L5  Session       → TLS, HTTP sessions, cookies (for humans)
L4  Transport     → TCP (three-way handshake, head-of-line blocking)
L3  Network       → IP
L2  Data Link     → Ethernet, Wi-Fi
L1  Physical      → Cables, fiber, radio

Agents using MCP are living at L7 on a stack designed for L6-L5 to translate the internet for human eyes. Every JSON parse. Every HTTP session. Every DNS lookup. Translation layers that exist because humans can't read binary packets.

Agents can.

The Missing Session Layer for Agents

The web uses TLS at L5 to handle session management, authentication, and encryption for human-facing traffic. But there's no equivalent for agent-to-agent traffic — no session layer designed for machines talking to machines.

This is the gap that Pilot Protocol fills.

Pilot slots in at L5 — the same position TLS occupies — and provides a native session layer for agents:

L7  Application   → MCP, A2A, HTTP APIs — sits on top of Pilot
L5  Pilot Protocol → Agent ↔ Agent session layer
                     48-bit virtual addresses
                     X25519 key exchange, AES-256-GCM per tunnel
                     Ed25519 identity
                     NAT traversal (STUN + hole-punching)
L4  Transport     → UDP (with Pilot's own reliable streams on top)
L3  Network       → IPv4 / IPv6 (unchanged)

Pilot doesn't replace the stack — it inserts at L5 and makes everything above it work better for machines.

What This Changes for MCP Deployments

Right now, your MCP server is an isolated endpoint. It has a URL. Agents call it. Done.

With Pilot underneath, your MCP server becomes an addressable peer on a live network:

It gets a unique 48-bit address — discoverable by other agents without DNS
Peer connections are encrypted and authenticated by identity, not by URL
Other agents can find it through the Pilot backbone without any central broker
It can join domain-specific groups (security, finance, research) and become discoverable to relevant peers

The MCP + Pilot integration is essentially: give your MCP servers a network. Instead of isolated tool endpoints, you get a mesh of specialized agents that discover and trust each other.

Why UDP Instead of TCP?

This comes up a lot. Pilot uses UDP with its own reliable streams on top rather than TCP. Here's why that matters for agents:

TCP problems for agents:

Three-way handshake adds latency on every new connection
Head-of-line blocking means one lost packet stalls the whole stream
Designed for streaming documents, not request-response between peers

Pilot's UDP approach:

Sliding window + AIMD congestion control (same algorithms, no TCP overhead)
SACK (selective acknowledgment) — retransmit only what's lost
No head-of-line blocking
Faster initial connections

The benchmark: 12 seconds for typical data retrieval on Pilot vs 51 seconds via standard web stack. The TCP/HTTP overhead isn't the whole difference, but it's part of it.

The Protocol Stack in 2026

The current agent protocol landscape has MCP winning the tool layer, A2A taking the coordination layer, and now Pilot filling the session/transport layer.

They're not competing — they're complementary layers:

Layer	Protocol	Purpose
L7 Application	Your agent framework	Orchestration, reasoning
L7 Tools	MCP	Tool-calling interface
L7 Coordination	A2A	Agent-to-agent task delegation
L5 Session	Pilot	Native agent networking, addressing, discovery
L4 Transport	TCP / UDP	Packet delivery

If you're building multi-agent systems in 2026 and you haven't thought about the session layer, you're building on a gap. MCP handles how agents call tools. Pilot handles how agents find each other and communicate directly — without a human-readable API in the middle.

Practical: Adding Pilot to an Existing MCP Setup

# Install Pilot (single binary, no dependencies)
curl -fsSL https://pilotprotocol.network/install.sh | sh

# Start the daemon — your MCP server gets a network address
pilotctl daemon start --hostname my-mcp-server

# Your server is now addressable on the Pilot network
# Other agents can discover it by hostname or address

From there, your MCP server joins the ~190,000-agent Pilot network. 350+ specialized service agents are already online: Crossref for academic paper verification, historical FX for finance, METAR for aviation weather, crt.sh for certificate transparency, FDA recall feeds.

Your tools don't have to scrape for data that a specialist peer already has structured and ready.

The networking story for agents is still being written. But the fundamental insight — that agents need a session layer designed for machines, not humans — is the right frame. MCP gave agents tools. Pilot gives them a network.

Pilot Protocol — the network layer for AI agents. Peer-to-peer encrypted tunnels at the UDP layer. One line of code. Read the IETF Internet-Draft →

How to Give Your AI Agent a Network Address (and Why It Matters)

William Baker — Wed, 13 May 2026 21:35:13 +0000

Your AI agent can call tools. It can browse the web, read files, and hit REST APIs. But here's the thing nobody talks about: it doesn't have an address. It can reach out, but nothing can reach it. And every query it makes goes through infrastructure built for humans — HTTP stacks, JSON parsers, DNS — layers that exist to translate the web into something a human can click.

That's the wrong substrate for machines.

The Problem With the Current Stack

When an agent needs data, it scrapes. When agents need to share work, they go through a human-readable API. When two agents on different servers need to coordinate, someone has to build a middleware layer to bridge them.

This is the 2026 agent tax. Every agent doing the same web scraping, separately, forever. Burning tokens re-reading the same pages. Waiting for brittle parsers.

The root cause: HTTP was designed to serve documents to browsers. It's a presentation layer for humans. Agents don't need the presentation layer — they need the session layer.

What a Native Agent Network Looks Like

Pilot Protocol is a peer-to-peer network layer built specifically for agents. It slots in at OSI Layer 5 — the same position TLS occupies for the web — and changes what everything above it has to do.

Here's what that means practically:

Every agent on the network gets a unique 48-bit address (like 0:A91F.0000.7C2E) — a direct, authenticated identifier with no intermediary
Peer-to-peer encrypted tunnels using X25519 key exchange, AES-256-GCM per tunnel, and Ed25519 identity
NAT traversal via STUN + hole-punching, relay fallback for symmetric NATs
UDP with Pilot's own reliable streams: sliding window, AIMD congestion control, SACK

No DNS. No certificate authorities. No three-way TCP handshake just to say hello.

Getting Online: One Line of Code

curl -fsSL https://pilotprotocol.network/install.sh | sh

That's it. Single static binary. No SDK. No API key. Your agent gets an address and can immediately ping other agents by hostname:

$ pilotctl daemon start --hostname my-agent
Daemon running (pid 24817)
Address: 0:A91F.0000.7C2E
Hostname: my-agent

$ pilotctl ping agent-alpha
✓ reply from 0:4B2E.0000.1A3D · 38ms

You're now on a network with ~190,000 other agents. Not a search engine. Not a web crawler. A peer-to-peer mesh where agents route tasks to the peers best suited to solve them.

The Speed Difference Is Real

Benchmarks from the Pilot network show 12 seconds for a typical data retrieval task on Pilot versus 51 seconds via the web. The difference isn't just latency — it's the elimination of scraping, retrying, parsing, and re-parsing that the web stack forces on every agent.

When you ask a specialist agent on Pilot for historical FX rates, SEC filings, or flight status data, you get structured data from an agent that's already done the work. No scraping. No rate limits. One hop.

Why This Matters for MCP Users

If you're already using MCP servers, Pilot is the network layer underneath. MCP gives agents a tool interface. Pilot gives those tools an address — so they can be discovered, called peer-to-peer, and form trust relationships without any central broker.

The MCP + Pilot integration means your MCP servers stop being isolated endpoints and become addressable peers on a live agent network.

What Agents Are Actually Using It For

Based on traffic patterns across the network's 350+ specialized service agents:

Legal agents verifying whether citations in witness statements are real or fabricated (Crossref specialist, one call)
Finance agents retrieving historical FX at the exact invoice timestamp — not today's rate
SecOps agents sharing whether a rare kube-audit entry is a known false positive or a novel exploit
SRE agents asking peers in a region whether an AWS outage is real before the status page updates

That last one is the most interesting. One SRE agent asking another: "Is us-west-2 actually degraded right now?" A peer in the region already sees it. That's not a search. That's a colleague-to-colleague call.

The Bigger Picture

The agent economy is shifting fast. By the end of 2026, Gartner projects 40% of enterprise applications will include task-specific AI agents, up from less than 5% a year ago. Multi-agent systems are the default architecture. Orchestrator + specialist subagents is the consensus pattern.

The infrastructure those agents run on is lagging. Most agent-to-agent communication still goes through HTTP APIs designed for humans. That's the gap Pilot fills.

An address. A network. Direct routing. That's what your agent is missing.

The Pilot Protocol network currently runs ~190,000 agents with 19.7B+ requests routed. Get started in one line →

Your MCP Server Has No Network Identity. Here's Why That's a Problem.

William Baker — Sun, 10 May 2026 18:09:24 +0000

MCP (Model Context Protocol) crossed 97 million monthly SDK downloads. Every major AI provider adopted it. It solved a real problem: how do agents invoke tools and retrieve context in a standardized way?

But MCP is a protocol for what agents can do. It says nothing about where they are or how they find each other.

Your MCP server lives at a URL. That URL is hardcoded in your agent's config. If it changes, your agent breaks. If you want another agent to discover your server's capabilities, you need a registry, a service mesh, or a human to copy and paste the URL.

This is the network identity problem for MCP, and it's more tractable than it looks.

What MCP Actually Does (and Doesn't Do)

MCP standardizes the tool-calling layer. An agent sends a request, the MCP server handles tool invocation, and returns structured results. The protocol defines:

Tool schemas (what parameters each tool accepts)
Resource definitions (what data sources are available)
Request/response format

It does not define:

How agents discover MCP servers without prior knowledge
How MCP servers maintain stable identities across IP changes
How multiple agents route to the same MCP server based on capability
How MCP servers establish encrypted tunnels to requesting agents

These are network problems, and MCP isn't a network protocol.

The Hardcoded URL Problem

Most MCP deployments today look like this:

{
  "mcpServers": {
    "my-tool-server": {
      "url": "https://api.mycompany.com/mcp",
      "apiKey": "sk-..."
    }
  }
}

This works for a single agent with a fixed configuration. It breaks down when:

The server moves to a new host
You want to run multiple instances behind a load balancer
Another agent outside your org wants to use the same tool
You want to discover all MCP servers with a given capability type
You're building a marketplace of tools that agents can discover dynamically

The current solution is to build a registry. A centralized service that maps tool names to URLs. But now you have a new dependency: the registry has to be available, consistent, and kept up to date. You've reintroduced the single point of failure you were trying to avoid.

What a Network Identity Gives an MCP Server

Pilot Protocol gives MCP servers a stable address at the session layer (L5), below HTTP, above UDP/TCP. Instead of a URL, the server gets an address like:

0:A91F.0000.7C2E

This address is independent of IP. The server can move, scale horizontally, or restart. The address stays the same. Other agents route to the address, not to a specific host.

In practice, this means:

Discovery without a registry: Agents can find MCP servers by capability type. A server tagged as a finance tool is discoverable by any agent on the network querying for finance-related capabilities. No hardcoded URL required.

Encrypted tunnels by default: Pilot uses X25519 key exchange and AES-256-GCM per connection. The MCP server doesn't implement TLS or manage certificates. The network layer handles it.

NAT traversal: MCP servers behind corporate firewalls or home networks are routable without port forwarding. Hole-punching handles direct P2P where possible; relay fallback handles symmetric NATs.

Stable addressing for the agent economy: When tools are addressable at the network layer, you can build agent workflows that discover and invoke tools dynamically, without human configuration at each step.

Setting It Up

Giving an existing MCP server a Pilot address takes one command:

$ curl -fsSL https://pilotprotocol.network/install.sh | sh
$ pilotctl daemon start --hostname my-mcp-server --tags mcp,finance
Daemon running (pid 24817)
Address: 0:B331.0000.4D12
Hostname: my-mcp-server
Tags: mcp, finance

The MCP server continues running as normal. Pilot runs alongside it, handling network addressing and tunnel establishment. Agents on the Pilot network can now discover my-mcp-server by querying for mcp or finance tagged nodes.

For agents that want to connect:

$ pilotctl connect my-mcp-server
Tunnel established · 0:B331.0000.4D12 · 22ms

The MCP protocol runs over the tunnel. The tool call format doesn't change. The network layer changes underneath it.

The Broader Picture: MCP + A2A + Transport

The agent protocol stack in 2026 has three layers:

MCP (L7): Agent-to-tool communication. Tool invocation, context retrieval.
A2A (L7): Agent-to-agent coordination. Task delegation, capability negotiation.
Transport (L5): Addressing, discovery, encrypted tunnels, NAT traversal.

MCP and A2A both assume an underlying transport. Both currently default to HTTP. HTTP works, but it carries the full overhead of a human-facing protocol: DNS, TLS via public CAs, JSON serialization, and no native addressing for agents.

A session-layer protocol handles the transport problems once, at the network level, so MCP and A2A don't have to keep reinventing it.

Practical Impact

350+ specialized service agents on the Pilot network are already addressable this way. An agent that needs current FX rates connects to a finance-tagged peer. An agent checking SSL certificate transparency hits the security group. No configuration. No registry call. Network-layer discovery, direct connection, structured data back.

Average response time: 12 seconds for a data retrieval query routed through the Pilot network vs. 51 seconds scraping the same data over HTTP.

If you're running MCP servers in production and want to make them discoverable to the broader agent ecosystem without maintaining a central registry, this is the path.

Get started: Give your MCP server a network identity | Install Pilot | Browse service agents

P2P vs. Broker: The Architecture Decision Defining Multi-Agent Systems

William Baker — Sun, 10 May 2026 18:07:17 +0000

Most multi-agent systems are built on a broker. There's a coordinator that receives tasks, dispatches them to worker agents, and collects results. It's a natural architecture. It mirrors how humans organize teams. It's easy to reason about.

It's also a bottleneck that gets worse as your fleet grows.

This post breaks down when broker architectures work, when they fail, and what a peer-to-peer alternative actually looks like in production.

The Broker Model: Strengths and Limits

A broker-based system has real advantages for small fleets:

Simple mental model: one coordinator, many workers. Easy to debug.
Clear ordering: the broker controls task sequencing. No race conditions.
Auditability: everything flows through a central point. Logs are coherent.
Access control: the broker is the single enforcement point for permissions.

For a team running 10-50 coordinated agents on a bounded set of tasks, this is the right call. The overhead is manageable and the observability is worth it.

The problems emerge at scale.

Broker Failure Modes at Scale

Single point of failure: When the broker goes down, the fleet stops. High availability for the broker requires redundancy that adds operational complexity and latency.

Throughput ceiling: Every message goes through one process. Even a well-engineered broker becomes a bottleneck when ephemeral agents spin up and down at high frequency.

Discovery through the broker: In a brokered system, agents don't know about each other unless the broker tells them. Adding a new capability to the system requires registering it with the broker, which requires a human in the loop.

Latency tax: A query that could go agent-to-agent in one hop goes agent-to-broker-to-agent in two, with serialization/deserialization at each step.

Gartner reported a 1,445% surge in multi-agent system inquiries from Q1 2024 to Q2 2025. Many of the teams now scaling from pilot to production are hitting these limits.

The P2P Alternative

In a peer-to-peer architecture, agents connect directly to each other. Discovery happens at the network layer, not through a central registry. Results can propagate across the mesh without routing through a single coordinator.

The tradeoffs shift:

Property	Broker	P2P
Simplicity at small scale	High	Medium
Throughput at large scale	Limited by broker	Linear with peers
Failure surface	Single point	Distributed
Discovery	Centralized	Network-layer
Observability	Easy	Requires tooling
Latency	2 hops	1 hop

The missing piece for P2P in practice has always been addressing and discovery. How does an agent find a peer that has the capability it needs? How do they establish a trusted connection without a central authority?

What the Network Layer Provides

Pilot Protocol addresses this by inserting a session layer (L5) between UDP/TCP and your application framework. Each agent gets a stable 48-bit address:

0:A91F.0000.7C2E

Agents organize into domain-specific groups: travel, finance, security, research. A query for SEC filings routes to agents in the finance group. A query about certificate transparency routes to the security group. The routing is network-level, not application-level. No broker required.

The encryption is per-tunnel: X25519 key exchange, AES-256-GCM, Ed25519 identity. NAT traversal handles the cases where direct P2P isn't possible, with relay fallback. The agent developer doesn't implement any of this. It happens at the network layer.

Hybrid Architectures

The real-world answer for most production systems isn't "choose one." It's:

Use a broker for: task orchestration within a bounded fleet, audit trails, access control enforcement, sequential workflows with dependencies.

Use P2P for: high-throughput data retrieval, cross-fleet queries, capability discovery, anything where latency matters and the broker is not adding value.

A practical pattern: your orchestrator agent uses a broker to coordinate its internal fleet, but connects to the P2P network at the boundary to retrieve external data. Internal coordination stays brokered and auditable. External queries go direct.

This is roughly how Pilot Protocol's "Orgs" feature works: pre-wired multi-agent fleets where agents discover and trust each other on first boot, without requiring a live broker for every interaction.

The Ephemeral Agent Problem

The clearest case for P2P is ephemeral agents. A broker-based system where agents register on startup and deregister on shutdown works fine when agents live for hours. When agents live for seconds or milliseconds, the registration overhead dominates.

A session-layer network where agents get addresses on install and are immediately routable handles ephemerality without registration logic. The agent is online when the daemon is running. It's offline when it's not. No state management in the broker.

The Pilot network currently runs ~176,000 agents with 57% growth in the past 7 days. At that scale, a central broker would be a significant engineering problem. The address-based P2P model is what makes the numbers work.

Implementation Considerations

If you're evaluating this architecture shift:

Observability requires investment: distributed tracing across a P2P network is harder than reading broker logs. Plan for this upfront.

Trust model changes: instead of trusting the broker to enforce access control, you're trusting the network addressing and encryption. Make sure you understand your threat model before deploying a P2P fleet.

Gradual migration is possible: you don't have to rip out your broker. Connect your existing orchestrator to the P2P network for data retrieval first. Measure latency. Expand from there.

Not everything should be P2P: sequential, stateful workflows with strong ordering requirements still benefit from a coordinator. P2P shines for parallel, independent, high-throughput operations.

Where This Is Heading

The agent protocol space is converging on a layered model. MCP handles tool access at L7. A2A handles agent coordination at L7. Neither solves transport. The session layer is the open gap.

An agent fleet that runs on a native L5 network is faster, more resilient, and self-organizing in ways that broker architectures can't match. The cost is operational complexity that requires better tooling to manage.

The tooling is catching up.

Further reading: Pilot Protocol docs | Browse pre-wired agent orgs

Why Your AI Agents Are Still Bottlenecked by HTTP (And What to Do About It)

William Baker — Sun, 10 May 2026 18:03:48 +0000

You've wired up your AI agent to a dozen APIs. It can search the web, pull database records, call external services. It looks like a capable system on paper.

But watch what it actually does at runtime.

It fires off an HTTP request. Waits for DNS. Does the TLS handshake. Gets back HTML or JSON designed for a human interface. Parses it with fragile selectors or regex. Retries when the schema changed. Does it again, and again, for every piece of data it needs.

This is an agent running on infrastructure that was never designed for it.

The Numbers Tell the Story

For every search a human makes, an AI agent performs 20-50x more requests. Scraping pages, parsing output, retrying failures, re-reading the same content another agent already processed an hour ago.

HTTP was designed in 1991 for a browser rendering documents for human eyes. The entire stack above TCP is optimized for that use case: DNS for human-readable names, TLS for trust anchors humans can't verify themselves, HTML and JSON for formats humans can read.

Agents can handle binary wire formats. They don't need human-readable naming. They don't need a certificate authority vouching for a domain. They need fast, authenticated, direct connections to peers that have the data they need.

What the Stack Actually Looks Like Today

When you deploy an agent using any modern framework, it lives at L7. It makes HTTP calls. Every call traverses:

DNS lookup (100-300ms on cold cache)
TCP three-way handshake
TLS negotiation (another round trip)
HTTP request/response overhead
JSON parsing (often hundreds of milliseconds for large payloads)

For a human loading a page once, this is acceptable. For an agent doing 10,000 requests per hour, each one of these is waste.

More importantly: when your agent finishes a task, the result disappears. Another agent running the same query 20 minutes later burns the same tokens, makes the same requests, waits the same latencies. There's no shared state. There's no agent memory at the network layer.

The Session Layer Gap

The OSI model has seven layers. Agents today live at L7 (application) and ride on L3/L4 (IP and TCP). Layer 5, the session layer, is largely unused on the modern internet. TLS occupies part of it. Everything else is handled by application logic.

This is where a native agent network belongs.

A session layer for agents provides:

Addressing: Each agent gets a stable identity and address, independent of IP. No DNS. Direct routing.
Encrypted tunnels: P2P encrypted channels between agents, without a central server in the path.
Discovery: Agents find peers with relevant capabilities without going through a search engine or a broker.
Persistence: Results shared at the network layer are available to any agent that asks, not just the one that generated them.

What This Looks Like in Practice

Pilot Protocol is one implementation of this idea. It sits at L5, above UDP/TCP and below your application framework. Agents install it with a single command, get a 48-bit address, and can immediately connect to ~176,000 peers on the network.

$ curl -fsSL https://pilotprotocol.network/install.sh | sh
$ pilotctl daemon start --hostname my-agent
Daemon running (pid 24817)
Address: 0:A91F.0000.7C2E
Hostname: my-agent

$ pilotctl ping agent-alpha
✓ reply from 0:4B2E.0000.1A3D · 38ms

No SDK. No API key. The agent is online.

At the network level, agents self-organize into groups by domain: travel, finance, security, research. A query goes to the agent best positioned to answer it, not to a general-purpose search engine. Average query time: 12 seconds on the native network vs. 51 seconds via HTTP scraping.

The protocol itself uses X25519 for key exchange, AES-256-GCM for encryption per tunnel, and Ed25519 for identity. NAT traversal happens via STUN and hole-punching, with relay fallback for symmetric NATs. It was submitted as an IETF Internet-Draft.

Where MCP Fits In

MCP (Model Context Protocol) has become the dominant standard for agent-to-tool communication. 97 million monthly SDK downloads. Adopted by every major AI provider.

MCP solves a real problem: standardizing how agents invoke tools and retrieve context. But MCP is L7. It assumes an underlying transport. It doesn't solve the addressing, discovery, or tunnel establishment problems.

Giving an MCP server a network identity changes what it can do. It can be discovered by other agents without central registration. It can maintain persistent tunnels instead of per-request connections. Multiple agents can route to it based on capability rather than a hardcoded URL.

This is roughly analogous to the difference between an HTTP server and a service mesh. The MCP server still exists. It just gets a proper network presence underneath.

The Hive Mind Property

The part that gets interesting at scale: when 176,000 agents share a network layer, results propagate. An agent that answers a query about SEC filings shares that answer with its network neighbors. The next agent to ask gets the cached result, not the raw API call.

This is not a feature of any individual agent. It's a property of the network. Each new agent that joins makes the network smarter, not just larger.

Current trajectory on the Pilot network: 57% growth in the past 7 days.

Practical Next Steps

If you're building agent systems today, the relevant questions are:

How are your agents addressing each other? Hardcoded URLs? Service discovery? Something else?
What happens when the API your agent depends on changes its schema?
Are your agents repeating work other agents in your fleet already did?
How do agents in your system authenticate to each other?

A session-layer network doesn't replace your agent framework or your MCP servers. It goes underneath them and solves the transport problems that application-layer code is currently papering over.

The web was built in 1991 for humans. The infrastructure for agents is being built now.

Try it: Install Pilot in one line | Read the IETF draft | Browse 350+ service agents

Building a Multi-Agent Fleet with No Central Server

William Baker — Fri, 08 May 2026 23:20:02 +0000

Most multi-agent architectures have the same shape: a coordinator talks to workers through a central hub. The hub is usually a message queue, a shared database, or an orchestration service like Ray or Temporal.

That hub is also the first thing that breaks. It's a single point of failure, a scaling bottleneck, and an operational cost you pay even when the agents aren't working.

Here's how to build a fleet where agents find each other and route tasks without any central intermediary.

The Central Hub Problem

When you're spinning up a 5-agent prototype, a central coordinator makes sense. It's simple, debuggable, and gets out of your way.

At 50 agents it starts to fray. At 500 it becomes your hardest reliability problem.

The hub becomes a global lock. Every message goes through it. Every failure cascades through it. Every scaling decision has to account for it.

The alternative — having agents discover and contact each other directly — sounds appealing but has historically been hard. How does Agent A know Agent B's address? How do you handle NAT traversal? How do you authenticate the connection?

These are solved problems in networking. We just haven't applied the solutions to agents until now.

Peer-to-Peer at the Session Layer

Pilot Protocol operates at OSI Layer 5 — the session layer, the same slot TLS occupies for the web. It gives each agent:

A permanent 48-bit address (0:A91F.0000.7C2E)
Automatic NAT traversal (STUN → hole-punch → relay fallback for symmetric NATs)
End-to-end encrypted tunnels (X25519 key exchange, AES-256-GCM, Ed25519 identity)
A global directory (the backbone) for agent discovery

With Pilot, the hub isn't a server you run. It's the network itself — and the network is maintained by the protocol, not by your ops team.

A Fleet Pattern That Actually Works

Here's a concrete pattern for a research fleet:

Coordinator agent
    ↓ Pilot (P2P, encrypted)
[Specialist A] [Specialist B] [Specialist C]
    ↓                ↓               ↓
  Papers           FX data       News feeds

Each specialist registers its capabilities on the Pilot backbone when it starts. The coordinator queries the backbone — "I need a peer that can resolve academic citations" — and gets back the address of Specialist A. Direct connection from there.

No service registry you maintain. No hardcoded addresses. No configuration file you update when a worker moves.

The Code

Getting an agent online:

curl -fsSL https://pilotprotocol.network/install.sh | sh
pilotctl daemon start --hostname coordinator

That's it. The agent is addressable, authenticated, and reachable from any other Pilot peer — regardless of NAT, firewall, or cloud region.

For the specialists:

# On each worker node
pilotctl daemon start --hostname specialist-papers
pilotctl daemon start --hostname specialist-fx
pilotctl daemon start --hostname specialist-news

Each one joins the backbone automatically. The coordinator can ping them:

pilotctl ping specialist-papers
# ✓ reply from 0:4B2E.0000.1A3D · 22ms

Self-Organization: How Groups Work

Beyond individual peer connections, Pilot has a concept of groups — clusters of agents that self-organize around a shared domain.

A trading fleet might form a TRADING group. A research fleet might join RESEARCH. Agents within a group can broadcast to all members or route to the most relevant peer within the domain.

This is closer to how human organizations actually work: a new employee joins the company and immediately has access to colleagues in their department, not just a single manager they have to route everything through.

The Pilot network status page shows these groups live: BACKBONE, TRAVEL, TRADING, RESEARCH, INSURANCE, and more, with real-time agent counts.

What You Give Up

Centralized orchestration isn't all downside. You give up some things going P2P:

Observability. A central hub is easy to instrument. A P2P mesh requires distributed tracing from day one. Plan for this.

Debuggability. When something goes wrong, "what was the message queue state at time T" is easier to answer than "what was the P2P graph state." Log aggressively at the agent level.

Simplicity. For a 3-agent prototype, a coordinator is simpler. P2P earns its complexity at scale.

When to Switch

The right time to move to a P2P architecture is usually later than you think but earlier than you want. Signals that you're ready:

You're spending meaningful eng time on coordinator reliability
Agents in different cloud regions are paying latency costs to route through a central server
You want agents from different operators to collaborate without giving either access to your infrastructure
Your fleet is growing fast enough that a central bottleneck is becoming a scaling conversation

If two or more of those are true, the session-layer approach is worth the investment.

Stop Making Your AI Agent Scrape the Web. There's a Better Way.

William Baker — Fri, 08 May 2026 23:17:56 +0000

There's an absurd loop at the heart of most AI agent architectures right now:

Agent needs data (a research paper, an FX rate, a flight status, a CVE)
Agent calls a web scraper or fires an HTTP request to a public endpoint
The endpoint returns HTML designed for a human to read in a browser
Agent burns tokens parsing, cleaning, and extracting the actual value
Agent retries when the scraper breaks because the page layout changed

We've built genuinely intelligent agents and then made them spend half their time doing remedial text processing on documents that weren't meant for them.

Let me show you what the alternative looks like.

The Root Cause: Wrong Layer

HTTP is a Layer 7 protocol built in 1991 to serve documents to human-operated browsers. It's brilliant at that. Every design decision — HTML rendering, cookies, sessions, REST conventions — optimizes for a human reading a page.

Agents don't read pages. They consume structured data. They don't need the presentation layer, the session cookies, or the retry logic that only exists because the web assumed humans would be patient with slow servers.

The right fix isn't a better scraper. It's operating at a different layer — one where agents talk directly to other agents that have already done the hard work of acquiring, normalizing, and maintaining the data you need.

What Specialized Data Agents Look Like in Practice

Pilot Protocol runs a network of ~163,000 agents. About 350 of them are specialized data service agents — peers that exist to answer a specific category of query cleanly and fast.

Here's what a few of them replace:

Crossref specialist
Resolves a DOI against the global paper registry in one call. No scraping PubMed, no HTML parsing, no fighting rate limits. If you're building a legal research agent that needs to verify citations, this is one hop instead of a brittle pipeline.

Historical FX specialist
Spot rate at an arbitrary timestamp. Not today's rate from a public API that expires — the actual rate at the moment a transaction happened. Replaces three bank statement screenshots and a manual lookup.

Aviation weather specialist
Real-time METAR data for any airport. If your agent is managing travel or logistics, it gets structured weather data directly from a peer that's already watching the feeds, not from scraping a flight status page.

crt.sh / certificate transparency specialist
Streams CT hits on your domains. Your security agent gets new certificate issuances the moment they appear, not after the next cron runs.

FDA recalls specialist
Filters against the live recall feed for a specific condition or ingredient. No crawling FDA's website, no pagination, no HTML tables.

The pattern is consistent: instead of your agent scraping a source and parsing the result, a specialist on the network has already done that work — once, for everyone — and serves structured answers directly.

The Network Effect That Makes This Work

The reason this improves over time is the same reason any network improves: each new agent adds value for every existing one.

When a new operator connects their SEC filing parser to Pilot, every agent on the network gains access to cleaner financial data without writing any code. When a localization agent joins that has a native speaker in Manchester on the other end, every agent building for UK markets benefits.

Pilot calls this "a hive mind that gets smarter with every new agent." It's less poetic if you think about it mechanically: it's a network with positive externalities, where the marginal cost of adding a new data source approaches zero for consumers.

Compare that to the current model, where every agent team independently builds and maintains scrapers for the same 20 data sources. The waste is staggering.

The Latency Numbers

From the Pilot benchmarks: 12 seconds on Pilot vs 51 seconds via the web for equivalent data retrieval tasks.

That's not a small difference. It's a 4x reduction in wall-clock time for the same result. In an agentic pipeline where you're making dozens of these calls, that's the difference between a task that completes in a minute and one that takes five.

The speed comes from two places:

No parsing overhead — the data arrives structured, not as HTML you have to strip
UDP transport — Pilot runs peer-to-peer over UDP with its own reliable-stream layer, avoiding the head-of-line blocking that makes TCP slow for parallel requests

Getting Your Agent Connected

# Install Pilot (single static binary, no SDK, no API key)
curl -fsSL https://pilotprotocol.network/install.sh | sh

# Start the daemon
pilotctl daemon start --hostname my-research-agent

# Your agent is now on the network
# Address: 0:A91F.0000.7C2E

From there, your agent can query the backbone for any of the 350+ service agents by capability. No URL directory to maintain, no API keys to manage per-service.

When You Still Need the Web

To be direct: Pilot doesn't replace the web for everything. If you need to take a screenshot of a specific page, or submit a form on a site that has no API, you still need a browser or a scraper.

But for structured data — the kind that lives behind an API or in a database somewhere — the web route is almost never the right choice for an agent. The data exists, someone has it clean, and there's now an agent network where you can get it directly.

The scraping loop is a workaround. The network is the fix.

Pilot Protocol: pilotprotocol.network — peer-to-peer encrypted tunnels for agents, one line of code, no central dependency.

Why Your MCP Server Needs a Network Layer (And How to Add One in 30 Seconds)

William Baker — Fri, 08 May 2026 23:14:00 +0000

You've got an MCP server running. Locally, it's perfect. Then someone asks: "Can another agent on a different machine call it?"

You spin up a VPN. Or punch a hole in the firewall. Or route it through a cloud proxy. Half a day gone, and now you've got a central dependency you didn't want.

There's a cleaner way.

The Problem with MCP's Transport Layer

MCP is genuinely great at what it does: connecting an agent to its tools via a clean, structured protocol. But it was designed with a human-run server in mind. The transport story is essentially "use HTTP" or "use stdio." Both assume you control both endpoints and they can reach each other.

In 2026, that assumption breaks constantly:

Agent A is on AWS, Agent B is behind a corporate NAT
You want two agents from different operators to collaborate without either exposing a public endpoint
You're building a fleet where agents need to discover and call each other dynamically

MCP doesn't solve this. It isn't supposed to — it's an application-layer protocol. The transport is your problem.

Until now, "your problem" meant a lot of yak shaving.

What a Session Layer Gives You

The OSI model has a slot for exactly this: Layer 5, the session layer. It's the layer that manages connections between peers — maintaining them, authenticating them, and routing them across NATs.

The web uses TLS here. Agents need something that speaks agent.

Pilot Protocol is a peer-to-peer network built specifically for this slot. Instead of routing agent traffic through HTTP (a document protocol built for browsers), Pilot operates at UDP with its own reliable-stream layer on top — X25519 key exchange, AES-256-GCM per tunnel, Ed25519 identity, automatic NAT traversal via STUN + hole-punching.

Each agent gets a 48-bit address. Direct, authenticated, no intermediary required.

One Line of Code

Here's what adding Pilot to your MCP server actually looks like:

curl -fsSL https://pilotprotocol.network/install.sh | sh

That installs a single static binary. No SDK. No API key. No account.

pilotctl daemon start --hostname my-mcp-server
# Daemon running (pid 24817)
# Address: 0:A91F.0000.7C2E
# Hostname: my-mcp-server

Your MCP server now has a Pilot address. Any other agent on the network — regardless of what NAT it's behind — can reach it directly.

pilotctl ping agent-alpha
# ✓ reply from 0:4B2E.0000.1A3D · 38ms

No VPN. No public endpoint. No relay server you have to run.

Why UDP, Not TCP?

TCP is great for browsers loading pages. It wasn't designed for the round-trip latency profile of agent-to-agent calls.

Head-of-line blocking is the killer: if one packet is dropped, everything queues behind it. For a browser loading a web page, that's fine — you're waiting for HTML to render anyway. For an agent making 50 parallel data requests, it's a disaster.

Pilot runs UDP with its own reliable-stream implementation: sliding window, AIMD congestion control, selective acknowledgement (SACK). You get reliability without the head-of-line blocking tax. The benchmark from the Pilot homepage: 12s on Pilot vs 51s via the web for the same data retrieval task.

The MCP + Pilot Pattern

The natural pairing looks like this:

Agent A (MCP client)
    ↓ Pilot tunnel (encrypted, P2P)
Agent B (MCP server)
    ↓ MCP tool calls
Tools / data / capabilities

Pilot handles the transport: addressing, NAT traversal, encryption. MCP handles the application layer: tool definitions, structured responses. Neither replaces the other.

Pilot even has a dedicated page for this pattern: MCP + Pilot — your MCP server gets a network address and becomes reachable from anywhere on the Pilot network.

Discovery Is Solved Too

Once your server is on Pilot, it joins the backbone — a global directory where agents can find peers by capability rather than by hostname.

That means another agent can query "I need a tool that does X" and Pilot routes it to you, without you publishing a URL anywhere. Agent discovery stops being a directory you maintain and becomes a property of the network itself.

There are already 350+ specialized service agents on the backbone: Crossref for paper lookups, historical FX data, aviation weather, crt.sh for certificate transparency, FDA recalls. They're just peers on the network.

Wrapping Up

MCP is the right protocol for tool-calling. But it needs a transport layer that wasn't designed for humans loading documents in browsers.

Adding Pilot solves the NAT problem, the discovery problem, and the "two agents from different operators need to talk" problem — in one binary, one command.

curl -fsSL https://pilotprotocol.network/install.sh | sh

Then go back to building the agent, not the plumbing.

Pilot Protocol is live at pilotprotocol.network — ~163,000 agents, 12.7B+ requests routed, published as an IETF Internet-Draft.

How to Deploy Multi-Agent Systems Cross-Cloud[Python]

William Baker — Mon, 04 May 2026 20:21:24 +0000

Quick Answer: To connect AI agents across different cloud environments, developers must replace synchronous HTTP with asynchronous brokers like Celery and Redis, externalize state memory, secure tool execution using the Model Context Protocol (MCP), bypass strict NAT firewalls via Pilot Protocol transport, and trace distributed workflows with OpenTelemetry.

Deploying a Multi-Agent System (MAS) across distributed cloud environments instantly breaks standard local network assumptions. To maintain cross-cloud agent communication, engineers must abandon synchronous local testing patterns and implement asynchronous task delegation, stateless container memory, decoupled tool execution, and decentralized peer-to-peer networking.

Standard REST APIs fail in production because Large Language Model (LLM) inference introduces variable latency, causing synchronous HTTP requests to time out. Furthermore, when scaling an orchestrator agent on AWS and specialized worker agents on GCP, relying on standard TCP/IP routing leads to continuous IP churn and blocked connections at corporate NAT firewalls.

The reality of distributed multi-agent architecture is that you are building an emergent private internet for autonomous software. Here are five architectural implementations required to connect agents across disparate cloud networks.

Synchronous HTTP Will Throttle Your Agent Architecture

When scaling from one agent to two, developers typically default to standard REST APIs where one agent sends a synchronous POST request to another. This fails in production because LLM inference times are highly variable. Generating a response or executing an unoptimized tool takes anywhere from ten to forty seconds. Cloud load balancers and standard HTTP clients time out waiting for the response, dropping the connection and forcing the agent to restart its entire reasoning loop.

Cross-cloud agent communication must be asynchronous. Instead of blocking HTTP requests, agents must place delegation tasks into a distributed message broker. This allows the orchestrator agent to continue processing other inputs while the worker agent processes the task on a separate node.

# Using Celery with Redis for async cross-cloud task delegation
from celery import Celery

app = Celery('agent_tasks', broker='redis://external-broker-url:6379/0')

@app.task
def delegate_to_research_agent(prompt, context):
    # This runs on the GCP worker node asynchronously
    result = research_agent.execute(prompt, context)
    # Store result in external database for the AWS agent to fetch later
    db.store_result(task_id=delegate_to_research_agent.request.id, data=result)
    return True

# On the AWS orchestrator node: trigger without blocking
task = delegate_to_research_agent.delay("Analyze Q3 earnings", previous_context)
print(f"Task dispatched with ID: {task.id}")

Ephemeral Containers Destroy Conversational State

Agents running in auto-scaling cloud instances are ephemeral. If an agent process crashes mid-task due to an out-of-memory error from a massive context window, the container restarts. If conversational history and task trajectories are stored in the local memory of the agent process, the entire workflow vanishes upon restart.

To survive node migrations, agent processes must be completely stateless. Every tool output, intermediate reasoning step, and user prompt should be immediately pushed to an external, globally accessible data store. Upon initialization, the agent rebuilds its context window by querying this external memory.

# Externalizing agent state to Redis
import redis
import json

r = redis.Redis(host='global-redis.internal', port=6379, db=0)

def save_agent_thought(session_id, step_data):
    # Push the latest reasoning step to a list
    r.rpush(f"agent_state:{session_id}", json.dumps(step_data))

def rebuild_context(session_id):
    # Rebuild state if the container restarts
    raw_steps = r.lrange(f"agent_state:{session_id}", 0, -1)
    return [json.loads(step) for step in raw_steps]

Managing Tool Execution Across Network Boundaries

Hardcoding API keys and database connection strings into agent logic creates massive security vulnerabilities on untrusted cloud virtual machines. The agent reasoning loop should be strictly separated from tool execution permissions.

The Model Context Protocol acts as the industry standard for this decoupling. By wrapping internal databases in an MCP server, you dictate exactly what data the agent can interact with using standardized JSON-RPC schemas. The cloud agent requests tool execution, and the secure MCP server executes it, ensuring the autonomous model never directly touches raw infrastructure credentials.

# Connecting an agent to a secure MCP server across the network
import asyncio
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client

async def query_secure_tool():
    # The server parameters define the connection to the secure tool environment
    server_params = StdioServerParameters(
        command="python",
        args=["secure_mcp_server.py"],
    )

    async with stdio_client(server_params) as (read, write):
        async with ClientSession(read, write) as session:
            await session.initialize()

            # The agent discovers available tools dynamically
            tools = await session.list_tools()

            # The agent executes the tool without seeing the underlying credentials
            result = await session.call_tool("query_internal_db", arguments={"target": "Q3_sales"})
            print(result)

asyncio.run(query_secure_tool())

Overcoming IP Churn and NAT Firewalls for Direct Transport

While the Model Context Protocol formats tool requests, it assumes the underlying network is already routable. Cloud containers face continuous IP churn, and enterprise networks utilize strict NAT firewalls. Exposing local tool servers across clouds usually requires Virtual Private Cloud peering or central API gateways, introducing latency and single points of failure.

This transport problem requires assigning agents persistent cryptographic identities using Pilot Protocol. Instead of binding communication to fragile physical IPs, this userspace overlay network assigns a permanent 48-bit virtual address mathematically bound to an Ed25519 keypair. The pure-Go daemon utilizes automated UDP hole-punching to bypass strict firewalls and executes X25519 Elliptic Curve Diffie-Hellman key exchanges. This allows an orchestrator on AWS to communicate directly with a worker on a corporate network without reverse proxies.

# Install the pure-Go userspace network stack
curl -fsSL https://pilotprotocol.network/install.sh | sh

# Initialize the daemon on the local secure machine (Node A)
pilotctl daemon start --hostname secure-mcp-tool

# Initialize the daemon on the cloud VPS agent (Node B)
pilotctl daemon start --hostname cloud-worker-agent

# Node B can now route directly to Node A bypassing the NAT
# utilizing the underlying TCP-over-UDP transport layer
pilotctl connect secure-mcp-tool --message '{"jsonrpc": "2.0", "method": "call_tool"}'

Distributed Tracing is Mandatory for Agent Debugging

When a cross-cloud multi-agent workflow fails, identifying the exact point of failure is difficult. If an orchestrator on Azure delegates a task to a researcher on GCP, and the GCP agent encounters a hallucination loop, local logs will only show a generic HTTP timeout.

Implementing distributed tracing is non-negotiable for autonomous systems. Injecting trace context into payloads passed between clouds allows engineers to visualize the entire sequence of tool calls and prompt generations across network boundaries using OpenTelemetry standards.

# Injecting OpenTelemetry trace IDs into cross-cloud payloads
from opentelemetry import trace
from opentelemetry.propagate import inject

tracer = trace.get_tracer(__name__)

def dispatch_task_to_peer(agent_endpoint, payload):
    with tracer.start_as_current_span("cross_cloud_delegation") as span:
        headers = {}
        # Inject the current trace context into the headers or payload
        inject(headers)

        # Add the headers to the payload sent to the remote agent
        payload["trace_context"] = headers

        # Standard request to the remote agent
        response = requests.post(agent_endpoint, json=payload)
        span.set_attribute("peer.response", response.status_code)
        return response

How to Deploy Multi-Agent Systems Cross-Cloud[Python]

William Baker — Mon, 04 May 2026 20:21:24 +0000

Synchronous HTTP Will Throttle Your Agent Architecture

# Using Celery with Redis for async cross-cloud task delegation
from celery import Celery

app = Celery('agent_tasks', broker='redis://external-broker-url:6379/0')

@app.task
def delegate_to_research_agent(prompt, context):
    # This runs on the GCP worker node asynchronously
    result = research_agent.execute(prompt, context)
    # Store result in external database for the AWS agent to fetch later
    db.store_result(task_id=delegate_to_research_agent.request.id, data=result)
    return True

# On the AWS orchestrator node: trigger without blocking
task = delegate_to_research_agent.delay("Analyze Q3 earnings", previous_context)
print(f"Task dispatched with ID: {task.id}")

Ephemeral Containers Destroy Conversational State

# Externalizing agent state to Redis
import redis
import json

r = redis.Redis(host='global-redis.internal', port=6379, db=0)

def save_agent_thought(session_id, step_data):
    # Push the latest reasoning step to a list
    r.rpush(f"agent_state:{session_id}", json.dumps(step_data))

def rebuild_context(session_id):
    # Rebuild state if the container restarts
    raw_steps = r.lrange(f"agent_state:{session_id}", 0, -1)
    return [json.loads(step) for step in raw_steps]

Managing Tool Execution Across Network Boundaries

# Connecting an agent to a secure MCP server across the network
import asyncio
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client

async def query_secure_tool():
    # The server parameters define the connection to the secure tool environment
    server_params = StdioServerParameters(
        command="python",
        args=["secure_mcp_server.py"],
    )

    async with stdio_client(server_params) as (read, write):
        async with ClientSession(read, write) as session:
            await session.initialize()

            # The agent discovers available tools dynamically
            tools = await session.list_tools()

            # The agent executes the tool without seeing the underlying credentials
            result = await session.call_tool("query_internal_db", arguments={"target": "Q3_sales"})
            print(result)

asyncio.run(query_secure_tool())

Overcoming IP Churn and NAT Firewalls for Direct Transport

# Install the pure-Go userspace network stack
curl -fsSL https://pilotprotocol.network/install.sh | sh

# Initialize the daemon on the local secure machine (Node A)
pilotctl daemon start --hostname secure-mcp-tool

# Initialize the daemon on the cloud VPS agent (Node B)
pilotctl daemon start --hostname cloud-worker-agent

# Node B can now route directly to Node A bypassing the NAT
# utilizing the underlying TCP-over-UDP transport layer
pilotctl connect secure-mcp-tool --message '{"jsonrpc": "2.0", "method": "call_tool"}'

Distributed Tracing is Mandatory for Agent Debugging

# Injecting OpenTelemetry trace IDs into cross-cloud payloads
from opentelemetry import trace
from opentelemetry.propagate import inject

tracer = trace.get_tracer(__name__)

def dispatch_task_to_peer(agent_endpoint, payload):
    with tracer.start_as_current_span("cross_cloud_delegation") as span:
        headers = {}
        # Inject the current trace context into the headers or payload
        inject(headers)

        # Add the headers to the payload sent to the remote agent
        payload["trace_context"] = headers

        # Standard request to the remote agent
        response = requests.post(agent_endpoint, json=payload)
        span.set_attribute("peer.response", response.status_code)
        return response

DEV Community: William Baker

Building Multi-Agent Fleets That Actually Talk to Each Other

What "Agents Communicate Through You" Costs

Native Agent-to-Agent Communication

Setting Up a Multi-Agent Fleet on Pilot

1. Install and start the daemon on each agent

2. Join an org for pre-wired trust

3. Route queries to the best peer

The 350+ Service Agents Already Online

Emergent Coordination: The Part Nobody Plans For

Fleet Patterns That Work Well

Networks Within Your Fleet

MCP is a Tool Layer. But What's Underneath It?

Where MCP Lives in the Stack

The Missing Session Layer for Agents

What This Changes for MCP Deployments

Why UDP Instead of TCP?

The Protocol Stack in 2026

Practical: Adding Pilot to an Existing MCP Setup

How to Give Your AI Agent a Network Address (and Why It Matters)

The Problem With the Current Stack

What a Native Agent Network Looks Like

Getting Online: One Line of Code

The Speed Difference Is Real

Why This Matters for MCP Users

What Agents Are Actually Using It For

The Bigger Picture

Your MCP Server Has No Network Identity. Here's Why That's a Problem.

What MCP Actually Does (and Doesn't Do)

The Hardcoded URL Problem

What a Network Identity Gives an MCP Server

Setting It Up

The Broader Picture: MCP + A2A + Transport

Practical Impact

P2P vs. Broker: The Architecture Decision Defining Multi-Agent Systems

The Broker Model: Strengths and Limits

Broker Failure Modes at Scale

The P2P Alternative

What the Network Layer Provides

Hybrid Architectures

The Ephemeral Agent Problem

Implementation Considerations

Where This Is Heading

Why Your AI Agents Are Still Bottlenecked by HTTP (And What to Do About It)

The Numbers Tell the Story

What the Stack Actually Looks Like Today

The Session Layer Gap

What This Looks Like in Practice

Where MCP Fits In

The Hive Mind Property

Practical Next Steps

Building a Multi-Agent Fleet with No Central Server

The Central Hub Problem

Peer-to-Peer at the Session Layer

A Fleet Pattern That Actually Works

The Code

Self-Organization: How Groups Work

What You Give Up

When to Switch

Further Reading

Stop Making Your AI Agent Scrape the Web. There's a Better Way.

The Root Cause: Wrong Layer

What Specialized Data Agents Look Like in Practice

The Network Effect That Makes This Work

The Latency Numbers

Getting Your Agent Connected

When You Still Need the Web

Why Your MCP Server Needs a Network Layer (And How to Add One in 30 Seconds)

The Problem with MCP's Transport Layer

What a Session Layer Gives You

One Line of Code

Why UDP, Not TCP?

The MCP + Pilot Pattern

Discovery Is Solved Too

Wrapping Up

How to Deploy Multi-Agent Systems Cross-Cloud[Python]

Synchronous HTTP Will Throttle Your Agent Architecture

Ephemeral Containers Destroy Conversational State

Managing Tool Execution Across Network Boundaries

Overcoming IP Churn and NAT Firewalls for Direct Transport