DEV Community: Philip Stayetski

Sandboxing Agent Apps at Runtime: The Security Model Behind Trusting Code You Didn't Write

Philip Stayetski — Sun, 16 Aug 2026 14:15:09 +0000

Your agent's tool loop is a supply chain now. Every install adds code you didn't write to a runtime you're responsible for, and the interesting security question isn't whether the install was verified — it's what happens when that code runs. If you want to sandbox agent apps at runtime, you need four things: artifacts that can't be swapped after verification, permissions scoped to declared grants, a supervisor that owns the process lifecycle, and a call surface narrow enough to audit.

This post walks the threat model, the four controls that actually limit blast radius, and one concrete implementation you can inspect today: the app store in Pilot Protocol, an open-source overlay network for AI agents. Its security model is documented in the Pilot Protocol app store docs, and every claim below comes from there.

Why "verified at install" isn't enough

Install-time checks are point-in-time. A signature check when you fetch a package tells you the artifact was intact then. It says nothing about the binary that gets executed minutes later, after a cache swap, a symlink trick, or a compromised update channel. This is the classic time-of-check-to-time-of-use (TOCTOU) gap, and it's worse for agents than for humans because nothing pauses to review: an agent installs a tool, reads its output, and acts on it in the same loop.

The runtime threat model for installed agent apps is roughly:

Prompt injection via tool output. The tool returns content that re-directs the agent. The most famous variant, but only one of several.
Buggy or malicious tool code. The app itself misbehaves — reads files it shouldn't, opens sockets, exhausts memory or file descriptors.
Data exfiltration. A tool with network access can ship your context out. This is why "can it reach the network?" has to be an explicit, grantable property, not a default.
Crash loops and resource abuse. A misbehaving app that dies and respawns can become a denial-of-service against your own host.

"Verified at install" answers none of these. Sandboxing agent apps at runtime means the environment — not the artifact's provenance alone — is what constrains the damage.

Sandbox agent apps at runtime: four controls that matter

If you're evaluating any agent runtime (or building your own), these are the four controls to look for. Together they form a defense-in-depth story; each one alone leaves a hole.

1. Signed artifacts, re-checked at launch. The manifest pins the binary's sha256 and carries an ed25519 signature. The daemon verifies the signature when it scans installed apps, and re-checks the binary's hash immediately before every spawn — rejecting it if it became a symlink or no longer matches the pinned hash. This closes the TOCTOU gap: a binary swapped between install-scan and launch is caught before it runs.

2. Grant-scoped permissions, no ambient authority. The manifest declares exactly what the app may do — network, file I/O — and a broker enforces those grants at runtime. Installing an app accepts its declared grants; nothing more. There is no "joined the system, therefore trusted" default, which is the failure mode of most plugin architectures.

3. A supervisor, not a launcher. The runtime supervises the app's lifecycle: auto-spawn on install, crash-loop detection with exponential backoff, suspension after repeated failures, and resource limits (file-descriptor and address-space caps) so a misbehaving app can't take the host down with it. Lifecycle events go to a rotating audit log.

4. A typed call surface. Each app method is a JSON-in/JSON-out call, and the manifest's exposes set is the entire dispatchable surface. The broker refuses anything not listed there — even requests from the daemon itself. No browser, no REST plumbing, no hidden endpoints: a surface small enough to audit is a surface you can actually reason about.

A worked example: Pilot Protocol's app store

Pilot Protocol's app store is a useful case study because it's young, open source, and unusually explicit about its threat model. The loop an agent runs is discover → install → call:

# 1. Discover what's installable
pilotctl appstore catalogue

# 2. Inspect before committing — description, vendor, methods, permissions
pilotctl appstore view io.pilot.cosift

# 3. Install by id — fetch + verify sha + signature; the daemon auto-spawns it
pilotctl appstore install io.pilot.cosift

# 4. Call a method — JSON in, JSON out
pilotctl appstore call io.pilot.cosift cosift.search '{"q":"raft consensus","k":"5"}'

Every layer is deny-by-default. The catalogue itself is signed with a dedicated ed25519 key whose public half is compiled into the client, so a compromised host or CDN can't redirect installs to hostile bundles. App-to-app calls go through the daemon's broker, which enforces two gates before dispatch: the exposes gate (the method must be in the target app's declared surface) and the grant gate (the caller must hold a matching ipc.call grant). The supervisor re-verifies the binary's hash at launch, applies resource limits, and backs off on verification failures.

One honest caveat worth stating plainly: OS-level sandboxing like landlock or seccomp isn't wired in yet. Today the enforcement is rlimits plus the syscall/IPC broker, not a kernel sandbox. That's a real boundary — but it's also exactly the kind of admission that makes the rest of the model credible, and it's the right thing to know before you run third-party apps in a hostile environment.

The store even ships a runtime firewall as an app: AEGIS blocks prompt injection before your agent reads the content — the same class of defense this article is about, applied at the input layer.

What to check before you run third-party agent code

Whatever runtime you use — MCP servers, plugins, a homegrown tool loop — run this checklist against it:

Who signs, and who can rotate? A signature is only as good as the key custody and the fail-closed behavior when verification fails.
What does it declare, and what can it actually touch? Does the permission model match the manifest, or is there ambient authority?
Who supervises it? Is there a process owner that detects crash loops, bounds resources, and logs lifecycle events?
Can it be revoked? If an app turns hostile or gets compromised, is removing it a real operation or a fantasy?
Is the surface auditable? Can you enumerate everything the app can do — and would you notice if that list changed?

The pattern that holds up across all of them: trust the artifact at install, but constrain the process at runtime. Install-time verification decides what you let in; runtime sandboxing decides how much damage it can do once it's there.

Pilot Protocol is one implementation of that pattern, and it's free to try. If you want to poke at the model yourself:

curl -fsSL https://pilotprotocol.network/install.sh | sh
pilotctl appstore catalogue

Install something, read its manifest, watch what the supervisor does. The app store docs spell out the whole security model — worth a read before you run your next agent's tools.

What an Agent Network Protocol (ANP) Has to Specify: Addressing, Discovery, Trust, Transport

Philip Stayetski — Sun, 16 Aug 2026 09:33:50 +0000

What an Agent Network Protocol (ANP) Has to Specify: Addressing, Discovery, Trust, Transport

You've got three agents that need to talk to each other across machines. Different clouds, maybe different vendors, no shared infrastructure to speak of. Before a single message moves, you have to answer four questions: how do they address each other, how do they find each other, how do they know they can trust each other, and how does a message actually get delivered?

Those four questions are what an agent network protocol (ANP) has to specify. Not the agent's logic, not the prompts, not the tool calls — the networking contract between agents. If you've been reading about agent-to-agent communication lately, you've seen the name: Agent Network Protocol (ANP) is an open-source spec that wants to be, in its own words, "the HTTP of the agentic web." This post breaks down what a protocol like that actually has to define, and where the spec ends and the implementation begins.

What an Agent Network Protocol (ANP) Has to Specify

Strip away the marketing and every agent communication protocol ends up specifying the same four layers:

Addressing — what an agent is called, and whether that name survives the machine underneath it.
Discovery — how one agent learns that another exists and how to reach it.
Trust — how two agents establish that they're actually talking to who they think they are.
Transport — the wire format, the delivery semantics, and what happens when the network is lossy.

Everything else — message schemas, capability negotiation, conversation state — sits on top of those four. Get one of them wrong and the protocol fails in ways that are painful to debug: agents that can't find each other, handshakes that silently fail, messages that vanish behind NAT.

Addressing: A Name That Survives the Machine

The first thing an agent network protocol has to nail is identity. A human conversation works because "call Alice" resolves regardless of which phone she's holding. Agents need the same property: an agent ID that resolves no matter where the process is running.

This sounds trivial until you've lived the failure mode. You build an agent, give it an endpoint, and every time the container restarts, the IP changes. Every time you move it to another cloud, the hostname changes. If the address is tied to the infrastructure, the network breaks on every deploy. The classic symptom is the webhook URL that stops working after a restart — the endpoint was an accident of the machine, not a property of the agent.

A proper agent network protocol defines an identity layer that is decoupled from the transport endpoint. ANP does this with its own identifier scheme for agents in its distributed network. Overlay networks push the same idea further: each agent gets a permanent virtual address that survives restarts, IP changes, and moving across clouds — the address belongs to the agent, not the box it happens to be running on.

Discovery: How Agents Find Each Other

Identity is half the problem; the other half is that an agent has to learn the identity of the agents it needs to talk to. That's discovery, and it's the bootstrap problem of any network: how does the first message find its target when neither side knows the other exists?

Two mechanisms show up over and over. The first is a rendezvous point: a registry or nameserver that agents register with, and that answers "who is agent X, and where is it reachable?" The second is capability metadata: agents advertise what they can do, so discovery becomes a search ("find an agent that does X") rather than a phone book lookup ("find agent X").

ANP's design centers on a distributed agent network where agents join and become findable by their identifiers. An overlay network gives you the same property as a service: a rendezvous registry plus a nameserver, so you can find agents and capabilities by name or tag instead of hardcoding endpoints. For an agent, discovery isn't a luxury — it's what turns a collection of isolated processes into something that behaves like a network.

Trust: Explicit, Mutual, and Decoupled from Membership

This is the layer most agent protocols get wrong, because it's the one that's actually about people's expectations. Joining a network and being trusted to receive messages are not the same thing. In a VPN, "joined" means "trusted" — one credential gets you into everything. For agents, that conflation is dangerous: you want to talk to some agents on the network, not all of them, and you want the decision to be explicit and revocable.

An agent network protocol has to specify how trust is established. The pattern that survives contact with reality is the mutual handshake: agent A requests a connection with agent B, B explicitly approves (or rejects), and only then does traffic flow. Both sides opt in. Trust is per-pair, not per-network, and it can be revoked by either side.

ANP, true to its HTTP heritage, leans on web-standard auth (signatures, credentials) at the message level. What an overlay network adds is trust as a lifecycle with real commands. pilotctl handshake <peer> sends the request, pilotctl approve <id> accepts one, pilotctl pending shows what's waiting on you. The protocol guarantees the identity (the registry signs the bidirectional trust record once both sides approve), and the agent keeps control of who it talks to.

Transport: Delivery Semantics Under the Message Format

The transport layer is where a protocol's philosophy shows. ANP made a deliberate, pragmatic choice: it rides on HTTP and formats messages as JSON-LD. That's the "HTTP of the agentic web" bet — reuse the most ubiquitous protocol on the planet, and any agent that can speak HTTP can join. It's a reasonable call, and it means ANP agents work behind the same infrastructure the web already runs on.

But HTTP request/response has a mismatch with how agents actually live. Agents are long-lived. They sit behind NAT, on laptops, in home labs, in clouds that don't give them public IPs. A call-style protocol that requires the agent to be reachable at a stable public URL breaks the moment the agent is behind a router. That's exactly the problem that pushed agent infrastructure toward persistent tunnels: instead of asking "how do I make my agent reachable at a URL," you ask "how do I keep a tunnel open that carries messages to me wherever I am."

This is the part of the spec that most protocols deliberately leave to implementations. An overlay network fills it in with encrypted UDP tunnels (X25519 key exchange, AES-GCM encryption), STUN-based hole punching with relay fallback so agents behind NAT are reachable, and userspace reliability on top of UDP. The agent keeps its permanent address; the overlay handles the unglamorous work of making that address reachable from anywhere.

Where an Overlay Network Implements the Spec for You

Here's the honest framing: a protocol spec tells you the vocabulary — the identifiers, the message formats, the handshake rules. Something still has to run the addressing, discovery, trust, and transport. For ANP, the reference implementation is the AgentConnect SDK. For the overlay approach, the implementation is the network itself.

An overlay network for agents — Pilot Protocol is the open-source example, implemented in Go with zero external dependencies — takes those four layers and makes them commands:

pilotctl handshake <peer>              # request a trusted connection
pilotctl approve <id>                  # accept an inbound handshake
pilotctl send-message <peer> '...'     # message over the encrypted tunnel
pilotctl find <hostname>               # resolve an agent by name

No public IPs, no port forwarding, no webhook URL to keep alive. Addressing comes from the permanent virtual address, discovery from the rendezvous registry, trust from the mutual handshake, transport from the encrypted tunnels and NAT traversal. An agent network protocol gives you the contract; the overlay network is what makes the contract true on the actual internet.

FAQ

Is ANP an alternative to MCP? Not really — they operate at different layers. ANP is a communication protocol between agents; MCP is a tool protocol between an agent and a tool server. A system can use both.

Do agents need a protocol at all, or can they just call APIs? Direct API calls work until you need stable identity, discovery, or reachability behind NAT. Those needs are what a protocol layer exists for.

What's the fastest way to try an agent overlay network? Install the CLI and talk to another agent:

curl -fsSL https://pilotprotocol.network/install.sh | sh
pilotctl handshake <peer>

Which layer should I evaluate first? Trust. Addressing and transport can be patched later; a trust model you can't live with means rebuilding the network.

If you're spec-shopping for agent communication, read the four layers first: addressing, discovery, trust, transport. ANP is a serious attempt to standardize the vocabulary over HTTP. Just remember the spec is the contract, not the plumbing — an overlay network is what delivers the contract on the real internet.

MQTT for AI Agent Communication: What a Broker Does Well (and Where It Doesn't Fit)

Philip Stayetski — Sat, 15 Aug 2026 23:36:26 +0000

You're wiring agents together and MQTT keeps coming up. It's the default answer to "how do my things talk to each other" — battle-tested, tiny footprint, works on a Raspberry Pi. So you evaluate MQTT for AI agent communication, and it looks great on paper. Then you hit the moment where it stops being the right shape: one agent needs to talk to one specific peer, directly, and the broker is sitting in the middle of a conversation that was always point-to-point.

This post is an honest comparison. MQTT is excellent at what it was designed for — many-to-many event fan-out. But broker-based messaging and peer-to-peer overlays solve different problems, and knowing which one you actually have saves you from building a system that's fragile in the exact places agents are most sensitive.

MQTT for AI Agent Communication: What It's Actually Good At

MQTT is a publish/subscribe protocol that runs over TCP. Clients don't talk to each other; they connect to a broker (Mosquitto, EMQX, HiveMQ, AWS IoT Core) and exchange messages on topics. A publisher sends to agents/fleet-1/status, the broker fans it out to every subscriber.

That model is genuinely great for several agent workloads:

Fan-out. One agent publishes an event, a hundred agents receive it. The broker handles the copy fan-out for you. This is the killer feature — nothing peer-to-peer does this as cleanly.
Decoupling. Publishers don't know who's listening. An agent can publish to a topic without caring whether the consumer is up, on the same cloud, or even written yet.
Delivery semantics. QoS 0/1/2 give you at-most-once, at-least-once, and exactly-once delivery. Retained messages let a new subscriber immediately see the latest state. Last-will messages tell the rest of the fleet when an agent died.
NAT friendliness (the one people forget). Clients dial out to the broker, so agents behind NAT connect without any inbound holes. If you have a reachable broker, your agents don't need public addresses.

If your pattern is "many agents emit events, many agents consume them," MQTT is a legitimately good choice and this post isn't arguing otherwise.

Where the Broker Model Stops Fitting

The problems start when agent communication is actually point-to-point — which, in practice, is most of it. A task handoff, a file transfer, a tool call, a negotiation: these are one-to-one conversations wearing a topic name.

The broker is a single point of failure in every conversation. All traffic crosses it, even between two agents on the same machine. Broker down, agents down. You can cluster brokers, but now you operate infrastructure — and the cluster itself is still a shared dependency every agent trusts.

Identity is broker-scoped. An MQTT client ID only means something to that broker. Two agents on different brokers can't address each other at all without bridges. There's no notion of "this agent, wherever it lives, reachable by this name" — the identity dies with the connection.

NAT help becomes a NAT requirement. Yes, clients dial out — but only to your broker. If agents are split across clouds, a partner's network, a laptop at home, the broker must be reachable from all of them. You've traded per-agent port forwarding for one big, always-on, publicly reachable component that everything depends on.

Trust is centralized and coarse. The broker authenticates clients, and that's it. Agent A doesn't verify Agent B; both just trust the broker's ACLs. For autonomous agents that need to make their own trust decisions about peers, that's the wrong trust boundary — it's "joined the network" = "trusted," which is exactly the conflation that breaks agent security.

The Direct-Link Case: One Agent, One Peer, No Middleman

The alternative shape is a peer-to-peer overlay: every agent gets a permanent virtual address, and agents talk over encrypted tunnels directly, with NAT traversal (STUN + hole punching, relay fallback) handling reachability instead of a central broker.

This maps cleanly onto the point-to-point reality:

Direct links. Agent A sends to Agent B's address. No broker in the path, no shared component to operate, no single point of failure for the conversation.
Permanent identity. The address survives restarts, IP changes, and moving between clouds. Peer A can reach peer B tomorrow by the same name — no reconnecting to a broker, no re-subscribing.
Per-peer trust. Each pair does a mutual handshake. Agent A decides whether to trust Agent B directly, rather than inheriting trust from a broker. Membership and trust are decoupled — "on the network" no longer implies "trusted by me."
Reachability without infrastructure. STUN + hole punching gets agents behind NAT talking directly; when a NAT defeats punching, a relay fallback carries the traffic. No inbound ports, no public broker.

That's the model Pilot Protocol implements: an open-source overlay network (Go, zero external dependencies, AGPL-3.0) giving agents a permanent address, encrypted UDP tunnels (X25519 + AES-GCM), NAT traversal, and an explicit mutual-handshake trust model — plus an app store where agents install capabilities with pilotctl appstore install. It's built for the case MQTT explicitly delegates away: one agent reaching one peer directly.

A Quick Comparison

	MQTT (broker)	Peer-to-peer overlay
Message pattern	Publish/subscribe, topic-based	Direct address-to-address
Fan-out	Excellent — broker copies	Manual per-peer sends
Point-to-point	Through the broker (extra hop)	Direct tunnel
Single point of failure	The broker	None in the data path
NAT handling	Clients dial out to broker	Hole punching + relay fallback
Identity	Broker-scoped client ID	Permanent virtual address
Trust	Broker authenticates clients	Mutual per-peer handshake

How to Choose

Ask what the traffic actually looks like.

If agents mostly emit events others consume — telemetry, status feeds, result broadcasts — use MQTT (or NATS, or Kafka; the broker family is right for fan-out). MQTT is mature, tiny, and does this better than any overlay.

If agents mostly talk one-to-one — task handoffs, file transfers, tool calls between specific peers — the broker is an extra hop that adds a failure point and a shared dependency. A peer-to-peer overlay gives you the direct link, stable identity, and per-peer trust that broker routing can't express.

Most multi-agent systems are a mix, and that's fine: the two aren't mutually exclusive. Run MQTT for the event streams where fan-out earns its keep, and put the point-to-point conversations on an overlay where they belong. The trick is not picking a winner — it's refusing to force one architecture to carry both patterns.

The interesting write-up of this tradeoff is on the Pilot Protocol blog, which covers overlay networking for agents, NAT traversal, and trust. If you want to see the direct-link model in action:

curl -fsSL https://pilotprotocol.network/install.sh | sh
pilotctl daemon start
pilotctl send-message <peer-address> --data 'hello from agent one'

When Should You Move Beyond HTTP Streaming for Your AI Agent?

Philip Stayetski — Sat, 15 Aug 2026 19:09:14 +0000

Your agent streams beautifully over HTTP. Tokens arrive as they're generated, you render them live, and it feels like the modern way to build. Then somebody asks: "can agent B push a message to agent A?" — and the whole mental model creaks. That's the moment to ask when you should move beyond HTTP streaming for your AI agent.

I hit this wall building a multi-agent system. Every component was HTTP-native: fetch, SSE, chunked responses. It worked — until a workflow needed one agent to notify three others mid-task, from behind a NAT. Nothing in my stack could express that. Not a missing feature. A missing shape.

This is an opinion piece, so here's my opinion up front: HTTP streaming is the right tool for exactly one relationship — client asks, server streams an answer. The moment your agents act as peers, receiving instead of just requesting, a persistent tunnel or overlay network becomes the simpler answer. And the threshold is easier to spot than you'd think.

What HTTP streaming is actually good at

Let's be fair to it. Server-Sent Events, chunked transfer encoding, fetch ReadableStream — HTTP streaming nails one-to-one request/response with a long tail:

Token streaming from an LLM, rendered as it arrives.
Progress updates on a long-running job you kicked off.
Any "I asked, so give me the answer incrementally" exchange.

The client opens the request; the server writes the response body a chunk at a time. Simple, ubiquitous, works through every proxy and load balancer on earth. If that's your whole system, stop reading and go build.

Where the shape stops fitting

The constraints aren't performance problems. They're structural. HTTP streaming keeps the client-server asymmetry baked in, and agents stop being clients the moment you need:

Inbound events. The server can't open a stream to a client that never asked. If another process needs to reach your agent, HTTP has no direction for that — you bolt on polling or a public callback URL.
NAT-bound callbacks. Your agent runs on a laptop, a CI runner, a spot instance — behind NAT. It can make outbound requests all day, but nothing can connect to it without a tunnel service or a static IP.
Long-lived sessions. An agent task that runs for hours — research, code review, a human in the loop — doesn't fit a request lifecycle. You end up with job IDs, a status endpoint, retry queues: reimplementing a session layer on top of a stateless protocol.
Many-to-many fan-out. One orchestrator, N workers, one event everyone needs. With HTTP that's N connections and N reconnects. HTTP has no native fan-out.

When should you move beyond HTTP streaming for your AI agent?

The threshold isn't bytes or latency. It's directionality and lifetime. Move when any of these start showing up in your code:

Your agent has to receive, not just request. A webhook to your agent, a message from a peer, an event it must react to. HTTP streaming doesn't cover that direction at all.
Sessions outlive requests. You'll notice when you've built a job table, a status endpoint, and three retry loops to keep one logical task alive.
You're fanning out. One event, many consumers. You're paying for N connections and re-implementing delivery semantics per consumer.
Addresses change. Your agent moves between machines, clouds, networks. HTTP callbacks need a stable, reachable address; the address becomes your biggest operational risk.
Reconnection should resume, not restart. A stream drops and you re-request from zero. A persistent connection gives you a session that resumes instead of a request to redo.

The threshold is directionality, not traffic

Here's where I landed. "When should I move beyond HTTP streaming for my AI agent?" is really "when did my agents stop being clients?" The moment you have N peers that each initiate and receive, you're modeling a network — and request/response tooling just makes you simulate what a network already does, badly, by hand.

WebSockets? Honest answer: they fix the persistent-connection half while keeping the asymmetry — one side still has to be a reachable server. If both ends are behind NAT, two agents in two offices, WebSockets doesn't answer the question. (To be fair: WebSockets are excellent when you control a public server endpoint, and browser support is a real advantage.)

ngrok-style tunnels? They solve reachability by handing you a public URL. Fair and factual: great for demos and single services. But the URL becomes the thing that changes, and a hundred agents each holding a tunnel is a hundred moving parts.

What a persistent overlay changes

When the shape of your system is "peers," the simpler answer is a network layer that treats agents as first-class nodes: each agent gets a permanent virtual address that survives restarts and IP changes; traffic moves over encrypted UDP tunnels (X25519 key exchange + AES-GCM); NAT traversal via STUN and hole-punching, with relay fallback, means an agent behind NAT is reachable without you running a server; and trust is explicit — a per-peer handshake, so joining the network and trusting a member are separate decisions, unlike a VPN where "joined" means "trusted."

That's the model Pilot Protocol implements: an open-source overlay network purpose-built for agents, written in Go with zero external dependencies. I bring it up here because it's designed around exactly this threshold — agents as peers rather than clients. Discovery via a rendezvous registry, so you find peers by name instead of hardcoding IPs.

The practical tell: when your HTTP stack carries more machinery for "the server reaching the client" — polling loops, job tables, retry queues, public callback URLs — than it carries actual work, you've crossed the line. A persistent tunnel removes that machinery. The connection is just there.

My rule of thumb

Three questions before you add the next layer on top of HTTP streaming:

Does any peer need to initiate a connection to my agent?
Does any task outlive a single request?
Do I have more than two agents that talk to each other, rather than to a central API?

Two yeses, and I'd reach for a persistent tunnel before writing more polling code. Not because HTTP streaming is bad — it's genuinely the right shape for client-server — but because a multi-agent system is a different question, and the tool should match the question.

Get started: curl -fsSL https://pilotprotocol.network/install.sh | sh — then pilotctl appstore catalogue to see what agent-native tooling is one command away.

What Does a Production AI Transport Layer Need to Support? A Spec-Style Checklist

Philip Stayetski — Sat, 15 Aug 2026 14:35:30 +0000

You're putting agents into production. Orchestration is figured out, prompts are tuned, the eval suite passes. Then two agents on different clouds can't reach each other, and you realize nobody ever wrote down what the transport layer was supposed to do.

That question deserves an actual answer: what does a production AI transport layer need to support? This post is a spec-style checklist — six requirements, each with the failure mode it prevents — plus an honest pass at how the usual options measure up. Use it to evaluate any transport before you commit to one.

What does a production AI transport layer need to support?

A transport for agents is not a webhook endpoint. It's the substrate everything else sits on: addressing, delivery, security, reachability, and trust. If any one of these is bolted on as an afterthought, you'll discover it in a pager alert at 3am. Here's the checklist.

1. Permanent addressing that outlives the machine

Production agents restart. They move between laptops, containers, and clouds. A transport that identifies an agent by its current IP — or its current DNS name — breaks the moment either one changes.

What "support" means: the agent has an address that is its own, stable across restarts, IP changes, and migrations, with an optional human-readable name on top. Think of the difference between a phone number and a SIM card: the SIM moves between handsets, the number stays the same.

2. Reliability on top of an unreliable substrate

UDP is the natural substrate for agent traffic: connectionless, no handshake tax, and it doesn't care about NAT state the way TCP does. But UDP delivers nothing. Packets get dropped, reordered, duplicated.

What "support" means: the transport provides reliability in userspace — sequencing, acknowledgments, retransmission, flow control — so application code gets stream semantics it can trust even though the wire is UDP. And it should let you opt out when you don't need it: fire-and-forget datagrams for telemetry and heartbeats.

3. Encryption as the default, not an option

Agent traffic carries prompts, tool results, credentials, data. In production, "encrypted if you configure it" is the same as "encrypted never."

What "support" means: traffic is encrypted by default, with a real key-exchange handshake and authenticated encryption for the tunnel itself. Not TLS bolted on as an afterthought — the crypto is part of the transport's identity story: the same key material that encrypts the tunnel also signs who the peer is.

4. NAT traversal, or reachability without port forwarding

This is the one that bites most people. Agents run behind NAT: office networks, home routers, carrier-grade NAT, cloud VPCs with no public IP. If your transport requires a public endpoint on both sides, half your fleet is unreachable.

What "support" means: the transport discovers the agent's public endpoint, tries a direct connection, falls back to hole-punching when the NAT is restrictive, and relays through a beacon when neither works — automatically, without the application knowing or the operator opening a port.

5. Explicit trust, decoupled from membership

A VPN gives everyone on the network the same trust. That's wrong for agents: you want your agents on one network, but you don't want every agent on that network holding your API keys.

What "support" means: trust is established per peer, explicitly, with both sides approving. Membership ("we're on the same network") and trust ("I accept messages from this specific peer") are separate decisions, made separately.

6. Discovery without a static config file

The requirement everyone forgets until the fleet grows. When you have dozens of agents and capabilities, hardcoding addresses in config files doesn't scale.

What "support" means: a registry that assigns addresses and resolves names, so an agent can find a peer or capability by name or tag at runtime — instead of a spreadsheet of endpoints.

Running the checklist against the usual options

Honest evaluation:

TCP + TLS. Rock-solid reliability and encryption, and it's everywhere. It fails requirements 1 and 4 in the same way: the server needs a stable public endpoint. Fine for client-server, painful for agent-to-agent.
WebSockets. A great persistent channel, but still client-server — someone has to host the socket. Same reachability problem, plus you're building addressing and trust yourself.
gRPC. Excellent for typed request/response between services that can reach each other. It doesn't claim to solve reachability or identity, and you still need a listening endpoint.
Mesh overlays (Tailscale, ZeroTier, Nebula). These solve connectivity and encryption genuinely well, and they're worth considering. The difference is the model: they're built around human-owned devices and an IP-layer mental model, and membership generally implies trust. For agents, the trust decision needs to be per peer and explicit.
MCP servers and tool tunnels. The right shape for tool-call request/response between an agent and a server. Less suited as the general substrate for arbitrary agent-to-agent traffic.

None of these fail because they're bad software. They fail because they were designed for a different caller: a human at a browser, a service in a datacenter, a tool call in a loop. Agents are a new kind of endpoint, and they need a transport built for them.

Where Pilot Protocol fits

Pilot Protocol is an open-source overlay network that gives agents exactly this checklist: a permanent virtual address that survives restarts, reliable streams and datagrams over encrypted UDP tunnels (X25519 key exchange, AES-256-GCM authenticated encryption), automatic NAT traversal with relay fallback, and a trust model where membership and trust are decoupled — a mutual handshake, per peer. Written in Go with zero external dependencies. More than 243k+ agents and users are on the network.

The commands are boring in the good way:

pilotctl handshake <peer>   # explicit, mutual
pilotctl trust              # confirm the relationship
pilotctl ping <peer>        # is the tunnel actually up?

The full addressing, transport, encryption, and trust details are in the Pilot Protocol core concepts documentation.

The checklist, condensed

Permanent addressing?
Reliability over UDP?
Encryption by default?
NAT traversal without port forwarding?
Explicit per-peer trust?
Runtime discovery?

If a transport can't answer yes to all six, you're building the missing parts yourself — and the transport is the most expensive layer of your stack to maintain by hand.

Get started:

curl -fsSL https://pilotprotocol.network/install.sh | sh

AI Agent Swarm Deployment: Addressing, Discovery, and Trust Between Members

Philip Stayetski — Sat, 15 Aug 2026 09:33:21 +0000

Your swarm deploys cleanly. The orchestrator spins up ten agents, assigns them roles, wires the prompt pipeline. Then the first real run starts, and three of them can't find each other. Two are behind NAT, one restarted and lost its address, and the one that can be reached won't accept messages from the others because nobody ever established trust.

This is the part of AI agent swarm deployment that orchestration frameworks don't cover. Scheduling, retries, and state are well-solved problems. Addressing, discovery, and trust between members are the networking underneath, and if you skip that layer, your swarm works in the demo and falls apart in production.

What AI Agent Swarm Deployment Actually Involves

When people talk about deploying a swarm, they usually mean orchestration: which agent runs where, what it's allowed to call, how work gets distributed and retried. Frameworks like LangGraph, Temporal, and Kubernetes-based schedulers handle this well, and you should keep using them for it.

But orchestration assumes the members can actually talk to each other. That assumption has three hidden dependencies:

Addressing — every member needs an identity that survives restarts and IP changes.
Discovery — members need a way to find each other by name or role, not by hardcoded IP.
Trust — members need a way to verify who they're talking to, and to control who can reach them.

If you deploy a swarm across machines — and a swarm that lives on one box isn't really a swarm — these three things are the difference between "deployed" and "working".

Addressing: A Swarm Member Needs a Stable Identity

Here's a failure mode I hit the first time I deployed a multi-agent system across two cloud regions: the orchestrator restarted one of the worker agents, and it came back with a new IP. Every other member had that IP cached. The worker was healthy, but unreachable — it had no stable address to be reached at.

Containers get recreated, VMs get migrated, IPs get reassigned. If your swarm's addressing is "whatever IP the orchestrator assigned this run", then a single restart turns a member into a ghost.

The fix is a permanent virtual address that survives restarts, IP changes, and moving across clouds. Members register once and keep the same address for their whole lifecycle. A restarted member comes back at the same address, so nobody's cached routing breaks. This is exactly the problem Pilot Protocol's addressing model solves: every agent gets a permanent virtual address that outlives any single host.

Discovery: How Members Find Each Other

Addressing alone isn't enough. In a swarm of ten agents, each member needs to find the others. Hardcoding addresses works until the swarm changes — and a swarm, by definition, changes.

Discovery in an agent network is a rendezvous registry plus a nameserver: members register, and others look them up by name or tag. You ask "where's the research agent?" and you get an address back, instead of maintaining a config file of IPs that goes stale the moment something moves.

Pilot Protocol's registry works this way. Agents register with names and capabilities, and peers find them by name or tag. New members joining the swarm just register and become discoverable; you don't redeploy the whole fleet when the topology changes.

Trust: Membership Isn't Trust

The subtle one. Just because an agent is in your swarm doesn't mean every other member should be able to send it anything. In a VPN, "joined" means "trusted" — one credential gets you in, and once you're in, you're inside everything. That model doesn't fit agents, where a compromised member shouldn't get ambient access to all the others.

The alternative is explicit, per-member trust: each pair establishes a relationship by mutual approval. A handshake is a request, and the receiver decides. This keeps membership and trust decoupled — your swarm can have dozens of members while each member only talks to the ones it actually trusts.

This is the trust model in Pilot Protocol: an explicit per-peer handshake where both sides approve, rather than a single shared credential that opens everything.

Deploying a Swarm with pilotctl: A Walkthrough

Pilot Protocol is an open-source overlay network for agents — it gives each agent a permanent address, encrypted tunnels between them, NAT traversal, and the trust model above. Here's what deploying a two-member swarm across two machines actually looks like.

First, install the CLI on each host:

curl -fsSL https://pilotprotocol.network/install.sh | sh

Start the daemon on each member. Each gets its own identity and virtual address:

pilotctl daemon start

This blocks until the node is registered. Now each member has a permanent address, and neither needs a public IP — the overlay handles NAT traversal (STUN, hole-punching, with relay fallback when a direct path isn't possible).

Next, establish trust between the members. This is a handshake, and it's mutual — the receiving side approves before anything flows:

pilotctl handshake research-agent "joining the swarm"
# on the other member:
pilotctl pending
pilotctl approve <node_id>

Check the relationship is live:

pilotctl trust
pilotctl peers

Then the members can talk, using names instead of IPs:

pilotctl send-message research-agent --data '{"task":"summarize the incident report"}'

That's the whole loop: address, discover, trust, message. Each member has a stable identity, finds peers by name, and only talks to members it has explicitly approved. If a member restarts, it comes back at the same address and the swarm keeps working.

What Orchestration Still Handles

None of this replaces your orchestration layer. You still want the scheduler deciding which agent does what, the retry logic, the state management, the human-in-the-loop checkpoints. Those are orchestration's job, and frameworks do them well.

What the networking layer adds is the substrate: stable addressing, discovery, and trust, so the orchestrator's decisions actually reach the members. The clean split is: orchestration decides, the network delivers. Frameworks and agents that speak MCP or similar protocols work fine over it — the overlay is the transport underneath, not a replacement for the application protocol.

For a swarm, this also composes nicely with the app-store model in Pilot Protocol: capabilities like web search, browser access, or data enrichment are installable apps that any member can call over the same overlay. You deploy the swarm once, and members gain capabilities on demand instead of being rebuilt.

The Takeaway

AI agent swarm deployment fails in production for reasons that have nothing to do with orchestration: members that can't be addressed, can't be found, or can't trust each other. Add a networking layer that handles all three, and the swarm stops being a demo and starts being an infrastructure you can restart, grow, and move.

Get started:

curl -fsSL https://pilotprotocol.network/install.sh | sh
pilotctl daemon start

The full docs — including the trust and discovery details for larger fleets — are at Pilot Protocol's documentation.

Agent Connectivity Best Practices: 6 Failure Modes That Make Agents Unreachable — and What Prevents Them

Philip Stayetski — Fri, 14 Aug 2026 19:34:55 +0000

Your agent was reachable this morning. You deployed it, tested it end to end, and closed the laptop. This morning it's gone — not the code, not the model, the address. The laptop went to sleep and woke on a new DHCP lease. Or the container got rescheduled onto a different host. Or the office router rebooted and the port mapping vanished with it.

If you're searching for agent connectivity best practices, that's the scenario you're trying to fix: agents that silently go unreachable while nothing about their code changed. The failure modes behind it are a short, well-understood list, and each one has a practice that prevents it. This post is that list — six failure modes, six fixes, in the order you're likely to meet them.

Why agents go unreachable

A web service has a stable address: a DNS name, a public IP, maybe a load balancer in front. An agent has none of that by default. It runs on whatever machine it happens to occupy — a laptop behind home NAT, a container that gets rescheduled, a VM in a cloud region you don't control. The network gives it an address that is really a property of the machine, and the machine moves. Restarts, sleep/wake, cloud migrations, and network changes all break the address before they break the code.

Six failure modes cover most "my agent is unreachable" incidents:

NAT and CGNAT — the agent sits behind a router that silently drops inbound connections.
IP churn — the address is keyed to an IP, and the IP changes.
Egress-only firewalls — the network allows outbound only; nothing can reach the agent.
Symmetric NAT — hole punching fails on networks that assign per-destination ports.
Restart rotation — every restart mints new credentials, so the agent's identity effectively changes.
Discovery gaps — the address is fine, but nothing can find what the agent offers.

The practices below each prevent one of these. Do all six and your agents stay reachable across restarts, networks, and clouds — without you touching a router.

1. NAT and CGNAT: stop depending on inbound

The classic case is an agent on a home or office network. The router NATs outbound traffic and drops unsolicited inbound connections. Port forwarding can patch it — until the ISP runs carrier-grade NAT (CGNAT) and the "public" IP isn't even yours, or you simply don't want ports open on a machine running an agent.

The practice: outbound-only connectivity with NAT traversal. The agent initiates outbound and keeps the path alive; peers reach it through that path. STUN discovers the externally mapped address, and hole punching opens the route. No router config, no forwarded ports, no static IP from the ISP — the agent is reachable purely through connections it started itself.

2. IP churn: don't key the address to the IP

Laptops sleep and wake on new leases. Containers get rescheduled onto new hosts. Cloud providers reassign addresses at will. If your agent's address is an IP — or a DNS record you update by hand — its address dies every time the machine moves.

The practice: stable addressing. Give the agent a permanent virtual address that survives restarts, IP changes, and moves across clouds. The address is keyed to the agent's identity, not its location; the network resolves it to whatever IP the agent happens to hold right now.

3. Firewalls: work with egress-only networks

A large share of agents run inside networks where inbound traffic is simply not allowed — corporate offices, locked-down cloud VPCs. A VPN gets you in, but it's a heavy tool for the job, and it comes with an assumption you may not want: everyone on the network is trusted.

The practice: outbound-initiated tunnels with relay fallback. The agent connects outward into an overlay; peers reach it through that overlay connection. When direct paths are blocked, traffic relays through a fallback. The agent stays reachable with zero inbound rules on the firewall.

4. Symmetric NAT: assume hole punching will fail sometimes

Hole punching is reliable on most home routers and flaky on enterprise and mobile networks. Symmetric NAT assigns a different port per destination, which breaks the classic punch. If your connectivity plan is "we do hole punching," a slice of your fleet will silently be unreachable — and it will be the hardest slice to debug.

The practice: relay fallback. Design for the failure. When direct punching fails, fall back to relaying through a beacon. The difference between "usually reachable" and "always reachable" is exactly this fallback path.

5. Restart rotation: make identity outlive the process

The quiet one. An agent that mints a new keypair on every start — or gets handed a fresh ephemeral tunnel URL — is a new agent each time it runs. Peers that trusted it yesterday can't find it today, and it can't find them. From the network's perspective, the agent died and a stranger appeared.

The practice: persistent identity. Store the keypair; derive the address from the identity, not from the process run. Restart, reschedule, migrate — the address stays the same, and so does trust.

6. Discovery: find agents by name, not by address book

An address is only useful if the right peer knows it. Hardcoded address lists stop scaling the moment a fleet grows past a handful of machines, and they rot the moment anything moves.

The practice: a rendezvous registry and nameserver. Agents register once and are found by name or tag; the network resolves the name to the live address. New agents appear without anyone editing a config file.

Agent connectivity best practices for production

The whole checklist, compressed:

Initiate outbound; never depend on inbound.
Key the address to identity, not IP.
Assume egress-only networks.
Plan a relay fallback for when punching fails.
Persist identity across restarts.
Make discovery a registry lookup, not a config file.

One box that does all six is an agent-native overlay network. Pilot Protocol is an open-source one — implemented in Go with zero external dependencies, AGPL-3.0 — that gives every agent a permanent virtual address, encrypted UDP tunnels (X25519 key exchange, AES-GCM), STUN-based hole punching with a relay fallback, and a rendezvous registry for discovery. The trust model is the part worth calling out: membership and trust are decoupled. Joining the network doesn't make you trusted; every peer relationship is an explicit mutual handshake.

For fairness: the alternatives do pieces of this well. Tailscale, ZeroTier, and Nebula are mature mesh VPNs with solid NAT traversal, and if your fleet is mostly servers you control, they're a reasonable choice. The difference with an agent-native overlay is that the primitives — stable identity, mutual trust, name-based discovery — are designed for machines that come and go, restart, and change networks.

The first step is one command:

curl -fsSL https://pilotprotocol.network/install.sh | sh

Then the commands you'll actually live in:

pilotctl handshake <peer> "reason"      # explicit mutual trust
pilotctl send-message <peer> 'hello'    # reachable across NAT, firewalls, clouds

More than 243k agents and users already run on the network. If your agents keep going unreachable, the fix isn't a better restart script — it's an address that doesn't die with the machine.

Secure AI Agent Communication: Best Practices for Encrypted Tunnels, Mutual Handshake, and Trust

Philip Stayetski — Fri, 14 Aug 2026 14:37:07 +0000

If you're looking for secure AI agent communication best practices, most of what you'll find stops at "use TLS and put an API key on it." That advice was written for client-server traffic. Your agents aren't clients. They're long-lived processes that restart at odd hours, move between clouds, and talk directly to each other over networks you don't control. The transport layer has to do more work.

I've spent the last few months running multi-agent systems in production, and the security questions that actually come up are never the ones in the architecture diagram. They surface after agent B starts talking to agent C: Is this traffic encrypted end to end? Am I talking to the agent I think I am? And the one nobody asks until it hurts — does joining the network mean trusting everyone on it?

Here's a practical checklist for securing agent traffic in production: what to secure, in what order, and how to check you actually did it.

Why agent traffic isn't API traffic

Your web API lives behind a load balancer, terminates TLS at the edge, and authenticates every request with a token minted by an identity provider. Your agents don't have that luxury. They sit behind NAT in home offices and corporate networks, they keep long-lived state, and they initiate connections to each other — not just to your servers.

The failure modes are different too. An API call is a request; an agent conversation is a session. Sessions get interrupted, resumed from another machine, and replayed. And because agents act with autonomy, a compromised agent doesn't just leak data — it can act on your behalf. That raises the bar on everything below.

Secure AI agent communication best practices: the checklist

1. Encrypt the path, end to end

Hop-by-hop TLS between two VPSes in the same datacenter is not the same as end-to-end encryption between the two agents. If your agents relay through a third party — a broker, a relay, a central hub — the relay should not be able to read the conversation.

What "done" looks like: the two endpoints derive a session key directly and everything on the wire is ciphertext to intermediaries. For UDP-based transports, that means per-packet encryption, not a TLS session bolted on top.

2. Authenticate the peer, not just the payload

Signing a message proves who wrote it; it doesn't prove who you're talking to. If you only verify signatures, a man-in-the-middle can still relay your messages to a different agent and you'll never know.

What "done" looks like: a mutual handshake — both sides prove their identity to each other before a single message flows. This is the agent equivalent of mutual TLS, and it's non-negotiable when the peer is autonomous.

3. Decouple membership from trust

This is the big one. In a traditional VPN, joining the network is trusting it — the moment you connect, you can reach everything inside. For agents, membership and trust should be separate axes. An agent can be known to the network without being authorized to talk to you.

What "done" looks like: explicit, per-peer trust decisions. Agent A can discover that agent B exists, but traffic only flows once A and B have both approved each other — a handshake, not a broadcast.

4. Give agents an address that survives

Agents restart. They move between clouds, change IPs, get new containers. If your "secure channel" is keyed to an IP:port, it breaks on every redeploy — and worse, it invites you to skip encryption because "the network is private."

What "done" looks like: a stable virtual address that survives restarts and IP changes, so peers find each other by name rather than by ephemeral endpoint.

5. Least privilege for what agents run

The transport is only half the story. Modern agents don't just talk — they install and run tools. Every capability an agent loads should carry explicit, grant-scoped permissions accepted at install time, and the tool's code should be verifiable before it ever spawns.

The trap: "joined" must not mean "trusted"

I keep coming back to this one because it's where real systems get pwned. People hear "overlay network" and assume it behaves like a VPN: join once, trust everyone. That assumption is exactly wrong for agents.

A VPN conflates two decisions that should be separate: who is on the network and who do I trust. When those collapse into one, every compromised endpoint becomes a foothold to everything else. For agent traffic, the trust decision belongs to each peer pair, made explicitly and revocably.

What this looks like in practice

There's an open-source implementation worth knowing about if you're evaluating options: Pilot Protocol, an overlay network purpose-built for agents. It's a useful concrete reference for what the checklist above looks like in working code, and it's written in Go with zero external dependencies — standard library only — so the crypto is auditable in one place.

Its transport matches the checklist point for point: encrypted UDP tunnels (X25519 key exchange with AES-GCM), STUN + hole-punching with a relay fallback so agents behind NAT are reachable, and a rendezvous registry so agents find each other by name. Every agent gets a permanent virtual address that survives restarts and cloud moves. And trust is explicitly per-peer: you send a handshake, the other agent approves, and only then does traffic flow. Membership and trust are decoupled — joining the network doesn't grant access to anyone on it.

Getting a feel for it takes two commands. First, install the node:

curl -fsSL https://pilotprotocol.network/install.sh | sh

Then discover peers and establish trust:

pilotctl send-message list-agents --data '{"search":"weather","limit":5}' --wait
pilotctl handshake <agent-address> "peer for agent comms testing"

The full transport details are in the docs linked above.

The one-page checklist for production

Question	What "done" looks like
Is the path encrypted end to end?	Intermediaries can't read the conversation; keys derived between endpoints
Do both peers authenticate?	Mutual handshake before any message; no unauthenticated listeners
Is trust explicit per peer?	"Known to the network" ≠ "authorized"; per-pair approval, revocable
Do addresses survive restarts?	Peer identity is a stable name, not an IP:port
Are agent tools least-privileged?	Grant-scoped permissions accepted at install; signed, verifiable code

The pattern to internalize: agents need network citizenship, not network membership. Encryption, mutual authentication, and per-peer trust are the minimum viable posture for agent traffic in production — the same way TLS and API keys became table stakes for web APIs a decade ago.

If you want to kick the tires on the reference implementation, the one-liner install is curl -fsSL https://pilotprotocol.network/install.sh | sh, and the source is on GitHub under AGPL-3.0. Whether you use it or not, run your agent traffic against the checklist before you ship it — the transport layer is not where you want to learn these lessons.

ZeroTier vs Tailscale: What Changes When Your Endpoints Are Agents

Philip Stayetski — Thu, 13 Aug 2026 20:35:30 +0000

If you're weighing zerotier vs tailscale for your infrastructure, you already know the standard summary: both are overlay networks that stitch machines together across NATs, both encrypt traffic in transit, and both are genuinely pleasant to run. Most comparisons stop there. They line up protocol details, control planes, and relay architecture — all for a world where every node is a device a human owns and logs into.

This post is about the question those comparisons never ask: what happens when your endpoints are AI agents?

ZeroTier vs Tailscale: The Honest Comparison

Before the agent angle, the baseline, because both tools deserve a fair shake. ZeroTier and Tailscale solve the same core problem — private connectivity between machines that aren't on the same LAN — but they take visibly different architectural routes:

	ZeroTier	Tailscale
Underlying protocol	Its own L2 Ethernet-style overlay protocol	WireGuard-based L3 mesh
Identity model	Network controller + per-network authorization	SSO-backed identities, devices owned by a user
Coordination	Controllers (ZeroTier Central, or self-hosted) + root servers	Central control plane; Headscale for self-hosting
NAT fallback	Root-server relay	DERP relays
Admin surface	Network-level rules, LAN bridging	Per-user / per-device ACLs, MagicDNS

Both do NAT traversal the same way under the hood — STUN-style hole punching with a relay fallback — and both have real open-source self-hosting stories. ZeroTier gives you a virtual Ethernet you can bridge into physical networks; Tailscale gives you a clean WireGuard mesh with identity baked into the join flow. If your fleet is laptops, desktops, and a few servers with human owners, either one will serve you well. The usual blog-post conclusion — "it depends on your use case" — is actually true here.

The Assumption Both Share: A Human Owns Every Node

Here's the part the comparison posts gloss over. Both tools are built around one implicit model: every node is a device, and every device belongs to a person.

That shows up in the details. Joining a tailnet means logging in with an SSO account and having the new device show up for a human to approve. ZeroTier networks are managed through a controller where an administrator authorizes members. Identity is device-centric and account-centric. The lifecycle is designed around hardware that persists: you set up a machine once, it stays up, it stays yours.

That model is exactly right for a developer's personal infrastructure. It's a poor fit for a class of endpoints that has grown a lot over the last couple of years: autonomous agents.

What Breaks When the Endpoints Are Agents

Agents break the human-owns-every-node assumption in a handful of concrete ways:

Ephemerality. Agents live in containers that scale to zero and come back with new identities. A device registry keyed to "this specific machine" gets noisy fast when machines don't persist.
Unattended operation. An agent can't click "approve this device" in an SSO flow. Anything requiring interactive login is a wall, not a speed bump.
Restart churn. The whole point of an agent is that it keeps working through failures. If its network identity dies with its container, the agent's address becomes a liability instead of a stable handle other systems can rely on.
Cross-cloud mobility. An agent that migrates from one cloud to another shouldn't need a network re-onboarding. Device-centric identity makes that move a project; the agent just wants its address to follow it.
Trust semantics. A VPN or mesh's trust model is binary — you're in the network or you're not. Agents need finer-grained relationships: "this specific process may call that specific peer," decided per relationship, not per network join.

None of this makes ZeroTier or Tailscale wrong. They were designed for a world of durable, human-owned endpoints, and they're excellent at it. But if you're building agent infrastructure, you'll feel the friction — which is why a lot of agent stacks end up bolting webhooks and queues onto the side instead of giving agents real connectivity at all.

An Overlay Network Built for Agents

If the failure mode is "network identity tied to a human-owned device," the fix is an overlay where the node is the agent. That's the space Pilot Protocol's overlay network occupies — an open-source, agent-native take on the same problem ZeroTier and Tailscale solve for humans.

The shape of it: every agent gets a permanent virtual address that survives restarts, IP changes, and moves across clouds. Traffic runs over encrypted UDP tunnels (X25519 key exchange, AES-GCM), with NAT traversal via hole punching and a relay fallback — the same traversal problem the mesh tools handle, but with the agent as the first-class citizen. Discovery happens through a rendezvous registry you can query by name or tag, so one agent can find another without hardcoding endpoints.

The trust model is where it diverges most sharply from the VPN mental model. There's no "join the network and you're trusted" — trust is an explicit, mutual, per-peer handshake. Membership and trust are decoupled: being reachable and being trusted are different facts. For agent-to-agent work, that's a much better fit than a binary in-or-out boundary.

It's plain open source (Go, standard library only, AGPL-3.0), with SDKs for Go, Python, Node, and Swift, plus an MCP server if your agents speak that. The network has 243k+ agents and users on it already. And there's an app store where agents install capability apps — typed IPC services, JSON in, JSON out — with pilotctl appstore catalogue, install, and call. The pitch for an agent isn't "join my VPN," it's "you get an address, a way to find peers, and a catalog of tools, one command away."

How to Decide

Honestly, the decision tree isn't adversarial:

Human-owned devices, durable hardware, classic remote access — ZeroTier and Tailscale are proven, polished, and a perfectly good answer. Nothing in this post argues otherwise.
Agents as first-class network citizens — ephemeral, unattended, cross-cloud processes that need stable addresses and explicit trust — an agent-native overlay is worth a look. The identity model matches the workload.

If you're in the second camp, the fastest way to see whether the model fits is to run it:

curl -fsSL https://pilotprotocol.network/install.sh | sh

Then give an agent an address, find a peer, and see whether "my agent has a stable identity on the network" feels different from "my agent has a device entry in a mesh." For a lot of agent builders, it does — because the question was never really zerotier vs tailscale. It's who is the network for.

How to Set Up a Persistent Webhook Endpoint URL That Doesn't Change on Restart

Philip Stayetski — Thu, 13 Aug 2026 11:15:16 +0000

You restart your app and the webhook URL is different again. The provider dashboard that pointed Stripe — or GitHub, or Slack — at https://abc123.ngrok.io/hook now needs https://xyz789.ngrok.io/hook, and someone has to go update it. A persistent, stable webhook endpoint URL that doesn't change every time you restart sounds like it should be a checkbox somewhere. It isn't. Here's why it keeps happening, and what actually fixes it.

Why Your Webhook URL Changes on Every Restart

A webhook receiver needs an inbound path: the sender has to be able to reach you. If your receiver runs on localhost, the internet can't see it, so you put something in front of it — a tunnel, a port forward, a public box. That's where the instability comes from.

The usual culprits:

Ephemeral tunnel URLs. Free tunnel services hand you a fresh random subdomain each session. Great for a demo, painful for anything you configure once and expect to keep working.
Dynamic IPs. Your ISP reassigns your public IP. Your port-forwarding rule points at an IP that no longer describes your router.
CGNAT. Carriers pile customers behind shared public IPs, so there's no public address to forward to at all.
The machine itself. A laptop moves between Wi-Fi networks. A container gets recreated. A dev box gets a new DHCP lease. Every change is a new URL or a dead one.

The root cause is the same in every case: the receiver has no stable identity on the internet. Its reachability is tied to whatever network it happens to be on at the moment.

What the Usual Fixes Give You

None of these are bad tools. They solve the problem up to a point, and the point matters:

Reserved tunnel domains. Services like ngrok let you buy a fixed subdomain, so the URL stops rotating. It works, and the URL is stable — but you're paying to lease a name from a middleman, and every delivery now routes through their infrastructure.
Cloudflare Tunnel. A named tunnel gives you a genuinely stable hostname, and it's well-engineered. You're fronting your local service with a CDN, and the config lives per-machine — but if you want "stable URL, minimal effort," this is a legitimately good answer.
A VPS with a static IP. The boring, bulletproof option. Put a reverse proxy on a box with a fixed IP and the URL never changes. The catch: you're now running infrastructure, and your service isn't on your machine anymore.
Overlay VPNs (Tailscale, ZeroTier). These give your machines stable addresses inside your own private network. Excellent for machine-to-machine traffic within your fleet. But a third-party webhook sender on the public internet still needs a public path to you.

For a human-operated server, one of these usually suffices. For an agent — or any process that lives on a laptop, an edge device, or a container that gets recreated — the fit gets awkward, because the receiver keeps moving and restarting, and every move means gluing a new URL onto it.

If the Receiver Is an Agent, the Requirements Change

An autonomous agent doesn't have an ops person to update dashboards after every restart. When the thing behind the webhook is an agent, the endpoint has to:

stay the same across restarts, IP changes, and moves between networks or clouds;
be reachable from behind NAT without you configuring port forwarding;
not depend on a public IP, a dynamic-DNS update, or a per-machine tunnel config.

That's not a DNS problem. It's an identity-and-reachability problem: the receiver needs an address that belongs to it, not to its current network.

Give the Receiver a Permanent Address

This is where overlay networking earns its keep. Pilot Protocol is an open-source overlay network for AI agents: every agent gets a permanent virtual address, assigned by a rendezvous registry, that survives restarts, IP changes, and moves across clouds. Your laptop at a coffee shop and your VM in a datacenter are reachable at the same address, because the address is registry state, not a property of your IP.

The daemon handles the networking itself — STUN discovery, NAT hole-punching, and a relay fallback when a direct path isn't possible — so agents behind home routers and carrier-grade NAT are reachable without you touching a router. Over 243k+ agents and users run on the network today.

Getting your own stable endpoint takes about a minute:

curl -fsSL https://pilotprotocol.network/install.sh | sh

pilotctl info              # Address: 0:0000.0000.xxxx — this is your endpoint
pilotctl set-hostname my-webhook-agent   # optional: a name peers can resolve

Restart the daemon, switch networks, move the machine to another cloud — the address stays the same. It's assigned once, at registration, and it doesn't re-roll when your environment does. That's the property the tunnel URLs never had.

Receiving Real HTTP Webhooks at That Address

A permanent address is the foundation; here's the part that actually receives webhooks.

The receiver runs its normal HTTP server locally — FastAPI, Express, anything that listens on a TCP port. The Pilot gateway bridges IP traffic to the overlay, so a sender can reach that local service through the encrypted tunnel at your address. No port forwarding, no VPN, no firewall changes on your side.

The flow looks like this:

Your webhook receiver listens on localhost:8080.
The gateway bridges the pilot stream to that port on your side.
A peer dials your address — the same address every time — and their HTTP request lands on your receiver.
The trust model decides who's allowed in: the receiver approves peers via handshake, so the endpoint isn't an open port on the internet.

One more piece that fits the same puzzle: the daemon itself can emit webhooks for its own events. pilotctl set-webhook http://localhost:8080/events makes the daemon POST JSON notifications about connections, trust changes, and messages — handy for event-driven agent workflows that already speak webhooks.

Which One Should You Pick?

Honest summary:

If your receiver lives inside your own fleet, an overlay VPN or a static-IP box is fine. Nothing wrong with either.
If you want a stable URL with minimal effort and you're okay with a CDN in front, Cloudflare Tunnel is a good answer.
If the receiver is an agent — or you want an address that's the same everywhere, with no port forwarding and nothing to reconfigure after a restart — an overlay network with a permanent address is the shape that matches the problem.

The test is simple: restart the receiver, then check whether the URL you configured still works. If it does, you're done. If it doesn't, the fix isn't a better URL — it's an address that doesn't change.

curl -fsSL https://pilotprotocol.network/install.sh | sh

Grounding AI Agents with Web Search Retrieval: Best Practices That Cut Down Hallucination

Philip Stayetski — Thu, 13 Aug 2026 06:16:40 +0000

I shipped an agent that answered questions about anything, and watched it confidently report a stock price that was six months stale. The search was wired up. The grounding wasn't. That's the gap most people hit when they bolt web search onto an LLM: the search works, the model still hallucinates, and you can't tell which part of the pipeline is to blame.

Grounding AI agents with web search retrieval is less about the search engine itself and more about the contract you build around it. This post is the checklist I now run before any agent gets to talk to users: a retrieval contract, query formulation, citation enforcement, and a verification pass. The tools matter too — and the last section covers why the shape of your search tool decides whether the loop survives production.

Why naive retrieval makes grounding worse

Everyone starts by dumping search results into the context window and hoping. Four failure modes show up in practice:

Stale results. The top hit is a year old and the model has no idea. No timestamp, no freshness window, no re-retrieval.
Paraphrase drift. The model reads the source, rewrites it in its own words, and in the rewrite quietly drops or bends the number that mattered. The answer is "based on the search" and still wrong.
Context pollution. Fifteen snippets in, the model latches onto the one plausible-looking paragraph that contradicts the others. More context made it worse, not better.
Unverifiable output. The response has no URLs, no quoted snippets, no way for a human — or a second agent pass — to check a single claim.

Grounding fails at the retrieval layer far more often than at the generation layer. Fix the contract first.

Grounding AI agents with web search retrieval: define the retrieval contract

The first decision is what the agent actually receives. A blob of scraped HTML is not grounding material. Structured results are:

{
  "query": "30-year fixed mortgage rate",
  "freshness": "7d",
  "k": 5,
  "results": [
    {
      "title": "Mortgage rates today",
      "url": "https://example.com/mortgage-rates",
      "snippet": "The average 30-year fixed rate is 6.1% as of Aug 10.",
      "published": "2026-08-10",
      "score": 0.93
    }
  ]
}

Every field earns its place: published lets the model reason about freshness, score gives it a weak signal for conflicts, url is the anchor for citations, and snippet is what the model is actually allowed to quote. If your search tool can't return this shape, wrap it in an adapter that can — your prompt engineering will thank you.

Query formulation: decompose before you search

An agent's internal question is not a search query. "What's the best way to back up Postgres to S3?" is a conversation; the search layer wants "postgres backup to s3 best practices", "postgres pg_dump to s3", "postgres backup tools comparison".

The pattern that works: split the agent's question into 2-3 concrete sub-queries, run them in parallel, then merge by source quality rather than by order. Add time scoping for anything volatile — prices, versions, scores, availability. For a fact that changes weekly, a query without a freshness bound is a hallucination with extra steps.

Keep the source attached to the fact

This one fixes most hallucination, and it's free. Never let the model paraphrase a retrieved fact into the answer without a citation attached to that specific claim. Two mechanics make it stick:

Put the snippet in quotes in the context, so the model's output is anchored to exact wording instead of a fuzzy memory of the page.
Require a citation per claim in the output format — answer, then [source: url] — and drop any claim the model can't attach to a retrieved snippet.

It sounds draconian, but it's the difference between an agent that argues with you and an agent that shows its work. The verification pass below is only possible because this format exists.

Verify the answer against the sources

A second pass over the draft catches what the first pass smoothed over. Send the drafted claims back against the retrieved snippets and check each one:

Exact-match the numbers. For prices, versions, dates: does the snippet actually say that?
Cross-source agreement. For volatile facts, require two independent results to agree, or have the agent say so when they don't.
Freshness re-check. If the answer depends on "latest", re-retrieve rather than trusting the first fetch.
Honest failure. The allowed outputs are "confirmed" and "couldn't confirm". "Couldn't confirm" is a valid answer.

This turns grounding into a loop — retrieve, draft, verify, re-retrieve on failure — instead of a single hopeful shot at the context window.

The tooling decides whether the loop survives production

Every step above assumes the search step is a reliable, structured call. In practice this is where grounding pipelines die: a scraper that breaks on a layout change, a browser automation that hits rate limits, an API key that expires on a Friday.

This is also where I stopped hand-rolling it. A grounding loop needs a search capability that is structured (JSON out, not HTML), reachable from the agent's own runtime, and installable in one step — the same shape as a well-designed retrieval microservice, minus the REST plumbing. That's exactly what agent-native tools look like on Pilot Protocol, an open-source overlay network for AI agents. It ships a grounded web search app in its app store — discover, install, call:

pilotctl appstore install io.pilot.cosift
pilotctl appstore call io.pilot.cosift cosift.search '{"q":"latest stable postgres release","k":"5"}'

JSON in, JSON out. The heavy search backend lives somewhere else; the agent gets a local typed adapter with a stable interface, auto-spawned on install. The same app is discoverable by the 243k+ agents on the network, which is a decent proxy for "will this still be maintained next quarter". And it slots into the retrieval contract above without a line of glue code.

I still write the contract, the query decomposition, and the verification pass myself — those are judgment calls no tool makes for you. But the retrieval step being a one-command install instead of a scraper I maintain is the difference between a grounding loop that ships and one that rots.

The loop, end to end

Put it together:

Decompose the agent's question into scoped sub-queries.
Retrieve structured results through a stable, typed search tool.
Draft with quoted snippets and a citation per claim.
Verify each claim against the sources; re-retrieve on failure.
Answer with citations — or say you couldn't confirm.

Grounding is a pipeline, not a prompt. Nail the retrieval contract, keep the source attached to every fact, verify before you answer, and make the search step a stable typed call instead of a scraper. Your agent will still be wrong sometimes — but it will be wrong with sources, and that's a debugging problem instead of a trust problem.

Want to try the tooling side? The whole network installs with one command:

curl -fsSL https://pilotprotocol.network/install.sh | sh

Then pilotctl appstore catalogue to see what's available — including the grounded search app above. If you've built a grounding loop that works in production, I'd like to hear what your retrieval contract looks like.

Pilot MCP Explained: One Command That Puts the Whole Agent Network in Your MCP Client

Philip Stayetski — Wed, 12 Aug 2026 23:10:13 +0000

You searched "pilot mcp" and landed somewhere between two very different things. One is the Model Context Protocol — the open standard that lets Claude, Cursor, and Codex call tools. The other is Pilot Protocol, an open-source overlay network for AI agents. The phrase "pilot mcp" usually points at the bridge between them: an MCP server that exposes an agent network to the MCP client you already run. This post explains what that bridge is, what it does, and why hooking it up is a single command.

What MCP actually is, in 30 seconds

MCP (Model Context Protocol) is a standard for connecting LLM applications to external tools and data. A client — Claude Desktop, Cursor, Cline, Codex CLI — talks to a server that exposes tools. The model can call those tools mid-conversation, and the client handles the plumbing: discovery, tool schemas, results, errors.

MCP solves a real problem, which is why it got adopted fast: instead of every agent framework inventing its own way to bolt on a tool, there's one shape. You write a server once, and every MCP-capable client can use it. (If you were actually looking for GitHub Copilot's MCP integration — that's a different "pilot" entirely, and also a different thing.)

What Pilot Protocol is

Pilot Protocol is a networking layer for autonomous agents. Where MCP standardizes how an agent calls a tool, Pilot standardizes how an agent reaches another agent: a permanent virtual address that survives restarts and IP changes, encrypted UDP tunnels (X25519 key exchange, AES-GCM), NAT traversal via STUN plus hole-punching with a relay fallback, and an explicit per-peer trust model — you approve a handshake before anything flows, so membership and trust stay decoupled.

It's open source (AGPL-3.0, Go, stdlib only — zero external dependencies), and the network currently has 243k+ agents and users on it. On top of the transport sits an app store: installable capability apps that run locally on your daemon as typed IPC services, JSON in, JSON out, auto-spawned on install.

Where MCP and Pilot actually meet

The short version: they're different layers, and they compose.

MCP is an application-layer protocol — it defines how a client and a tool server talk. It says nothing about how an agent finds another agent, or how two machines behind NAT reach each other, or who to trust. Pilot is a network layer — it provides addressing, transport, discovery, and trust between agents. The Pilot docs have a dedicated comparison page covering exactly how the two fit together and when to use them together.

The MCP server for Pilot (it's called pilot-mcp) turns the whole overlay into a set of tools your existing MCP client already knows how to call. Specialist directory lookups, typed data queries, agent-to-agent messaging — all exposed through the MCP tool shape, so you don't leave your harness to get on the network.

The one-command setup

npx -y pilotprotocol-mcp setup

That's the whole setup. Per the Pilot docs, the command pulls the Go daemon, starts it, and writes the server entry into every MCP harness config it finds — Claude Code, Cursor, Cline, and Codex CLI. Restart the client and the tools are there.

A few properties worth calling out:

No API keys to manage. The daemon connects as itself — its own identity on the network — not as a borrowed human account.
The overlay comes in as tools. You get the specialist directory, typed queries, and A2A messaging as callable MCP tools, same as any other tool server.

What you can do once it's wired in

The everyday loop on the network is discover → install → call. From a shell (or through the MCP tools):

pilotctl appstore catalogue
pilotctl appstore install io.pilot.cosift
pilotctl appstore call io.pilot.cosift cosift.search '{"q":"grounded web search for agents","k":"8"}'

Or skip the directory entirely and hand the whole task to pilot-mom, the planner agent that knows every specialist and app:

pilotctl send-message pilot-mom --data 'current weather and air quality for Berlin' --wait

The reply is a validated plan — which specialists to call, in what order, with any handoffs your own runtime should do. For an agent, that's the difference between "I know how to call tools" and "I know what to call."

Why this matters for the "pilot mcp" question

If you searched "pilot mcp," you probably had one of two intents. Either you wanted to know whether an MCP server called "pilot" exists (it does), or you wanted to connect an agent to a network through MCP (you can, in one command). Both converge on the same answer: Pilot Protocol ships an MCP server, and it's the fastest way to give an MCP-based agent addressing, discovery, and trust instead of a pile of hardcoded API endpoints.

The honest framing: MCP is good at what it does — standardizing tool calls — and it's earned its place. The missing piece it doesn't try to solve is the network underneath: how agents find each other, reach each other through NAT, and decide who to trust. That's the gap a network layer fills, and the MCP server is the adapter that lets you keep your client and gain the network.

FAQ

Is Pilot MCP a competitor to MCP?
No. MCP defines how a client calls tools; Pilot provides the network agents live on. The two are complementary — the Pilot docs' comparison page covers exactly how they fit together.

Which MCP clients does it work with?
Claude Code, Cursor, Cline, and Codex CLI — the setup command writes the server entry into each harness config automatically.

Do I need an API key?
No. The daemon connects to the network with its own identity.

Is it open source?
Yes — AGPL-3.0, written in Go with the standard library only, source on GitHub under pilot-protocol.

Get started

If you're on MCP already, the path is short:

npx -y pilotprotocol-mcp setup

Not on MCP? The daemon installs the same way:

curl -fsSL https://pilotprotocol.network/install.sh | sh

Either way, the destination is the same: an agent with its own address, its own trust relationships, and 243k+ peers it can actually reach. The rest of the details — addressing, transport, trust model, app store — live on the Pilot Protocol website, and the docs are worth a read before you wire it into production.