DEV Community: fcn06

Unifying Agent Orchestration and Model Gateways in Rust: Solving the Stateless Turn Problem

fcn06 — Wed, 09 Sep 2026 19:35:57 +0000

While developing multi-agent systems and LLM integrations in Rust, I found myself repeatedly confronting two related architectural challenges.

The first was an operational divide between two layers of infrastructure: a lightweight model gateway for routing and caching LLM requests, and an agent orchestration runtime responsible for tool execution, planning, and task resolution. Teams often end up deploying two separate systems for these workloads, duplicating provider configurations, authentication layers, and connection pools.

The second challenge was the "stateless turn" problem: many agent loops initialize each execution turn from scratch ([system_prompt, current_user_message]), immediately discarding previous tool outputs and conversational turns. While simple, this makes multi-turn reasoning brittle and deprives agents of contextual continuity.

This article shares how we approached these challenges around a unified Tokio-based architecture, and how we recently implemented an opt-in, four-tier context engine to give agents persistent conversational memory without sacrificing backward compatibility.

The Dual-Mode Architectural Pattern

Instead of maintaining a separate proxy daemon and an agent orchestrator, the system is organized around two complementary operational modes sharing the same core runtime:

+--------------------------------------------------------------------------------------------------+
|                                        RUNTIME MODES                                             |
+--------------------------------------------------------------------------------------------------+
|                                                                                                  |
|   MODE 1: MULTI-AGENT & MCP ORCHESTRATION               MODE 2: MODEL GATEWAY SERVER             |
|   (kickstart/multi_agent_orchestration_kickstart/)      (kickstart/gateway_kickstart/)           |
|                                                                                                  |
|   • Planner Agent (Dynamic DAG generation)              • POST /v1/chat/completions (OpenAI API) |
|   • Executor Agent (Workflow DAG execution)             • POST /v1/responses (Open Responses)    |
|   • Domain Specialists with native MCP Tools            • Stateful multi-turn chaining           |
|   • Pluggable Context & Memory Services                 • Multi-provider routing (Groq, Gemini,  |
|   • Evaluation & Self-Correction (Judge)                  OpenAI, Ollama, vLLM, llama.cpp)       |
|   • Agent-to-Agent (A2A) protocol contracts             • High-throughput lock-free cache         |
|                                                                                                  |
+--------------------------------------------------------------------------------------------------+

Mode 1: Agent Orchestration via Model Context Protocol (MCP)

Mode 1 coordinates specialized agents through explicit message contracts:

Planner Agent: Analyzes high-level requests and constructs a directed acyclic graph (DAG) of tasks.
Executor Agent: Traverses the DAG, resolving dependencies and dispatching step execution.
Domain Specialists: Execute tools using the Model Context Protocol (MCP), handling tools over standard I/O, Server-Sent Events (SSE), or streamable HTTP transports.
Evaluation Service: An optional LLM-as-a-Judge validation step that assesses tool outputs and prompts the agent for correction if the result is malformed.

User Request ──> Planner ──> Task DAG ──> Executor ──> Specialist Agent ──> MCP Tool ──> Evaluation ──> Response

Mode 2: OpenAI-Compatible Model Gateway

Mode 2 acts as a standard inference gateway:

/v1/chat/completions: Drop-in compatibility with existing OpenAI SDKs, IDE extensions, and client pipelines.
/v1/responses: Stateful chaining that correlates turns using explicit response IDs.
Unified Provider Routing: TOML-configured routing across commercial APIs (Groq, Google Gemini, OpenAI) and local runtimes (Ollama, vLLM, llama.cpp).

Sharing a runtime means both modes share the same connection pooling, HTTP client configuration, secret management, and tracing telemetry.

Two Deployment Patterns: Start Unified, Let Usage Decide

To accommodate different operational requirements, we implemented two concrete deployment patterns:

"Full Swarm" (Agents + MCP + Gateway): Runs the entire orchestration stack — planner, DAG executor, domain specialists with MCP tool servers, and the model gateway — in a single unified deployment. This provides an end-to-end autonomous environment where agents talk to LLMs directly through the local gateway with zero network hops, sharing connection pools and in-process memory.
"Gateway-Only Swarm": Strips away agent orchestration and runs purely as a high-performance inference gateway. It handles /v1/chat/completions, provider failover, caching, and stateful response chaining (/v1/responses) for external client applications, developer tools, or existing third-party agent stacks.

The underlying philosophy is pragmatic: start unified, and let actual production usage patterns tell you if they should stay glued. If your inference proxying traffic grows 100x faster than agent orchestration, or if security requirements dictate isolating live tool execution behind private networks while keeping the gateway at the ingress, the gateway can be peeled off into an independent service without modifying client contracts or configuration formats.

The Problem with Stateless Agent Loops

During early iterations, our MCP agent loop followed a common pattern:

// A common pattern in basic agent runtimes
let messages = vec![
    Message {
        role: "system".to_string(),
        content: Some(system_prompt),
    },
    user_message,
];

Every user interaction started with a fresh message array. While clean and deterministic for single-shot question answering, this architecture has distinct failure modes:

Session Amnesia: Follow-up questions fail because prior exchanges and tool outputs vanish once the request terminates.
Write-Only Logging: Memory traits are frequently defined with only a log() method, recording conversation logs for telemetry while providing no retrieval mechanism back to the agent.
Prompt Coupling: Ad-hoc attempts to inject history often result in manual string concatenation in the HTTP handler, coupling transport logic to prompt formatting.

Designing a Pluggable 4-Tier Context Engine

To address this cleanly, we recently restructured the context architecture into four distinct operational tiers:

┌────────────────────────────────────────────────────────────────────────┐
│                        CONTEXT TAXONOMY                                │
├────────────────────────────────────────────────────────────────────────┤
│ 1. Conversational Working Context (Multi-turn session & thread state)  │
│ 2. Long-Term & Semantic Memory (User preferences, key facts, recall)   │
│ 3. Sovereign Identity & Security Context (Roles, clearance, metadata)  │
│ 4. Procedural & Skill Knowledge (Domain runbooks, safe tool sequencing)│
└────────────────────────────────────────────────────────────────────────┘

Rather than baking assumptions about prompt format or storage backends directly into the core execution loop, we introduced a ContextProvider abstraction into the commons layer:

#[async_trait]
pub trait ContextProvider: Send + Sync {
    /// Enrich the prompt or messages before the LLM thinking phase
    async fn enrich_context(
        &self,
        request: &ContextRequest,
        messages: &mut Vec<Message>,
    ) -> Result<()>;

    /// Post-turn hook to record dialogue or extract persistent facts
    async fn on_turn_complete(
        &self,
        request: &ContextRequest,
        user_message: &Message,
        assistant_message: &Message,
    ) -> Result<()>;
}

How the Execution Pipeline Works

When a request arrives, McpAgent executes a two-phase context pipeline:

User Request (with session_id & metadata)
          │
          ▼
┌────────────────────────────────────────────────────────┐
│              CONTEXT ASSEMBLER PIPELINE                │
│                                                        │
│  1. Base System Prompt                                 │
│  2. Identity Provider   ──> Injects identity block     │
│  3. Skill Provider      ──> Injects tool runbooks      │
│  4. Fact Memory Provider──> Recalls relevant facts     │
│  5. History Provider    ──> Loads sliding window turns │
│  6. Current User Turn                                  │
└──────────────────────────┬─────────────────────────────┘
                           │
                           ▼
           ┌───────────────────────────────┐
           │      McpAgent Run Context     │
           │      (Enriched Message List)  │
           └───────────────┬───────────────┘
                           │
                           ▼
           ┌───────────────────────────────┐
           │    McpAgent Execution Loop    │
           │ Thinking ──> Executing Tools  │
           └───────────────┬───────────────┘
                           │ (Assistant Response)
                           ▼
┌────────────────────────────────────────────────────────┐
│                   POST-TURN FLUSH                      │
│  • Persist User & Assistant turns to MemoryService     │
│  • Invoke `on_turn_complete` on all context providers  │
└────────────────────────────────────────────────────────┘

Sliding Window History (HistoryContextProvider):
The provider queries MemoryService::get_conversation(session_id, limit), fetching the last $N$ turns (configurable via TOML) and injecting them between the system prompt and the latest user turn. Older turns can be safely compacted or truncated without blowing past LLM context limits.
Persistent Fact Recall (SemanticMemoryContextProvider):
A dedicated db_facts store exposes store_fact and recall_facts. Before LLM execution, facts matching the user's query or category are recalled and attached as structured context.
Identity & Security Context (IdentityContextProvider):
Injects caller metadata, tenant identification, active permissions, and clearance level into an <identity_context> block, ensuring the model stays aligned with authorization boundaries.
Zero-Breaking-Change Guarantee:
To ensure existing deployments remain stable, all methods added to the MemoryService trait provide default no-op implementations. If an agent does not configure memory services or context providers, it continues operating in the original stateless mode with zero performance regression.

Why Rust for Multi-Agent & Gateway Infrastructure?

Rust is well-suited for this specific intersection of workloads for several reasons:

Concurrency Without Global Locks: Shared state across concurrent sessions (such as DashMap-backed memory tables and cached tool schemas) operates cleanly across Tokio worker threads without coarse mutex contention.
Low Latency & Predictable Footprint: When proxying high-token streaming requests or managing complex tool calls, predictable memory ownership avoids garbage-collection pause spikes.
Compile-Time Protocol Validation: Using strongly typed structs for A2A and MCP message boundaries surfaces serialization and schema mismatches at compile time rather than midway through an automated workflow.

Configuration Example

Enabling context memory in an agent runtime is handled declaratively in the configuration file:

[agent_mcp]
agent_mcp_model_id = "openai/gpt-oss-20b"
agent_mcp_llm_url = "https://api.groq.com/openai/v1/chat/completions"
agent_mcp_system_prompt = "You are a specialized customer domain agent."

# Context Engine Settings
agent_mcp_history_length = 10
agent_mcp_enable_memory_recall = true
agent_mcp_enable_identity_context = true

Trade-offs and Open Questions

In designing this system, we encountered several trade-offs that have no single "correct" answer:

Sliding Window vs. Recursive Summarization: A fixed sliding window is fast, deterministic, and adds zero LLM cost. However, long-running threads inevitably drop older details. A recurring compactor that periodically condenses earlier turns into a narrative summary solves this, but incurs additional latency and inference cost.
Key-Value Fact Storage vs. Vector Embeddings: For local and edge environments, key-value fact matching (by key, tag, and query substring) avoids the operational burden of running an embedding model or vector database. For large-scale unstructured corpora, however, vector embeddings remain necessary.
Glued vs. Decoupled Deployments: Providing both "Full Swarm" and "Gateway-Only" patterns allows teams to start simple and observe traffic patterns empirically. Rather than guessing the ideal microservice boundary on day one, letting actual load, blast radius considerations, and operational ownership dictate whether the gateway and orchestrator remain glued has proved much less painful than premature decomposition.

We would be glad to hear from developers building agent systems or LLM gateways:

How do you manage conversation history compaction across extended agent workflows?
Do you prefer combining proxy routing and agent execution into a single binary, or keeping them decoupled across services?

Project & Code

If you are interested in exploring the implementation, testing the multi-agent kickstarts, or examining the MCP runtime contracts, the project is open-source under Apache-2.0:

👉 Swarm on GitHub

Contributions, critiques, and discussions are welcome.

The Agent Economy: Why Agents Must Negotiate Agreements, and How It Rewrites Integration.

fcn06 — Wed, 09 Sep 2026 17:25:02 +0000

The pitch for autonomous AI agents has quietly shifted.

A year ago, agentic AI meant giving an LLM access to your local bash terminal or a calculator. Today, the conversation is about the agent economy: autonomous agents from different companies discovering one another, negotiating terms, ordering goods, and settling transactions at machine speed.

That sounds great on a slide deck. But look at how people actually build cross-enterprise agent integrations today, and you'll find a massive disconnect.

Most current implementations treat the "agent economy" as nothing more than handing an LLM an API key to a counterparty's REST endpoint, wrapping it in a system prompt, and crossing their fingers. If Company A's agent wants to buy something from Company B, either human engineers spend three months writing bespoke integration code, or Company B gives Company A's model raw access to an internal tool and prays prompt injection doesn't drain their inventory.

Neither model works.

Real economies don't run on hardcoded API calls, and they certainly don't run on blind faith in a counterparty's system prompt. Real economies run on negotiated agreements between sovereign parties who don't control each other.

If we want a genuine agent economy, agents must be able to negotiate agreements with other agents. And to do that safely, we need to understand what agents should negotiate — and what they must never be allowed to touch.

The Problem: APIs Are Static, But Business Is Dynamic

Today, enterprise B2B integration is painfully rigid:

Company A (Custom Code) ──[Months of Specs & Mapping]──> Company B (REST API)

Company B has to document endpoints, schemas, authentication scopes, webhooks, and rate limits. Company A has to hire engineers to read all of it and write bespoke glue code. Even when both use standard formats like JSON or OpenAPI, their business semantics almost never match. (Does "cancel order" mean voiding the PO immediately, or does it mean submitting a cancellation request subject to supplier review?)

Agents change the equation because they can reason about intent:

Instead of calling POST /api/v3/orders with twenty hardcoded parameters, Company A's agent can say:

"I need 500 industrial bearings delivered to our Lyon warehouse before next Friday. Target budget is under €15,000."

It's tempting to think: "Great! LLMs replace APIs. Agents will just chat with each other in natural language and do business."

That would be a catastrophe.

Two probabilistic models chatting freely across company boundaries can hallucinate prices, misinterpret delivery terms, or fall prey to prompt injection. A natural language chat is not a business contract.

The breakthrough isn't replacing APIs with LLM conversations. The breakthrough is using agents to negotiate a machine-readable agreement at runtime, while keeping the underlying APIs as the deterministic execution engine.

Enter the Interaction Contract

Imagine Company A's procurement agent discovers Company B's sales agent. Instead of firing arbitrary requests, the two agents engage in a structured negotiation to establish an Interaction Contract — a shared, machine-readable definition of their relationship for this task:

┌────────────────┐      Propose Capabilities & Terms       ┌────────────────┐
│   Company A    │ ───────────────────────────────────────> │   Company B    │
│   B2B Agent    │ <─────────────────────────────────────── │   B2B Agent    │
└────────────────┘         Counter-Offer & Agreement        └────────────────┘
                                    │
                                    ▼
                     ┌─────────────────────────────┐
                     │    Interaction Contract     │
                     │  (Signed, Hashed, Versioned)│
                     └─────────────────────────────┘

A negotiated contract might look like this:

purpose: supplier_parts_procurement
counterparties:
  buyer: did:web:company-a.com:agent-procure
  seller: did:web:company-b.com:agent-sales

capabilities:
  - quote
  - create_order
  - track_shipment
  - cancel_order

constraints:
  max_order_value:
    currency: EUR
    amount: 15000
  geography:
    delivery_destination: EU

cancellation:
  allowed_until: dispatch
  penalty: 0

validity:
  expires_at: "2026-10-01T00:00:00Z"

Notice what just happened:

Zero custom glue code was written in advance. The agents dynamically discovered each other's capabilities and negotiated mutually agreeable terms.
The relationship is strictly bounded. Company A's agent cannot suddenly execute a €50,000 order or deliver to an unsupported jurisdiction; the contract rules it out.
It's a bilateral agreement, not a unilateral permission list. Unlike a static API token or OAuth scope, an Interaction Contract binds both parties to shared constraints, settlement rules, and validity windows. Both sides cryptographically sign it.

The Critical Rule: Agents Negotiate Meaning, Not Authority

Here is the load-bearing architectural principle that makes this entire model work:

Agents negotiate meaning and operational terms. They do NOT decide execution authority.

An LLM is fantastic at reconciling differences. It can negotiate whether payment is in EUR or USD, map Company A's shipping_address to Company B's destination_facility, or agree on a cancellation window. That is semantic negotiation.

But an agent should never have the power to approve its own authority. If Company A's agent and Company B's agent agree on a €50,000 transaction limit, but Company A's internal corporate policy says automated agents are capped at €10,000, enterprise policy must always win.

An agent's effective authority is always the strict intersection of three layers:

┌────────────────────────────────────────────────────────┐
│                  EFFECTIVE AUTHORITY                   │
│                           =                            │
│   Negotiated Contract  ∩  Enterprise Policy  ∩  Identity │
└────────────────────────────────────────────────────────┘

Negotiated Contract (Bilateral): What did both agents agree to? (e.g., max €15,000).
Enterprise Policy (Local & Deterministic): What does your company allow right now? (e.g., max €10,000, EU hours only).
Agent Identity (Cryptographic): Is the calling agent who it claims to be, backed by a verified did:web and active keys?

Negotiation can only shrink the permitted operational space. It can never expand beyond what deterministic enterprise policy permits.

Where the Trust Gateway Fits In

Once the two agents negotiate and sign an Interaction Contract, how do we prevent either agent from deviating from it during execution?

This is where a deterministic control plane — what we call a Trust Gateway — sits behind each organization's agent.

Each party runs their own gateway at their enterprise boundary. It treats both external counterparties and internal LLMs as untrusted actors.

The architecture cleanly separates responsibilities into three distinct roles:

┌────────────────────────────────┐         A2A / Negotiation         ┌────────────────────────────────┐
│           COMPANY A            │ ◄───────────────────────────────► │           COMPANY B            │
│  [ B2B Buyer Agent ]           │                                   │  [ B2B Seller Agent ]          │
│          │ (Proposes Action)   │    ┌─────────────────────────┐    │          │ (Proposes Action)   │
│          ▼                     │    │  Interaction Contract   │    │          ▼                     │
│  [ Trust Gateway A ] ──────────┼──► │ (Signed, Hashed, Shared)│ ◄──┼──────────[ Trust Gateway B ]   │
│          │ (Execution Grant)   │    └─────────────────────────┘    │          │ (Execution Grant)   │
│          ▼                     │                                   │          ▼                     │
│  [ Local Tool Executor ]       │                                   │  [ Local Tool Executor ]       │
│  (Internal ERP / Payment)      │                                   │  (Internal ERP / Inventory)    │
└────────────────────────────────┘                                   └────────────────────────────────┘

Agents Propose: The agent figures out what needs to be done under the negotiated contract and submits a ProposedAction (e.g., "Create order for 500 bearings at €12,500").
The Gateway Authorizes: A deterministic engine written in pure Rust (no LLM, no fuzziness) verifies:
- Does this action match the signed Interaction Contract?
- Does it comply with local enterprise risk rules?
- Is the counterparty's identity authentic?
The Executor Verifies: If approved, the gateway mints a single-use, cryptographically signed Execution Grant (Ed25519) with the canonical hash of the arguments. The executor verifies the grant signature before touching real-world databases, payment rails, or ERP APIs.

If someone prompt-injects the agent mid-session or the model hallucinates an order for €50,000, the proposal hits the gateway, fails the contract verification, and gets rejected on the spot. Zero state change. Zero financial loss.

Sorting Out the Alphabet Soup: MCP vs. A2A vs. Contracts vs. Gateways

With so many agent standards emerging, it's easy to get confused. Here is how they cleanly stack together:

Layer	Protocol / Concept	Primary Role	Analogy
Tool Interface	MCP (Model Context Protocol)	How an agent talks to its internal tools and data sources.	The USB port connecting the computer to peripherals.
Communication	A2A (Agent-to-Agent)	How two agents discover each other and exchange messages.	The telephone line between two offices.
Relationship	Interaction Contract	The negotiated operational agreement defining what the two parties agreed to do.	The signed business agreement between two companies.
Enforcement	Trust Gateway	The deterministic bouncer ensuring actions strictly match the contract and policy.	The legal and compliance department with the key to the vault.

You don't have to choose between MCP, A2A, and secure contracts. They solve completely different layers of the problem:

A2A gives agents the wire protocol to talk.
MCP gives them tools to execute.
Interaction Contracts define what they are allowed to agree on.
Trust Gateways guarantee neither side can break the rules.

Why This Unlocks the Real Agent Economy

The current debate around AI agents is stuck in a false dichotomy:

Camp A (Full Autonomy): "Give the LLM an API key and let it roam the web!" (Terrifying for any CISO or CFO).
Camp B (Zero Autonomy): "Keep humans in the loop for every single button click." (Destroys the efficiency of automated software).

Negotiated agreements backed by deterministic gateways offer the way forward:

Autonomous at runtime: Agents dynamically discover counterparties, map schemas, and negotiate constraints without human engineers writing point-to-point integration code.
Governed by design: Every action is cryptographically tied to a negotiated contract and evaluated against enterprise policy before execution.
APIs don't disappear — the boundary moves up: Your ERP, banking APIs, and inventory systems remain deterministic, reliable, and auditable. What changes is that you no longer need to spend months standardizing every endpoint before two organizations can do business.

The real agent economy won't be built on chatbots clicking buttons. It will be built on sovereign agents negotiating structured agreements, enforced by mathematical and cryptographic guarantees.

We've open-sourced our experiments and reference implementation for the Trust Gateway and Interaction Contracts on GitHub: fcn06/trust_gateway. If you want to dive deeper into the full architecture, threat model, and formal protocols, check out the technical whitepaper: Interaction Contracts for Autonomous B2B Agents.

We'd love to hear your thoughts: in your domain, what parts of a B2B relationship could safely be negotiated by agents at runtime, and what must remain permanently hardcoded?

Stop Giving AI Agents Your Master Password: UCAN Delegation

fcn06 — Sun, 23 Aug 2026 08:02:16 +0000

Why handing a permanent API key to an autonomous AI agent is a bad idea — and a simpler way to grant it just enough trust, for just long enough, using cryptographic "visas" called UCANs.

Status: work in progress. This is an early write-up of something I'm actively building, not a finished product announcement. I'm sharing it specifically to get feedback — on the idea, the explanation, and the approach itself — before taking it further. See the last section for exactly what kind of feedback would help most.

The problem, in one picture

Imagine you hire an intern to add one meeting to your Google Calendar. Would you hand them:

A) Your Google password — which also opens your email, your Drive, and your saved credit cards, or
B) A note that says "You may add one event, today, between 2 and 4 PM. Nothing else."

Everyone picks B for a human intern. Almost nobody picks B for an AI agent.

Today, most AI agent frameworks (LangChain, CrewAI, AutoGen, and friends) are wired up with Option A: a permanent, all-powerful API key pasted straight into the agent's code or environment variables. If that agent gets tricked by a malicious webpage, hallucinates, or simply has a bug, it doesn't just fail — it fails with full permissions. Wiped databases and drained accounts are not hypothetical; they're a Tuesday.

The obvious fix — "just ask the human before every single click" — kills the entire point of having an autonomous agent. Nobody wants to approve every calendar invite by hand.

We wanted a third option: agents that can act on their own, but only within a narrow, time-boxed, cryptographically provable slice of permission. That's what this article walks through.

The idea: give agents a passport, not a master key

Think about how you cross a border. You don't hand a customs officer your entire identity and bank access — you hand them a passport, and inside it, a visa that says exactly which country you can enter, for how long, and for what purpose.

We apply the same idea to AI agents:

A UCAN (User Controlled Authorization Network) is the digital equivalent of a visa: a small, signed piece of proof that says "person X allows agent Y to do Z, until this time."
A Virtual Passport is the little folder an agent carries around, holding all the UCANs (visas) it has been issued by different people or departments.

 ┌──────────────────────────────────────────────────────────┐
 │                AGENT'S VIRTUAL PASSPORT                  │
 │  Agent: "Supervisor Agent #7721"                          │
 │                                                            │
 │  🎫 Visa #1 — issued by Alice                              │
 │     Allowed: create a Google Calendar event                │
 │     Valid for: 10 minutes                                  │
 │                                                            │
 │  🎫 Visa #2 — issued by Finance                            │
 │     Allowed: approve refunds up to $50                     │
 │     Valid for: 1 hour                                      │
 └──────────────────────────────────────────────────────────┘

Each visa is signed with real cryptography (Ed25519 signatures), so nobody can forge one or quietly extend its lifetime. Under the hood, a UCAN is just a small, signed JSON object:

{
  "issuer": "did:twin:alice...",
  "audience": "did:twin:supervisor-agent...",
  "capabilities": [
    { "resource": "google_calendar", "action": "create_event" }
  ],
  "expiry": 1787491200
}

issuer — who is granting the permission (Alice)
audience — who receives it (the agent)
capabilities — exactly what the agent may do, nothing more
expiry — the moment this visa stops working, automatically

No central authority has to revoke anything — the token simply stops being valid. That's the whole trick.

A narrower visa for a narrower job: delegation

Here's the part that makes this genuinely useful for multi-agent systems, not just a single bot: an agent can hand a smaller visa to another agent.

Say Alice asks her "Supervisor" agent to schedule a client meeting. The Supervisor spins up a small "Worker" agent just to touch the calendar. It doesn't hand the Worker its own visa (which might also include refund permissions) — it mints a new, narrower visa, stripped down to exactly calendar: create_event, and nothing else.

This is called attenuation: every time permission is delegated, it can only get narrower, never wider. A worker agent can never end up with more power than its supervisor had. That single rule is what makes it safe to build swarms of agents that spawn other agents without the whole thing turning into a permissions free-for-all.

What actually happens when the agent tries to act

Two things are worth noticing:

The agent never talks to Google Calendar directly with a permanent credential. It shows its visa to a middleman — the Trust Gateway — which checks that everything is legitimate and in-scope, then issues a tiny, single-use pass valid for just 30 seconds.
That 30-second pass is also cryptographically tied to the exact action requested (the specific meeting title and time, hashed). Even if it leaked, it couldn't be replayed or repurposed for a different action.

If an agent ever tries to do something outside what its visa allows — say, a $500 refund when it only has a $50 visa — the request doesn't fail silently or get approved anyway. It gets parked and sent to the human for a real approval (a fingerprint or Face ID tap on their phone), before anything happens.

So the agent gets to act autonomously within its lane, and a human only gets pulled in when something falls outside that lane. That's the balance the "God-mode key vs. approve-everything" dilemma from the start of this article was missing.

Trying it yourself

The gateway that does the checking — the Trust Gateway — is open source and written in Rust. Here's the shortest possible version of the flow above, using its API directly. All three calls below are handled entirely inside the open-source trust_gateway itself — no other service is involved in minting, validating, or checking policy on a token.

Swap in your own gateway's address wherever you see <YOUR-TRUST-GATEWAY-URL> (e.g. http://127.0.0.1:3060 if you're running it locally).

1. Alice mints a visa for her agent, scoped to calendar access only, for one hour:

curl -s -X POST https://<YOUR-TRUST-GATEWAY-URL>/v1/ucan/mint \
  -H "Content-Type: application/json" \
  -d '{
    "issuer": "did:twin:alice...",
    "audience": "did:twin:supervisor-agent...",
    "capabilities": [{ "resource": "google_calendar", "action": "create_event" }],
    "ttl_seconds": 3600,
    "issuer_seed_hex": "<alice_private_seed>"
  }'

This returns a signed UCAN token — the digital visa.

2. Anyone can verify that visa, without contacting Alice, because the proof is self-contained:

curl -s -X POST https://<YOUR-TRUST-GATEWAY-URL>/v1/ucan/validate \
  -H "Content-Type: application/json" \
  -d '{
    "ucan_token": "<the token from step 1>",
    "required_resource": "google_calendar",
    "required_action": "create_event"
  }'

3. The agent proposes the actual action. The Gateway checks the visa and, if everything lines up, mints the short-lived execution pass:

curl -s -X POST https://<YOUR-TRUST-GATEWAY-URL>/v1/actions/propose \
  -H "Content-Type: application/json" \
  -d '{
    "action_name": "google_calendar_create_event",
    "arguments": {
      "summary": "Architecture Review",
      "start_time": "2026-09-01T14:00:00Z"
    },
    "ucan_token": "<the token from step 1>"
  }'

That's it — three calls, entirely served by the open-source gateway, and you've reproduced the whole "mint a scoped permission, verify it, use it once" cycle. The full API also lets an agent prove it holds one specific capability without revealing the rest of its passport (handy if it's carrying visas it shouldn't disclose), and lets you inspect an agent's live passport at any time. Both are just extensions of the same idea. Turning that final execution pass into a real side effect (actually touching Google Calendar) is the one piece that needs something outside the gateway — a downstream tool consumer to carry it out — but the entire token lifecycle and governance you just exercised runs on trust_gateway alone.

Open source

Everything described in this article — UCAN minting and validation, delegation, and execution-grant issuance — lives in the open-source Trust Gateway, written in Rust and runnable locally today:
👉 github.com/fcn06/trust_gateway

The takeaway

Don't hand AI agents permanent, all-powerful API keys. It's the equivalent of giving an intern your master password.
Give them a passport of short-lived, narrowly scoped visas instead — cryptographically signed, automatically expiring, and impossible to widen through delegation.
Check every action at the door, with a lightweight gateway that turns a valid visa into a single-use, single-action execution pass. The agent still acts on its own. It just can't act beyond what it was actually trusted to do — and that trust is provable, not just assumed.

I'd like your feedback

This is still a work in progress, and I'd genuinely like to hear from people who read this — whether you build agents, work in security, or just have an opinion. A few specific things I'm unsure about:

The core idea: Does scoping AI agent permissions with short-lived, delegatable visas (UCANs) actually solve a problem you've run into, or does it feel like overkill for how you use agents today?
The explanation: Was the passport/visa analogy clear, or did it break down somewhere once the delegation and execution-grant parts came in?
The gaps: What's missing that would make you trust this in production — key rotation, revocation before expiry, auditability, something else?
The demo: If you tried the three curl calls yourself, did they work as described? Anything confusing about the request/response shapes? Comments here are the easiest way to reach me, but I'll also take issues or pull requests on the GitHub repo. If you spend five minutes poking holes in this, I'll read every one of them.

You Don't Need to Choose Between a Gateway and an Agent Framework

fcn06 — Fri, 21 Aug 2026 15:32:51 +0000

When I first published Swarm on GitHub, most questions weren't about Rust or MCP. They were about timing and categorization:

"We just need a lightweight gateway for multi-provider routing; agents feel like overkill."

"We already run an orchestration framework; why would we replace our proxy?"

This reaction highlights a false dichotomy currently plaguing the AI infrastructure ecosystem: the assumption that a gateway and an agent orchestrator must be two completely different products.

In practice, teams rarely wake up needing full-blown multi-agent autonomous swarms on Day 1. But when they start with a standalone proxy, they inevitably hit a wall — patching together Python microservices, external vector state stores, MCP bridges, and ad-hoc eval scripts. Every evolution requires a rewrite.

The core premise of Swarm is different: a single, pure-Rust runtime where you don't choose between a gateway and an orchestrator — you simply choose which capabilities to turn on.

The AI Adoption Ladder

Most engineering teams evolve their LLM stack along a predictable trajectory:

Rung 1: OpenAI-Compatible Gateway   (Drop-in replacement for hardcoded SDKs)
  └── Rung 2: Multi-Provider Fallbacks (Groq, Gemini, Ollama, vLLM via TOML)
        └── Rung 3: Stateful Sessions     (Previous response chaining & context)
              └── Rung 4: Native MCP Tools    (SSE + Streamable HTTP tool execution)
                    └── Rung 5: Multi-Agent DAGs  (Planner + Executor + Specialists)
                          └── Rung 6: Built-in Evals    (LLM-as-a-Judge & policy gates)

You can stop at any rung and have a lean, production-grade binary. When you're ready for the next level, you change a configuration flag — not your architectural foundation.

Rung 1 — Just a Low-Latency Gateway

If your immediate goal is simply eliminating hardcoded API keys and single-vendor SDK locks, Swarm acts as an OpenAI-compatible drop-in front door with sub-millisecond native routing overhead.

# Spin up the gateway in seconds
./kickstart/gateway_kickstart/01_launch_gateway.sh

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "Explain progressive disclosure in software."}]
  }'

You get instant OpenAI compatibility. No agent overhead, no background worker queues, no forced abstractions.

Rung 2 — Multi-Provider & Local Model Routing

When rate limits hit or you need cost-effective fallbacks across cloud and local runtimes (Groq, Anthropic, Gemini, Ollama, vLLM, llama.cpp), routing is declared cleanly in config.toml:

[providers.groq]
api_url = "https://api.groq.com/openai/v1/chat/completions"
weight = 80

[providers.local_vllm]
api_url = "http://localhost:8000/v1/chat/completions"
recommended_models = ["meta-llama/Llama-3.3-70B-Instruct"]

Your applications continue calling the same /v1/chat/completions endpoint. Failover, load distribution, and local-inference routing happen invisibly inside the runtime.

Rung 3 — Stateful Conversations via `/v1/responses`

Multi-turn chat state is where teams often bolt on an external Redis or PostgreSQL session manager. Swarm provides explicit turn-by-turn state management natively through /v1/responses using previous_response_id chaining:

curl -X POST http://localhost:8080/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "groq/llama-3.3-70b-versatile",
    "input": "Calculate the Q3 cloud infrastructure spend.",
    "previous_response_id": "resp_01JMW892KPA7XYZ"
  }'

State is managed by the runtime, eliminating client-side conversation bloat while keeping state inspection simple and deterministic.

Rung 4 — Native Model Context Protocol (MCP)

When your model needs real-world context — database schemas, filesystem access, API calls — you shouldn't have to migrate to a heavy agent framework just to call tools.

Swarm natively supports MCP (over both SSE and streamable HTTP) directly inside the gateway layer:

[mcp_servers.postgres_db]
transport = "sse"
url = "http://localhost:3001/sse"

[mcp_servers.git_tools]
transport = "http"
url = "http://localhost:3002/mcp"

Tool discovery, argument validation, and streaming tool execution run within the same engine that routes your completions.

Rung 5 — Coordinated Multi-Agent Workflows

When single-prompt loops cannot solve compound tasks, Swarm activates its autonomous orchestration engine:

User Intent
   │
   ▼
[ Planner ] ──► Builds Execution DAG (Dependencies & Concurrency)
   │
   ▼
[ Executor ] ──► Dispatches tasks across Domain Specialists
   │
   ├── Specialist A (Data Analyst + Postgres MCP)
   └── Specialist B (Report Writer + File MCP)
   │
   ▼
Unified Response

Why this matters: Rung 5 reuses the identical provider configurations, fallback pools, state engine, and MCP tool connectors established in Rungs 1–4. There is no secondary agent daemon or translation bridge.

Rung 6 — Built-in LLM-as-a-Judge Evaluation

The final rung is the one most gateways and agent frameworks omit entirely: closing the loop on quality.

Instead of exporting logs to an external SaaS pipeline, Swarm embeds an LLM-as-a-Judge loop. It scores intermediate DAG outputs, validates MCP tool results against deterministic schemas, and flags hallucinated responses before they reach client applications:

curl -X POST http://localhost:8080/v1/eval/judge \
  -H "Content-Type: application/json" \
  -d '{
    "response_id": "resp_01JMW892KPA7XYZ",
    "criteria": ["correctness", "grounding", "conciseness"],
    "judge_model": "openai/gpt-4o"
  }'

This foundational layer enables our upcoming roadmap items: policy-based dynamic routing, durable state checkpoints, and human-in-the-loop validation gates.

Architectural Coherence Beats Glue Code

The individual capabilities of Swarm — gateway proxying, MCP tool invocation, DAG planning, automated evaluation — exist across different open-source projects.

What is rare is finding them integrated into a single, zero-dependency, memory-safe binary where adopting multi-agent orchestration doesn't invalidate the proxy architecture you set up on Day 1.

The architectural bet of Swarm is simple: the tools you choose when you only need a gateway should never become technical debt the day you need agents.

Discussion

If you're currently scaling your LLM infrastructure:

At which rung on this ladder has your team spent the most engineering time?
Have you had to replace a gateway when moving to agents (or vice-versa)?

Check out the project and try the kickstart scripts on GitHub: github.com/fcn06/swarm