When you need an Agent Gateway, not just another LLM proxy

Victor García — Sun, 23 Aug 2026 04:11:23 +0000

When you need an Agent Gateway, not just another LLM proxy

If your problem is "one OpenAI-compatible endpoint for 50 providers," a proxy like LiteLLM is usually the right default. Huge catalog, huge community, good enough ops for most teams.

If your problem is "agents that call tools (MCP) with the same identity, audit, budgets, and policy as model calls," a thinner Agent Gateway starts to matter — one control plane so auth doesn't drift between chat completions and tool traffic.

Honest fit table (no fake benches)

Need	Usually start with	Why
Broadest provider catalog, ecosystem glue	LiteLLM	Depth beats novelty
Hosted control plane, zero ops	Portkey / Helicone-class	Ops off your plate
Self-host, low overhead Go proxy	Bifrost / similar	Latency + governance focus
Self-host LLM + MCP same policy/identity	Agent Gateway (e.g. TrustGate)	One plane for models + tools
K8s/service-mesh native	Envoy AI Gateway / Kong	Fits existing mesh

What "Agent Gateway" should mean

OpenAI-compatible path for apps you already have
MCP / tool gateway on the same identity + audit trail
Fail-closed budgets / rate limits before the vendor 429s you
Self-hostable so prompts stay in your perimeter

Consumers + registries (the MCP aggregation plane)

In practice the useful model is:

Registries hold upstream MCP servers and model providers (keys stay server-side).
Consumers get their own credentials into the gateway (per app / team / agent).
tools/list is filtered per consumer so agents never see tools they cannot call.
The same consumer identity covers chat completions and tools/call, so spend and audit line up.

That is the wedge: not "another 100 providers," but one aggregation plane for models + tools.

Try TrustGate (OSS, Apache-2.0)

Repo: https://github.com/NeuralTrust/TrustGate

Built in Go by NeuralTrust. Quickstart and "Why TrustGate" live in the README — including where it is not the right pick vs LiteLLM/Helicone/Portkey.

Disclosure: TrustGate / NeuralTrust DevRel.

Agent Gateway in 60 Seconds: Governed LLM Traffic with TrustGate

Victor García — Sat, 22 Aug 2026 17:20:35 +0000

Agent Gateway in 60 Seconds: Governed LLM Traffic with TrustGate

Most teams start with a direct OpenAI (or Anthropic) SDK call. That works until you have three apps, two providers, and a security review asking who can call which model, at what rate, with what audit trail.

An AI / Agent Gateway sits between your apps (and agents) and every upstream model or tool. One place for routing, policy, and observability — without rewriting clients.

TrustGate is NeuralTrust’s open-source, security-first Agent Gateway (Go). It fronts LLM APIs and MCP tool servers behind Admin / Proxy / MCP planes, so you change a base URL and two headers instead of scattering keys and rate limits across services.

What you get in one binary

Plane	Port	Job
Admin	`:8080`	Gateways, registries, consumers, auth, policies
Proxy	`:8081`	OpenAI-compatible LLM traffic + plugins
MCP	`:8082`	Aggregated Model Context Protocol endpoint for agents (e.g. Cursor)

Policies (rate limit, token rate limit, request size, semantic cache, CORS) run in the data path before traffic hits providers.

60-second bring-up

Requires Docker (and git). From a clean machine:

curl -fsSL https://raw.githubusercontent.com/NeuralTrust/TrustGate/main/scripts/install.sh | bash

Or, if you prefer the explicit path:

git clone https://github.com/NeuralTrust/TrustGate.git && cd TrustGate
cp .env.example .env
make up

Health checks:

curl localhost:8080/healthz   # Admin
curl localhost:8081/healthz   # Proxy
curl localhost:8082/healthz   # MCP

First governed chat completion

With TrustGate up and OPENAI_API_KEY set, the repo ships a script that creates a demo gateway, registers OpenAI, mints a consumer key, and sends a chat completion:

export OPENAI_API_KEY="sk-..."
./examples/curl-first-request/first-request.sh

From an app, keep the OpenAI SDK — point it at the proxy:

from openai import OpenAI
import os

client = OpenAI(
    base_url="http://localhost:8081/my-app",  # /{consumer_slug}
    api_key="unused",  # provider key lives in the gateway registry
    default_headers={
        "X-AG-Gateway-Slug": "demo",
        "X-AG-API-Key": os.environ["CONSUMER_API_KEY"],
    },
)

print(
    client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "Hello through TrustGate"}],
    )
    .choices[0]
    .message.content
)

Full example: examples/openai-sdk/.

Bonus: one MCP endpoint for agents

Agents shouldn’t each hold a private mesh of tool servers. TrustGate’s MCP plane (:8082) aggregates registered MCP targets behind one endpoint with the same tenancy and policy model. Cursor setup notes live in examples/mcp-cursor/.

When TrustGate is the right fit

Choose a gateway when you need central policy (who / what / how fast), multi-provider routing with fallback, and increasingly MCP aggregation for agents — not only request logs.

If you mainly want a thin multi-provider proxy, LiteLLM is a strong developer default. If you mainly want analytics, Helicone leads with observability. TrustGate’s wedge is governance and security in the data path, including MCP.

Docs: docs.neuraltrust.ai · Repo: github.com/NeuralTrust/TrustGate

If this was useful, the repo is here — stars help others find it.

Disclosure: TrustGate / NeuralTrust DevRel.

DEV Community: Victor García

When you need an Agent Gateway, not just another LLM proxy

When you need an Agent Gateway, not just another LLM proxy

Honest fit table (no fake benches)

What "Agent Gateway" should mean

Consumers + registries (the MCP aggregation plane)

Try TrustGate (OSS, Apache-2.0)

Agent Gateway in 60 Seconds: Governed LLM Traffic with TrustGate

Agent Gateway in 60 Seconds: Governed LLM Traffic with TrustGate

What you get in one binary

60-second bring-up

First governed chat completion

Bonus: one MCP endpoint for agents

When TrustGate is the right fit