DEV Community

Victor García
Victor García

Posted on

Agent Gateway in 60 Seconds: Governed LLM Traffic with TrustGate

#ai #mcp #devops #opensource

Agent Gateway in 60 Seconds: Governed LLM Traffic with TrustGate

Most teams start with a direct OpenAI (or Anthropic) SDK call. That works until you have three apps, two providers, and a security review asking who can call which model, at what rate, with what audit trail.

An AI / Agent Gateway sits between your apps (and agents) and every upstream model or tool. One place for routing, policy, and observability — without rewriting clients.

TrustGate is NeuralTrust’s open-source, security-first Agent Gateway (Go). It fronts LLM APIs and MCP tool servers behind Admin / Proxy / MCP planes, so you change a base URL and two headers instead of scattering keys and rate limits across services.

What you get in one binary

Plane Port Job
Admin :8080 Gateways, registries, consumers, auth, policies
Proxy :8081 OpenAI-compatible LLM traffic + plugins
MCP :8082 Aggregated Model Context Protocol endpoint for agents (e.g. Cursor)

Policies (rate limit, token rate limit, request size, semantic cache, CORS) run in the data path before traffic hits providers.

60-second bring-up

Requires Docker (and git). From a clean machine:

curl -fsSL https://raw.githubusercontent.com/NeuralTrust/TrustGate/main/scripts/install.sh | bash
Enter fullscreen mode Exit fullscreen mode

Or, if you prefer the explicit path:

git clone https://github.com/NeuralTrust/TrustGate.git && cd TrustGate
cp .env.example .env
make up
Enter fullscreen mode Exit fullscreen mode

Health checks:

curl localhost:8080/healthz   # Admin
curl localhost:8081/healthz   # Proxy
curl localhost:8082/healthz   # MCP
Enter fullscreen mode Exit fullscreen mode

First governed chat completion

With TrustGate up and OPENAI_API_KEY set, the repo ships a script that creates a demo gateway, registers OpenAI, mints a consumer key, and sends a chat completion:

export OPENAI_API_KEY="sk-..."
./examples/curl-first-request/first-request.sh
Enter fullscreen mode Exit fullscreen mode

From an app, keep the OpenAI SDK — point it at the proxy:

from openai import OpenAI
import os

client = OpenAI(
    base_url="http://localhost:8081/my-app",  # /{consumer_slug}
    api_key="unused",  # provider key lives in the gateway registry
    default_headers={
        "X-AG-Gateway-Slug": "demo",
        "X-AG-API-Key": os.environ["CONSUMER_API_KEY"],
    },
)

print(
    client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "Hello through TrustGate"}],
    )
    .choices[0]
    .message.content
)
Enter fullscreen mode Exit fullscreen mode

Full example: examples/openai-sdk/.

Bonus: one MCP endpoint for agents

Agents shouldn’t each hold a private mesh of tool servers. TrustGate’s MCP plane (:8082) aggregates registered MCP targets behind one endpoint with the same tenancy and policy model. Cursor setup notes live in examples/mcp-cursor/.

When TrustGate is the right fit

Choose a gateway when you need central policy (who / what / how fast), multi-provider routing with fallback, and increasingly MCP aggregation for agents — not only request logs.

If you mainly want a thin multi-provider proxy, LiteLLM is a strong developer default. If you mainly want analytics, Helicone leads with observability. TrustGate’s wedge is governance and security in the data path, including MCP.

Docs: docs.neuraltrust.ai · Repo: github.com/NeuralTrust/TrustGate

If this was useful, the repo is here — stars help others find it.

Disclosure: TrustGate / NeuralTrust DevRel.

Top comments (0)