DEV Community: Muralidharan Deenathayalan

What Is AgentGateway? The AI-Native Gateway, Explained for Newbies and Pros

Muralidharan Deenathayalan — Fri, 03 Jul 2026 04:29:41 +0000

What Is AgentGateway? The AI-Native Gateway, Explained for Newbies and Pros

Spend a week building with AI agents and you hit the same wall I did. The moment there's more than one agent, model, or tool in play, nothing is actually in charge of the traffic moving between them.

Two protocols took a swing at the chaos. MCP (Model Context Protocol) standardizes how an agent talks to tools; think of it as a USB-C port for tools. A2A (Agent-to-Agent) standardizes how agents hand work off to each other. Both are genuinely useful. The problem is that they only describe how the messages are shaped. They say nothing about who's allowed to call what, what it costs you, or how you'd debug any of it in production. No authentication, no authorization, no spend control, no audit trail.

AgentGateway is the layer that fills that gap. This is Post 1 of the series, so I'll keep it grounded: what AgentGateway actually is, the four modes it runs in, and the two bits of jargon you'll trip over everywhere else, data plane and control plane.

The short version

AgentGateway is an open-source, Rust-based proxy that sits between your apps and everything they talk to (LLMs, tools, and other agents) and adds the security, governance, and observability the raw protocols leave out.

A few things worth knowing before we go deeper:

It's open source under Apache 2.0, and it's written in Rust, which matters here because a lot of agent traffic rides on long-lived connections that a slower proxy would choke on.
It's hosted by the Linux Foundation and recently joined the Agentic AI Foundation (AAIF).
It speaks MCP and A2A natively, but it also handles ordinary HTTP and gRPC. So it isn't a bolt-on "AI sidebar" living next to your real infrastructure.

That last point is really the whole pitch. One gateway for your AI traffic and your normal API traffic, instead of running and securing two separate things.

The four modes

I find the easiest way to hold AgentGateway in your head is "three agentic patterns, plus the boring traditional one." But before the picture of it working, look at the picture of life without it.

With no gateway, every app wires straight to every backend. Each one carries its own API keys, its own auth, its own retry logic, its own logging, all of it duplicated, and nothing anywhere has a full view of what's calling what. With N apps and M backends, you've built yourself an N × M tangle:

AgentGateway collapses that mesh down to a single governed front door. Same proxy, same policies, four kinds of backend behind it:

1. LLM mode (agent to LLM)

This is the mode most teams reach for first. You point your apps at one OpenAI-compatible endpoint, and AgentGateway deals with the mess behind it: spreading requests across OpenAI, Anthropic, Gemini, Bedrock, or whatever model you're self-hosting, and failing over when one of them falls down.

The load balancing is smarter than round-robin. It uses Power of Two Choices (P2C): it samples two providers and sends the request to whichever looks healthier on latency and pending load, so a struggling provider quietly sheds traffic on its own. On top of that you get token-based rate limits per user, team, or key. That's your guard against what folks have started calling "denial-of-wallet," a runaway agent loop chewing through your budget while you sleep.

Here's a minimal two-provider config, OpenAI and Gemini behind a single route:

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
binds:
- port: 3000
  listeners:
  - routes:
    - backends:
      - ai:
          groups:
          - providers:
            - name: openai
              provider:
                openAI:
                  model: gpt-3.5-turbo        # optional; overrides the model in requests
              backendAuth:
                key: "$OPENAI_API_KEY"
            - name: gemini
              provider:
                gemini:
                  model: gemini-1.5-flash-latest
              backendAuth:
                key: "$GEMINI_API_KEY"

Your app code never changes. It keeps calling a normal /chat/completions endpoint, and the failover, key handling, and cost attribution all happen at the gateway.

2. MCP mode (agent to tool)

Picture an agent that needs GitHub, a database, and a search API. The obvious move is to wire it directly to all three MCP servers. AgentGateway lets it talk to one MCP endpoint instead, and federates the servers behind that.

The security payoff is per-session tool filtering. A given client only ever sees the tools it's allowed to use, which is where RBAC and defenses against tool-poisoning attacks come into play.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
binds:
- port: 3000
  listeners:
  - routes:
    - policies:
        cors:
          allowOrigins: ["*"]
          allowHeaders: ["*"]
          exposeHeaders:
            - "Mcp-Session-Id"   # required so sessions persist across calls
      backends:
      - mcp:
          targets:
          - name: mcp
            mcp:
              host: http://localhost:3005/mcp/

Worth a note for anyone who's run gateways before, because this is exactly where the older ones fall over. MCP is stateful. A client and server hold a long-lived session, messages travel both directions, and one "list tools" call can fan out to several backends and come back aggregated. AgentGateway pins each session to a backend and packs the session state into the Mcp-Session-Id (encrypted with AES-256-GCM), so every follow-up call lands back on the same server. A gateway designed around stateless request/response simply can't model that, which is why so many of them buckle the first time you point real MCP traffic at them.

3. A2A mode (agent to agent)

Agent A hands a long-running task to Agent B. A2A defines the shape of that handoff, but not the governance around it: who's allowed to delegate to whom, and how you trace the thing once it's running. Put AgentGateway in the middle and you get authentication, authorization, and end-to-end tracing across that boundary. Those happen to be the parts A2A leaves entirely up to you.

4. HTTP/gRPC mode (the unified part)

This is the mode people forget, and it's arguably the reason adopting AgentGateway pays off at all. Your plain REST and microservice traffic runs through the same gateway, with the same load balancing, timeouts, retries, TLS, and authorization you've configured everywhere else. You can even take an existing REST API and expose it as an MCP tool. No standing up a separate "AI gateway" beside your "real" one. It's one proxy and one policy set covering both.

Data plane vs. control plane

These two terms turn up in every gateway and service-mesh doc going, and they reliably trip people up. Here's the version that finally stuck for me.

Picture a restaurant:

The data plane is the floor staff and the kitchen. They take every order and carry every plate, handling each table (each request) in real time.
The control plane is the manager in the back office. They never serve a table themselves. They write the menu, set the rules, decide which section each server covers, update the specials, then push all that out to the floor.

Mapped onto the product: AgentGateway itself is the data plane, the Rust proxy doing the real-time work on every request, from routing to auth to rate limiting to tracing. The control plane is whatever configures those proxies. AgentGateway accepts config dynamically over an xDS interface with no downtime, and it conforms to the Kubernetes Gateway API. On Kubernetes, the recommended control plane for spinning up and managing AgentGateway proxies is kgateway.

If you remember one sentence, make it this: the data plane handles and secures every request in real time, and the control plane decides the rules and pushes them down. You can run AgentGateway standalone with a static config.yaml (the snippets above are exactly that) or let a control plane drive it inside Kubernetes. Same proxy underneath, either way.

Key terms, in one place

Term	What it means here
MCP	Model Context Protocol. How agents call tools and data. Stateful, JSON-RPC.
A2A	Agent-to-Agent. How agents delegate tasks to each other.
Data plane	The proxy that moves and inspects every request. AgentGateway itself.
Control plane	The component that configures the proxies, e.g. kgateway on Kubernetes.
xDS	The interface for pushing config to the proxy live, with no restart.
P2C	Power of Two Choices. The load-balancing strategy across LLM providers.
Tool federation	Exposing many MCP servers through one endpoint, filtered per client.
Denial-of-wallet	A cost attack via runaway token usage. Stopped with token rate limits.

Where this is going

So that's the idea. AgentGateway gives your agents, models, tools, and ordinary APIs one governed front door, running in four modes (LLM, MCP, A2A, HTTP/gRPC), with a clean split between the proxy that does the work and the control plane that sets the rules.

Next time I'll get hands-on: install the binary, run the LLM config from above, and actually watch failover and spend limits kick in. Bring a couple of API keys.

Understanding DORA Metrics: A Beginner's Guide to Measuring DevOps Performance

Muralidharan Deenathayalan — Sat, 02 Nov 2024 09:38:03 +0000

DORA (DevOps Research & Assessment) metrics consist of essential measurements used to evaluate a software development team’s performance. Developed by Google’s DORA team over six years, these metrics are based on insights gathered from more than 31,000 professionals worldwide.
The key metrics are below.

1. Deployment Frequency (DF)

2. Lead Time for changes (LT)

3. Mean Time To Recovery (MTTR)

4. Change Failure Rate (CFR)

Why DORA metrics are important?

Helps you to predict software delivery performance
It focuses on Outcomes and not on Outputs
It is language and technology agnostic
Helps you to identify improvement areas
Let's start look into each metric in detail.

1. Deployment Frequency (DF)

Deployment Frequency measures how often code changes are deployed to production, reflecting the agility of the development team in delivering updates, new features, and bug fixes. This metric directly indicates the team’s ability to bring value to end-users quickly and is essential for assessing the efficiency of the deployment process. This can be daily, daily, weekly, or monthly. This metrics gives insight on how frequently the team can deliver incremental improvements to users.

Why DF is important ?
The higher value of Deployment Frequency shows that the team can deliver new features, bug fixes, and enhancements will be quick and consistent. Frequent deployments suggest a team can respond quickly to customer feedback, market changes, critical bug fixes and security vulnerabilities.

How to Calculate?
To determine this metric value, count the total number of deployments made into production for a given time period and this can be daily, weekly, or monthly.
Performance Levels

Elite: Teams in this category perform at the highest level of efficiency, deploying multiple times per day. This level indicates a highly automated and well-maintained deployment pipeline with minimal or no friction.

High: Teams deploying between once per day to once per week. High-frequency deployments demonstrate the team’s capability to roll out updates regularly without needing daily releases.

Medium: Teams deploying between once per week to once per month. This level suggests some stability in the deployment process but may indicate manual steps or longer testing cycles that slow down deployment.

Low: Teams deploying less than once per month or higher. This low value indicates that the team uses more traditional approach, with longer release cycles. There are opportunities for increasing automation, testing speed, or process improvement.

2 .Lead Time for Changes (LT)

Lead Time for Changes measures the time it takes for code changes to progress from the first commit to production deployment. This metric reflects the team's ability to respond to business needs effectively and deliver updates quickly.

Why it is Important?
A shorter Lead Time for Changes indicates that the team can address business requirements, bug fixes, customer feedback, and market changes more swiftly.

How to Measure ?
To calculate this metric, determine the median time from code commit to production deployment. Using the median minimizes the influence of outliers, providing a clearer view of typical lead times.
Performance Levels:

Elite: Less than one hour. This level indicates a highly optimized process with efficient pipelines and minimal bottlenecks.

High: Between one day and one week. Teams at this level balance speed with quality, demonstrating a well-managed release process.

Medium: Between one week and one month. This level reflects a stable process with potential for improvement, such as reducing manual steps or dependencies that may be causing delays.

Low: More than one month. higher lead times at this level highlight areas for improvement, including opportunities for enhanced automation, dependency reduction, or more streamlined testing.

3. Mean Time to Recovery (MTTR)

Mean Time to Recovery (MTTR) measures the average time required to restore service after an incident is detected or reported. This metric indicates the team's effectiveness in managing and resolving issues promptly.

Why it is important?
A shorter MTTR reflects a team’s readiness and capability to manage unexpected failures, minimizing downtime and its impact on users and the business. Lower recovery times demonstrate a robust incident management process and enhance reliability.

How to Calculate?
To calculate MTTR, find the average time taken from the identification of a problem to the resolution or fix. This average provides a realistic view of the typical recovery time, helping teams identify and reduce bottlenecks in their incident response process.

Performance Levels

Elite: Less than one hour. This level represents a highly effective and streamlined incident response process, ensuring rapid recovery.

High: Less than one day. Teams at this level demonstrate solid recovery processes, restoring service within a reasonable timeframe.

Medium: Less than one week. This level indicates a moderate response time, but there may be opportunities to improve processes and reduce dependencies causing delays.

Low: More than one week. Extended recovery times suggest a need for process improvements, such as refining detection, response procedures, or streamlining incident management workflows.

4. Change Failure Rate (CFR)

Change Failure Rate (CFR) measures the percentage of code changes that result in service degradation or require remediation. This metric indicates the stability and reliability of the deployment process.

Why it is important?
A lower Change Failure Rate reflects a stable and mature delivery process with minimal disruptions caused by new deployments. CFR helps identify areas for improvement, guiding teams to enhance testing, deployment practices, and quality assurance.

How to Calculate?
To calculate CFR, divide the number of failed changes by the total number of changes and multiply by 100. This percentage shows the likelihood of failures occurring with each deployment, highlighting process quality.

Performance Levels

Elite: 0–15%. This level demonstrates an exceptionally stable process, with most changes deployed smoothly and minimal need for rollbacks or fixes.

High: 16–30%. Teams at this level maintain a strong process, though occasional issues may arise that need attention.

Medium: 31–45%. This level suggests that while deployments are generally stable, there is room for process and quality improvements to reduce failures.

Low: 46% and above. High failure rates indicate the need for significant improvements in testing, quality control, or deployment practices to enhance stability.

Summary:

By understanding and optimizing Deployment Frequency (DF), teams can better align their practices with DevOps principles, promoting agility, rapid feedback, and faster delivery of value.

Lead Time for Changes (LT) is crucial for aligning development speed with business agility, helping teams deliver high-impact updates faster.

Mean Time to Recovery (MTTR) is a key metric for measuring reliability and responsiveness, helping teams focus on improving resilience and reducing downtime.

Change Failure Rate (CFR) is a key indicator of delivery reliability and supports continuous improvement efforts by reducing disruptions caused by deployments.

DEV Community: Muralidharan Deenathayalan

What Is AgentGateway? The AI-Native Gateway, Explained for Newbies and Pros

What Is AgentGateway? The AI-Native Gateway, Explained for Newbies and Pros

The short version

The four modes

1. LLM mode (agent to LLM)

2. MCP mode (agent to tool)

3. A2A mode (agent to agent)

4. HTTP/gRPC mode (the unified part)

Data plane vs. control plane

Key terms, in one place

Where this is going

Further reading

Understanding DORA Metrics: A Beginner's Guide to Measuring DevOps Performance

1. Deployment Frequency (DF)

2. Lead Time for changes (LT)

3. Mean Time To Recovery (MTTR)

4. Change Failure Rate (CFR)

1. Deployment Frequency (DF)

2 .Lead Time for Changes (LT)

3. Mean Time to Recovery (MTTR)

4. Change Failure Rate (CFR)