DEV Community: Michael Levan

Managing MCP Servers and Tools With Agentregistry OSS

Michael Levan — Sat, 04 Apr 2026 22:38:18 +0000

Three big topics when it comes to MCP:

How do you know the MCP Server is secure?
Where is it stored?
Is it version-controlled, or can anyone just change it at any time?

And that's where having an MCP registry comes into play.

In this blog post, you'll learn how to securely store your MCP Server, and it's available tools to be used later within your Agents.

Quick Recap: What Is MCP?

Model Context Protocol (MCP) is a spec/standard created by and open-sourced Anthropic. The goal of MCP is to have a server that hosts tools, and these tools are able to implement certain functionality for what you're working on. For example, you can use a Kubernetes MCP Server that can do everything from list/describe/log Pods and deploy objects to Kubernetes. MCP uses JSON-RPC 2.0 for it's communication layer underneath the hood for communication between an Agent (the client) and MCP tools (hosted on the server).

The "Is MCP Dead" Debate

I was at MCPDevSummit in NY this week, and I caught a keynote that explained the need for MCP Server tools pretty nicely from a theoretical perspective. Right now, it may be easier for Agents to talk to MCP Server tools vs having them talk tens or hundreds of APIs directly. The reason why is that it's simpler for an Agent to call a tool and have that tool (because a tool, underneath the hood, is simply a function/method) call the APIs instead. What this could come down to is less tokens used and less context bloat, along with hopefully, better results.

Configuring Agentregistry Locally

With an understanding of what MCP is at a high level, let's dive into the hands-on portion of this blog post. In this section, you'll get agentregistry deployed, which takes around 30 seconds.

Pull down the latest version of agentregistry.

curl -fsSL https://raw.githubusercontent.com/agentregistry-dev/agentregistry/main/scripts/get-arctl | bash

Run the following command, which starts the agentregistry daemon.

 arctl daemon start

You'll see an output similar to the following:

Starting agentregistry daemon...
✓ agentregistry daemon started successfully

Open Docker and you'll see agentregistry running along with a link you can click to reach the UI.

You should now see the agentregistry UI.

Sidenote: if you have a remote registry, you can connect to it with the following:

arctl --registry-url http://YOUR-HOST:12121 version

Adding An MCP Server To Agentregistry

With agentregistry deployed, you can now add an MCP Server to the registry to ensure it's stored and secured. For testing purposes, lets use the filesystem MCP Server that's stored on GitHub.

Using arctl mcp publish, you'll pass in the following flags.
1. MCP Server: server-filesystem
2. Type: NPM
3. Version: 0.6.3

arctl mcp publish io.github.modelcontextprotocol/server-filesystem --type npm --package-id
  @modelcontextprotocol/server-filesystem --version 0.6.3 --description 'MCP server for filesystem access' --git
  https://github.com/modelcontextprotocol/servers.git -v

The MCP Server will now show in your registry.

You can also add your MCP Server via the UI.

Click the purple + Add button and choose Server.

Add in the details about your MCP Server.

Conclusion

Having a safe, secure, and reliable place to store something as prone to security incidents as MCP Servers is key to creating a proper posture for you and your organization when using AI. This is why agentregistry can also be used to store Agent Skills and prompts. Because the majority of what you're using is either a function/method (an MCP Server tool) or .MD files/text files, shadow AI can easily occur.

Running OpenClaw on Kubernetes

Michael Levan — Sat, 14 Mar 2026 17:00:08 +0000

The "new and exciting" way of interacting with Agents is the personal assistant method (from iMessage, WhatsApp, or whatever else) and this interest is taking the industry by storm. OpenAI "bought" OpenClaw, Nvidia is investing in its own version of personal assistants, and several other organizations are trying to figure out how to implement this in production.

The question is - does it run on your infra?

In this blog post, you'll learn how to implement OpenClaw in Kubernetes and observe/secure it with agentgateway.

Prerequisites

To follow along with this blog post from a hands-on perspective, you will need the following:

A Kubernetes cluster running with at least 2 vCPUs and 2–4 GB RAM, though 8 GB RAM and higher are recommended for smoother performance.
agentgateway installed, which you can find here.

Containerization Setup

There are two different ways that you can use OpenClaw in a containerized fashion:

Build your own container image. There's a Dockerfile in the OpenClaw repo which you can find here.
Use a container image that was already built. There is an official Alpine image which you can find here.

The first option will, of course, be the most secure, as you can build the container image yourself and ensure you know what is within the Dockerfile. In air-gapped environments, this would be the ideal setup.

Agentgateway Config To Observe and Secure Agentic Traffic

Once the containerization setup is complete, you can begin the agentgateway setup.

Create a Gateway using the Kubernetes Gateway API CRDs and the agentgateway Gateway class.

kubectl apply -f- <<EOF
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1
metadata:
  name: agentgateway-oc
  namespace: agentgateway-system
  labels:
    app: agentgateway
spec:
  gatewayClassName: agentgateway
  listeners:
  - protocol: HTTP
    port: 8081
    name: http
    allowedRoutes:
      namespaces:
        from: All
EOF

Set an env variable with an Anthropic API key

export ANTHROPIC_API_KEY=

Create a Kubernetes Secret with the API key.

kubectl apply -f- <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: anthropic-secret
  namespace: agentgateway-system
  labels:
    app: agentgateway-oc
type: Opaque
stringData:
  Authorization: $ANTHROPIC_API_KEY
EOF

Create an agentgatewaybackend, which tells the Gateway what to route to. In this case, it's using Anthropic as the LLM Provider.

Please note: When using an ai agentgatewaybackend with the Anthropic provider, agentgateway attempts to parse and re-marshal the request body as a structured LLM message, which fails on OpenClaw's native Anthropic format due to a missing type field in complex message content. Switching to a static backend pointing directly at api.anthropic.com:443 tells agentgateway to forward the request as-is without any LLM-specific processing, while still providing routing, observability, and logging on all traffic. The tls: {} policy is required because api.anthropic.com listens on HTTPS (port 443), and without it, agentgateway sends plain HTTP, which Cloudflare rejects.

kubectl apply -f- <<EOF
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  labels:
    app: agentgateway-oc
  name: anthropic
  namespace: agentgateway-system
spec:
  static:
    host: api.anthropic.com
    port: 443
  policies:
    tls: {}
EOF

Create a route that points to the path /v1/messages, which is the format that Anthropic expects.

kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: ocroute
  namespace: agentgateway-system
spec:
  parentRefs:
    - name: agentgateway-oc
      namespace: agentgateway-system
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /v1/messages
    backendRefs:
    - name: openclaw
      namespace: agentgateway-system
      group: agentgateway.dev
      kind: AgentgatewayBackend
EOF

Deploy OpenClaw On Kubernetes

With the Gateway deployed, let's set up OpenClaw.

Please note: This deployment is just for testing and does not include anything for persistent volumes for data that is not ephemeral. If you want that configuration, you can create a PVC and mount it on /home/node/.openclaw and /home/node/workspace.

Create a ConfigMap to map the configuration that's needed for OpenClaw to route traffic through agentgateway. Please remember you will need to replace the baseUrl with the hostname or IP of your Gateway.

kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: openclaw-agw-config
  labels:
    app: openclaw
data:
  agw-overlay.json: |
    {
      "gateway": {
        "bind": "lan"
      },
      "models": {
        "mode": "merge",
        "providers": {
          "anthropic": {
            "baseUrl": "http://YOUR_AGENTGATEWAY_HOSTNAME_OR_IP:8081",
            "models": []
          }
        }
      }
    }
EOF

Create a Kubernetes Deployment that points to the OpenClaw Alpine container image.

Please note: When deploying OpenClaw in Kubernetes with agentgateway, the openclaw.json config file needs to include the agentgateway baseUrl to route LLM traffic through the gateway. However, OpenClaw auto-generates its base config (including auth tokens and default settings) at startup, and any config modification, from the initial overlay or from running openclaw onboard triggers OpenClaw's built-in hot-reload, which performs a full process restart that kills PID 1 and causes the container to crash. The solution uses a wrapper shell script that pre-creates openclaw.json with the agentgateway overlay before OpenClaw starts

(so initial startup merges cleanly), and runs OpenClaw inside a while true loop so the shell remains PID 1 and automatically restarts OpenClaw whenever a config change triggers its internal restart, preventing the container from exiting.

Please note: The models: [] parameter is required by the schema, but it also causes the a ANTHROPIC_MODEL_ALIASES error. This is a known bug in v2026.3.12. The ANTHROPIC_MODEL_ALIASES error is a temporal dead zone issue that affects any config using an Anthropic primary model. The workaround is to use v2026.3.11 instead. That's why you see that image pinned in the deployment below.

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: openclaw
  labels:
    app: openclaw
spec:
  replicas: 1
  selector:
    matchLabels:
      app: openclaw
  template:
    metadata:
      labels:
        app: openclaw
    spec:
      containers:
        - name: openclaw
          image: alpine/openclaw:2026.3.11
          ports:
            - name: gateway
              containerPort: 18789
              protocol: TCP
            - name: bridge
              containerPort: 18790
              protocol: TCP
          command:
            - sh
            - -c
            - |
              mkdir -p /home/node/.openclaw
              cp /tmp/agw-overlay.json /home/node/.openclaw/openclaw.json
              trap 'kill $(jobs -p) 2>/dev/null' EXIT
              while true; do
                docker-entrypoint.sh node openclaw.mjs gateway --allow-unconfigured
                echo "OpenClaw process exited, restarting..."
                sleep 2
              done
          volumeMounts:
            - name: agw-config
              mountPath: /tmp/agw-overlay.json
              subPath: agw-overlay.json
          resources:
            requests:
              cpu: "4"
              memory: "8Gi"
            limits:
              cpu: "4"
              memory: "8Gi"
      volumes:
        - name: agw-config
          configMap:
            name: openclaw-agw-config
EOF

Create a Service for OpenClaw. This Service will be used in the next section when implementing agentgateway for secure and observable OpenClaw traffic.

kubectl apply -f -<<EOF
apiVersion: v1
kind: Service
metadata:
  name: openclaw
  labels:
    app: openclaw
spec:
  type: ClusterIP
  selector:
    app: openclaw
  ports:
    - name: gateway
      port: 18789
      targetPort: gateway
      protocol: TCP
    - name: bridge
      port: 18790
      targetPort: bridge
      protocol: TCP
EOF

You'll now see that OpenClaw is running.

openclaw-bf55866b7-s7wn6   1/1     Running       0          4m36s

Onboard OpenClaw

For OpenClaw to work, you need to set configurations like how you want to interact with OpenClaw (iMessage, Telegram, etc.) and the LLM Provider you want to use. To do that, you need to run the openclaw onboard command. Because this is running in Kubernetes, you can exec into the Pod.

kubectl exec -it YOUR_OPENCLAW_POD_NAME -n default -- openclaw onboard

You'll see an output similar to the below and you can get started with the onboarding process.

After the onboarding, you can test and ensure that OpenClaw is passing traffic through agentgateway.

kubectl exec OPENCLAW_POD_NAME -n default -- openclaw agent --message "Say hi"

And you'll see traffic routing through agentgateway similar to the below:

2026-03-14T15:41:26.634010Z     info    request gateway=agentgateway-system/agentgateway-oc listener=http route=agentgateway-system/ocroute endpoint=api.anthropic.com:443 src.addr=10.224.0.149:62282 http.method=POST http.host=52.241.254.163 http.path=/v1/messages http.version=HTTP/1.1 http.status=200 protocol=http duration=2936ms

Route and Secure OpenAI Azure Foundry Traffic Through Your AI Gateway

Michael Levan — Tue, 10 Mar 2026 15:21:50 +0000

As you begin to expland into various Agentic frameworks, there's a good chance you will end up choosing the one that exists within the cloud provider you're already using. If you're in Azure, that's Azure Foundry.

The question then becomes "How do I securely route and observe the traffic?".

In this blog post, you'll learn how to route Foundry traffic through a secure, reliable, and performant AI Gateway with agentgateway.

Prerequisites

To follow along with this blog post in a hands-on fashion, you'll need the following:

An Azure account.
Agentgateway installed (OSS), which you can find here.

What Is Microsoft Foundry

Foundry is the Agentic framework within Azure. If you use AWS and have heard of Bedrock before or GCP and have heard of Vertex AI, it's all very similar. They allow you to host Models from various providers (OpenAI, Anthropic, etc.) and connect to those Models from a centralized endpoint with the same API key/token (so you don't have to worry about various keys per provider). Some of the services, like Foundry, also allow you to connect to tools and fine-tune the Models you're working with.

The "tldr" is that it's an Agentic hosting platform to connect to various LLMs.

Azure Foundry Setup

With the knowledge around what Foundry is in place, let's dive into the setup. You'll start with setting up Foundry.

Within the Azure porta, search for foundry.

In the Foundry portal, click the blue + Create button.

Create the Foundry resource within your respective subscription and resource group.

Once Foudnry is created, you'll see a UI similar to the belo. Save the project API key. You'll need it for the next section when you create the Gateway configuration.

Within Foundry, search for gpt-5-mini. Realistically, you can use any Model, but the mini Models will save you some money.

Deploy the Model with the default settings.

With the Model deployed, you will now be able to reach it with agentgateway.

Gateway Configuration

Create an environment variable with the Foundry API key that you saved in the previous section in step 4.

export AZURE_FOUNDRY_API_KEY=

Create a Gateway object listening on port 8081.

kubectl apply -f- <<EOF
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1
metadata:
  name: agentgateway-azureopenai-route
  namespace: agentgateway-system
  labels:
    app: agentgateway-azureopenai-route
spec:
  gatewayClassName: agentgateway
  listeners:
  - protocol: HTTP
    port: 8081
    name: http
    allowedRoutes:
      namespaces:
        from: All
EOF

Save the ALB IP of the Gateway in an environment variable. If you're not using a k8s cluster that can create a public ALB IP, you can use localhost when connecting to the Gateway as long as you port-forward the k8s Gateway svc.

export INGRESS_GW_ADDRESS=$(kubectl get svc -n agentgateway-system agentgateway-azureopenai-route -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
echo $INGRESS_GW_ADDRESS

Create a k8s secret that stores the Foundry API key.

kubectl apply -f- <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: azureopenai-secret
  namespace: agentgateway-system
  labels:
    app: agentgateway-azureopenai-route
type: Opaque
stringData:
  Authorization: $AZURE_FOUNDRY_API_KEY
EOF

The agentgateway backend will tell the Gateway what to route to. In this case, it's the gpt-5-mini Model. You'll also point to the Foundry endpoint.

kubectl apply -f- <<EOF
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  labels:
    app: agentgateway-azureopenai-route
  name: azureopenai
  namespace: agentgateway-system
spec:
  ai:
    provider:
      azureopenai:
        endpoint: mlevantesting-resource.services.ai.azure.com
        deploymentName: gpt-5-mini
        apiVersion: 2025-01-01-preview
  policies:
    auth:
      secretRef:
        name: azureopenai-secret
EOF

The last step is to create a route. Because you're using a GPT Model, the path will be /v1/chat/completions, but you can set a custom route to shorten the path.

kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: azureopenai
  namespace: agentgateway-system
  labels:
    app: agentgateway-azureopenai-route
spec:
  parentRefs:
    - name: agentgateway-azureopenai-route
      namespace: agentgateway-system
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /azureopenai
    filters:
    - type: URLRewrite
      urlRewrite:
        path:
          type: ReplaceFullPath
          replaceFullPath: /v1/chat/completions
    backendRefs:
    - name: azureopenai
      namespace: agentgateway-system
      group: agentgateway.dev
      kind: AgentgatewayBackend
EOF

Test the route to the OpenAI Model via agentgateway. Swap out $INGRESS_GW_ADDRESS with localhost if your Gateway doesn't have a public ALB IP.

curl "$INGRESS_GW_ADDRESS:8081/azureopenai" -v -H content-type:application/json -d '{
  "messages": [
    {
      "role": "system",
      "content": "You are a skilled cloud-native network engineer."
    },
    {
      "role": "user",
      "content": "Write me a paragraph containing the best way to think about Istio Ambient Mesh"
    }
  ]
}' | jq

You should see an output similar to the below.

Intercept, Inspect, Secure: Proxying Claude Code CLI Traffic

Michael Levan — Fri, 20 Feb 2026 12:39:27 +0000

Architecture diagrams always look something like this:

Agent -> Gateway -> LLM (or MCP Server).

The Agents that organizations are typically referring to are Agents that perform an action via prompts by a user or autonomously, and those Agents are usually running in a system somewhere in production. That is, however, not where the majority of Agentic traffic originates. The larger chunk of traffic comes from Agentic clients (Claude Code CLI, Cursor, Copilot, etc.) and because of that, we now must think about Agents not only running in production systems, but on someone's laptop.

In this blog post, you'll learn how to secure that traffic within an Agentic client with agentgateway.

Gateway Configuration

The first thing to ensure is that you have a proper AI Gateway configured so traffic from the Agentic CLI client can get from point A to point B securely. In this case, you can use agentgateway, which is an AI Gateway built from the ground up specifically for AI traffic.

Generate an API key and put it into an environment variable so a k8s Secret can be created with it later.

export ANTHROPIC_API_KEY=

Create the Gateway object.

kubectl apply -f- <<EOF
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1
metadata:
  name: agentgateway-route
  namespace: agentgateway-system
  labels:
    app: agentgateway
spec:
  gatewayClassName: enterprise-agentgateway
  infrastructure:
    parametersRef:
      name: tracing
      group: enterpriseagentgateway.solo.io
      kind: EnterpriseAgentgatewayParameters
  listeners:
  - protocol: HTTP
    port: 8080
    name: http
    allowedRoutes:
      namespaces:
        from: All
EOF

Create the secret for Anthropic. This way, you have proper access to Anthropic via your Gateway for LLM calls.

kubectl apply -f- <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: anthropic-secret
  namespace: agentgateway-system
  labels:
    app: agentgateway-route
type: Opaque
stringData:
  Authorization: $ANTHROPIC_API_KEY
EOF

Create an Agentgateway Backend.

Two things to keep in mind with the AgentgatewayBackend config.

The first is that notice the routes are going through /v1/messages and not /v1/chat/completions like you'd normally see in an OpenAI API format spec route. The reason is that Agentgateway can handle the translation (from Anthropic spec to OpenAI spec), but because you're routing traffic directly through Claude, no translation occurs, which is why the Anthropic spec is needed.

The second thing is with the two configurations below, you'll see either a Model specified (Opus) or an open bracket to specify any Model you want. The reason why is because if you specify a Model in your AgentgatewayBackend and then use a different Model in Claude Code CLI, you will get a 400 error that says something along the lines of "thinking mode isn't enabled", which isn't the error that Claude Code should be showing you, but that's what you'll most likely see. If you specify Opus, you must use Opus in your Claude Code CLI configuration. If you specify no Model and just a Provider (anthropic: {}), you can use any Model you'd like.

kubectl apply -f- <<EOF
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  labels:
    app: agentgateway-route
  name: anthropic
  namespace: agentgateway-system
spec:
  ai:
    provider:
        anthropic:
          model: "claude-opus-4-6"
  policies:
    ai:
      routes:
        '/v1/messages': Messages
        '*': Passthrough
    auth:
      secretRef:
        name: anthropic-secret
EOF

Or without a specified Model

kubectl apply -f - <<EOF
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  labels:
    app: agentgateway-route
  name: anthropic
  namespace: agentgateway-system
spec:
  ai:
    provider:
      anthropic: {}
  policies:
    auth:
      secretRef:
        name: anthropic-secret
    ai:
      routes:
        '/v1/messages': Messages
        '*': Passthrough
EOF

Create the routing configurations that point to your Gateway and use the Agentgateway Backend you created in the previous step as the reference.

kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: claude
  namespace: agentgateway-system
  labels:
    app: agentgateway-route
spec:
  parentRefs:
    - name: agentgateway-route
      namespace: agentgateway-system
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: anthropic
      namespace: agentgateway-system
      group: agentgateway.dev
      kind: AgentgatewayBackend
EOF

Test Connectivity

With the Gateway, Backend, and Route configured, let's ensure that the Claude Code CLI traffic can successfully go through agentgateway.

Grab your ALB IP from the Gateway within an environment variable. If you're running this locally and don't have access to an ALB IP, you can skip this test and just use localhost after port-forwarding the Gateway service.

export INGRESS_GW_ADDRESS=$(kubectl get svc -n agentgateway-system agentgateway-route -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
echo $INGRESS_GW_ADDRESS

Test the LLM connectivity through your Gateway with a single prompt.

ANTHROPIC_BASE_URL="http://$INGRESS_GW_ADDRESS$:8080" claude -p "What is a credit card"

Or with localhost.

ANTHROPIC_BASE_URL="http://127.0.0.1:8080" claude -p "What is a credit card"

You can also go into Claude Code CLI if you just run ANTHROPIC_BASE_URL="[http://127.0.0.1:8080](http://127.0.0.1:8080/)" claude or ANTHROPIC_BASE_URL="http://$INGRESS_GW_ADDRESS$:8080" and you'll be able to prompt it with whatever you'd like.

With the traffic connectivity tested, let's implement Prompt Guards.

Prompt Guards

Connectivity through agentgateway with Claude Code CLI has been tested and confirmed, so now, let's move into the security piece.

The number 1 thing organizations want to be able to secure is what can actually get prompted via an Agent. For example, the last thing you want is to have someone prompt an Agent with Delete all of the Kubernetes clusters in production and it actually does it. To avoid this, you need to ensure that what a user can prompt is something that they should be able to prompt.

Modify the AgentgatewayBackend with a prompt guard. Notice how this is a regex and for the test, we want to block any traffic that has the words credit card in it.

kubectl apply -f- <<EOF
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  labels:
    app: agentgateway-route
  name: anthropic
  namespace: agentgateway-system
spec:
  ai:
    provider:
        anthropic:
          model: "claude-opus-4-6"
  policies:
    ai:
      routes:
        '/v1/messages': Messages
        '*': Passthrough
      promptGuard:
        request:
        - response:
            message: "Rejected due to inappropriate content"
          regex:
            action: Reject
            matches:
            - "credit card"
    auth:
      secretRef:
        name: anthropic-secret
EOF

You can also do the same thing without a Model specified:

kubectl apply -f - <<EOF
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  labels:
    app: agentgateway-route
  name: anthropic
  namespace: agentgateway-system
spec:
  ai:
    provider:
      anthropic: {}
  policies:
    auth:
      secretRef:
        name: anthropic-secret
    ai:
      routes:
        '/v1/messages': Messages
        '*': Passthrough
      promptGuard:
        request:
        - response:
            message: "Rejected due to inappropriate content"
          regex:
            action: Reject
            matches:
            - "credit card"
EOF

Run the check again by running either of the following:

ANTHROPIC_BASE_URL="http://$INGRESS_GW_ADDRESS:8080" claude -p "What is a credit card"
ANTHROPIC_BASE_URL="http://$INGRESS_GW_ADDRESS:8080" claude -p and then prompting within Claude Code What is a credit card

You'll get an output similar to the one below.

With traffic routing through agentgateway from Claude Code CLI and the knowledge of how prompt guards can work in this scenario, you can now secure traffic from anyones laptop/desktop when they're using an Agentic CLI client.

Build AI Agents on Kubernetes: Kagent + Amazon Bedrock Setup Guide

Michael Levan — Sat, 24 Jan 2026 13:19:06 +0000

Managing various LLM provider accounts, subscriptions, and cost can get cumbersome for many organizations in a world where multiple LLMs are used. To avoid this, you can use what can be called a "middle ground" between your Agent and the LLM provider.

With AWS Bedrock, you can set up an API key and access various LLMs from Claude to GPT to Llama from one place. Instead of having multiple API keys and various accounts, you can route all of your Agentic traffic from your Agent to an LLM via Bedrock.

In this blog post, you'll learn how to set up an Agent via kagent to access Bedrock Models and use them to perform any action you'd like.

Prerequisites

To follow along with this blog post from a hands-on perspective, you should have the following:

A Kubernetes cluster.
Kagent installed, which you can find here.

Configuring Access To AWS

The first step is ensuring that you have proper access to AWS so you can use the Model that you'd like to implement within your Agent.

Create environment variables with your AWS access key, secret, and region. To retrieve an AWS access key and secret, you'll need to create them in AWS IAM.

export AWS_ACCESS_KEY_ID=<your-access-key-id>
export AWS_SECRET_ACCESS_KEY=<your-secret-access-key>
export AWS_REGION=us-west-1

Once you have access, you can run the command below which will show you what Models are available in your region of choice.

aws bedrock list-inference-profiles --region us-east-1 \
  --query "inferenceProfileSummaries[?contains(inferenceProfileId, 'claude')].{id:inferenceProfileId,name:inferenceProfileName}" \
  --output table

Here's an example of the output you should see on your terminal.

----------------------------------------------------------------------------
|                                  ListInferenceProfiles                                  |
+---------------------------------------------------+----------------------
|                        id                         |                name                 |
+---------------------------------------------------+-----------------------
|  us.anthropic.claude-sonnet-4-20250514-v1:0       |  US Claude Sonnet 4                 |
|  global.anthropic.claude-sonnet-4-5-20250929-v1:0 |  Global Claude Sonnet 4.5           |
|  us.anthropic.claude-haiku-4-5-20251001-v1:0      |  US Anthropic Claude Haiku 4.5      |
|  global.anthropic.claude-haiku-4-5-20251001-v1:0  |  Global Anthropic Claude Haiku 4.5  |
|  us.anthropic.claude-opus-4-5-20251101-v1:0       |  US Anthropic Claude Opus 4.5       |
|  global.anthropic.claude-opus-4-5-20251101-v1:0   |  GLOBAL Anthropic Claude Opus 4.5   |
|  us.anthropic.claude-sonnet-4-5-20250929-v1:0     |  US Anthropic Claude Sonnet 4.5     |
+---------------------------------------------------+-----------------------

Next, go into AWS Bedrock and generate an API key. Although you have access to your AWS account, there's a separate API key needed to access LLMs via AWS Bedrock.

Create an environment variable with the API key.

export BEDROCK_API_KEY=

With this configuration, you can now begin the Model and Agent setup so you can access LLMs via Bedrock through kagent.

Model And Agent Setup

The next phase is to create a Model Config which will be how the Agent knows what Model to access. In this case, the Model called to within the Model Config will be an OpenAI GPT Model.

Create a Kubernetes secret that contains your AWS access key, secret, and Bedrock API key.

kubectl create secret generic kagent-bedrock-aws -n kagent \
  --from-literal=AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
  --from-literal=AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
  --from-literal=BEDROCK_API_KEY=$BEDROCK_API_KEY \
  --from-literal=AWS_SESSION_TOKEN=""

Implement a Model config that calls out to the openai.gpt-oss-20b-1:0 Model using your Bedrock API key secret. You'll also see the base URL which is the URL where the Model and provider exist via Bedrock.

kubectl apply -f - <<EOF
apiVersion: kagent.dev/v1alpha2
kind: ModelConfig
metadata:
  name: bedrock-model-config
  namespace: kagent
spec:
  apiKeySecret: kagent-bedrock-aws
  apiKeySecretKey: BEDROCK_API_KEY
  model: openai.gpt-oss-20b-1:0
  provider: OpenAI
  openAI:
    baseUrl: "https://bedrock-runtime.us-east-1.amazonaws.com/openai/v1"
EOF

Check that the Model was accepted.

kubectl get modelconfig bedrock-model-config -n kagent -o jsonpath='{.status.conditions}' | jq

You'll see an output similar to the below:

[
  {
    "lastTransitionTime": "...",
    "message": "",
    "reason": "ModelConfigReconciled",
    "status": "True",
    "type": "Accepted"
  }
]

With the Model config set up, you can now create the Agent and test it.

Using Bedrock With Kagent

With kagent installed, you have access to various CRDs like the ModelConfig object you created in the previous section. Within the kagent CRDs, you also have access to the Agent object, which allows you to define everything from what Model Config to use to the prompt to MCP Server tools and Agent Skills.

Create a new Agent with the YAML below. It includes all of the secrets needed, a prompt, and a few MCP Server tools.

kubectl apply -f - <<EOF
apiVersion: kagent.dev/v1alpha2
kind: Agent
metadata:
  name: bedrock-agent-test
  namespace: kagent
spec:
  description: Kubernetes troubleshooting agent powered by Claude via Bedrock
  type: Declarative
  declarative:
    modelConfig: bedrock-model-config
    deployment:
      env:
        - name: AWS_ACCESS_KEY_ID
          valueFrom:
            secretKeyRef:
              name: kagent-bedrock-aws
              key: AWS_ACCESS_KEY_ID
        - name: AWS_SECRET_ACCESS_KEY
          valueFrom:
            secretKeyRef:
              name: kagent-bedrock-aws
              key: AWS_SECRET_ACCESS_KEY
        - name: AWS_SESSION_TOKEN
          valueFrom:
            secretKeyRef:
              name: kagent-bedrock-aws
              key: AWS_SESSION_TOKEN
    systemMessage: |
      You're a friendly and helpful agent that uses Kubernetes tools to help with troubleshooting and deployments.

      # Instructions
      - If user question is unclear, ask for clarification before running any tools
      - Always be helpful and friendly
      - If you don't know how to answer the question, respond with "Sorry, I don't know how to answer that"

      # Response format
      - ALWAYS format your response as Markdown
      - Include a summary of actions you took and an explanation of the result
    tools:
      - type: McpServer
        mcpServer:
          name: kagent-tool-server
          kind: RemoteMCPServer
          toolNames:
          - k8s_get_available_api_resources
          - k8s_get_cluster_configuration
          - k8s_get_events
          - k8s_get_pod_logs
          - k8s_get_resource_yaml
          - k8s_get_resources
          - k8s_check_service_connectivity
EOF

Wait until the Agent is up and operational.

kubectl get pods -n kagent --watch

Open the kagent dashboard
Go to your new Agent.
Prompt it with something like What can you do?. You'll see an output similar to the one below.

Routing Observable and Secure Traffic Through Claude

Michael Levan — Sun, 18 Jan 2026 15:38:08 +0000

AI traffic that goes through enterprise systems should include everything from servers, cloud environments, and even laptops, desktops, and mobile devices. This level of observability and security isn't "new"; the industry has had it for years with Mobile Device Management (MDM) software. With AI workloads, however, the concepts of properly observing and securing local systems seem to have been forgotten.

And we can't forget about AI traffic.

In this blog post, you'll learn how to route local AI traffic through agentgateway when tools like Claude desktop are interacting with MCP Servers.

Prerequisites

To follow along from a hands-on perspective, you'll need the following:

A Kubernetes cluster.
Claude Desktop.
Agentgateway installed.

The Low-Hanging Fruit

Organizations, enterprises, teams, and engineers are working on consistent ways to implement Agentic infrastructure, whether that be on systems, domain-specific Agents, generic Agents, MCP, and everything in between. This is typically happening in many places today at the, what we can call "backend layer". The "backend layer" are the cloud environments, servers running AI workloads, and networks.

However, there's one piece to the puzzle that seems to be overlooked - the "frontend layer". These are the user devices (laptops, desktops, mobile devices) within the organization that are being used at work.

In the engineering space, that typically falls into the LLM, Agents, or desktop software that engineers are using (Claude Code, Claude Desktop, Gemini CLI, etc.). With these "frontend layer" tools, it's open to all with zero observability or security. Now, the goal isn't to completely lock everything down to where no one can use AI, but there needs to be defense in depth, security practices, and perhaps most importantly, observability for all AI traffic even, and especially, when it's coming from a local machine.

Much like all systems (laptops, desktops, mobile devices) go through networks within the enterprise that are the internal networks (traffic through a router and rules in place by a firewall and observed at the packet level), AI traffic needs to be looked at the same way.

Deploying An MCP Server

The first step in the journey is to give Claude Code desktop "something" to route to. This could be another Agent, various Models, or an MCP Server for specific tool selection needs. This section will walk you through how to deploy an MCP Server on a Kubernetes cluster.

Deploy the following configuration which contains a configmap that has the MCP Server configuration, a Kubernetes Deployment, and a Kubernetes Service.

kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: mcp-math-script
  namespace: default
data:
  server.py: |
    import uvicorn
    from mcp.server.fastmcp import FastMCP
    from starlette.applications import Starlette
    from starlette.routing import Route
    from starlette.requests import Request
    from starlette.responses import JSONResponse, Response

    mcp = FastMCP("Math-Service")

    @mcp.tool()
    def add(a: int, b: int) -> int:
        return a + b

    @mcp.tool()
    def multiply(a: int, b: int) -> int:
        return a * b

    async def handle_mcp(request: Request):
        try:
            data = await request.json()
            method = data.get("method")
            msg_id = data.get("id")
            result = None

            if method == "initialize":
                result = {
                    "protocolVersion": "2024-11-05",
                    "capabilities": {"tools": {}},
                    "serverInfo": {"name": "Math-Service", "version": "1.0"}
                }

            elif method == "notifications/initialized":
                # Notifications are fire-and-forget, return empty 202 response
                return Response(status_code=202)

            elif method == "tools/list":
                tools_list = await mcp.list_tools()
                result = {
                    "tools": [
                        {
                            "name": t.name,
                            "description": t.description,
                            "inputSchema": t.inputSchema
                        } for t in tools_list
                    ]
                }

            elif method == "tools/call":
                params = data.get("params", {})
                name = params.get("name")
                args = params.get("arguments", {})

                # Call the tool
                tool_result = await mcp.call_tool(name, args)

                # --- FIX: Serialize the content objects manually ---
                serialized_content = []
                for content in tool_result:
                    if hasattr(content, "type") and content.type == "text":
                        serialized_content.append({"type": "text", "text": content.text})
                    elif hasattr(content, "type") and content.type == "image":
                         serialized_content.append({
                             "type": "image",
                             "data": content.data,
                             "mimeType": content.mimeType
                         })
                    else:
                        # Fallback for dictionaries or other types
                        serialized_content.append(content if isinstance(content, dict) else str(content))

                result = {
                    "content": serialized_content,
                    "isError": False
                }

            elif method == "ping":
                result = {}

            else:
                return JSONResponse(
                    {"jsonrpc": "2.0", "id": msg_id, "error": {"code": -32601, "message": "Method not found"}},
                    status_code=404
                )

            return JSONResponse({"jsonrpc": "2.0", "id": msg_id, "result": result})

        except Exception as e:
            # Print error to logs for debugging
            import traceback
            traceback.print_exc()
            return JSONResponse(
                {"jsonrpc": "2.0", "id": None, "error": {"code": -32603, "message": str(e)}},
                status_code=500
            )

    app = Starlette(routes=[
        Route("/mcp", handle_mcp, methods=["POST"]),
        Route("/", lambda r: JSONResponse({"status": "ok"}), methods=["GET"])
    ])

    if __name__ == "__main__":
        print("Starting Fixed Math Server on port 8000...")
        uvicorn.run(app, host="0.0.0.0", port=8000)
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mcp-math-server
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mcp-math-server
  template:
    metadata:
      labels:
        app: mcp-math-server
    spec:
      containers:
      - name: math
        image: python:3.11-slim
        command: ["/bin/sh", "-c"]
        args:
        - |
          pip install "mcp[cli]" uvicorn starlette &&
          python /app/server.py
        ports:
        - containerPort: 8000
        volumeMounts:
        - name: script-volume
          mountPath: /app
        readinessProbe:
          httpGet:
            path: /
            port: 8000
          initialDelaySeconds: 5
          periodSeconds: 5
      volumes:
      - name: script-volume
        configMap:
          name: mcp-math-script
---
apiVersion: v1
kind: Service
metadata:
  name: mcp-math-server
  namespace: default
spec:
  selector:
    app: mcp-math-server
  ports:
  - port: 80
    targetPort: 8000
EOF

The MCP Server should now be running in a Pod via the default Namespace with the mcp-math-server k8s Service sitting in front of the Pod.

Configuring A Gateway

With the MCP Server deployed, you need a way to pass traffic through to it. If you think about when Agents communicate to other Agents, MCP Servers, or LLMs, there's a "middle layer", which is how the Agent gets from point A (itself) to point B (the MCP Server in this case), that "middle layer" is where the packets flow, which is the Gateway.

If you aren't running on a Kubernetes cluster that has the ability to create a public ALB with an IP address that's accessible externally, you can use something like Metallb or port-forward the Gateway in your terminal.

Create a new Gateway, which will use the agentgateway Gateway Class. It will be listening on port 8080 and allow traffic from the same Namespace as where the Gateway is deployed (agentgateway-system).

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: agentgateway-mcp
  namespace: agentgateway-system
spec:
  gatewayClassName: enterprise-agentgateway
  listeners:
  - name: http
    port: 8080
    protocol: HTTP
    allowedRoutes:
      namespaces:
        from: Same
EOF

Implement an agentgateway backend, which is what tells the Gateway what to route to. In this case, it's the MCP Server that you deployed in the previous section.

kubectl apply -f - <<EOF
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  name: demo-mcp-server
  namespace: agentgateway-system
spec:
  mcp:
    targets:
      - name: demo-mcp-server
        static:
          host: mcp-math-server.default.svc.cluster.local
          port: 80
          path: /mcp
          protocol: StreamableHTTP
EOF

Create an HTTP route so there's a path for the Gateway to route to. In this case, the "path" is the MCP Server via the agentgateway backend.

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: mcp-route
  namespace: agentgateway-system
spec:
  parentRefs:
  - name: agentgateway-mcp
  rules:
  - backendRefs:
    - name: demo-mcp-server
      namespace: agentgateway-system
      group: agentgateway.dev
      kind: AgentgatewayBackend
EOF

Retrieve the IP address of the Gateway. If an external one doesn't exist, you can port-forward the Gateway service.

export GATEWAY_IP=$(kubectl get svc agentgateway-mcp -n agentgateway-system -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo $GATEWAY_IP

Open MCP Inspector to test the traffic to the MCP Server.

npx modelcontextprotocol/inspector#0.16.2

Add the following URL into MCP Inspector. If you're port forwarding the Gateway service, use localhost instead of an IP address.

http://YOUR_ALB_LB_IP:8080/mcp

If you search for tools, you should see an add and multiply tool.

Configure Claude Desktop With An MCP Server

The last step is to configure Claude Desktop to route through/use the AI gateway (agentgateway) that you deployed in the previous section. This will ensure that the traffic flowing from Claude Desktop to the MCP Server is observable, has the ability to be secured, and is going through a properly built Gateway designed specifically for AI workloads.

Create a new file called claude_desktop_config.json in the path where Claude exists (like in the following example).

mkdir -p ~/Library/Application\ Support/Claude
cat > ~/Library/Application\ Support/Claude/claude_desktop_config.json << 'EOF'
{
  "mcpServers": {
    "math-service": {
      "command": "npx",
      "args": ["-y", "supergateway", "--streamableHttp", "http://YOUR_ALB_LB_IP:8080/mcp"]
    }
  }
}
EOF

After saving the config, restart Claude Desktop for changes to take effect. If you don't see any errors when opening Claude Desktop, that means the configuration that you added in step 1 worked as expected.
With Claude Desktop open, ask it a simple question like What is 2 + 2.

Traffic is now routing through agentgateway via Claude Code!

Running Any AI Agent on Kubernetes: Step-by-Step

Michael Levan — Sat, 13 Dec 2025 23:35:58 +0000

There are many Agentic creation frameworks ranging from CrewAI to kagent to langchain and several others which are typically written in Python or JS. If you're an engineer working on Kubernetes, you may be thinking "What about a declarative Agent deployment method?"

In this blog post, you'll see how to create your own Agent in an Agent framework and then deploy it to kagent in a declarative fashion.

Prerequisites

To follow along with this blog post, you should have the following:

A Kubernetes cluster deployed with kagent installed. If you've never installed kagent, you can find the how-to here.
Python3.10 or above installed.
Docker desktop (or just the Docker engine) installed to build the container image.

What Are BYO Agents

BYO (Bring Your Own) means you can create an Agent in any of the supported providers from kagent. You can also create your agent fully along with connect it to MCP sercers in kagent, but if you're already used to writing your Agents in Python using CrewAI, ADK, langchain, or any other framework, kagent gives you the ability to import those Agents. The only thing you need to do is containerize the Agent, which is straightforward with a Dockerfile (you'll see an example in the section on creating Agents).

Building An Agent

With the previous section giving you knowledge around BYO Agents, it's time to start creating an Agent and see it run within Kubernetes. The next two sections will walk you through how to build a custom Agent with Agent Development Kit (ADK), which is an Agent creation framework and use an existing Agent to see the process of getting one that's readily available for Kubernetes deployed.

Creating An Agent

Install the Google ADK library. Depending on where you're running the below, you may need to use pip3 instead of pip.

pip install google-adk

With the adk command, use the create subcommand to create a scaffolding for an ADK Agent in Python.

adk create NAME_OF_YOUR_AGENT

You should see an output similar to the one below (with the name of your Agent).

You can cd into the directory and use the run subcommand to see it in action as with the scaffolding, you'll have an Agent template.

cd adk/NAME_OF_YOUR_AGENT && adk run NAME_OF_YOUR_AGENT

Using An Existing Agent

To make life a bit easier, instead of having to go and build out everything that is needed for the Agent to be containerized, you can use one that was already built and tested (by myself). If you're wondering "Well, why did I build an Agent then?" it's because with that Agent, you'll be able to containerize it and run it yourself after seeing the example in this section as you can use it as a reference.

Clone the agentic-demo-code repo and cd into the adk/troubleshoot-agent directory.
Open the Dockerfile and you should see the file contents below.

### STAGE 1: base image
ARG DOCKER_REGISTRY=ghcr.io
ARG VERSION=0.7.4
FROM $DOCKER_REGISTRY/kagent-dev/kagent/kagent-adk:$VERSION

WORKDIR /app

COPY troubleshootagent/ troubleshootagent/
COPY pyproject.toml pyproject.toml
COPY uv.lock uv.lock
COPY how-it-works.md how-it-works.md

RUN uv sync --locked --refresh

CMD ["troubleshootagent"]

Run the following command to build the container image.

docker build . -t troubleshootagent:latest

If you see an error about a "uv sync", run the following command to create a lock file for library versions and dependencies.

uv lock

You should see that the image was fully built.

With the Agent container image local, you'll need to push it to a container registry of your choosing. Considering Docker Hub is free, you can use that if you'd prefer. Below is an example with my GitHub org.

docker tag troubleshootagent:latest adminturneddevops/troubleshootagent:latest

docker push adminturneddevops/troubleshootagent:latest

If you don't want to push the container image to your container registry, you can use adminturneddevops/troubleshootagent:latest in the next section since the container image will be public.

Deploying An Agent On Kubernetes

With the Agent fully built, it's time to deploy it on Kubernetes using the kagent framework. This will give you a declarative method of running Agents in a mature orchestration platform like Kubernetes.

For the Agent to work, it'll connect to an LLM. You need authentication/API access to an LLM of your choosing. In this scenario, Google Gemini is used, but you can swap it for any AI Provider you'd like to use.

Use an env variable to expoert the API key.

export GOOGLE_API_KEY=

Create a Kubernetes Secret with the API key.

kubectl apply -f- <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: kagent-google
  namespace: kagent
type: Opaque
stringData:
  GOOGLE_API_KEY: $GOOGLE_API_KEY
EOF

Use the Agent object via the kagent CRDs to add the Agent to kagent.

kubectl apply -f - <<EOF
apiVersion: kagent.dev/v1alpha2
kind: Agent
metadata:
  name: troubelshoot-agent
  namespace: kagent
spec:
  description: This agent is used to be a Platform Engineering troubleshoot expert.
  type: BYO
  byo:
    deployment:
      image: adminturneddevops/troubleshootagent:latest
      env:
        - name: GOOGLE_API_KEY
          valueFrom:
            secretKeyRef:
              name: kagent-google
              key: GOOGLE_API_KEY
EOF

Confirm that the Agent is running by looking at the Pod in the kagent Namespace.

kubectl get pods -n kagent

You can now begin using the Agent in kagent.

Context-Aware Networking & Runtimes: Agentic End-To-End

Michael Levan — Sat, 06 Dec 2025 14:33:18 +0000

AI network traffic can very much feel like a black box. You open an AI provider console or an Agent, ask a question or perform a task, and then what happens? Where does that traffic go? Is the traffic secure? Is it going to the appropriate destination? Where’s the context?

There are, what feels like hundreds of questions that need to be answered from when an Agent makes a call to an LLM to when you get a response.

For those questions to be answered, you need an end-to-end workflow for when traffic leaves the Agent to when you get a response or a task is completed.

In this blog post, you’ll learn how to, from a hands-on perspective, accomplish answering those questions.

Install Kagent

The first step in the process is the installation of the two platforms you'll be using, which is kagent and agentgateway. Agentgateway will be installed in the next section and in this section, you'll deploy kagent, which is an Agent framework that runs on Kubernetes.

Install the kagent CRDs and create the kagent Namespace.

helm install kagent-crds oci://ghcr.io/kagent-dev/kagent/helm/kagent-crds \
    --namespace kagent \
    --create-namespace

Specify your AI provider API key. For the purposes of this blog post, Anthropic is used. However, you can use whichever provider you'd like that's supported.

export ANTHROPIC_API_KEY=your_api_key

Install kagent with your specified provider.

helm upgrade --install kagent oci://ghcr.io/kagent-dev/kagent/helm/kagent \
    --namespace kagent \
    --set providers.default=anthropic \
    --set providers.anthropic.apiKey=$ANTHROPIC_API_KEY \
    --set ui.service.type=LoadBalancer

Ensure that kagent is installed successfully.

kubectl get svc -n kagent

You should see an output similar to the below.

Install Agentgateway + Kgateway

The next step in the process is to install kgateway with agentgateway enabled. Kgateway is the control plane (think of it as the brains of the operation) and agentgateway is the AI-enabled data plane/proxy for all agentic traffic. You'll see how you can track, observe, and secure all traffic going through an AI Agent with agentgateway.

Installed the Kubernetes Gateway API CRDs as the Gateway is build with these CRDs to ensure flexibility and agnostic compatibilty.

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/standard-install.yaml

Install the kgateway CRDs.

helm upgrade -i --create-namespace --namespace kgateway-system kgateway-crds oci://cr.kgateway.dev/kgateway-dev/charts/kgateway-crds  \
--version v2.1.1 \
--set controller.image.pullPolicy=Always

Install kgateway as the control plane along with agentgateway enabled for AI-related data plane/proxy traffic.

helm upgrade -i -n kgateway-system kgateway oci://cr.kgateway.dev/kgateway-dev/charts/kgateway \
    --version v2.1.1 \
    --set agentgateway.enabled=true \
    --set controller.image.pullPolicy=Always

Ensure that kgateways control plane was installed successfully by checking to see if the kgateway Pod is running.

kubectl get pods -n kgateway-system

The gateway classes for both kgateway (standard Envoy/stateless traffic) and agentgateway (AI-related/stateful traffic) should now be available.

kubectl get gatewayclass

You should see an output similar to the below.

NAME           CONTROLLER                  ACCEPTED   AGE
agentgateway   kgateway.dev/agentgateway   True       20h
kgateway       kgateway.dev/kgateway       True       20h

LLM Gateway Creation

Now that the installation of both the AI Agent framework (kagent) and the AI gateway (agentgateway) is installed, it's time to start the configuration for LLM-related traffic. Any time that you use an AI Agent in kagent, you can observe and secure that traffic via agentgateway. Think of agentgateway as the path that gets your AI traffic from point A to point B.

Create the secret for your AI provider. In this case, Anthropic is used.

kubectl apply -f- <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: anthropic-secret
  namespace: kagent
  labels:
    app: agentgateway
type: Opaque
stringData:
  Authorization: $ANTHROPIC_API_KEY
EOF

Create a Gateway using agentgateway as the data plane/proxy. When the Gateway is fully up and running, it should have a public IP address if you're on a managed Kubernetes cluster. If you aren't, you'll want to port-forward the Gateway service to be used in the next section. You can find out by running kubectl get svc -n kgateway-system.

kubectl apply -f- <<EOF
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1
metadata:
  name: agentgateway
  namespace: kgateway-system
  labels:
    app: agentgateway
spec:
  gatewayClassName: agentgateway
  listeners:
  - protocol: HTTP
    port: 8080
    name: http
    allowedRoutes:
      namespaces:
        from: All
EOF

The Backend is used to tell the gateway what/where to route to. In this case, you'll be routing to an LLM.

kubectl apply -f- <<EOF
apiVersion: gateway.kgateway.dev/v1alpha1
kind: Backend
metadata:
  labels:
    app: agentgateway
  name: anthropic
  namespace: kgateway-system
spec:
  type: AI
  ai:
    llm:
        anthropic:
          authToken:
            kind: SecretRef
            secretRef:
              name: anthropic-secret
          model: "claude-3-5-haiku-latest"
EOF

Ensure that the backend was deployed successfully.

kubectl get backend -n kgateway-system

The last step is to create a route. The gateway and the backend are both configured, but when you hit the Gateway, there's no configuration to tell the Gateway where to route to. This is where the HTTPRoute object comes into play.

kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: claude
  namespace: kgateway-system
  labels:
    app: agentgateway
spec:
  parentRefs:
    - name: agentgateway
      namespace: kgateway-system
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /anthropic
    filters:
    - type: URLRewrite
      urlRewrite:
        path:
          type: ReplaceFullPath
          replaceFullPath: /v1/chat/completions
    backendRefs:
    - name: anthropic
      namespace: kgateway-system
      group: gateway.kgateway.dev
      kind: Backend
EOF

In the next section, you'll see how to create and connect an Agent.

Creating and Connecting An Agent

With the Gateway created and traffic flow enabled to flow through the agentgateway proxy, you can create an Agent which will route all packets through agentgateway.

Create the Model Config which adds the usage of Claude (or any other LLM) to an Agent that can consume it in kagent. Notice how the baseUrl points to the agentgateway IP. The agentgateway IP was created when you ran the Gateway object in the previous section. As mentioned in the previous section, if you aren't using a managed k8s cluster, you'll want to port-forward the Gateway service so it can be accessed as the proxy.

kubectl apply -f - <<EOF
apiVersion: kagent.dev/v1alpha2
kind: ModelConfig
metadata:
  name: anthropic-model-config
  namespace: kagent
spec:
  apiKeySecret: anthropic-secret
  apiKeySecretKey: Authorization
  model: claude-3-5-haiku-latest
  provider: OpenAI
  openAI:
    baseUrl: http:YOUR_AGENTGATEWAY_IP:8080/anthropic
EOF

💡

You'll notice that although Anthropic is used, OpenAI is the provider spec. The reason why is because the route for /v1/chat/completions is an OpenAI schema. We're using the schema, but the backend is still connecting to a Claude model.

Create the Agent which connects to the Model Config.

kubectl apply -f - <<EOF
apiVersion: kagent.dev/v1alpha2
kind: Agent
metadata:
  name: testing-agentgateway
  namespace: kagent
spec:
  description: This agent can use a single tool to expand it's Kubernetes knowledge for troubleshooting and deployment
  type: Declarative
  declarative:
    modelConfig: anthropic-model-config
    systemMessage: |-
      You're a friendly and helpful agent that uses the Kubernetes tool to help troubleshooting and deploy environments

      # Instructions

      - If user question is unclear, ask for clarification before running any tools
      - Always be helpful and friendly
      - If you don't know how to answer the question DO NOT make things up
        respond with "Sorry, I don't know how to answer that" and ask the user to further clarify the question

      # Response format
      - ALWAYS format your response as Markdown
      - Your response will include a summary of actions you took and an explanation of the result
EOF

Open kagent and find the testing-agentgateway Agent.

Prompt the Agent with What can you do?. You should see an output similiar to the below.

Open a terminal and run the following commands:

kubectl get pods -n kgateway-system

kubectl logs YOUR_AGENTGATEWAY_POD_NAME -n kgateway-system

You'll now be able to see that the traffic routed through the agentgateway proxy.

request gateway=kgateway-system/agentgateway listener=http route=kgateway-system/claude endpoint=api.anthropic.com:443 src.addr=192.168.46.17:29426 http.method=POST http.host=a1e5a3b9a8eba4aa09517966f1777763-34157947.us-east-1.elb.amazonaws.com http.path=/anthropic/chat/completions http.version=HTTP/1.1 http.status=200 protocol=llm gen_ai.operation.name=chat gen_ai.provider.name=anthropic gen_ai.request.model=claude-3-5-haiku-latest gen_ai.response.model=claude-3-5-haiku-20241022 gen_ai.usage.input_tokens=183 gen_ai.usage.output_tokens=240 duration=4797ms

Security Holes in MCP Servers and How To Plug Them

Michael Levan — Tue, 02 Dec 2025 12:35:43 +0000

Model Context Protocol (MCP), has officially hit one year old as of November 25th and although there have been some amazing innovations within MCP, one issue still persists - the gaping security hole. This is no secret as just about every organization is talking about it. The long-running joke so far has been “The S in MCP stands for security”.

Aside from prompt injections, MCP Server security is arguably the biggest issue in the AI security world right now.

In this blog post, you’ll learn how to fix the gaps.

Prerequisites

To follow along with this blog post, you should have:

A Kubernetes cluster running (it can be local).
Kubernetes Gateway API CRDs installed, which you can find here.

Why Security Matters For MCP

There are two forms of MCP servers:

stdio
StreamableHTTP

stdio (the communication method) stands for standard input/output. When using standard input/output, you’re typically targeting a pre-built MCP Server that’s written in, typically, Python or JS (Go is up and coming in this space) that’s called to locally via a command like uvx or npx (depending on how the MCP Server is built). It’s not a standard library download (like a pip install), but instead stored in cache (e.g - ~/.local/share/uv).

StreamableHTTP is an external (or even internal) server that you’re reaching out to that’s not cached locally. A good example of this is the GitHub Copilot MCP Server. It’s an MCP Server that you’re reaching out to over the HTTP protocol instead of via a local cache.

💡 There was another option called SSE which you may see, but it is now deprecated as of June 2025.

Regardless of which option you choose, there are many security holes.

One (of the many) problem with stdio MCP Servers is that you can't run them through a Gateway. That means no AuthN/Z, no rate limiting, and no tool control. The MCP Servers are effectively open and usable by anyone in an organization unless you’re manually locking each computer down with Claude Desktop configurations (which, spoiler alert: no ones doing).

With Streamable HTTP, you’re in the dark. You’re connecting Agents to some black box running in someones datacenter with who knows what (if any) security protocols, and even if there are security protocols, that doesn’t help from an overall AuthN/Z perspective for your organization. There’s also no way to even test the security without a proper pentest, which wouldn’t be legal without explicit permission from the organization hosting the MCP Server. The only way to do it would be to put an Agent in front of the MCP Server, but then you're not actually securing the MCP Server, you're securing the Agent.

As Model Context Protocol stands right now, there’s only one true way to secure it - with a proper AI Gateway.

Solo Enterprise For Agentgateway implements everything from locking down tools to proper user and system-level authentication to MCP Servers with or without Agents. In the following sections, you’ll see how to configure security for MCP.

Deploy An MCP Server and Agentgateway

The first step is to deploy an MCP Server and a Gateway via agentgateway enterprise so we not only have an MCP Server to test with, but a proper AI gateway to secure our MCP connectivity.

Create a new Kubernetes Deployment pointing to the test MCP Server that is containerized. You’ll also see a Service that gets deployed so the Gateway can properly connect to it.

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mcp-website-fetcher
  namespace: default
spec:
  selector:
    matchLabels:
      app: mcp-website-fetcher
  template:
    metadata:
      labels:
        app: mcp-website-fetcher
    spec:
      containers:
      - name: mcp-website-fetcher
        image: ghcr.io/peterj/mcp-website-fetcher:main
        imagePullPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: mcp-website-fetcher
  namespace: default
spec:
  selector:
    app: mcp-website-fetcher
  ports:
  - port: 80
    targetPort: 8000
    appProtocol: kgateway.dev/mcp
EOF

Deploy the Backend so agentgateway knows what to route to. In this case, it’s routing to the MCP Server service that you deployed in step 1.

kubectl apply -f - <<EOF
apiVersion: gateway.kgateway.dev/v1alpha1
kind: Backend
metadata:
  name: mcp-backend
  namespace: gloo-system
spec:
  type: MCP
  mcp:
    targets:
    - name: mcp-target
      static:
        host: mcp-website-fetcher.default.svc.cluster.local
        port: 80
        protocol: StreamableHTTP
EOF

Deploy the Gateway using the agentgateway enterprise class. This Gateway is what will be used for MCP Inspector to connect to (more on Inspector coming up).

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: agentgateway
  namespace: gloo-system
spec:
  gatewayClassName: agentgateway-enterprise
  listeners:
  - name: http
    port: 8080
    protocol: HTTP
    allowedRoutes:
      namespaces:
        from: Same
EOF

Create an HTTP route so you can route the traffic to the backend, which you created in step 3.

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: mcp-route
  namespace: gloo-system
spec:
  parentRefs:
  - name: agentgateway
  rules:
  - backendRefs:
    - name: mcp-backend
      group: gateway.kgateway.dev
      kind: Backend
EOF

Capture the Gateway ALB IP in an environment variable to be used when connecting to the MCP Server. If you’re running this locally, you can do a port-forward on the Gateway and use localhost within the IP address section when connecting to it.

export GATEWAY_IP=$(kubectl get svc agentgateway -n gloo-system -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo $GATEWAY_IP

Open MCP Inspector to connect to the MCP Server.

npx modelcontextprotocol/inspector#0.16.2

The URL to put into MCP Inspector is: http://YOUR_ALB_LB_IP:8080/mcp or if you’re running locally, http://localhost:8080/mcp

You’re now connected to the MCP Server via Inspector (the MCP client), but as you can see, it’s fully open. There’s no security at all. In the next section, that’ll be fixed.

Secure MCP Server Auth

With a properly deployed MCP Server and agentgateway in front of it, let’s begin the journey of securing MCP Server connectivity. The first step is to enable token-based authentication. In this case, you’ll use a JWT token.

Add in a traffic policy for auth based on a JWT token.

kubectl apply -f- <<EOF
apiVersion: gloo.solo.io/v1alpha1
kind: GlooTrafficPolicy
metadata:
  name: jwt
  namespace: gloo-system
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: agentgateway
  glooJWT:
    beforeExtAuth:
      providers:
        selfminted:
          issuer: solo.io
          jwks:
            local:
              key: '{"keys":[{"kty":"RSA","kid":"solo-public-key-001","use":"sig","alg":"RS256","n":"AOfIaJMUm7564sWWNHaXt_hS8H0O1Ew59-nRqruMQosfQqa7tWne5lL3m9sMAkfa3Twx0LMN_7QqRDoztvV3Wa_JwbMzb9afWE-IfKIuDqkvog6s-xGIFNhtDGBTuL8YAQYtwCF7l49SMv-GqyLe-nO9yJW-6wIGoOqImZrCxjxXFzF6mTMOBpIODFj0LUZ54QQuDcD1Nue2LMLsUvGa7V1ZHsYuGvUqzvXFBXMmMS2OzGir9ckpUhrUeHDCGFpEM4IQnu-9U8TbAJxKE5Zp8Nikefr2ISIG2Hk1K2rBAc_HwoPeWAcAWUAR5tWHAxx-UXClSZQ9TMFK850gQGenUp8","e":"AQAB"}]}'
EOF

Save the token for auth via MCP Inspector.

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6InNvbG8tcHVibGljLWtleS0wMDEifQ.eyJpc3MiOiJzb2xvLmlvIiwib3JnIjoic29sby5pbyIsInN1YiI6ImJvYiIsInRlYW0iOiJvcHMiLCJleHAiOjIwNzQyNzQ5NTQsImxsbXMiOnsibWlzdHJhbGFpIjpbIm1pc3RyYWwtbGFyZ2UtbGF0ZXN0Il19fQ.GF_uyLpZSTT1DIvJeO_eish1WDjMaS4BQSifGQhqPRLjzu3nXtPkaBRjceAmJi9gKZYAzkT25MIrT42ZIe3bHilrd1yqittTPWrrM4sWDDeldnGsfU07DWJHyboNapYR-KZGImSmOYshJlzm1tT_Bjt3-RK3OBzYi90_wl0dyAl9D7wwDCzOD4MRGFpoMrws_OgVrcZQKcadvIsH8figPwN4mK1U_1mxuL08RWTu92xBcezEO4CdBaFTUbkYN66Y2vKSTyPCxg3fLtg1mvlzU1-Wgm2xZIiPiarQHt6Uq7v9ftgzwdUBQM1AYLvUVhCN6XkkR9OU3p0OXiqEDjAxcg

Try to reconnect to the MCP Server and you’ll see an error similar to the below:

Connection Error - Check if your MCP server is running and proxy token is correct

Within MCP Inspector, click on Authentication and add in the following:
Header Name: Authorization
Bearer Token: Bobs Token from step 2

You should now be able to connect to the MCP Server successfully.

With proper auth set up, you now know that not just anyone can use your agentgateway to connect to an MCP Server. This allows you to ensure that the traffic you’re observing from an AuthN/Z perspective is valid in comparison to the “anyone can do whatever they want” nature of MCP Servers without agentgateway in place.

Locking Down MCP Tool Lists

The final step is to specify what MCP Tools are available. One of the main issues for organizations is they want to use Tools within an MCP Server, but not all Tools. For example, maybe a person or an AI Agent connecting to an MCP Server only needs the ability to view/list/get resources (like a readonly Agent), but with the current architecture out of the box available for MCP, that’s not doable.

However, with traffic policies via agentgateway, it is.

Create a policy that specifies no tools available for use. This will help in testing the ability to lock down tools via the Gateway.

kubectl apply -f- <<EOF
apiVersion: gateway.kgateway.dev/v1alpha1
kind: TrafficPolicy
metadata:
  name: jwt-rbac
  namespace: gloo-system
spec:
  targetRefs:
    - group: gateway.kgateway.dev
      kind: Backend
      name: mcp-backend
  rbac:
    policy:
      matchExpressions:
        - 'mcp.tool.name == ""'
EOF

Disconnect and reconnect via the MCP Inspector and you should see no tools available.

Update the policy to include the fetch tool.

kubectl apply -f- <<EOF
apiVersion: gateway.kgateway.dev/v1alpha1
kind: TrafficPolicy
metadata:
  name: jwt-rbac
  namespace: gloo-system
spec:
  targetRefs:
    - group: gateway.kgateway.dev
      kind: Backend
      name: mcp-backend
  rbac:
    policy:
      matchExpressions:
        - 'mcp.tool.name == "fetch"'
EOF

Reconnect to the MCP Server via Inspector and you’ll now see the tool available.

Conclusion

With all of the security concerns around MCP Servers, it reminds us of a very important aspect of cyber security - it’s not about trying to block all bad actors, it’s about mitigating as much risk as possible. That should be the goal for every organization and with these implementations, your MCP security posture should be in a much better place.

FinOps For Agentic: How To Capture Token Usage Cost Across LLMs

Michael Levan — Tue, 25 Nov 2025 15:25:48 +0000

There's one major topic that every organization is talking about right now when it comes to Agentic workloads:

How am I going to track cost?

Tracking cost comes down to Agentic traffic, LLM traffic, and overall Token usage. The problem is that right now, it's scattered. Everyone is using an Agent and that Agent is tied to an API key, not a Gateway or a user. There's no way to track the cost.

Until now.

In this blog post, you'll learn how to track all AI/LLM/Token usage/cost across all Gateways at once with agentgateway.

Prerequisites

To follow along with this blog post from a hands-on perspective, you'll want to have the following:

A Kubernetes cluster
An Anthropic API key

If you don't have these readily available, you can follow along from a theoretical perspective. If you have another AI provider like OpenAI and not Anthropic, you can swap out the Model/LLM calls in the Backend object for whichever Model you want to use.

Gateway Installation

In this section, you will use two open-source tools - kgateway as the control plane and agentgateway as the data plane/proxy.

Install the Kubernetes Gateway API CRDs.

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/standard-install.yaml

Install the CRDs for kgateway.

helm upgrade -i --create-namespace --namespace kgateway-system --version v2.2.0-main \
kgateway-crds oci://cr.kgateway.dev/kgateway-dev/charts/kgateway-crds \
--set controller.image.pullPolicy=Always

Install kgateway with agentgateway enabled.

helm upgrade -i --namespace kgateway-system --version v2.2.0-main kgateway oci://cr.kgateway.dev/kgateway-dev/charts/kgateway \
  --set gateway.aiExtension.enabled=true \
  --set agentgateway.enabled=true  \
  --set controller.image.pullPolicy=Always

Confirm that the kgateway control plane is operational.

kubectl get pods -n kgateway-system

Grafana Configuration

For the purposes of this blog post, you'll use the kube-prometheus Stack to set up metric collection and showcase those metrics via a dashboard within Grafana to see the AI/LLM/Token usage/cost. To accomplish that, you'll install kube-prometheus and configure ServiceMonitors/PrometheusRules

Install Kube-Prometheus

Install kube-prometheus.

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

helm upgrade --install kube-prometheus-stack prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --create-namespace \
  --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false \
  --set prometheus.prometheusSpec.podMonitorSelectorNilUsesHelmValues=false \
  --set prometheus.prometheusSpec.ruleSelectorNilUsesHelmValues=false \
  --set prometheus.prometheusSpec.retention=30d \
  --set prometheus.prometheusSpec.storageSpec.volumeClaimTemplate.spec.resources.requests.storage=50Gi

Access the Grafana dashboard.

kubectl port-forward -n monitoring svc/kube-prometheus-stack-grafana 3000:80

Username: admin

Password retrieval: kubectl get secret -n monitoring kube-prometheus-stack-grafana -o jsonpath='{.data.admin-password}' | base64 -d && echo

Configure Monitors

Go to this repo, copy the monitoring.yaml config, save it, and apply it.

kubectl apply -f monitoring.yaml

Configure The Dashboard

The last step is to configure a proper dashboard so you can see all of the Token/LLM/AI costs via the Gateways.

Open Grafana
Go to Dashboards > Import
Upload grafana-dashboard.json or paste its contents. You can find the dashboard here.
Select "Prometheus" as the data source and click Import

Create Gateways

Now that kgateway is installed, you can create Gateways with backends to LLMs. In this section, you'll create 3 very similar Gateways. The reason you'll create 3 is to showcase how you can collect LLM/AI/Token costs across multiple Gateways at once instead of having to collect the cost for each Gateway one at a time, which saves engineering/FinOps cycles.

Configure Gateways

Set your Anthropic API key as an environment variable.

export ANTHROPIC_API_KEY=

Create a Secret for your API key.

kubectl apply -f- <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: anthropic-secret
  namespace: kgateway-system
  labels:
    app: agentgateway
type: Opaque
stringData:
  Authorization: $ANTHROPIC_API_KEY
EOF

Create Gateway/Route/Backend number 1.

kubectl apply -f- <<EOF
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1
metadata:
  name: agentgateway1
  namespace: kgateway-system
  labels:
    app: agentgateway1
spec:
  gatewayClassName: agentgateway
  listeners:
  - protocol: HTTP
    port: 8080
    name: http
    allowedRoutes:
      namespaces:
        from: All
---
apiVersion: gateway.kgateway.dev/v1alpha1
kind: Backend
metadata:
  labels:
    app: agentgateway1
  name: anthropic1
  namespace: kgateway-system
spec:
  type: AI
  ai:
    llm:
        anthropic:
          authToken:
            kind: SecretRef
            secretRef:
              name: anthropic-secret
          model: "claude-3-5-haiku-latest"
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: claude1
  namespace: kgateway-system
  labels:
    app: agentgateway1
spec:
  parentRefs:
    - name: agentgateway1
      namespace: kgateway-system
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /anthropic
    filters:
    - type: URLRewrite
      urlRewrite:
        path:
          type: ReplaceFullPath
          replaceFullPath: /v1/chat/completions
    backendRefs:
    - name: anthropic1
      namespace: kgateway-system
      group: gateway.kgateway.dev
      kind: Backend
EOF

Number 2.

kubectl apply -f- <<EOF
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1
metadata:
  name: agentgateway2
  namespace: kgateway-system
  labels:
    app: agentgateway2
spec:
  gatewayClassName: agentgateway
  listeners:
  - protocol: HTTP
    port: 8080
    name: http
    allowedRoutes:
      namespaces:
        from: All
---
apiVersion: gateway.kgateway.dev/v1alpha1
kind: Backend
metadata:
  labels:
    app: agentgateway2
  name: anthropic2
  namespace: kgateway-system
spec:
  type: AI
  ai:
    llm:
        anthropic:
          authToken:
            kind: SecretRef
            secretRef:
              name: anthropic-secret
          model: "claude-3-5-haiku-latest"
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: claude2
  namespace: kgateway-system
  labels:
    app: agentgateway2
spec:
  parentRefs:
    - name: agentgateway2
      namespace: kgateway-system
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /anthropic
    filters:
    - type: URLRewrite
      urlRewrite:
        path:
          type: ReplaceFullPath
          replaceFullPath: /v1/chat/completions
    backendRefs:
    - name: anthropic2
      namespace: kgateway-system
      group: gateway.kgateway.dev
      kind: Backend
EOF

Number 3.

kubectl apply -f- <<EOF
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1
metadata:
  name: agentgateway3
  namespace: kgateway-system
  labels:
    app: agentgateway3
spec:
  gatewayClassName: agentgateway
  listeners:
  - protocol: HTTP
    port: 8080
    name: http
    allowedRoutes:
      namespaces:
        from: All
---
apiVersion: gateway.kgateway.dev/v1alpha1
kind: Backend
metadata:
  labels:
    app: agentgateway3
  name: anthropic3
  namespace: kgateway-system
spec:
  type: AI
  ai:
    llm:
        anthropic:
          authToken:
            kind: SecretRef
            secretRef:
              name: anthropic-secret
          model: "claude-3-5-haiku-latest"
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: claude3
  namespace: kgateway-system
  labels:
    app: agentgateway3
spec:
  parentRefs:
    - name: agentgateway3
      namespace: kgateway-system
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /anthropic
    filters:
    - type: URLRewrite
      urlRewrite:
        path:
          type: ReplaceFullPath
          replaceFullPath: /v1/chat/completions
    backendRefs:
    - name: anthropic3
      namespace: kgateway-system
      group: gateway.kgateway.dev
      kind: Backend
EOF

Test Gateways

Add the load balancer IP into an environment variable for each Gateway.

export INGRESS_GW_ADDRESSONE=$(kubectl get svc -n kgateway-system agentgateway1 -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
echo $INGRESS_GW_ADDRESSONE

export INGRESS_GW_ADDRESSTWO=$(kubectl get svc -n kgateway-system agentgateway2 -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
echo $INGRESS_GW_ADDRESSTWO

export INGRESS_GW_ADDRESSTHREE=$(kubectl get svc -n kgateway-system agentgateway3 -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
echo $INGRESS_GW_ADDRESSTHREE

Test each Gateway.

curl "$INGRESS_GW_ADDRESSONE:8080/anthropic" -v \ -H content-type:application/json -H x-api-key:$ANTHROPIC_API_KEY -H "anthropic-version: 2023-06-01" -d '{
  "model": "claude-sonnet-4-5",
  "messages": [
    {
      "role": "system",
      "content": "You are a skilled cloud-native network engineer."
    },
    {
      "role": "user",
      "content": "Write me a paragraph containing the best way to think about Istio Ambient Mesh"
    }
  ]
}' | jq

curl "$INGRESS_GW_ADDRESSTWO:8080/anthropic" -v \ -H content-type:application/json -H x-api-key:$ANTHROPIC_API_KEY -H "anthropic-version: 2023-06-01" -d '{
  "model": "claude-sonnet-4-5",
  "messages": [
    {
      "role": "system",
      "content": "You are a skilled cloud-native network engineer."
    },
    {
      "role": "user",
      "content": "Write me a paragraph containing the best way to think about Istio Ambient Mesh"
    }
  ]
}' | jq

curl "$INGRESS_GW_ADDRESSTHREE:8080/anthropic" -v \ -H content-type:application/json -H x-api-key:$ANTHROPIC_API_KEY -H "anthropic-version: 2023-06-01" -d '{
  "model": "claude-sonnet-4-5",
  "messages": [
    {
      "role": "system",
      "content": "You are a skilled cloud-native network engineer."
    },
    {
      "role": "user",
      "content": "Write me a paragraph containing the best way to think about Istio Ambient Mesh"
    }
  ]
}' | jq

Metrics Testing

Now that everything is deployed, you can start checking the metrics for Token cost/usage, LLM cost, and see it all for every Gateway within the same dashboard.

Port-forward Prometheus

kubectl port-forward -n monitoring svc/kube-prometheus-stack-prometheus 9090:9090

Check Token usage.

curl -s 'http://localhost:9090/api/v1/query?query=agentgateway:input_tokens:total' | jq '.data.result[0].value'

Check the overall cost.

curl -s 'http://localhost:9090/api/v1/query?query=agentgateway:cost_usd:total_daily' | jq '.data.result[0].value'

You should see outputs similar to the below:

[
  1763214193.134,
  "41.31458333333333"
]

[
  1763214214.598,
  "0.0008821471428571428"
]

Go to the Grafana Dashboard and you'll now see all of the Gateway costs per Gateway.

Deploying Local AI Agents In Kubernetes

Michael Levan — Sat, 08 Nov 2025 16:09:44 +0000

There are two types of Models/LLMs you see in today's Agentic world:

"SaaS-based Models", which are Models that are managed for you (Claude, Gemini, GPT, etc.)
Local Models, which you manage yourself.

From a security, governance, and overall data control perspective, some organizations want to go with local Models.

In this blog post, you'll learn how to manage and deploy a local Model using Kubernetes primitives and kagent.

Prerequisites

To follow along with this blog post in a hands-on fashion, you should have the following:

A Kubernetes cluster. If you're using a local cluster, ensure that your local machine has enough CPU/memory for a more resource-intensive environment.
An Anthropic API key. If you don't have one and/or prefer to use another AI provider, there are several providers supported by kagent.

Deploying Ollama

The first step is to deploy your local Model. In this case, you'll see Ollama, which is a popular Model for local deployments.

Create a Kubernetes Namespace for your Llama Model.

kubectl create ns ollama

Deploy the Ollama Model as a Kubernetes Deployment and attach a Service to it. Notice how there's a fair amount of CPU and memory given to the Deployment. The reason is that local models are typically slower. The goal with more CPU and memory (when a GPU doesn't exist) is that it'll be faster to use.

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ollama
  namespace: ollama
spec:
  selector:
    matchLabels:
      name: ollama
  template:
    metadata:
      labels:
        name: ollama
    spec:
      initContainers:
      - name: model-puller
        image: ollama/ollama:latest
        command: ["/bin/sh", "-c"]
        args:
          - |
            ollama serve &
            sleep 10
            ollama pull llama3
            pkill ollama
        volumeMounts:
        - name: ollama-data
          mountPath: /root/.ollama
        resources:
          requests:
            memory: "8Gi"
          limits:
            memory: "12Gi"
      containers:
      - name: ollama
        image: ollama/ollama:latest
        ports:
        - name: http
          containerPort: 11434
          protocol: TCP
        volumeMounts:
        - name: ollama-data
          mountPath: /root/.ollama
        resources:
          requests:
            memory: "8Gi"
          limits:
            memory: "12Gi"
      volumes:
      - name: ollama-data
        emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
  name: ollama
  namespace: ollama
spec:
  type: ClusterIP
  selector:
    name: ollama
  ports:
  - port: 80
    name: http
    targetPort: http
    protocol: TCP
EOF

Give the Pod a few minutes to get up and running, as it's fairly large and it's downloaded the Llama Model.

Confirm that the Model was downloaded.

kubectl exec -n ollama deployment/ollama -- ollama list

You should see an output similar to the one below, indicating that the Model has been downloaded successfully.

Defaulted container "ollama" out of: ollama, model-puller (init)
NAME             ID              SIZE      MODIFIED
llama3:latest    365c0bd3c000    4.7 GB    About a minute ago

Deploying kagent

Now that the Llama Model is on your Kubernetes cluster, you can deploy kagent to manage that Model and attach an Agent to it.

Install the kagent CRDs.

helm install kagent-crds oci://ghcr.io/kagent-dev/kagent/helm/kagent-crds \
    --namespace kagent \
    --create-namespace

Set an environment variable for your Anthropic API key.

export ANTHROPIC_API_KEY=your_api_key

💡

Install kagent.

helm upgrade --install kagent oci://ghcr.io/kagent-dev/kagent/helm/kagent \
    --namespace kagent \
    --set providers.default=anthropic \
    --set providers.anthropic.apiKey=$ANTHROPIC_API_KEY \
    --set ui.service.type=LoadBalancer

💡

Retrieve the IP address of your Agent.

kubectl get svc -n kagent

If you're running locally and don't have a way to retrieve a public IP address, you can port-forward the kagent UI service.

kubectl port-forward svc/kagent-ui -n kagent 8080:8080

Open the kagent UI and either go through the wizard or click the skip button on the bottom left (going through the wizard isn't needed for the purposes of this blog post).

You should see the UI similar to the screenshot below.

Create A Model Config

With kagent installed on your Kubernetes cluster, you can manage Agents, Models, and MCP Servers in a declarative fashion. One object you can use is the ModelConfig, which allows you to import a Model into kagent. In this case, you'll import the Llama Model that you created.

Run the following configuration (notice how it's pointing to the Ollama Kubernetes Service).

kubectl apply -f - <<EOF
apiVersion: kagent.dev/v1alpha2
kind: ModelConfig
metadata:
  name: llama3-model-config
  namespace: kagent
spec:
  model: llama3
  provider: Ollama
  ollama:
    host: http://ollama.ollama.svc.cluster.local:80
EOF

Get the Model config to ensure that it deployed successfully.

kubectl get modelconfig -n kagent

You'll see the AI Provider that you did the kagent installation with and Ollama.

NAME                   PROVIDER    MODEL
default-model-config   Anthropic   claude-3-5-haiku-20241022
llama3-model-config    Ollama      llama3

Go to the UI and click on View > Models

You should now see the Model within kagent.

You'll also now see Llama as an option within kagent when you create an Agent.

AWS EKS Model Context Protocol (MCP): How It Improves Kubernetes Reliability

Michael Levan — Tue, 08 Jul 2025 12:51:32 +0000

The closer we get to using AI in our day-to-day, the more we'll need to ensure that the data we're interacting with from the LLMs that are used is accurate. This way, engineers and technical leadership teams can ensure the usage is worth the time and expense (expense of using LLMs and time for engineers to get trained up on them).

The majority of organizations cannot spend millions of dollars to train a Model to help with AIOps and Kubernetes workloads, but MCP Servers can be used to ensure accurate and reliable information.

In this blog post, you'll learn the key aspects of the EKS MCP Server, how to use the EKS MCP server for your Elastic Kubernetes Service cluster, and how to interact with an MCP Server using Python.

Prerequisites

If you want to follow along from a hands-on perspective, you'll need:

An AWS account and an EKS cluster deployed. If you don't have an EKS cluster deployed, this blog post will still be very helpful as you'll have a code example for when you do have an EKS cluster deployed.
Python3 installed.

Model Context Protocol (MCP) Recap

When thinking about what MCP is, think "An API call to a collection of needed data". As the name suggest, it could be an actual server/VM that's hosting the data. Programmatically, it could be a client/server architecture where the "server" in the case of MCP is something like a PyPi package that you can "download" and call upon. You'd then use a client (e.g - a script) to access the tools within the MCP Server.

The overall goal of MCP is simple; accurate information. Sometimes, LLMs don't give you accurate information and because everyone wants as close to "accurate" as an LLM can get, an MCP Server can be used to store accurate information for the job you're trying to do. That's why MCP Servers are popping up for everything from Kubernetes to documentation to CICD.

💡I see MCP Servers as an almost "universal RAG". When building a RAG into an application, it's doing the same thing - calling upon accurate information (blog links, articles, etc.). An MCP Server is doing the same thing. The only difference is it isn't tied to a particular application like a RAG is.

The AWS EKS MCP Server

The EKS MCP Server, as the name suggests, is an MCP Server that's designed for EKS.

It has the following tools/functionality:

list_k8s_resources: List k8s resources.
list_api_versions: List k8s API versions.
manage_k8s_resource: CRUD operations for k8s resources
apply_yaml: Like kubectl apply -f
get_k8s_events: Like kubectl get eventssuggests
get_pod_logs: Like kubectl get pods

💡Here's the part that kind of still has me scratching my head. Things like listing Kubernetes resources and applying Kubernetes Manifests, at least in my opinion, don't really seem like something an MCP Server needs to do. My only assumption is (and this is an early assumption because MCP is only about 6 months old at the time of writing this) is maybe it'll be used for some autonomous actions (e.g - AI, go deploy this thing for me).

manage_eks_stacks: CloudFormation templates for EKS clusters.
search_eks_troubleshoot_guide: Search EKS docs for info.
get_cloudwatch_logs: Logs from CloudWatch for Pods or the Control Plane.
get_cloudwatch_metrics: Metrics from CloudWatch for Pods and Clusters.

Combining all of this functionality in an MCP Server, you have a way to troubleshoot, deploy, and receive information about workloads running in EKS.

Using The AWS EKS MCP Server

There are a few different ways to interact with an MCP Server. In the case of the EKS MCP Server, you'll need a client. That client could be anything from Cline for VS Code to a Python script that interacts with the mcp library.

In this case, we'll see how it's done with Python.

First, below is what an MCP Server configuration looks like. It's use uvx, which uses uv to execute Python tools. Notice how it also takes some environment variables and arguments.

💡uv is a Python package manager that's written in Rust. It's gaining a ton of popularity for replacing pip.

{
    "mcpServers": {
        "awslabs.eks-mcp-server": {
            "command": "uvx",
            "args": [
                "awslabs.eks-mcp-server",
                "--allow-write"
            ],
            "env": {
                "AWS_PROFILE": "default",
                "AWS_REGION": "us-west-2",
                "FASTMCP_LOG_LEVEL": "INFO"
            }
        }
    }
}

First, import the libraries that are needed. In this case, it'll be the MCP libraries, the async library for concurrent tasks, and the JSON library for clean output to the terminal.

import asyncio
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
import json

Next, specify a function. In this case, it'll be an async function. MCP Servers are I/O centric, so having concurrency actions for handling I/O bound tasks is important.

async def main()

The first piece of logic in the function is to call out to the AWS EKS MCP Server. In this case, it's hard-coded into the Python script. However, you could save it as a .json file and call upon it instead of hard-coding it.

    server_params = StdioServerParameters(
        command="uvx",
        args=["awslabs.eks-mcp-server", "--allow-write"],
        env={
            "AWS_PROFILE": "default",
            "AWS_REGION": "us-east-1",
            "FASTMCP_LOG_LEVEL": "INFO"
        }
    )

Once the EKS MCP Server is defined, you can call upon it via the MCP library and specify some functionality. In the example below, it's using the list_k8s_resources tool and specifying the name of a cluster and the kind/object that you want to call upon within the list_k8s_resources tool

    async with stdio_client(server_params) as (read, write):
        async with ClientSession(read, write) as session:
            await session.initialize()

            result = await session.call_tool("list_k8s_resources", arguments={
                'cluster_name': 'k8squickstart-cluster',
                'kind': 'Pod',
                'api_version': 'v1',
            })

The last step is to output the results to the terminal.

            for c in result.content:
                data = json.loads(c.text)
                print(json.dumps(data, indent=2))

Putting the code together, it'll look like the below.

import asyncio
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
import json

async def main():
    server_params = StdioServerParameters(
        command="uvx",
        args=["awslabs.eks-mcp-server", "--allow-write"],
        env={
            "AWS_PROFILE": "default",
            "AWS_REGION": "us-east-1",
            "FASTMCP_LOG_LEVEL": "INFO"
        }
    )

    # Connect to EKS MCP server
    async with stdio_client(server_params) as (read, write):
        async with ClientSession(read, write) as session:
            await session.initialize()

            result = await session.call_tool("list_k8s_resources", arguments={
                'cluster_name': 'k8squickstart-cluster',
                'kind': 'Pod',
                'api_version': 'v1',
            })

            for c in result.content:
                data = json.loads(c.text)
                print(json.dumps(data, indent=2))

if __name__ == "__main__":
    asyncio.run(main())

Save the code above in a file called eksmcp.py.

Running The Client

Now that the client is built, the only thing left is to run it and confirm that it works as expected.

From the terminal where you saved the script, run the following command:

python3 eksmcppy

💡Depending on your OS, you may have to use python instead of python3. It all depends on how you configired your alias.

The first output you'll see is the authentication/authorization step, which looks at your AWS config locally for proper access.

2025-07-08 08:45:33.682 | INFO     | awslabs.eks_mcp_server.server:main:140 - Starting EKS MCP Server in restricted sensitive data access mode
[07/08/25 08:45:33] INFO     Processing request of type CallToolRequest                                                   server.py:619
                    INFO     Found credentials in shared credentials file: ~/.aws/credentials                       credentials.py:1352
                    INFO     Found credentials in shared credentials file: ~/.aws/credentials                       credentials.py:1352

The second output is a quick list of what resources (in this case, Pods) are deployed.

2025-07-08 08:45:35.392 | INFO     | awslabs.eks_mcp_server.logging_helper:log_with_request_id:49 - [request_id=1] Cleaned up resource responses for Pod resources
2025-07-08 08:45:35.392 | INFO     | awslabs.eks_mcp_server.logging_helper:log_with_request_id:49 - [request_id=1] Listed 8 Pod resources in all namespaces

You'll then see a JSON output of the Pods running within your environment.

Congrats! You've successfully implemented a solid use case of the AWS EKS MCP Server with Python.