Why You Need Rate Limiting for MCP Servers (And How to Do It)

Without proper controls for how many times an Agent/LLM can hit an MCP Server, you open yourself up to potential DOS attacks, memory hogging, insane API bills, and server/system overload. Luckily, this can be mitigated quickly by rate-limiting the number of requests an Agent can make to an MCP Server.

In this blog post, you'll learn how to implement rate limiting for MCP using agentgateway.

Prerequisites

To follow along with this blog post from a hands-on perspective, you will need:

1. A k8s cluster (local with something like Kind or Minikube is perfect).
2. Agentgateway installed.
3. A GitHub account.

Rate Limiting in a Nutshell

If you think about what a memory leak in an application is conceptually, it's a bug in the software that effectively allows the application to continue to consume RAM/memory to perform tasks. The app silently takes more and more RAM it no longer needs, which forces the system that the software is running on to watch total available RAM in the system until a crisis occurs (e.g - the system runs out of memory).

In short, resources are consumed until a system crash occurs.

MCP Servers can have the same impact/outcome without proper rate limiting. LLMs retry if an error occurs.

This is part of its programming to ensure it gets you at least some answer, even if the answer isn't correct. It wants to satisfy whoever is using it, so some answer is better than no answer. What this means, however, especially when making MCP tool calls, is that a non-rate-limiting loop can cause huge API bills or crash your system because the Agent hitting the LLM/MCP Server will eat up all available memory.

With rate limiting for MCP, you can configure how many calls/requests can be made in a particular timeframe.

MCP Requests

Before jumping into rate limiting MCP requests, it's important to understand what the workflow looks like.

A typical MCP client HTTP tool call looks like the following:

[ Client ] -- HTTP POST /mcp (JSON-RPC: tools/call) --> [ Server ]
[ Client ] <-- HTTP 200 OK (JSON-RPC: result) --------- [ Server ]

When the client or Agent begins a request, it hits tools/list one time to have an understanding of what MCP tools are available. The client/Agent then caches the tool schemas (name, description, input parameters) locally. That way, it's not always making a call to tools-list as that would be incredibly inefficient and waste a ton of tokens.

When you tell an Agent to make a call, it looks at the local cache to see which tools fit. The tool can then be used by the Agent to perform an action (e.g - search_repositories if you're using the GitHub Copilot MCP Server).

LLM vs MCP Rate Limiting

When you limit LLM calls, you can specify the number of requests to an LLM based on a particular timeframe or limit tokens in a particular timeframe (e.g - 100 tokens every minute). MCP rate limiting is all about the number of requests to the MCP server in a specific timeframe.

Implementing MCP Rate Limiting

With the theory of MCP rate limiting and MCP tool calls/requests discussed, let's get hands-on and learn how to implement rate limiting for MCP. This section will cover two configurations:

1. Configuring a Gateway to use an MCP server.
2. The policy for rate limiting.

In the example, you'll use the GitHub Copilot MCP Server purely because the majority of people have access to GitHub, so it makes the demonstration universal.

MCP Gateway Configuration

1. Create a Kubernetes Secret that contains your GitHub PAT.

export GITHUB_PAT=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

kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: github-pat
  namespace: agentgateway-system
type: Opaque
stringData:
  Authorization: "Bearer ${GITHUB_PAT}"
EOF

1. Create a Gateway object that uses agentgateway as the Gateway Class.

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: mcp-gateway
  namespace: agentgateway-system
  labels:
    app: github-mcp-server
spec:
  gatewayClassName: agentgateway
  listeners:
    - name: mcp
      port: 3000
      protocol: HTTP
      allowedRoutes:
        namespaces:
          from: Same
EOF

1. Create an agentgatewaybackend so the Gateway knows what to route to. In this case, it's the GitHub Copilot MCP Server.

kubectl apply -f - <<EOF
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  name: github-mcp-server
  namespace: agentgateway-system
spec:
  mcp:
    sessionRouting: Stateless
    targets:
      - name: github-copilot
        static:
          host: api.githubcopilot.com
          port: 443
          path: /mcp/
          protocol: StreamableHTTP
          policies:
            tls: {}
            auth:
              secretRef:
                name: github-pat
EOF

1. Create a route/path with the reference as the agentgatewaybackend.

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: mcp-route
  namespace: agentgateway-system
  labels:
    app: github-mcp-server
spec:
  parentRefs:
    - name: mcp-gateway
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /mcp
      backendRefs:
        - name: github-mcp-server
          namespace: agentgateway-system
          group: agentgateway.dev
          kind: AgentgatewayBackend
EOF

1. Capture the Gateways IP in the GATEWAY_IP environment variable.

export GATEWAY_IP=$(kubectl get svc mcp-gateway -n agentgateway-system -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo $GATEWAY_IP

1. Test the connection.

curl -v -sS -X POST "http://${GATEWAY_IP}:3000/mcp" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{
    "jsonrpc": "2.0",
    "id": 3,
    "method": "tools/call",
    "params": { "name": "get_me", "arguments": {} }
  }'

You should see an output similar to the below:

MCP Rate Limit Policy

With the Gateway created, let's create and test a rate limit policy.

1. Run the below. Out of the box/by default, you'll see that all 11 requests to the MCP Server work just fine.

for i in {1..11}; do
  curl -v -sS -o /dev/null -w "req $i: %{http_code}\n" \
    -X POST "http://${GATEWAY_IP}:3000/mcp" \
    -H "Content-Type: application/json" \
    -H "Accept: application/json, text/event-stream" \
    -d "{\"jsonrpc\":\"2.0\",\"id\":$i,\"method\":\"tools/list\"}"
done

1. Create a policy that blocks more than 10 requests

kubectl apply -f- <<EOF
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayPolicy
metadata:
  name: mcp-rate-limit
  namespace: agentgateway-system
spec:
  targetRefs:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: mcp-route
  traffic:
    rateLimit:
      local:
      - burst: 0
        requests: 10
        unit: Minutes
EOF

1. Try running the curl again without the v flag for verbose.

for i in {1..11}; do
  curl -sS -o /dev/null -w "req $i: %{http_code}\n" \
    -X POST "http://${GATEWAY_IP}:3000/mcp" \
    -H "Content-Type: application/json" \
    -H "Accept: application/json, text/event-stream" \
    -d "{\"jsonrpc\":\"2.0\",\"id\":$i,\"method\":\"tools/list\"}"
done

You'll see that the 11th request failed.

Congrats! You've officially set up and configured rate limiting for an MCP Server.