MCP Rate Limiting: Lessons from Three Production Blocks

Honestly, I thought adding rate limiting to an MCP server would be boring. Like, just drop in a library, set some limits, call it a day. How wrong was I.

After getting blocked three times in production over three weeks, I learned that rate limiting for MCP isn't quite the same as it is for regular HTTP APIs. The interaction model is different, the clients expect different things, and small mistakes can break your entire server in really confusing ways.

Let me walk you through what got blocked, what I broke, and what I finally got working after three weeks of pain.

The Backstory: Why I Even Needed Rate Limiting

I've been building Papers — a personal knowledge base that exposes everything through an MCP server. That means any AI client (Claude Desktop, Cursor, ChatGPT) can connect directly to it and search my notes, pull context, add new notes, all without me copying and pasting everything manually.

After writing 70+ articles about building MCP servers and getting this thing into production, I started actually using it. And guess what? I kept getting blocked. Not by GitHub, not by OpenAI — by myself. Well, by my own AI client keeping open connections and hammering my server with multiple concurrent requests when I wasn't expecting it.

Let me show you what I started with, and what broke.

The First Block: Too Many Concurrent Requests

When I first deployed, I just used the default Spring Boot configuration. No rate limiting, no connection limiting. I was thinking "it's just me using this, how much traffic can I generate?"

Turns out: when you have an AI client that keeps a persistent connection open, and it automatically retries when things time out, and you have multiple AI clients connected at the same time... you can absolutely overwhelm your own tiny server.

I got hit with:

• 15 concurrent tools/list requests in 10 seconds
• 8 concurrent tools/call requests all hitting the database at once
• Connection pool exhaustion in my reverse proxy
• 5xx errors from my hosting provider because I hit the connection limit

The whole thing went down. And honestly? I was shocked. It's just me using the server. How did this happen?

Turns out: MCP clients keep connections open for streaming, and they can have multiple ongoing conversations. Each conversation can keep a connection alive. If you have Claude Desktop + Cursor + Claude Code all connected... that's multiple connections right there. Each one can fire multiple requests.

Add automatic retries on timeout, and you've got a thundering herd before you know it.

The Second Block: I Rate Limited the Wrong Thing

Alright, so I added rate limiting. Easy, right? Just throw in a filter that counts requests per client, block anything over the limit, done.

I used Spring Security's bucket4j integration, set it up to limit 10 requests per minute per IP, and called it a day.

Got blocked again. What happened this time?

MCP does streaming. Long-lived streaming responses. One tools/call can stream content for 10+ seconds. If you rate limit by requests, you can still get flooded with concurrent long-running streams that eat all your connection count and memory.

I had 8 concurrent streaming requests each taking 15 seconds — that still took down my server. Rate limiting counts how many requests, but it doesn't count how long they're connected or how much bandwidth they're using.

So I fixed that: I added concurrent connection limiting. Now you can only have X concurrent requests at a time. Great.

Got blocked a third time.

The Third Block: I Didn't Handle the Disconnect Properly

This one was really subtle.

What happens when you hit the rate limit and you reject the connection? For a normal HTTP API, you just return 429 Too Many Requests, and the client handles it. Done.

But MCP uses JSON-RPC over a persistent streaming connection. If you reject the connection at the HTTP level before the JSON-RPC handshake even completes... the client doesn't get a JSON-RPC error response. It just gets disconnected.

And what does the client do when it gets disconnected? It retries.

So I got into this loop:

1. Client connects → I reject at HTTP level → client disconnects → client retries → reconnect → reject → retry → repeat
2. This creates a thundering herd of retries that still takes down the server
3. Because every failed connection attempt still consumes a connection slot

I couldn't believe I messed this up. Three times now.

Alright, let's start over and do this properly.

The Final Solution: Layered Rate Limiting for MCP

After three weeks of getting blocked, I ended up with three layers of rate limiting that actually work for MCP:

1. Concurrent connection limiting at the HTTP level — limit how many active connections you'll accept
2. Request rate limiting at the JSON-RPC level — limit how many requests per client
3. Proper error handling — send JSON-RPC error instead of dropping the connection

Let me show you the code. This is all production tested, and you can drop it straight into your own Spring Boot MCP server.

Step 1: Concurrent Connection Filter

First, we need to limit how many active concurrent connections we'll accept. This protects against the thundering herd before any request even gets processed.

@Component
public class ConnectionCountFilter extends OncePerRequestFilter {

    private final AtomicInteger activeConnections = new AtomicInteger(0);
    private final int maxConnections = 20; // adjust based on your server size

    @Override
    protected void doFilterInternal(
            HttpServletRequest request,
            HttpServletResponse response,
            FilterChain filterChain
    ) throws ServletException, IOException {

        int current = activeConnections.incrementAndGet();
        if (current > maxConnections) {
            activeConnections.decrementAndGet();
            // 👉 Important: DO NOT close the connection here!
            // We need to send a proper JSON-RPC error to the client
            // If you close it here, the client will retry forever
            response.setStatus(HttpStatus.TOO_MANY_REQUESTS.value());
            response.setContentType("application/json");
            // MCP uses JSON-RPC, so we need to send a valid error response
            // Even though this is HTTP-level, some clients handle this better than disconnecting
            PrintWriter writer = response.getWriter();
            writer.write("{\"jsonrpc\":\"2.0\",\"id\":null,\"error\":{\"code\":-32000,\"message\":\"Too many concurrent connections. Please try again later.\"}}");
            writer.flush();
            return;
        }

        try {
            filterChain.doFilter(request, response);
        } finally {
            activeConnections.decrementAndGet();
        }
    }
}

This is straightforward, but the key mistake I made earlier was just returning 429 without a body. Sending a proper JSON-RPC error message lets the client handle it gracefully instead of retrying blindly.

Step 2: Per-Request Rate Limiting with Bucket4j

Next, we need rate limiting per client. I use Bucket4j with Redis for distributed rate limiting, but if you're running a single instance you can do it in-memory.

@Component
@Order(Ordered.HIGHEST_PRECEDENCE)
public class McpRateLimitingFilter extends OncePerRequestFilter {

    private final BucketConfig bucketConfig;
    private final RateLimiterRegistry rateLimiterRegistry;

    public McpRateLimitingFilter(RateLimiterRegistry rateLimiterRegistry, BucketConfig bucketConfig) {
        this.rateLimiterRegistry = rateLimiterRegistry;
        this.bucketConfig = bucketConfig;
    }

    private Bucket createNewBucket(String clientIp) {
        Bandwidth limit = Bandwidth.builder()
                .capacity(60) // 60 tokens = 60 requests
                .refillGreedy(60, Duration.ofMinutes(1)) // refill 60 tokens per minute
                .build();
        return Bucket.builder().addLimit(limit).build();
    }

    @Override
    protected void doFilterInternal(
            HttpServletRequest request,
            HttpServletResponse response,
            FilterChain filterChain
    ) throws ServletException, IOException {

        String clientIp = request.getRemoteAddr();
        Bucket bucket = rateLimiterRegistry.getOrCreate(
                "mcp-rate-limit-" + clientIp,
                () -> createNewBucket(clientIp)
        );

        if (bucket.tryConsume(1)) {
            filterChain.doFilter(request, response);
        } else {
            response.setStatus(HttpStatus.TOO_MANY_REQUESTS.value());
            response.setContentType("application/json");
            PrintWriter writer = response.getWriter();
            writer.write("{\"jsonrpc\":\"2.0\",\"id\":null,\"error\":{\"code\":-32000,\"message\":\"Rate limit exceeded. You are limited to 60 requests per minute. Please try again later.\"}}");
            writer.flush();
        }
    }
}

For in-memory rate limiting (good enough for personal servers):

@Configuration
public class BucketConfig {
    // Default token expiration after 1 minute of inactivity
    public static final int DEFAULT_CAPACITY = 60;
    public static final int REFILL_INTERVAL_MINUTES = 1;
}

// Register as a bean:
@Bean
public RateLimiterRegistry rateLimiterRegistry() {
    return new InMemoryRateLimiterRegistry();
}

Step 3: Handle Timeouts Properly with Graceful Degradation

One thing I didn't anticipate: MCP tools can take a while to run. A search might take 10+ seconds if it's pulling lots of context. If you have a timeout that's too short, you get half-completed requests and clients retry, which creates duplicate work.

I added this filter to handle timeouts properly:

@Component
public class McpTimeoutFilter extends OncePerRequestFilter {

    @Override
    protected void doFilterInternal(
            HttpServletRequest request,
            HttpServletResponse response,
            FilterChain filterChain
    ) throws ServletException, IOException {

        // MCP streaming can take time, give it 5 minutes
        // adjust based on what your tools do
        request.setAttribute("org.apache.catalina.ASYNC_TIMEOUT", TimeUnit.MINUTES.toMillis(5));
        filterChain.doFilter(request, response);
    }
}

And configure your connector to allow async:

@Configuration
public class MvcConfig implements WebMvcConfigurer {

    @Override
    public void configureAsyncSupport(AsyncSupportConfigurer configurer) {
        // 5 minute timeout for async requests (MCP streaming)
        configurer.setDefaultTimeout(TimeUnit.MINUTES.toMillis(5));
    }
}

Step 4: The Complete Picture — Filter Ordering

The order matters! You want to check connection count first, then rate limit, then process the request. This way, you block early before wasting resources on requests you're going to reject anyway.

1. ConnectionCountFilter (check concurrent connections first)
   → if over limit, reject immediately with JSON-RPC error
2. McpRateLimitingFilter (check request rate limit)
   → if over quota, reject with JSON-RPC error
3. McpTimeoutFilter (set long timeout for streaming)
4. Your MCP controller (process the request)

What I Got Wrong: Pros & Cons of This Approach

Alright, let's be honest. This isn't perfect. Let me tell you what works and what doesn't.

Pros ✅

1. It actually stops the thundering herd — after adding this three-layer approach, I haven't gotten blocked again. Three weeks later, still running strong.
2. Clients get meaningful error messages — instead of just disconnecting, they get "rate limit exceeded" and can show it to the user instead of retrying forever.
3. It's simple — you don't need any fancy distributed stuff for a personal server. In-memory rate limiting works fine.
4. Drop-in code — you can literally copy these three filters into your project and it just works. I did it, you can too.

Cons ❌

1. Rate limiting by IP isn't perfect — if you're behind NAT, multiple clients on the same IP share the limit. For a personal server that's fine, but for multi-user you'd want to rate limit by API key instead.
2. No token bursting — the greedy refill I use allows full bursting, which is what I want for personal use. If you need stricter limiting you'd want a different refill strategy.
3. Doesn't limit message size — you can still get hit with huge payloads that eat your bandwidth. If that's a problem you'd need to add size checking too. I haven't needed it yet for my personal use.

My Actual Numbers: What Works for a Personal MCP Server

After experimenting, here's what settled on that works for me:

• Max concurrent connections: 20 — that's enough for 3-4 AI clients all connected at once, with room for multiple requests per client. If you're the only user using it, this is plenty.
• Rate limit: 60 requests per minute — each request is a tools/list or tools/call, so 60 per minute is way more than you'll ever need as a human, but low enough to prevent run-away retries from killing your server.
• Timeout: 5 minutes — gives your tools plenty of time to run even complex queries. I've never hit this in practice, but it's there to stop hung requests from sticking around forever.

What Would I Do Differently for a Multi-User MCP Server?

If this was a public server with multiple users, I'd change a few things:

1. Rate limit by API key, not IP — each user gets their own rate limit bucket. MCP supports API key auth, so this fits naturally.
2. Lower concurrent connection limit per user — 10 per user instead of 20 total.
3. Strict rate limiting with token bucket instead of greedy refill — to prevent one user from hogging all the resources.
4. Add request size limiting — prevent extremely large requests from eating your bandwidth.
5. Monitoring and alerts — alert me when rate limiting is kicking in often, so I can adjust limits before users get mad.

Final Thoughts: MCP Rate Limiting Isn't Rocket Science, But It's Different

Honestly, I went into this thinking "rate limiting is rate limiting, how different can it be?"

But it is different. The persistent streaming connections, the JSON-RPC error handling, the automatic retries from clients when you disconnect — all these things trip you up if you just copy-paste regular HTTP rate limiting.

Three blocks later, I finally got it right. The key lessons:

1. Limit concurrent connections first — before you even count requests. Connections are your most constrained resource on a small server.
2. Send JSON-RPC errors, don't just drop the connection — dropping causes retries, which causes more blocking. Send a proper error and clients can handle it gracefully.
3. Give it time — MCP tool calls can take a while, set your timeout longer than you think you need.

I've been running this configuration for a month now after those three blocks, and it hasn't let me down. No more blocks, no more random server outages, just works.

---

Questions for You

Have you deployed an MCP server to production? Gotten blocked unexpectedly? What kind of rate limiting are you doing? Did you run into the same retry loop issues I did, or did you solve it a different way? Drop a comment below and let me know your stories!

---

This article is part of my series building MCP servers for personal knowledge bases. Check out the project on GitHub to see the full code, including all these filters in production use.