DEV Community: Sindhuja Nagaraja Sudhakar

When AI Agents Hammer tools/list: Building a Caching-Aware MCP Server (SEP-2549)

Sindhuja Nagaraja Sudhakar — Mon, 21 Sep 2026 03:30:00 +0000

Part 2 of 2. Part 1 covered statelessness; here I tackle caching.

Part 1 was about statelessness, now shipped in the 2026-07-28 spec. This time it's another feature from that same spec: caching (SEP-2549, "TTL for List Results"). It adds two tiny fields — ttlMs and cacheScope — to cacheable results, giving clients a way to avoid re-fetching the same discovery data. So I built a server that emits them, pointed real AI clients at it, and watched what actually happened. The result surprised me.

The problem: repeated discovery

AI agents plan in loops. A human calls a tool because they decided to; an agent can call — and re-discover — the same tools dozens of times inside a single planning loop. That changes the economics of the protocol: one user request fans out into many MCP calls. A ReAct-style agent can repeatedly re-check the available tools during a multi-step task, calling my discovery endpoint (tools/list) over and over.

In a toy server that's harmless — the tool list is a constant. In a real one it isn't. Building a genuine tools/list means querying a service registry, filtering tools through RBAC, assembling input schemas from config, applying feature flags — one or more backend round-trips per call. Now multiply that by a recursive planning loop across many agents, and my discovery endpoint has quietly become a self-inflicted DoS risk for my own databases.

flowchart LR
    A["AI agent<br/>(recursive planner)"] -->|tools/list x9| S[MCP Server]
    S -->|registry + RBAC + schema| DB[(Corporate DB)]
    style DB fill:#f8d7da,stroke:#842029

This isn't hypothetical. When I pointed GitHub Copilot CLI at my server, it fired nine tools/list calls in ~40 seconds in one short session — plus repeated re-initializes. That's the storm SEP-2549 is meant to tame.

The protocol answer: `ttlMs` and `cacheScope`

SEP-2549 lets the server advertise, right in the discovery result, how long the schema may be cached and by whom. The fields sit at the top level of the result (a CacheableResult):

// a tools/list result
{ "tools": [ /* ... */ ], "ttlMs": 300000, "cacheScope": "private" }

ttlMs — how long the result stays fresh, in milliseconds. Semantics are analogous to HTTP Cache-Control: max-age. Tool schemas rarely change, so a 5-minute window collapses a burst of planning turns into a single fetch.
cacheScope — who may cache it. The spec defines exactly two values: "public" (identical for everyone; a shared proxy may cache one copy) and "private" (varies per authorization context; must never be shared across callers). Its semantics are analogous to HTTP Cache-Control: public vs private.

And a rule worth underlining: the spec marks read/discovery results cacheable — server/discover, tools/list, prompts/list, resources/list, resources/templates/list, even resources/read — but not actions. tools/call is always no-store. You cache "what can I do," never "do it."

⚠️ Schema note: In my server I group the hint under _meta.cache for readability, but the spec puts ttlMs and cacheScope at the top level of the result (the CacheableResult shape shown above), not under _meta. The values and semantics are what matter for this walkthrough; check the caching spec for the exact field placement.

The code: two coordinated cache layers

My FastAPI server exposes three read-only repo tools (get_git_diff,
inspect_file_structure, fetch_logs) and publishes the hint in two places from one source of truth, so both smart clients and dumb proxies can act on it:

TOOLS_TTL_MS = int(os.getenv("MCP_TOOLS_TTL_MS", str(5 * 60 * 1000)))
TOOLS_CACHE_SCOPE = os.getenv("MCP_TOOLS_CACHE_SCOPE", "private")

def cache_meta():
    return {"ttlMs": TOOLS_TTL_MS, "cacheScope": TOOLS_CACHE_SCOPE}

def cache_control_header(ttl_ms=TOOLS_TTL_MS, scope=TOOLS_CACHE_SCOPE):
    visibility = "public" if scope == "public" else "private"
    return f"{visibility}, max-age={ttl_ms // 1000}"

_meta.cache rides inside the JSON-RPC result — for MCP-aware clients/proxies.
Cache-Control maps the same values to an HTTP header — for gateways that only read headers.

But here's the layer that actually saved my database. Because I can't trust the caller to honor ttlMs (more on that in a second), the server caches its own expensive discovery build, keyed by cacheScope:

class DiscoveryCache:
    def get(self, client_id, build):
        key = "*" if self.scope == "public" else f"client:{client_id}"
        now = time.monotonic()
        with self._lock:
            entry = self._entries.get(key)
            if entry and now < entry[0]:
                self.cache_hits += 1
                return entry[1], True            # served from memory, no DB hit
            self.backend_builds += 1             # pay the backend cost ONCE
            value = build()
            self._entries[key] = (now + self.ttl_s, value)
            return value, False

Wired into tools/list, the expensive build_tool_list() (my stand-in for the registry + RBAC + schema queries) now runs at most once per TTL window, no matter how often the client re-discovers.

Worth being precise: SEP-2549 gives the consumer a caching hint; this DiscoveryCache is a separate server-side defense I control regardless of whether any consumer honors that hint. They share the same ttlMs value but do different jobs.

Two caveats I'd flag in a real build. First, I key the private scope by client_id, but the spec's private semantics are per authorization context — so client_id is really a stand-in for "the auth principal and scopes that decide who may share a response." Second, keeping the cache in-process (no shared Redis) preserves Part 1's statelessness, but it doesn't give me globally shared cache state — it trades cross-instance sharing for duplication: a burst spread across four replicas can trigger up to four backend builds instead of one. No Redis lock — but not free either.

flowchart LR
    A["AI agent<br/>tools/list x9"] --> S[MCP Server]
    S -->|1st call only| DB[(Corporate DB)]
    S -.->|calls 2-9: served from memory| A
    style DB fill:#f8d7da,stroke:#842029

Experiment: does the cache actually stop the storm?

Hypothesis: if the server refuses to rebuild the tool list more than once per TTL window, an agent's repeated tools/list calls should collapse to a single backend hit — regardless of whether the client caches anything.

This is where it got interesting.

Finding 1 — the server-side cache works. Replaying the nine-call storm collapses it to a single backend build:

Replaying a 9-call discovery storm from one client...
  tools/list #1: servedFromServerCache=False
  tools/list #2..9: servedFromServerCache=True
discovery cache stats: {'backendBuilds': 1, 'cacheHits': 8}
=> 9 client discovery requests collapsed to 1 backend build(s)  (88% avoided)

I confirmed it end-to-end with real Copilot CLI: it re-requested tools/list repeatedly, yet my server log showed fromCache: true and backendBuilds pinned at 1 (8 of 9 builds avoided, 88.9%). The DB was queried once and shielded after that.

Finding 2 — the clients I tested didn't honor ttlMs. My server advertised ttlMs: 300000, but both Copilot CLI and Cursor re-requested tools/list repeatedly and didn't cache it client-side. (Their handshakes differ — Copilot CLI is dual-era, probing server/discover before falling back to initialize; Cursor took the legacy initialize path — but neither honored the cache hint in my runs.) I'll be careful with this one: it doesn't mean MCP clients ignore ttlMs in general — SEP-2549 is final in the 2026-07-28 spec, and client support is still landing. The two I tested just hadn't wired it up yet, which is exactly why the server can't rely on the hint being honored.

Finding 3 — client-side caching is timing-sensitive. I wrote a simulated edge proxy that does honor ttlMs; normally it collapses 20 planning turns to ~2 fetches. But in a deliberately adversarial timing setup — a slower laptop where each round-trip took ~2 seconds, matching the proxy's own 2-second window — the client cache expired just before every next call, giving 0% avoided. The same run showed the server-side cache still serving 19 of 20 calls from memory. TTL effectiveness depends on the relationship between cache TTL, request frequency, and
round-trip latency — so client-side caching is dependent on consumer behavior and timing, while server-side caching stays effective even when the client ignores the hint.

Whose database, and who protects it

Worth stating plainly: this protects the server's backend, not the client's. The agent is the source of the load; the corporate database behind my server is the victim. And there are three places a cache can step in — each reduces the load reaching the DB, and the further upstream it lives, the more work it eliminates:

flowchart LR
    A[Agent] -->|1. client cache| P[Edge Proxy]
    P -->|2. Cache-Control| S[MCP Server]
    S -->|3. server-side cache| DB[(Corporate DB)]
    style DB fill:#f8d7da,stroke:#842029

Layer	Mechanism	Protects
Client	honors `ttlMs` / `cacheScope`	DB + network + server CPU
Gateway / proxy	`Cache-Control` / protocol-aware caching	DB + server CPU
Server	local discovery cache	DB

Since I can't count on the clients I tested to honor the hint, the server-side cache is the only layer I fully control — and therefore the only protection I can enforce unilaterally today. So my server does both jobs at once: it advertises ttlMs / cacheScope for the ecosystem that's catching up, and it enforces the same ttlMs
internally for right now.

💡 Key takeaway: A cache hint is only advice — it protects nothing until a consumer acts on it. Advertise ttlMs / cacheScope for the ecosystem you wish you had, and enforce the same TTL inside the server for the ecosystem you actually have.

What surprised me

I assumed the hard part would be designing the cache hint. It wasn't — that's two fields. The surprise was that advertising it changed nothing in the clients I tested: both kept requesting tools/list despite the ttlMs I returned. The protection had to come from the one place I fully control — the server enforcing its own advertised TTL. The lesson mirrors Part 1: a protocol capability can ship before every client has adopted it, so a server that wants the benefit today has to implement both sides of the contract itself.

Takeaways

Recursive AI planning turns tools/list into a self-inflicted DoS — every planning turn re-runs the registry/RBAC/schema queries behind discovery.
ttlMs / cacheScope are the right vocabulary, but they're advice for consumers — and the versions of Copilot CLI and Cursor I tested didn't honor it, so a server-side discovery cache is the protection I can enforce unilaterally.
Cache read-only results, never actions, and key the cache by cacheScope (public vs private) so per-tenant tool lists never leak.

Tying both parts together:

Statelessness removes protocol-level affinity without any client cooperation. Caching has two sides — the server can advertise a policy and defend its own backend alone, but the biggest savings only arrive when clients and proxies honor the hint too.

References

MCP Specification (2026-07-28) — Server Utilities: Caching (ttlMs, cacheScope = public / private, cacheable methods). https://modelcontextprotocol.io/specification/2026-07-28/server/utilities/caching
MCP Specification (2026-07-28) — Basic / Overview. https://modelcontextprotocol.io/specification/2026-07-28/basic
SEP-2549 — TTL for List Results (incorporated into the 2026-07-28 spec). https://modelcontextprotocol.io/seps/2549-TTL-for-list-results
SEP-2575 — Make MCP Stateless. https://modelcontextprotocol.io/seps/2575-stateless-mcp
The 2026-07-28 MCP Specification Release Candidate — MCP Blog. https://blog.modelcontextprotocol.io/posts/2026-07-28-release-candidate/
HTTP Caching — Cache-Control (RFC 9111). https://www.rfc-editor.org/rfc/rfc9111
FastAPI documentation. https://fastapi.tiangolo.com/
JSON-RPC 2.0 Specification. https://www.jsonrpc.org/specification

Why MCP Dropped the Handshake: Building a Bare-Metal Stateless Client (SEP-2575)

Sindhuja Nagaraja Sudhakar — Tue, 15 Sep 2026 18:30:00 +0000

Part 1 of 2.

Here's something that snuck up on me: as of the 2026-07-28 MCP specification, MCP is stateless by default. The proposal that started it — SEP-2575, "Make MCP Stateless" — has been folded into the spec itself. The spec now calls the old handshake-based design legacy and the new per-request design modern: there is no initialize handshake and no Mcp-Session-Id; every request carries its own protocol version and capabilities in _meta.

So I did two things: built the smallest possible modern (stateless) client to feel how it works, then pointed today's real AI clients at it to see how they negotiate. Spoiler: it's messier than "they've caught up" or "they haven't" — and the spec's own negotiation rules explain exactly why.

The problem: session state creates affinity

In the legacy MCP flow, a client opens a session with initialize, the server may hand back an Mcp-Session-Id, and the client echoes that id on every later request. That one header is what pins you to a box.

Once a request carries a session id, your load balancer has to send every follow-up call back to the same server instance (sticky routing), or you have to replicate that session across instances — usually with a Redis session store. Either way you pay: sticky routing wrecks even load distribution, and shared session state adds a network hop, a new failure mode, and a scaling bottleneck to every single call.

Here's the stateful world SEP-2575 wants to leave behind:

sequenceDiagram
    participant Agent
    participant LB as Load Balancer
    participant S1 as Instance 1
    participant Redis

    Agent->>LB: initialize
    LB->>S1: (creates session abc123)
    S1->>Redis: store session abc123
    S1-->>Agent: Mcp-Session-Id: abc123
    Note over Agent,S1: every later call MUST return to Instance 1<br/>(or read the session back from Redis)
    Agent->>LB: tools/call  (Mcp-Session-Id: abc123)
    LB->>S1: pinned by session id

The question I wanted to answer: can I design the client so no session ever needs to be pinned or synchronized?

The idea: make every request self-describing

The modern move is to stop negotiating context once and remembering it, and instead put that context into every request. Each call carries its protocol version and capabilities inline in _meta. Nothing is remembered between requests, so nothing has to be routed to a particular instance. (The spec uses namespaced keys like _meta.io.modelcontextprotocol/* for this — more on the exact schema below.)

sequenceDiagram
    participant Agent
    participant LB as Load Balancer
    participant Pool as Any Instance

    Agent->>LB: tools/call  (_meta: version + capabilities)
    LB->>Pool: route to ANY node
    Pool-->>Agent: result (echoes clientId)
    Note over Agent,Pool: no session id, nothing to pin,<br/>no Redis in the path

The code: a handshake-free JSON-RPC client

There's deliberately no initialize step. Every request is a plain JSON-RPC 2.0 object with the protocol metadata riding along in params._meta:

PROTOCOL_VERSION = "2026-07-28"

CLIENT_CAPABILITIES = {
    "tools": {"listChanged": False},
    "stateless": True,
}

def build_meta(client_id=None, capabilities=None):
    return {
        "protocolVersion": PROTOCOL_VERSION,
        "clientCapabilities": capabilities or CLIENT_CAPABILITIES,
        "clientId": client_id or str(uuid.uuid4()),
    }

def build_jsonrpc_request(method, params=None, client_id=None, capabilities=None):
    params = dict(params or {})
    params["_meta"] = build_meta(client_id, capabilities)   # inline, every call
    return {"jsonrpc": "2.0", "id": next(_id_counter),
            "method": method, "params": params}

⚠️ Schema note: I flattened the metadata (protocolVersion,
clientCapabilities, clientId) to isolate the statelessness property and keep the snippet readable. The real wire format namespaces these under
_meta.io.modelcontextprotocol/* (e.g. io.modelcontextprotocol/protocolVersion). Treat my _meta as illustrative, not the literal spec schema.

The transport is just an HTTP POST — no session object, no connection state. I also surface the protocol version as a header so a gateway can inspect it without parsing the body:

def _post(self, payload):
    resp = requests.post(self.base_url, json=payload, headers={
        "Content-Type": "application/json",
        "MCP-Protocol-Version": PROTOCOL_VERSION,
    }, timeout=self.timeout)
    resp.raise_for_status()
    data = resp.json()
    if data.get("error"):
        raise MCPError(f"{data['error'].get('code')}: {data['error'].get('message')}")
    return data.get("result", {})

Because list_tools() and call_tool() both go straight through _post, any instance can serve any request. There's nothing sticky to preserve.

Experiment: does statelessness remove affinity?

Hypothesis: if every request carries its own protocol context, the same logical client should be able to hit different replicas on consecutive calls with no shared session state.

To test that, I wrote a test that fires 50 concurrent tool calls, each carrying its own randomized _meta, through a deliberately awful transport (35% dropped packets, latency jitter, random instance selection) at a pool of stateless replicas that share no session state:

NUM_REQUESTS = 50
POOL_SIZE = 4
DROP_RATE = 0.35

async def chaotic_send(request, pool):
    for _ in range(MAX_RETRIES):
        await asyncio.sleep(random.uniform(0, 0.005))   # jitter
        if random.random() < DROP_RATE:
            continue                                    # dropped -> retry
        instance = random.choice(pool)                 # no affinity
        return await instance.handle(request)
    raise ConnectionError("all packets dropped after retries")

Each replica answers using only the per-request _meta — never remembered state:

meta = params["_meta"]          # the ONLY context the server relies on
return {..., "_meta": {"serverInstance": self.instance_id,
                       "echoClientId": meta["clientId"]}}

The suite asserts all 50 requests completed, that reused client ids landed on different instances (proving no stickiness snuck in), and that every response echoed the exact protocol version its request supplied. Here's the actual run:

  Stateless MCP concurrency test  (50 concurrent calls)
protocol version      : 2026-07-28
simulated drop rate   : 35% (with up to 10 retries)
server pool           : ['srv-0', 'srv-1', 'srv-2', 'srv-3']
requests completed    : 50/50
client -> instances (proves no sticky routing):
    client-0: ['srv-0', 'srv-1', 'srv-2', 'srv-3']  <-- moved between instances
    client-1: ['srv-0', 'srv-1', 'srv-2', 'srv-3']  <-- moved between instances
    client-2: ['srv-0', 'srv-1', 'srv-2', 'srv-3']  <-- moved between instances
PASS: every instance served any request independently - no session locks.

Every client id was served by all four instances. The property we were after: request routing becomes independent of client identity — same logical client, different servers, no shared protocol session. This doesn't prove a production load balancer will never add affinity; it shows the protocol doesn't require any.

The catch: negotiation is a two-sided handshake

Then I pointed two real MCP clients at my server — GitHub Copilot CLI and Cursor. My first cut failed hard:

-32601: method not found: 'initialize'

The logs told a subtler story than "clients are behind." The spec gives a modern server two jobs: it MUST implement a server/discover RPC (so clients can learn its versions and capabilities without a handshake), and it serves requests carrying per-request _meta. My first server implemented neither server/discover nor the old initialize — so there was simply no way in.

The clients then revealed their eras in the request stream:

Copilot CLI is dual-era. It tried the modern server/discover first, and only when my server answered -32601 did it fall back to the legacy initialize (which also -32601'd) before giving up.
Cursor went straight to initialize — the legacy path, no probe.

So the fix had two valid shapes. The spec-correct one: implement server/discover and be a proper modern server. The quick one I took for this experiment: add a legacy initialize shim so both clients connect via the handshake path — but crucially, my shim stores nothing and never mints an Mcp-Session-Id. Even on the legacy path, with no session id handed back the client has nothing sticky to resend, so any instance can still serve any later call. Backward compatibility and load-balancer freedom.

That's the subtle bit I want you to take away: protocol-level stickiness is server-minted. The server issues the id; the client just echoes it. Decline to mint it, and the client naturally sends nothing to pin with — no client adoption required. Statelessness at the protocol layer is something the server unlocks on its own. (That's the exact opposite of caching, which — as I'll show in Part 2 — only pays off when the consumer cooperates.)

💡 Key takeaway: Protocol-level stickiness is server-minted. As long as your server answers initialize without returning an Mcp-Session-Id, the client has nothing to echo — so any instance can serve any request, and you stay backward-compatible with today's legacy clients. Apps that need cross-call state use explicit state handles (server-minted IDs like basket_id, threaded through tool arguments) — and because any replica can resolve a handle from shared storage, they keep application state without bringing stickiness back.

Stateful vs. stateless, at a glance

Attribute	Legacy MCP (`2025-11-25` and earlier)	Modern MCP (`2026-07-28`)
Session tracking	`Mcp-Session-Id` header required	Protocol metadata inline via `_meta`
Load balancing	Sticky routing required	Any instance / round-robin
Shared protocol-session state	Redis / session store	None required
Instance rotation	Restarts can break active sessions	Instances rotate freely

What statelessness does not solve

Protocol statelessness only removes implicit session state — it does not make your application stateless. You still own auth, authorization, rate limiting, idempotency, long-running tasks, and real conversation memory. In the modern model, cross-call state travels through explicit state handles — server-minted IDs (a basket_id, a connection_id) that the model threads through later tool calls — rather than an implicit session. That's the pattern SEP-2567 ("Sessionless MCP via Explicit State Handles") formalizes; notably it's a tool-design convention, not a new wire construct.

Stateless protocol ≠ stateless application. Statelessness buys free horizontal routing, not a free pass on application state.

flowchart TD
    MCP[MCP request] --> P[Protocol state]
    MCP --> A[Application state]
    P --> P1["Stateless: version + capabilities<br/>in _meta, no session"]
    A --> A1["Explicit state handle<br/>(basket_id, connection_id)"]
    P1 --> R[Any replica serves it]
    A1 --> ST[(Shared storage)]
    R --> App[Your application]
    ST --> App

What surprised me

I expected the hard part to be removing the session. It wasn't — that was a few lines. The harder problem was negotiation and compatibility: a server can be perfectly capable of handling stateless requests and still be unreachable by a client that expects a different protocol path. The real lesson wasn't "statelessness is easy" — it was that protocol design is about how two independently-evolving implementations discover what the other supports.

Takeaways

Protocol-level statelessness removes implicit session affinity — sticky routing and Redis session sync disappear because there's no session id to pin to.
Application state still exists, but through explicit state handles, not a stateful protocol. Stateless protocol ≠ stateless application.
The real production problem isn't just statelessness — it's compatibility (Copilot probes server/discover, Cursor wants initialize), plus caching, retries, and the sheer volume of tool calls an agent generates.

Part 1 was about routing. Part 2 is about agent behaviour — caching, and why advertising ttlMs / cacheScope isn't enough to stop AI planning loops from thrashing your database.

References

MCP Specification (2026-07-28) — Basic / Overview. https://modelcontextprotocol.io/specification/2026-07-28/basic
MCP Specification (2026-07-28) — Versioning and Compatibility (Modern / Legacy / Dual-era, per-request _meta, compatibility matrix). https://modelcontextprotocol.io/specification/2026-07-28/basic/versioning
MCP Specification (2026-07-28) — Transports / Overview (_meta.io.modelcontextprotocol/*, MCP-Protocol-Version). https://modelcontextprotocol.io/specification/2026-07-28/basic/transports
SEP-2575 — Make MCP Stateless (Final; incorporated into the 2026-07-28 spec). https://modelcontextprotocol.io/seps/2575-stateless-mcp
The 2026-07-28 MCP Specification Release Candidate — MCP Blog. https://blog.modelcontextprotocol.io/posts/2026-07-28-release-candidate/
SEP-2567 — Sessionless MCP via Explicit State Handles (cited in-text). https://modelcontextprotocol.io/seps/2567-sessionless-mcp
JSON-RPC 2.0 Specification. https://www.jsonrpc.org/specification

DEV Community: Sindhuja Nagaraja Sudhakar

When AI Agents Hammer tools/list: Building a Caching-Aware MCP Server (SEP-2549)

The problem: repeated discovery

The protocol answer: ttlMs and cacheScope

The code: two coordinated cache layers

Experiment: does the cache actually stop the storm?

Whose database, and who protects it

What surprised me

Takeaways

References

Why MCP Dropped the Handshake: Building a Bare-Metal Stateless Client (SEP-2575)

The problem: session state creates affinity

The idea: make every request self-describing

The code: a handshake-free JSON-RPC client

Experiment: does statelessness remove affinity?

The catch: negotiation is a two-sided handshake

Stateful vs. stateless, at a glance

What statelessness does not solve

What surprised me

Takeaways

References

The protocol answer: `ttlMs` and `cacheScope`