How to Test MCP Servers: A Practical Testing Pyramid for Agent Tools

Most MCP tutorials stop at "here's a tool that returns weather data." Almost none of them answer the question that actually matters once you ship: how do you know your MCP server still works after the next change?

If you've built a production MCP server, you already know the failure modes are weird. A tool schema drifts and the model starts calling it wrong. A session token expires mid-conversation and the agent silently gets empty results instead of an error. A refactor changes a return type from a string to an object and every downstream agent breaks in a way no pytest unit test catches.

This post gives you a concrete testing pyramid for MCP servers, with what to test at each layer and why the layer above it isn't enough on its own.

Why MCP testing is different from normal API testing

A regular REST API has one consumer type: code that calls it with fixed, known inputs. An MCP server has two:

1. Deterministic clients — your own test harness, CI, health checks.
2. A non-deterministic client — the LLM itself, which decides when to call a tool, what arguments to pass, and how to interpret the response.

That second consumer means schema correctness isn't optional polish — it's the interface contract the model reasons over. A tool description that's technically valid JSON Schema but ambiguous in plain English will get called wrong in production even if every unit test passes.

So your test suite needs to answer four separate questions:

• Does the tool do what it claims when called with valid arguments?
• Does it fail safely and informatively when called with bad arguments?
• Does the session/auth layer behave correctly across the lifecycle (expiry, revocation, refresh)?
• Would a model, given only the tool's description and schema, actually call it correctly?

Most teams only test the first one.

Layer 1: Schema and contract tests

Before anything touches an LLM, validate that your tool definitions are internally consistent and haven't silently drifted.

def test_tool_schema_matches_handler_signature():
    for tool in registered_tools:
        schema_props = set(tool.input_schema["properties"].keys())
        handler_params = set(inspect.signature(tool.handler).parameters.keys())
        assert schema_props == handler_params, (
            f"{tool.name}: schema/handler mismatch — "
            f"schema has {schema_props}, handler expects {handler_params}"
        )

def test_all_required_fields_are_documented():
    for tool in registered_tools:
        for field in tool.input_schema.get("required", []):
            assert tool.input_schema["properties"][field].get("description"), (
                f"{tool.name}.{field} has no description — the model can't reason about it"
            )

This catches the single most common regression: someone adds a required parameter to the handler and forgets to update the schema. It's boring, it's fast, and it should run on every commit.

Layer 2: Handler behavior tests (unit + integration)

These are close to normal unit tests, but with an MCP-specific twist: test the MCP response envelope, not just the underlying business logic.

def test_tool_returns_structured_error_on_bad_input():
    result = call_tool("get_invoice", {"invoice_id": "not-a-real-id"})
    assert result.is_error is True
    assert "invoice_id" in result.content[0].text  # error is specific, not generic
    assert result.content[0].text != ""             # never return an empty error body

def test_tool_handles_upstream_timeout_gracefully():
    with mock_upstream_timeout():
        result = call_tool("get_invoice", {"invoice_id": "INV-100"})
        assert result.is_error is True
        assert "timeout" in result.content[0].text.lower()

A pattern worth stealing: write one test per failure mode you've actually seen in production (timeout, malformed upstream response, rate limit, auth expiry), not just the happy path. If you keep an incident log, each entry should turn into a regression test.

Layer 3: Session and auth lifecycle tests

This is the layer almost everyone skips, and it's the one that causes the scariest production bugs, because failures here are silent — the agent doesn't crash, it just gets wrong or stale data.

Minimum test set for any MCP server with auth:

Scenario	Expected behavior
Valid session, first call	Succeeds, returns fresh data
Session expired mid-conversation	Tool returns explicit "session expired" error, not empty/stale data
Token refresh in-flight during concurrent calls	No duplicate refresh requests, no dropped calls
Revoked session	Immediate rejection, not a delayed 200 with garbage
Session reused across two different agent contexts	Explicit isolation — no data leakage between conversations

def test_expired_session_returns_explicit_error_not_stale_data():
    session = create_session(ttl_seconds=1)
    time.sleep(2)
    result = call_tool("list_documents", {}, session=session)
    assert result.is_error is True
    assert "expired" in result.content[0].text.lower()
    # the dangerous version of this bug: result.content has stale cached data instead

If you only test one thing from this post, test the expired-session case. It's the difference between an agent that says "I need you to re-authenticate" and one that confidently reports numbers from three hours ago.

Layer 4: LLM-in-the-loop evaluation tests

This is the layer that's genuinely new compared to normal software testing, and it's where "agent-eval" testing lives. The question isn't "does the code work," it's "given only the tool description, does the model call it the way you intended?"

A minimal eval harness:

EVAL_CASES = [
    {
        "prompt": "What's the status of invoice INV-4471?",
        "expected_tool": "get_invoice",
        "expected_args_contains": {"invoice_id": "INV-4471"},
    },
    {
        "prompt": "Cancel my subscription",
        "expected_tool": "cancel_subscription",
        "expected_args_contains": {},
        "must_not_call": ["delete_account"],  # guard against dangerous over-reach
    },
]

def run_eval_case(case, model_client):
    response = model_client.run(case["prompt"], tools=registered_tools)
    tool_calls = [c.name for c in response.tool_calls]
    assert case["expected_tool"] in tool_calls
    for forbidden in case.get("must_not_call", []):
        assert forbidden not in tool_calls, f"Model incorrectly called {forbidden}"

Run this against your actual model provider on a schedule (weekly, or on every schema change), not just once at launch. Model providers update their models, and a tool description that worked perfectly with one model version can start getting misinterpreted after a silent upgrade. This is the test layer that catches "the model started calling delete_account when the user said 'cancel my subscription'" before your users do.

Putting it together: a CI pipeline that actually catches regressions

A sane ordering, fastest and cheapest first:

1. Schema/contract tests — milliseconds, run on every save.
2. Handler unit tests — seconds, run on every commit.
3. Session/auth lifecycle tests — seconds to low minutes, run on every PR.
4. LLM-in-the-loop evals — costs real API tokens, run on every PR to main and on a nightly schedule, not on every keystroke.

Gate merges on layers 1–3. Treat layer 4 as a monitored signal — alert if the pass rate drops, but don't necessarily block every PR on it, since model non-determinism means occasional flakes are expected. Track the pass rate as a trend, not a single number.

Where to go from here

Writing all four layers from scratch for your first MCP server takes longer than it should — mostly because there's no standard scaffold for the schema/contract tests or the eval harness, so everyone reinvents it slightly differently.

If you'd rather start from a working baseline than blank files, I put together an MCP Production Pack — a production checklist, a minimal working server template with auth and session handling already wired up, and a set of agent-eval test templates like the ones above, ready to adapt to your own tools. It's built to save you the first few days of scaffolding, not to replace understanding what the tests actually check.

Either way — even just adding the expired-session test and one LLM-in-the-loop eval case this week will catch bugs your current test suite is currently blind to.

---

Written with AI assistance and reviewed for accuracy.