DEV Community

Cover image for The bug FastMCP's own CI could not see
Yonyon
Yonyon

Posted on

The bug FastMCP's own CI could not see

#python #typing #opensource #mcp

I opened a pull request to FastMCP (27k stars) at 23:12:32 UTC. A bot closed it at 23:12:47. Fifteen seconds.

A second bot then labeled it too-long, with the comment: "condense this issue. We'll triage it once it's trimmed down."

It was merged the next day, unedited. Nobody condensed anything. The label is still on the merged PR.

That is the funny part. The bug underneath it is the useful part, because it is a failure mode that any project with a type-checking gate can be sitting on right now without knowing.

The bug

Context.elicit is FastMCP's "ask the human a question" primitive. It declares six alternative call signatures as @overload stubs, one per supported response_type.

Each stub's body was ..., and the explanatory prose sat after the body. Verbatim from fastmcp/server/context.py at 3.4.5, lines 1022 to 1061:

    @overload
    async def elicit(
        self,
        message: str,
        response_type: None,
        *,
        response_title: str | None = None,
        response_description: str | None = None,
    ) -> (
        AcceptedElicitation[dict[str, Any]] | DeclinedElicitation | CancelledElicitation
    ): ...

    """When response_type is None, the accepted elicitation will contain an
    empty dict"""                                          # <-- not a docstring

    @overload
    async def elicit(
        self,
        message: str,
        response_type: type[T],
        *,
        response_title: str | None = None,
        response_description: str | None = None,
    ) -> AcceptedElicitation[T] | DeclinedElicitation | CancelledElicitation: ...

    """When response_type is not None, the accepted elicitation will contain the
    response data"""

    @overload
    async def elicit(
        self,
        message: str,
        response_type: list[str],
        *,
        response_title: str | None = None,
        response_description: str | None = None,
    ) -> AcceptedElicitation[str] | DeclinedElicitation | CancelledElicitation: ...

    """When response_type is a list of strings, the accepted elicitation will
    contain the selected string response"""
Enter fullscreen mode Exit fullscreen mode

Three more follow the same pattern, six in total.

That string is not a docstring. A docstring has to be the first statement inside a function or class body. Here the body already ended at ..., so the string is a bare expression statement sitting in the class body, between two overloads.

And a statement between overloads terminates the overload chain for mypy.

Why only mypy

Here is the reduced shape. No FastMCP install needed, paste it into a file and run two checkers:

from typing import overload

@overload
def f(x: None) -> str: ...
"""Doc for the None case."""

@overload
def f(x: list[str]) -> int: ...
"""Doc for the list case."""

def f(x):
    return x

f(["a", "b"])   # mypy: error.  pyright: fine.
Enter fullscreen mode Exit fullscreen mode
checker overloads registered verdict on the list[str] call
mypy 1 of 6 error, no call-site workaround
pyright 6 of 6 fine
ty 6 of 6 fine

pyright and ty tolerate the interleaved statement and keep collecting overloads. mypy stops.

FastMCP's own gate is ty. So the project's CI was green on a file that was broken for a large share of its users, and it stayed that way through a release.

Why it actually bit

Two details turn this from a style nit into a live problem.

First: the one stub mypy could still see is the deprecated one. In the 3.x line the first overload is response_type: None, which the library's own docstring marks as deprecated. So mypy users were being funneled toward exactly the call shape the library is retiring. There is no way out at the call site either. I tried five formulations (explicit annotation, cast, a typed local, a Sequence[str] alias, direct literal). All five fail under mypy. All five pass under pyright.

Second: I needed the invisible signature for a security control.

response_type=None compiles to an empty JSON schema:

{"type": "object", "properties": {}}
Enter fullscreen mode Exit fullscreen mode

The library documents this itself, in the very docstring that broke the overload chain: "When response_type is None, the accepted elicitation will contain an empty dict."

An empty dict means the accept payload carries no data. A client that auto-accepts produces bytes that are indistinguishable, on the wire, from a human clicking approve. I proved that with a live probe a week earlier, when a delete tool ran its full delete path with confirm=False.

The hardening is to stop treating "accepted" as a signal and require an affirmative value:

CONFIRM: Final[list[str]] = ["cancel", "confirm"]
...
result = await ctx.elicit(message, response_type=CONFIRM)
Enter fullscreen mode Exit fullscreen mode

list[str]. One of the five overloads mypy could not see.

The fix

Move each literal inside its stub body, where it becomes an actual docstring:

    @overload
    async def elicit(
        self,
        message: str,
        response_type: None = None,
    ) -> AcceptedElicitation[None] | DeclinedElicitation | CancelledElicitation:
        """The accepted elicitation will contain no data"""
Enter fullscreen mode Exit fullscreen mode

+15 / -24, one file. No runtime change, no API change. The implementation function is byte-identical before and after (I hashed it). The deletion count is larger than the addition count only because dropping the trailing ... let ruff format collapse two return annotations onto fewer lines.

The timeline

Every timestamp below is from the GitHub timeline API.

08-05 22:36:16Z  issue opened
08-05 23:12:32Z  PR opened
08-05 23:12:47Z  auto-CLOSED, 15 seconds later
                 (missing-issue-link: external PRs must reference an
                  issue ASSIGNED to their author)
08-05 23:14:09Z  bot labels it `too-long`
                 "Excessively verbose or unedited LLM output.
                  Condense before triage."
                 ... ~14 hours of silence ...
08-06 13:29:05Z  maintainer assigns the issue
08-06 13:29:17Z  label removed, PR auto-REOPENS
08-06 13:29:30Z  APPROVED, 13 seconds later
08-06 13:34:12Z  a second PR merges, see below
08-06 13:35:50Z  MERGED
Enter fullscreen mode Exit fullscreen mode

The part I did not expect

Twenty-eight seconds after approving my PR, the project lead opened a branch named codex/review-closed-contributor-prs and merged it four minutes later. It adds a line to FastMCP's own CLAUDE.md, the file that instructs their review agents:

Review closed contributor PRs. External PRs may be closed as part of the issue-link workflow, so closure alone is not a negative signal.

Then my PR merged.

I want to be precise about what I am claiming here: there is no explicit cross-reference between those two pull requests. I am reporting the order of events and the twenty-eight second gap. Draw your own conclusion.

It is a good line to add either way. When a repo automates triage, "this PR is closed" stops meaning "a human rejected this" and starts meaning "a bot ran." Anything downstream that reads closure as a signal, human or agent, is now reading a stale convention.

Status, honestly

Merged to main, which is the 4.x line, 289 commits ahead of the latest release. It is not in any release.

v3.4.6   released 08-05   still has all six stray literals   <- PyPI serves this
main     merged   08-06   fixed
Enter fullscreen mode Exit fullscreen mode

There is no 3.x maintenance branch on the repo. If you pin fastmcp>=3.4.5,<4, as I do across eleven packages, you consume the broken version until someone backports it or you move to 4.x.

Also worth saying plainly: this never broke my own CI, because my gate is ty, not mypy. It breaks downstream consumers of my package who run mypy. I found it by running a checker my project does not run.

The takeaway

A green gate proves that the checker you ran agrees with you. It does not prove the code is right.

Three type checkers looked at this file. One saw the bug. FastMCP happened to run one of the two that did not, and shipped it. I happened to run the third, for unrelated reasons, on code I needed for a security fix.

If you maintain a typed Python library, the cheap version of this lesson is: run a second checker in a non-blocking job. You do not have to fix what it finds. You just have to be able to see it.

Issue: PrefectHQ/fastmcp#4773
PR: PrefectHQ/fastmcp#4774

Links

Next in this series: Elicitation didn't die in the MCP stateless rewrite. It's the only one that survived. — why the empty-schema trap and this type bug are the same bug, one layer apart.

Top comments (1)

Collapse
 
mads_hansen_27b33ebfee4c9 profile image
Mads Hansen •

The key distinction here is source-tree correctness vs consumer compatibility. A library does not really support a type checker because its own repository passes that checker; it supports it when a downstream project can consume the built artifact and get the promised public types.

A practical gate would build the wheel, install it into tiny consumer fixtures, and run mypy, pyright, and ty across the supported Python-version matrix. Each public overload family gets a reveal_type/expected-error fixture, including deprecated and recommended call shapes. Running against the installed wheel also catches packaging omissions that an in-repo check cannot see. Add a small runtime-versus-typing contract check for public callables, and this class of “green library, broken user” release becomes a deterministic compatibility failure rather than a checker-specific surprise.