DEV Community

Cover image for The bug report was wrong, and that was the interesting part
Germán Massello
Germán Massello

Posted on

The bug report was wrong, and that was the interesting part

Summer Bug Smash: Clear the Lineup 🐛🛹

This is a submission for DEV's Summer Bug Smash: Clear the Lineup powered by Sentry.

Project Overview

getsentry/sentry-python is the official Sentry SDK for Python. You call sentry_sdk.init() once, and it auto-instruments whatever libraries it finds installed — Django, Celery, Redis, and a growing set of AI integrations: OpenAI, Anthropic, LangChain, LangGraph, LiteLLM, and MCP.

When one of those integrations catches an error, it attaches a mechanism to the event before sending it. The mechanism carries two fields:

  • type — which integration captured this
  • handled — whether the error was caught, or escaped into the user's code

Sentry uses both. type drives attribution and integration adoption metrics. handled decides whether an error counts toward your crash-free rate and whether it trips unhandled-issue alerts.

I picked issue #5242: "No mechanism on MCP and LangChain exceptions", opened by a Sentry engineer, labeled Bug, unassigned.

Bug Fix or Performance Improvement

The report says the mechanism is missing. Before touching anything, I wrote the assertions that would prove it and ran them against the existing test suite. The repo already spins up real MCP servers over stdio and makes tools fail, so the reproduction was two lines added to tests that already existed:

assert error["exception"]["values"][0]["mechanism"]["type"] == "mcp"
assert not error["exception"]["values"][0]["mechanism"]["handled"]
Enter fullscreen mode Exit fullscreen mode

Ten tests failed. But not the way the issue implied:

AssertionError: assert 'generic' == 'mcp'
Enter fullscreen mode Exit fullscreen mode

Not KeyError. The mechanism was there — with the default value from sentry_sdk/utils.py:692:

exception_value["mechanism"] = (
    mechanism.copy() if mechanism else {"type": "generic", "handled": True}
)
Enter fullscreen mode Exit fullscreen mode

So this is two defects, not one.

type: "generic" is the one the issue describes: the error carries no fingerprint of the integration that captured it. In the UI, an error from an MCP tool is indistinguishable from one your own code raised.

handled: True is the one that isn't in the issue, and it's the one that actually costs you something. Every MCP capture site looks like this:

except Exception as e:
    sentry_sdk.capture_exception(e)
    raise
Enter fullscreen mode Exit fullscreen mode

That raise means the exception keeps going and lands in your application. Nothing handled it. But with no mechanism supplied, Sentry defaults to handled: True — so these errors don't count as unhandled, don't move your crash-free rate, and don't trip the alerts built on that signal. Your service is breaking and the dashboard says it's fine.

Code

PR: getsentry/sentry-python#7226 (draft)

The fix is a module-level helper in each integration, matching the pattern the other eight AI integrations already use:

def _capture_exception(exc: "Any") -> None:
    event, hint = event_from_exception(
        exc,
        client_options=sentry_sdk.get_client().options,
        mechanism={"type": "mcp", "handled": False},
    )
    sentry_sdk.capture_event(event, hint=hint)
Enter fullscreen mode Exit fullscreen mode

Seven call sites route through it — six in mcp.py, one in langchain.py. About 30 lines of production code.

A note on the PR's state, since it's visible on GitHub: it's open as a draft, because the repo converts non-draft PRs automatically. The upstream test matrix hasn't run yet — GitHub gates workflows on pull requests from first-time contributors until a maintainer approves them. The three checks that don't need approval (Socket Security ×2, Semgrep) pass. Everything I could verify myself, I ran locally and listed below.

CONTRIBUTING also asks contributors to discuss the approach with a maintainer before opening a PR. The challenge deadline didn't allow for that round trip, so I opened the PR as a draft and left a comment on the issue explaining what I found. If the approach doesn't fit, it costs the maintainers one comment to close it.

My Improvements

The fix is small. Everything interesting is in what I had to verify to be sure it was right.

The reproduction is the regression test

I didn't build a separate harness. The suite already drives real MCP servers, so the assertions that prove the bug are the same ones that will guard against it. Red before, green after, one artifact.

Coverage measured, not assumed

mcp.py has six capture sites. The obvious assumption is that the three error tests cover them. They do — but only across two versions of the mcp package, which are mutually exclusive code paths: v1 patches decorators, v2 installs middleware.

I ran the suite under --cov and checked which lines actually executed:

call site mcp v1.29.0 mcp v2.0.0
397 _tool_handler_wrapper
493 _instrument_v2_tool_call
618 _prompt_handler_wrapper
767 _instrument_v2_prompt_get
922 _resource_handler_wrapper
987 _instrument_v2_resource_read

Run one version and half the fix is untested. Worth knowing before claiming the fix works.

What I ran locally, all green:

env result
py3.14-mcp-v1.29.0 100 passed, 7 skipped
py3.14-mcp-v2.0.0 105 passed, 2 skipped
py3.14-langchain-base-v1.3.14 626 passed
py3.14-langgraph-v0.6.11 146 passed
py3.14-fastmcp-v1.0 88 passed
py3.13-fastmcp-v4.0.0b2 42 passed, 3 skipped

Plus mypy sentry_sdk (strict, 193 files) and ruff clean.

LangGraph is in there for a reason: LangchainIntegration._ignored_exceptions is populated only by the LangGraph integration, and test_graph_bubble_up_ignored asserts that ignored exceptions produce zero events. The is_ignored branch never reaches the capture, so the fix shouldn't touch it — but "shouldn't" isn't "doesn't", so I ran it.

A path with zero coverage

langchain.py has three entry points into its error handler. Two were exercised. on_tool_error — the one that fires when an agent's tool raises — had no test at all: the only tool defined in the 7,000-line test file always succeeds.

I wrote test_langchain_tool_error, and confirmed with coverage that it hits on_tool_error and the capture, and neither of the other two entry points. That one goes into the PR on its own merit — it covers a real path nothing covered before.

handled=False at all seven sites

The MCP sites re-raise, so the exception escapes. flask.py:240 sets the precedent: errors a framework later converts into a response are still unhandled.

For LangChain, _handle_error is a notification callback, not a swallow point — the existing test wraps the whole thing in pytest.raises(ValueError).

I deliberately did not add a handled parameter. The only handled=True among the AI integrations lives in pydantic_ai, gated behind an option that has no equivalent here. YAGNI.

The regression I introduced, and caught

sentry_sdk.capture_exception wraps its internal capture_event in a safety net (scope.py:1593-1596). If Sentry's own serialization blows up, it swallows that quietly rather than replacing your exception with its own.

My helper dropped that net. The sibling integrations restore it at the call site (openai.py:845-846); mine didn't. Inside except Exception as e: followed by raise, that's a real regression: an internal SDK failure would surface to the user instead of their own error. The repo's contract is explicit — "Don't crash applications."

except Exception as e:
    with capture_internal_exceptions():
        _capture_exception(e)
    raise
Enter fullscreen mode Exit fullscreen mode

The raise stays outside the with, so the original exception always propagates. I verified it by forcing _capture_exception to throw and checking the user's ValueError still came through on both MCP versions.

One thing I found and did not fix

While extending the tests to FastMCP, one environment failed differently:

AssertionError: assert 'logging' == 'mcp'
Enter fullscreen mode Exit fullscreen mode

With fastmcp==4.0.0b2 + mcp==2.0.0, a failing tool never reaches any MCP capture site — I measured it, zero of the six executed. FastMCP catches and logs the error itself, and the only reason Sentry sees it at all is LoggingIntegration picking up that log line.

Which means test_fastmcp_tool_with_error isn't testing MCP error capture on that environment. It passes by accident, and its assert len(error_events) >= 1 hides it.

I left test_fastmcp.py alone. Adding the mechanism assertions there would go permanently red on that env for a cause unrelated to this bug. It's documented in the PR as a separate finding.

That's the decision I'd defend hardest: a static reading said "FastMCP routes through MCP, so it gets the mechanism." The measurement said otherwise, and the measurement wins.

Best Use of Sentry

I instrumented before fixing, so the broken state is on the record.

The demo runs a real MCP server whose lookup_order tool fails with ValueError: upstream order service returned 503. I sent one event with the fix stashed (release: before-fix) and one with it applied (release: after-fix).

Both land in the same issue with the same stack trace, which makes the comparison read itself.

Before — Sentry renders the mechanism right under the exception:

Sentry showing the exception with mechanism generic and handled true

After — same error, same place in the UI:

Sentry showing the same exception with mechanism mcp and handled false

The contrast that makes the case. The same trace, two panels of it.

The span the failing tool produced:

Sentry trace preview showing the mcp.server span for tools/call lookup_order

And that span's own metadata:

Sentry trace details showing Operation Name mcp.server and Origin auto.ai.mcp

The instrumentation always knew how to label its spans. auto.ai.mcp was sitting right there the whole time, on the very same tool call that produced the error. Its errors went out anonymous. The fix just brings them up to the standard the spans already met.

The issue view tells the story on its own too: First seen in release before-fix, Last seen in release after-fix, and the Unhandled badge that only appears once handled is correct.

Sentry credits for the challenge: code bugsmash26.

Best Use of Google AI

I used Gemini during the investigation, mainly to pressure-test my own reading of the code:

  • Mapping the capture paths. mcp.py is ~1,100 lines with two parallel instrumentation strategies for mcp v1 and v2. I had Gemini trace which functions own each capture site and which version reaches them, then verified the answer with --cov rather than trusting it. The map was right; the measurement is what made it evidence.
  • Challenging the handled call. I asked it to argue against handled=False — the strongest case being that MCP v1 turns a tool exception into an isError result the host never sees, so arguably it was handled. That's what sent me to flask.py:240 for the precedent, and it's why that decision is argued in the PR instead of asserted.
  • Edge cases for the test list. Chained exceptions, ExceptionGroup under anyio task groups, errors with no active transaction. Most turned out to be already covered or out of scope, but the FastMCP question came out of that pass — and that's the one that found something real.

What I'd tell the next person

The valuable part wasn't the fix. Thirty lines, and the pattern was already in the repo eight times over.

The valuable part was refusing to trust three things: the issue title (the mechanism wasn't missing, it was wrong), my own coverage assumption (measured it instead), and my own patch (a reviewer pass caught that I'd dropped a safety net the original code had).

Two of the three findings that matter here weren't in the bug report at all.

Top comments (0)