MCP tool metrics: counters, histograms and failure rate

MCP tool metrics answer three questions that traces alone handle poorly: how many calls per tool, what the typical latency is, and what share of calls fail. An MCP server exposes tools that a language model calls in a loop; without aggregates, the first sign of a failing tool is a user complaint, not an alert.

The article on observing MCP agents with OpenTelemetry covers spans. This one focuses on metrics: what the OpenTelemetry semantic conventions define for MCP, what they do not define (there is no dedicated counter), and how to get a per-tool failure rate without blowing up cardinality. The content is backend-independent; it applies to Dynatrace SaaS and Managed alike as long as metrics arrive over OTLP.

The SRE goal: turn « the agent is slow » into an SLI measured per tool, on which an SLO and an error budget can rest, and cut MTTR by identifying the faulty tool in one query.

MCP tool metrics: what the convention provides

The MCP conventions live in the open-telemetry/semantic-conventions-genai repository (the former page on the OpenTelemetry site points there). The file model/mcp/metrics.yaml, read at the repository state of 30 October 2026, declares four metrics, all histograms in seconds, at « recommended » level and « development » stability:

• mcp.client.operation.duration: duration of a request or notification, measured on the sender, from sending to receiving the response;
• mcp.server.operation.duration: duration measured on the receiver, from receiving to sending the result;
• mcp.client.session.duration and mcp.server.session.duration: session duration.

Two practical consequences. First, « development » stability means names and attributes may still change: pin the convention version you follow and read the changelog before upgrading. Second, the convention defines no call counter and no error counter: the number of calls and of failures is derived from the count built into each histogram, filtered by the error.type attribute.

The attributes that give the measurement its value

For the operation metrics, the docs/gen-ai/mcp.md page of the same repository lists these attributes:

• mcp.method.name: required (for example tools/call, tools/list, initialize);
• gen_ai.tool.name: conditionally required, when the operation concerns a specific tool;
• error.type: required if, and only if, the operation fails;
• rpc.response.status_code: required if the response contains a JSON-RPC error code;
• gen_ai.operation.name: recommended, with the value execute_tool for a tool call;
• mcp.resource.uri: opt-in only.

The last point deserves attention. A resource URI can take millions of values, so the convention leaves it off by default. The tool name, by contrast, is bounded by the server’s catalogue. That is the right dimension: mcp.method.name × gen_ai.tool.name × error.type stays low-cardinality. For how to measure and prune unused series, see Kubernetes metrics cardinality.

Counting failures: three cases to tell apart

The failure rate depends on what the convention classifies as an error. The definition of error.type in model/mcp/common.yaml distinguishes:

• a response carrying a JSON-RPC error code: error.type receives that code as a string;
• an HTTP error response without a JSON-RPC code: the HTTP status code;
• a timeout or transport error: the canonical name of the exception or error.

A fourth case is specific to tools: the JSON-RPC call succeeds, but the result (CallToolResult) carries isError set to true. The convention then asks for error.type to be tool_error. Without this rule, a tool that fails « cleanly » would stay invisible in a failure rate built on status codes.

On the server side, the convention states that some codes signal a request the receiver could not serve and must not be counted as errors: -32700 (Parse error), -32600 (Invalid Request), -32601 (Method not found), -32602 (Invalid params, also returned when the requested tool does not exist) and -32002 (Resource not found). Any other code is an error. On the client side, every JSON-RPC error code counts. As a result, the failure rate seen by the client is usually higher than the one seen by the server, and the two cannot be compared without this caveat.

Step 1: record the histogram with the recommended bounds

If the server is instrumented by hand, the convention says each duration metric should be declared with the explicit bounds [0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10, 30, 60, 120, 300] (in seconds).

The OpenTelemetry Python API accepts these bounds through the explicit_bucket_boundaries_advisory parameter of create_histogram (present in the opentelemetry-python repository read while writing). A minimal server-side example:

import time
from opentelemetry import metrics

meter = metrics.get_meter("my-mcp-server")
duration = meter.create_histogram(
    "mcp.server.operation.duration",
    unit="s",
    description="MCP request or notification duration as observed on the receiver",
    explicit_bucket_boundaries_advisory=[0.01, 0.02, 0.05, 0.1, 0.2, 0.5,
                                         1, 2, 5, 10, 30, 60, 120, 300],
)

def call_tool(tool_name, function, **arguments):
    attrs = {"mcp.method.name": "tools/call",
             "gen_ai.tool.name": tool_name,
             "gen_ai.operation.name": "execute_tool"}
    start = time.perf_counter()
    try:
        result = function(**arguments)
        if getattr(result, "isError", False):
            attrs["error.type"] = "tool_error"
        return result
    except Exception as exc:
        attrs["error.type"] = type(exc).__name__
        raise
    finally:
        duration.record(time.perf_counter() - start, attrs)

The critical lines: the unit is the second (not the millisecond), error.type is set only on failure, and no call argument enters the attributes. A tool name carries no user identifier and no free-form value.

Step 2: derive counters and histograms from traces with the Collector

When MCP spans already exist, there is no need to duplicate the instrumentation: the Collector’s span_metrics connector (contrib distribution) aggregates spans into metrics. According to its README, it produces a call counter (traces.span.metrics.calls) and a duration histogram (traces.span.metrics.duration), with a status.code dimension to tell errors apart. Its status is « alpha » and the default unit is the millisecond, per the same README: check this against the version you deploy.

connectors:
  span_metrics:
    histogram:
      unit: s
      explicit:
        buckets: [10ms, 20ms, 50ms, 100ms, 200ms, 500ms, 1s, 2s, 5s, 10s, 30s, 60s]
    dimensions:
      - name: mcp.method.name
      - name: gen_ai.tool.name
    aggregation_cardinality_limit: 500

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [span_metrics]
    metrics:
      receivers: [span_metrics]
      exporters: [otlp]

The keys histogram.unit, histogram.explicit.buckets, dimensions and aggregation_cardinality_limit appear in the connector README. The value 500 is an example to adapt. The README describes this limit as a circuit breaker: beyond it, extra combinations are grouped under otel.metric.overflow="true". The resulting names (traces.span.metrics.*) differ from mcp.server.operation.duration: pick one of the two families for your dashboards, not both. For setting up the Collector, see the OpenTelemetry Collector on Kubernetes.

How to check that it works

Deliberately call a tool with an invalid argument, then a tool that returns isError: true. In the backend, the series error.type="tool_error" must appear for the second, and the histogram count of the first must increase. If the Collector runs with the debug exporter, the metric points show up in its logs before they even reach the backend.

Step 3: define the SLI per tool

Two indicators are enough to start:

• Availability per tool: calls without error.type divided by all tools/call calls, per gen_ai.tool.name;
• Latency per tool: a percentile (for example the 95th) estimated from the buckets; the estimate is only as precise as the bucket width, hence the value of the recommended bounds.

Set the objective with the team that owns the tool, and alert on error-budget consumption rather than on an isolated spike, as described in alerting on symptoms rather than causes. How these measurements differ from plain monitoring is covered in observability versus monitoring.

Best practices

• One measurement point per side: server for real load, client for the agent’s experience (network included).
• No free-form value as a dimension: no resource URI, no argument, no session identifier.
• Write the rule « -32602 is not an error on the server side » into the runbook, to avoid debates during an incident.
• Cap cardinality upstream (aggregation_cardinality_limit) and watch for overflow.
• Blameless review: a high failure rate points to a tool to improve, not to a person.

Frequently asked questions

Is a call counter needed in addition to the histogram?

No. The MCP convention defines no counter: the number of observations in a histogram gives the number of calls. The span_metrics connector, however, provides a dedicated counter derived from spans.

Does a tool returning isError set to true count as a failure?

Yes, per the convention: error.type must be tool_error. Without that attribute, the call looks successful at protocol level.

Are these metrics stable?

No, they are marked « development » in the repository read. Pin the version you follow and read the changelog before each upgrade.

Conclusion

MCP tool metrics come down to four histograms in seconds, three useful attributes (method, tool, error type) and one precise rule for tool_error. The rest is derived: calls, failures, percentiles. Sources: the GenAI and MCP conventions repository and the span_metrics connector README. For the security side of these servers, see auditing MCP servers.

span.observer is an independent community blog, not affiliated with Dynatrace LLC or the projects cited. These conventions are in development: check names, attributes and parameters in the official documentation before applying them.

---

Originally published on span.observer. French version: https://span.observer/fr/metriques-outils-mcp/.