DEV Community

Cover image for The Retry Loop That's Tripling Your Bedrock Bill
Thirumalaiboobathi B
Thirumalaiboobathi B

Posted on

The Retry Loop That's Tripling Your Bedrock Bill

#ai #mcp #opensource #aws

Six green spans, one broken tool

A few weeks after shipping cost attribution, I was staring at a trace that looked healthy and made no sense.

Six calls to the same tool inside one session. All HTTP 200. All span status OK. Total duration about four seconds. Nothing in the trace suggested anything was wrong.

The tool had failed all six times. Same error, every call. The agent just couldn't tell.

I only found it because the cost attribution I'd built in v0.5.0 showed six InvokeModel charges for one logical operation. The traces said everything succeeded. The dollars said otherwise.

That gap is what v0.6.1 closes.

Why the agent keeps retrying

MCP has two error channels, and only one of them behaves the way you'd expect.

Protocol errors — unknown method, malformed request — come back as JSON-RPC error objects. Those propagate normally.

Tool errors don't. When a tool executes and fails, the server returns a successful JSON-RPC response with an isError flag inside the result:

HTTP/1.1 200 OK

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "isError": true,
    "content": [
      { "type": "text", "text": "connection refused: 10.0.0.5:5432" }
    ]
  }
}
Enter fullscreen mode Exit fullscreen mode

Nothing throws. Nothing rejects. Generic instrumentation reads the transport status, sees 200, and marks the span OK.

Now think about what the agent sees. It gets that error text back as a normal tool result. From the model's point of view nothing failed — it just received content that reads like a complaint about its input. So it does the sensible thing: rephrases the arguments and tries again.

The tool fails identically. The agent tries again.

Each retry resends the accumulated context to Bedrock, and the context has grown by the previous failed result. So the retries get progressively more expensive:

Attempt Input tokens Output tokens
1 8,000 300
2 8,400 300
3 8,800 300
4 9,200 300
5 9,600 300
6 10,000 300

Illustrative token growth for one retry loop — actual figures depend entirely on your context size and the agent's retry behaviour.

Six billed InvokeModel calls producing nothing, and not a single error span to point at.

What thrash detection emits

v0.6.1 watches for the same tool failing with the same failure fingerprint repeatedly inside one session. When that crosses a threshold, it emits one event carrying the whole loop instead of leaving it scattered across N indistinguishable spans.

Span event: mcp.loop.detected

mcp.loop.length              6
mcp.loop.wasted_tokens_in    54000
mcp.loop.wasted_tokens_out   1800
mcp.loop.wasted_cost_usd     0.19
mcp.loop.duration_ms         4310
mcp.loop.first_span_id       7b2e...
mcp.loop.first_trace_id      c81a...
mcp.loop.session_id          sess-4471
mcp.failure.fingerprint      a3f8c21d94b06e77
Enter fullscreen mode Exit fullscreen mode

first_span_id is the one that saves you time at 2 AM. It points back to the span where the loop started, which is where the actual root cause lives — the other five are just echoes.

Five metrics

mcp.tool.loop.detected          Counter
mcp.tool.loop.length            Histogram
mcp.tool.loop.wasted_tokens     Histogram   tokens
mcp.tool.loop.wasted_cost_usd   Histogram   USD
mcp.tool.loop.duration          Histogram   ms
Enter fullscreen mode Exit fullscreen mode

Same division of labour as v0.5.0: the counters give you alerting and trend lines, the span event gives you the drill-down when an alert fires.

The fingerprint is what makes grouping work

Detection keys off the failure fingerprinting from v0.4.0, and this is the part that makes it useful rather than noisy.

Raw error strings don't group. The same broken connection produces a different message every time — different IP, different request ID, different path. So the fingerprint normalises the error through a pipeline that strips UUIDs, paths, numbers, and hex strings, then hashes the result and truncates to 16 hex characters:

connection refused: 10.0.0.5:5432   →  a3f8c21d94b06e77
connection refused: 10.0.0.7:5432   →  a3f8c21d94b06e77
timeout after 30000ms on req_88a1   →  6d10b4e7c2f3a915
timeout after 30000ms on req_91c4   →  6d10b4e7c2f3a915
Enter fullscreen mode Exit fullscreen mode

Same root cause, same fingerprint, regardless of the incidental detail. Six failures with six different messages collapse into one detected loop — which is the correct reading, because it is one problem.

This is also why v0.6.1 couldn't have shipped before v0.4.0 and v0.5.0. The loop event needs the fingerprint to know the failures are the same, and the cost attribution to say what the loop cost. Neither feature could produce this alone.

Instrumenting a Bedrock-backed MCP server

No new setup. If you're already running the library, thrash detection is on by default:

import { instrumentMcpServer } from "opentel-mcp";

const server = instrumentMcpServer(mcpServer, {
  serviceName: "mcp-server"
});
Enter fullscreen mode Exit fullscreen mode

Any tool that fails three times with the same fingerprint inside 60 seconds gets flagged automatically.

Here's a Bedrock tool that will thrash if the downstream table goes missing:

import { BedrockRuntimeClient, InvokeModelCommand } from "@aws-sdk/client-bedrock-runtime";
import { instrumentMcpServer } from "opentel-mcp";

const bedrock = new BedrockRuntimeClient({ region: "us-east-1" });

server.tool("lookup_customer", async ({ query }) => {
  try {
    const rows = await db.query(query);
    const response = await bedrock.send(new InvokeModelCommand({
      modelId: "amazon.nova-pro-v1:0",
      body: JSON.stringify({
        messages: [{ role: "user", content: [{ text: summarize(rows) }] }]
      })
    }));

    const result = JSON.parse(new TextDecoder().decode(response.body));

    return {
      content: [{ type: "text", text: result.output.message.content[0].text }],
      _meta: { usage: result.usage, model: "amazon.nova-pro-v1:0" }
    };
  } catch (err) {
    // This is the shape that goes unnoticed: HTTP 200, isError in the payload
    return {
      isError: true,
      content: [{ type: "text", text: `query failed: ${err.message}` }]
    };
  }
});

instrumentMcpServer(server, { serviceName: "customer-tools" });
Enter fullscreen mode Exit fullscreen mode

Rename the underlying table and the agent will retry this four to six times before giving up, charging Bedrock for each attempt. With v0.6.1 you get one mcp.loop.detected event with the total.

Configuration

Every field is overridable in code or by environment variable, which makes it tunable per environment without a redeploy:

Option Env var Default Description
enabled OTEL_MCP_THRASH_ENABLED true Disables detection entirely
threshold OTEL_MCP_THRASH_THRESHOLD 3 Same-fingerprint failures before triggering
windowMs OTEL_MCP_THRASH_WINDOW_MS 60000 Rolling window failures must fall inside
maxTrackedKeys OTEL_MCP_THRASH_MAX_TRACKED_KEYS 1000 LRU cap on the tracking store
entryTtlMs OTEL_MCP_THRASH_ENTRY_TTL_MS 900000 How long an idle key survives
reEmitAfter OTEL_MCP_THRASH_RE_EMIT_AFTER 3 Re-emit every N failures past threshold
assumeSingleSession OTEL_MCP_THRASH_ASSUME_SINGLE_SESSION false Opt-in fallback for undetectable transports

Invalid environment values fall back to the default silently. They never throw.

Four design decisions worth explaining

High-cardinality attributes never touch metric labels

Fingerprints are unbounded — every new bug is a new fingerprint, permanently. Session IDs are worse. Put either on a metric dimension and you create a new time series per bug and per session, forever. In CloudWatch that gets expensive fast; in any backend it eventually falls over.

So every metric carries only gen_ai.tool.name. The fingerprint and session ID live on the span event, where high cardinality is safe and you get the full detail anyway when you drill in. The allowlist is enforced in code, not by convention, because "remember not to add attributes here" is not a strategy that survives contact with a future version of yourself.

The tracking store is bounded, and there's no timer

Loop detection has to remember recent failures, which means it holds state. MCP servers on stdio transport run for the lifetime of the process — sometimes weeks. An unbounded map here would be a slow memory leak that only shows up in the longest-running deployments, which are exactly the ones you least want to debug.

So it's an LRU with a TTL: capped keys, lazy expiry on read, and an amortised sweep on write. Deliberately no setInterval — a live timer keeps the Node event loop alive and stops the server exiting cleanly, which is its own subtle bug.

Sessions are never merged on a guess

This is the setting most likely to get misconfigured, so it's worth being explicit.

Loop detection needs a session boundary. Merge two clients' failures into one bucket and you get a phantom loop: three unrelated clients each failing once looks identical to one client failing three times.

Session-oriented transports provide a real session ID. Stdio doesn't — there's exactly one connection for the process lifetime instead. The resolution order: a real session ID always wins and permanently marks the server as session-aware. Once a server has been observed handing one out, a later call without one is skipped rather than merged. A generated fallback is used only when the transport is structurally confirmed single-connection, or when you explicitly opt in via assumeSingleSession.

Otherwise detection is skipped silently rather than guessing. Skipping produces missing data; guessing produces wrong data. Wrong is worse.

It observes. It does not intervene.

Same principle as the budget flags in v0.5.0. Detecting a loop sets attributes and emits metrics. It does not cancel the request, break the connection, or refuse the next call.

An instrumentation library that can interrupt agent execution is a library that can take down production in a way nobody predicted. Loop-breaking belongs in the agent framework or an AI gateway, where it's a deliberate part of the request path and can be disabled without redeploying your MCP server.

Checking it without a collector

If you just want to know whether anything is thrashing right now, there's an in-process accessor that involves no OpenTelemetry at all:

console.log(server.getThrashSummary());

// {
//   activeLoops: 1,
//   totalLoopsDetected: 4,
//   totalWastedCostUsd: 0.09,
//   totalWastedTokensIn: 3600,
//   totalWastedTokensOut: 900,
//   topOffenders: [
//     { toolName: 'lookup_customer', fingerprint: 'a3f4c8e2b1d09f77',
//       loops: 3, wastedCostUsd: 0.03 }
//   ]
// }
Enter fullscreen mode Exit fullscreen mode

Nothing is sent anywhere. It's safe to call from a health check handler, and it works before you've stood up any backend.

One caveat worth stating: activeLoops and topOffenders reflect only what's currently in the bounded store, so an evicted or expired loop won't appear even though it really happened. The cumulative totals survive both and answer "how much has this process wasted since it started." Read the totals for accounting, not the offender list.

Common questions

Does this add latency? Detection is a hash-map lookup and a counter increment on the failure path only. Successful calls do a single clear operation. No network calls, no async work.

What if the same tool legitimately fails for different reasons? Different reasons produce different fingerprints, so they don't group and no loop fires. That's the intended behaviour — a tool failing three distinct ways is a different problem from a tool failing the same way three times.

Does it work without fingerprinting enabled? No. Detection keys off the fingerprint, so with fingerprinting: false it silently never fires. Both are on by default.

What about schema validation errors? Those travel the JSON-RPC error channel rather than arriving as isError: true, so they're a separate detection path and aren't covered here. Worth knowing, since "the agent is calling the tool wrong" is arguably the failure mode most likely to loop. It's on the list.

Is the loop event emitted once or repeatedly? Once at the threshold, then again every reEmitAfter failures past it — so a 12-call loop emits at 3, 6, 9, and 12 rather than nine times. The wasted-cost figures are delta-accounted so re-emission doesn't double-count.

The gap I haven't closed

Worth stating plainly, because it's the most interesting unsolved problem in the library right now.

If the observation path itself is broken — no provider registered, collector unreachable, exporter silently dropping — then a tool failure and a clean run produce identical output. Zero. "Nothing failed" and "nothing was observed" collapse into the same state.

This came out of a review from an external reader on the v0.3.0 release post, and he was right. A liveness signal can't travel over the channel whose liveness is in question, so this has to surface on the in-process path rather than through spans.

The proposed contract is three states rather than two: OBSERVED_CLEAN, OBSERVED_FAILING, OBSERVATION_UNAVAILABLE. It's currently in the repo as a skipped specification test with the reasoning documented, rather than quietly ignored. Detecting an unbound observation path without coupling to unstable SDK internals is the part I don't have an answer for yet.

Anyone who has chased a collector that looked perfectly healthy while binding no ports will recognise the shape.

What's next

  • Splitting protocol-level failures from execution failures in the fingerprint, so a schema error and an upstream outage read differently
  • Tool schema drift detection — hashing tools/list schemas and flagging silent changes
  • The observation liveness contract above
  • Cost-aware sampling, so expensive traces survive sampling decisions that cheap ones don't

Try it 

npm install opentel-mcp
Enter fullscreen mode Exit fullscreen mode

If you're running MCP servers on AWS, the thing worth checking today is whether your tool failures are actually reaching your traces. Mine weren't — and I'd like to know what your retry patterns look like, because the edge cases people report are what shape the next release.

Top comments (0)