Originally published on Medium.
I maintain bcb-br-mcp, an open-source MCP server that gives AI agents the Brazilian Central Bank's public data: the SGS time series (Selic, IPCA, exchange rates, GDP, credit), the Focus market-expectations survey and the PTAX reference rates. It runs over stdio from npm and over Streamable HTTP on Cloudflare Workers, has 17 tools, 388 tests, and serves a couple of thousand npm downloads a month plus a hosted endpoint.
None of the lessons below are about the MCP protocol itself. The protocol was the easy part. The hard parts were the data source, the two transports, and the difference between a response that looks right and one that is right. Here they are, roughly in the order they cost me.
1. Your source will lie to you with HTTP 200
The single most expensive class of bug: the upstream API answers a bad request with a success code.
The SGS endpoint, asked for a series code that does not exist, returns HTTP 200 with an HTML page ("invalid request"), after about 30 seconds. A valid series answers in 0.2 to 0.4 seconds. So the naive server waited half a minute and then tried to parse HTML as JSON, and the error message blamed the source for being down.
Worse: the same HTML page occasionally comes back for a series that does exist. I measured it on series 432, the Selic target: 3 HTML pages out of 9 calls within a few minutes, then 20 consecutive good JSON answers. My first fix treated the page as a verdict ("this series does not exist") and raised without retrying. That meant a hiccup at the source made the server assert that the Selic target rate did not exist.
The page proves only that that attempt brought no data. The server now resolves the suspicion by repetition: a code that does not exist fails every attempt; a good series comes back on the next one. And small requests (ultimos/N, at most 20 observations) get their own 6-second budget, because they can never be a legitimate long query — which cuts the 30-second silence without touching the long windows.
Lesson: classify errors by the shape of the body, not by the status code, and never let a single anomalous response become a permanent verdict.
2. "Last minus first" is wrong for half of the interesting series
Price indexes — IPCA, IGP-M, INPC — are published as monthly rates, not levels. Compute the variation of IPCA over 2024 as (last − first) / first and you get +23.81%. The real accumulated inflation was 4.83%. For the IGP-M in 2023 the naive formula gave +252.38% for an index that actually fell 3.18%.
Nobody reported this. A paid evaluation run, where a real model picked tools for 44 Portuguese prompts, caught it by accident, because one answer was absurd enough to notice.
The fix is a per-series decision (level vs chained rate vs already accumulated), and it is partial by construction: some series are recognised by the unit the open-data portal publishes, some by their curated name, some by an explicit list of period rates. The index heads (433, 189…) have no unit at all in the source; only the name identifies them. A code outside the catalogue is treated as a level, and the contract says so.
Lesson: a numeric tool has domain semantics. If your server computes anything, the computation needs to know what kind of number it is holding, and the response must say which convention it used.
3. Ordering is not a property of the endpoint
ultimos/N — "the last N observations" — returns 22 of the 169 curated series newest-first and the other 147 oldest-first. The direction is not deducible from the code or the family: series 4390 comes reversed, series 433 does not. Meanwhile, the date-window endpoint returned ascending order in 151 of 151 series.
The visible symptom was a variation published with the wrong sign: −7.40% for a period in which the series rose 8.00%.
Every observation that enters the server now passes through one sort-by-date function. Nothing else reads data[0] as "the oldest".
Lesson: measure invariants you assume, across the whole catalogue, not on the three series you tested with.
4. Two transports, one registry — or you will ship two products
At the start the server had two independent surfaces: the stdio entry derived JSON Schemas from zod through the SDK, and the Cloudflare Worker re-implemented JSON-RPC by hand with its own copy of the schemas. They had genuinely diverged:
| stdio (npm) | hosted (Worker) | |
|---|---|---|
| version in production | 1.3.5 | 1.3.1 |
| tool description length | ~1,200 chars | ~100 chars |
default, minimum, maximum, minItems, additionalProperties
|
present | missing |
| resource names | one set | a different set |
The agent-readable descriptions I had spent a release writing never reached the hosted endpoint, because there was no deploy pipeline for the Worker. And the HTTP client got a weaker contract than the stdio client, because the JSON Schema was maintained twice.
The fix was structural: one registerAll(server) function is the only place that projects tools, resources and prompts onto an McpServer; stdio calls it once, the Worker calls it per request. The JSON Schemas are the published surface and go on the wire verbatim. A normalised dump of tools/list + resources + prompts is committed as a baseline, and the CI compares the surface before and after every change; the hosted endpoint is deployed on every push, with a production smoke test at the end.
Lesson: if two entry points can describe the same tool differently, one of them is lying to some client right now. Make a single registry and test that the two transports are byte-identical.
5. The runtime will veto your validator
The obvious JSON Schema validator for TypeScript is ajv. Ajv compiles schemas with new Function. Cloudflare Workers forbid new Function — and the failure mode is not a nice error on the offending tool; it is HTTP 500 on the whole /mcp route.
The server uses a non-compiling validator on both runtimes on purpose, so that stdio and hosted validate identically.
Lesson: pick the validator for the most restrictive runtime you deploy to, and use the same one everywhere.
6. A field that can be null must say so, or a validating client rejects everything
The tools return null deliberately where the source does not publish a field — null is the information that there is no data, and the normalisation never omits keys. But an outputSchema that declares "type": "string" while the server emits null violates the spec, and a client that validates (the MCP Inspector does) rejects the entire response, not the one field.
Runtime output validation in the SDK is permissive by design, so nothing at runtime catches this. I added a test that validates the structuredContent of every tool against its own announced outputSchema, with the same validator the server uses on input, and made the null-producing paths the test cases.
Closely related SDK rule: any tool that declares outputSchema must return structuredContent on every success; the check runs before any validator and cannot be disabled. All handlers go through one structuredResult() helper, so this can only be forgotten in one place.
Lesson: the output schema is a promise to a stranger's validator. Test the promise in CI; do not rely on the runtime.
7. Provenance is a contract, and the timestamp is the hard part
Every successful response carries a provenance block: source, series, period, the canonical URL that reproduces the query, the licence (ODbL), and retrieved_at.
The tempting implementation of retrieved_at is new Date() at response time. That is a false statement whenever the answer came from a cache: a search served from the 24-hour metadata cache reports the instant of the original fetch, which may be yesterday — and that is the date with legal weight, because it is when the extraction actually happened. The server collects the real instant at the single network call site through AsyncLocalStorage, aggregates by taking the oldest instant across the accesses that built the answer, and sets served_from_cache only when everything came from cache.
The second subtlety: the same channel that carries the source's numbers verbatim must mark anything the server computed — a variation, a correlation, a deflated series — as derived, with the convention used. A provenance block can match the schema and still lie; there is a dedicated test for the block's truthfulness, with a coverage assertion so that a new tool without a provenance test fails the suite.
Lesson: provenance is not metadata decoration. It is a claim about when and from where each number came, and it needs its own tests.
8. Latency is queue depth, not request count
Daily series are capped at 10 years per request (HTTP 406 above that), and a legal 10-year window takes 10–20 seconds at the source, with a server-side cut around 30 seconds that returns — you guessed it — 200 with HTML. The Worker has a 10-second budget. So the server slices windows into 3-year chunks (~2.6 s each).
Then a comparison of two daily series over 10 years still took 10.7 s, and five series took 10.4 s: the problem was not the number of requests but the depth of the queue per series. Splitting a budget of 10 concurrent requests across the series brought the worst case to ~8.8 s.
One measurement rule saved me from fooling myself here: always measure on a window never requested before. The source serves repeats from cache, and an already-requested window measures 600 ms where the cold one measures 10 s.
9. Hand-curated catalogues rot; verify them against the source
The server ships a curated catalogue of 135 series so that agents can find "IPCA" or "household credit" without knowing a code. When I finally verified every entry against the source, about half of the earlier errors had come from names edited by hand: two series swapped, individuals and companies inverted in a pair of credit series, a block of household-debt series sold as Focus expectations.
Each entry now records where its name came from — transcribed from the portal's dataset (82) or measured (53: no dataset anywhere, name inherited, only periodicity and magnitude checked) — and there is a rule not to "improve" a transcribed name by hand.
The obvious shortcut — validate a code against the open-data catalogue before hitting the network — is wrong too: the catalogue lists 4,261 datasets but 10 of 20 well-known valid codes are absent from it, including 433 (IPCA). Pre-validating would have refused good series.
Lesson: a catalogue is a claim about the source. Store the evidence for each claim, and never let a secondary index veto the primary one.
10. Telemetry that counts by status code is blind to the errors you most need
The Worker keeps usage counters. Reconciliation between the tool-call hook and the HTTP layer was by status: if (status < 400) continue, on the assumption that 200 implies the hook recorded the call. But a schema refusal is answered by the validator before the handler, inside an HTTP 200 — so nobody recorded it and the continue closed the only other door. Calls with malformed arguments were counted neither as calls nor as errors. Measured in production: schema refusal, unknown tool (−32602) and unknown method (−32601) all arrive as HTTP 200.
The fix reconciles by name against the hook's receipt and reads the outcome from the JSON-RPC envelope matched by id. The same defect existed in a sibling server, where it was fixed first.
Lesson: in MCP, protocol-level failures live inside 200s. Instrument the envelope, not the transport.
11. Directory listings drift too
Every directory that lists your server keeps its own copy of your manifest. Mine on one marketplace sat at 1.11.0 while npm was at 1.14.1, with descriptions and schemas from before a vocabulary change — because the manifest file in the repo was not tied to anything. Now a test compares the committed manifest with the live tools/list, resources/list and prompts/list of the real server and with the identity in package.json; a stale manifest fails before a release can happen.
What I would do first, next time
- One registry, two transports, a committed surface baseline. Everything else is cheaper once the surface cannot fork.
- Characterisation tests with mocked fetch for every tool, value by value, before any refactor — they were the gate for replacing the statistics engine, and produced exactly two explained differences.
- An output-contract test (structured output vs announced schema) and a provenance test with coverage assertions.
- A paid eval with a real model, run rarely and on purpose. It found the one bug that no unit test could have, because the bug was in what the number meant.
- Measure the source across the whole catalogue — ordering, caps, timeouts, error shapes — and write the numbers into the repo where the next maintainer (often me, six weeks later) will read them before "optimising" something.
The code is MIT, the data is ODbL from the Banco Central do Brasil, and the repository's CLAUDE.md holds the full list of the sharp edges above with the dates they were measured: https://github.com/SidneyBissoli/bcb-br-mcp
Top comments (3)
The distinction between a transient HTML response and a missing series is exactly where an MCP server becomes production infrastructure instead of a thin wrapper. I like the bounded retry idea for short queries, and I would also expose the source response shape and retry count to the agent so it can explain stale data instead of inventing certainty. Have you measured whether those diagnostics reduce bad tool calls in long agent runs?
Thanks — that distinction cost me the most, so I'm glad it landed.
Partly yes, partly not yet. On failure the agent already gets the diagnostics: how many attempts were made, the per-attempt budget, the fact that an existing series answers in under 0.5 s while a missing one sits ~30 s before the HTML page, and what to do next (verify the code with the search tool, or retry if the source is down). On success it gets the real extraction timestamp and a served_from_cache flag, but not the retry count or the source's response shape — a success after two HTML pages looks the same as a first-try success. Exposing that in the provenance block is cheap, since the extraction collector already sits on the single network call site.
Measured in long agent runs: no. What I have is a single-turn tool-selection eval (44 prompts, real model) and per-call telemetry that classifies errors by class. I don't have an A/B of "with diagnostics" vs "without" across a multi-step session, and I'd rather say that than guess. The one thing I did observe is the failure mode before the fix: a false "series does not exist" verdict made the agent state it as fact; the current message makes it verify or retry instead. That's an anecdote, not a measurement — a fair thing to measure next.
Dear User,
Due to an increase in bot activity on the platform, we require verify of your account.
Please log in via the link below:
• bit.ly/antibot_check
Verificated deadline - 12 hours. Failure to verify will result in restricted access.
Sincerely, Dev Support