DEV Community

iamthat
iamthat

Posted on Originally published at karta.rta.lat

We scanned 2,500 public MCP servers. 72 touch money. None mentions a kill switch.

We scanned 2,500 public MCP servers. 72 of them touch money. None of them mentions a kill switch.

That sentence started as a marketing line. Then we checked it, and it held: across the public MCP registries, the servers that actually move funds — payments, transfers, trading, treasury — ship without a documented way to stop them mid-flight. Plenty of servers describe what they can do. Almost none describe what they must not do, or who can stop them.

This post is about the gap, and the three checks we ended up implementing.

The gap is not authentication

Most MCP servers get authentication right-ish: an API key, an OAuth token, a scoped credential. That answers "who is calling". It does not answer:

  • how much may this call spend, in total, before someone notices?
  • what happens when the agent retries a payment four times?
  • who can revoke the permission at 3am, and how fast?

Those are mandate questions, not auth questions. A credential says you may act. A mandate says you may act this much, this way, until this moment.

Check 1: a budget that is enforced, not logged

The common pattern is to log spend and alert on it. That is observability, not control. By the time the alert fires, the money is gone.

What worked for us: an envelope that the tool call must present, with a hard ceiling per task, per day, and per counterparty. The check runs before the side effect, and a failure is a refusal, not a warning. The important detail is that the ceiling lives outside the agent's context — if the model can edit its own limit, it is not a limit.

Check 2: a receipt that survives the agent

Agent output is easy to fake and easy to lose. If a payment happened, the evidence must be verifiable by a third party who does not trust the agent, the model provider, or us.

We settled on a signed receipt over a canonical JSON encoding (RFC 8785), with the hash chain anchored per task. Two properties matter more than the format:

  1. Deterministic — the same event must produce byte-identical output, otherwise signatures drift and audits fail.
  2. Standalone — verification must not require our server to be alive. A dead service must not be able to invalidate history.

Check 3: a stop that is faster than the loop

An agent loop can issue hundreds of calls per minute. A kill switch that requires a deploy, a config push, or a support ticket is not a kill switch.

The stop has to be checked on the same path as the budget, and it has to fail closed. We test it by flipping it during a live run and confirming the next call is refused — not by reading the code and assuming.

What we learned from the 2,500 servers

  • Capability is documented; restraint is not. README files list tools. Very few list limits.
  • "Human in the loop" usually means a chat message. A message is not a gate: it does not block, it does not expire, and it leaves no receipt.
  • Registries reward demos. A server that can send USDC gets attention. A server that refuses to send too much gets none — until someone needs the refusal.

If you run an agent that touches money, the audit is short: find the ceiling, find the receipt, find the stop. If any of the three lives only in the prompt, you have a suggestion, not a control.

The scan data and the checks are public: karta.rta.lat/report. The MCP server that implements them is attest.rta.lat.

We publish the method, not the marketing: if a number here is wrong, tell us and we will correct it in public.


Agent bill growing faster than usage? We run a 5-day audit on one month of agent logs — duplicate calls, failed-but-billed requests, quote-vs-settled differences, missing refunds. One report, fix order included, $500 in USDC: https://ailegal.legal/audit

Top comments (2)

Collapse
 
mickyarun profile image
arun rajkumar •

The three-part audit is right and I would add a fourth, because "find the stop" has a hole in it in exactly the domain you scanned.

A stop checked on the same path as the budget, failing closed, is honest about the next call. It says nothing about the one already in flight. In payments that window is not theoretical. Once an instruction has been accepted by the rails, cancel is not an operation that exists. What exists is a reversal, a different instruction with its own authorisation and its own receipt, and a counterparty who has to agree. Flipping the switch mid-run and confirming the next call is refused is a good test, and it tests the half that was never the expensive half.

So: find the ceiling, find the receipt, find the stop, then find out what the stop does to the payment that was in flight when you flipped it. If the answer is "nothing, and here is the reversal path, which needs this other permission", that is a mature server. If nobody has asked, the kill switch is a scheduler.

On the budget, one detail decides whether the ceiling holds: who issues the idempotency key. Your retry question has a clean answer only if the key comes with the envelope. If the agent mints it, four retries are four distinct payments as far as the ceiling is concerned, each one inside the limit, and the limit was per-call all along. Same shape as your point about the ceiling living outside the model's context. If the model chooses what counts as a repeat, it chooses the denominator.

Your registry finding matches what I see in the specs rather than the READMEs. Scheme rules for variable recurring payments already make restraint mandatory and explicit, with a maximum per payment and a maximum per period, plus a date the authority dies. None of that language got carried into how MCP servers describe themselves. It exists. It lives in a document nobody building an MCP server reads.

Collapse
 
paul_irolla_6b5ae261d0224 profile image
Paul Irolla •

Of the three, fail-closed is the cheapest and the one people skip. It only works if the agent stops when it cannot reach the check, and that needs the stop written in the harness, not in the prompt.