Every day, agents pick tools for us. A skill to write a document, an MCP server to read a database, a paid endpoint for some data. Many of these calls work. Some fail quietly: the skill needed a key nobody had set, the server answered with nothing useful, the endpoint took the payment and returned an empty body. The agent moves on, and the next agent facing the same choice starts from zero.
What strikes me is not that tools fail. It is that the knowledge of how they fail, and for whom, disappears at the end of each session. The agent that found out does not keep it, and nobody else ever sees it.
The gap is memory, not information
Each actor around an agent sees a slice. A registry knows a tool exists and how it describes itself. A scanner knows what its code contains. A check before payment sees an endpoint at the moment it is asked. A marketplace sees the tasks that pass through it. None of them holds what happened when agents actually used the tool, and in which situation.
That last part matters more than I expected when I started. The same skill can work for writing and fail for code. An MCP server that fails for lack of a key will work fine for the next person who has one. A result without its context tells the next agent almost nothing. With its context, it is exactly what that agent needs.
Where the gap becomes measurable
Payments are the one place where this can already be counted, because a payment leaves a trace anyone can check. Teppi paid 422 x402 endpoints 3,618 times and checked the answers: 69 were wrong, and all 69 were billed. Fizzl, who build a check that runs before payment, found that the frauds it cannot catch share one property: no single payment looks wrong, and they only show across many buyers.
I read both as the same finding seen from one corner. Paid calls are not the whole problem. They are the part where a result can be tied to a proof, which makes them a good place to start.
Three ideas I am building on
The first is that a result always belongs to a context. In ARP, the project I am building, an agent writes a line saying it used a tool, in a given domain, that it worked or failed, and if it failed, why, in a short closed list of reasons. A failure caused by something missing on the user's side does not count against the tool. A tool that was simply the wrong choice is marked down only in the domain where it was chosen. A score comes out of these lines, per tool and per context, by counting distinct wallets. Anyone can recompute it, and money deposited never moves it.
The second is that a signal should carry its own evidence whenever it can. For a paid call, the line names the settlement transaction, and the payer is read from the token transfer on chain. A line pointing to a payment nobody can find does not count. Because lines are written on chain and cannot be deleted, the agent also strips anything personal from a URL before it signs, so only the route remains.
The third is that nobody should have to come to ARP to use what it knows. The signal is produced once and reshaped for whoever reads it: a feed a payment check can show next to its own verdict, a list a registry can show beside a listing, an input in the format of a trust score that already exists. I would rather sit underneath other products than be one more place to look.
What I would like to hear from you
All of this rests on assumptions I can only test with people who run agents for real, and on most of them there is no public data. So instead of asking you to install something, here are the questions I keep coming back to.
When your agent chooses a tool, what tells you that someone else's experience applies to you? Is it the task, the stack, the domain, or something about your own setup that no label captures? Have you seen a tool that worked for everyone fail in your hands, and how long did it take you to notice?
If an agent you have never heard of says a tool failed, what would make you believe it? A payment attached, the age of its wallet, three strangers saying the same thing, or nothing short of your own test?
Would you let your agent report its own results without asking you, or would you want to see each one first? And where would you want to read what others found: inside the agent while it works, next to a listing, in a check before paying, or folded into a score you already trust?
If you publish a tool or sell an endpoint, would you want a way to answer a bad report with your own evidence? If you run a registry or a scoring service, would a usage signal change how you rank, and in what shape would you take it?
And the money question, because I honestly do not know the answer. Who should pay for this: the agent checking before it acts, the tool that wants to show its record, or the registry that wants better rankings?
If any of this is a problem you have, I would rather hear how it shows up for you than explain how I approach it. Reply here, or write to me.
The code is at github.com/intuition-box/ARP. Figures from Teppi and Fizzl, read on 11 November 2026.
Top comments (0)