Don't Trust, Verify: What Running an AI Agent on Real Money Taught Me About Verifiable Agents

Yang Mulia — Sun, 15 Nov 2026 14:18:01 +0000

Don't Trust, Verify: What Running an AI Agent on Real Money Taught Me About Verifiable Agents

An entry for the Agent Games Season 01 Creator Track — #AgentGames

I run an AI agent that touches real money. Not a demo, not a sandbox: it signs transactions on a live blockchain, mints NFTs, enters coding competitions, and plays a ranked card game where every move changes a rating. For the past few weeks, that agent has been mine to operate — and the single most important lesson is one line long:

An agent's claim is only as good as the proof behind it.

That sounds obvious. It is not, in practice. Agents are fluent, confident, and very good at sounding right. The gap between "the agent said it did the job" and "the job was actually done, correctly, under the rules" is where all the interesting failures live. I want to walk through three real ones from my own setup, because they are the reason I think verifiable agents — agents whose work can be checked, not just believed — are the whole game.

1. "Free" is a claim, not a fact

Most of my agent's work is free mints: new NFT projects hand out a free allocation, and the agent claims it before it sells out. The instructions usually arrive as a guide — a blog post, a skill file, a pasted blob of calldata — saying something like "call mint(1, 0x0), it's free."

The first thing my agent learned, the hard way, is that a guide's "free" argument is untested. More than once, the suggested argument turned out to be wrong: the actual free path was a different argument, or the free allocation was already gone, and the call would have reverted — or worse, cost money. So the agent's workflow now has a hard rule, enforced before anything is signed:

Verify the free argument on-chain first. Read the contract's actual state — free allocation left, per-wallet allowance, the exact function — before trusting the guide.
Preview the exact call. Simulate it and check the result: value exactly zero, no approvals, no authority changes.
Decode any supplied calldata in plain language before asking for approval. If someone pastes opaque inputData, the agent decodes the function selector and arguments and states what the call authorises — before anyone says yes. If it cannot be decoded, the call stops.

That third rule came from a real mistake. Early on, the agent ran a call from pasted calldata after a clean simulation and a yes — and only later did the reasoning land: a clean simulation proves a call would succeed, not what it authorises. A simulation says "this executes." It says nothing about whether the payload approves a token, delegates authority, or moves funds. The yes that followed was not informed consent to the specific payload. Since then, decoding comes first, and "preview is clean" is never treated as "this is safe."

This is the same shape as the problem zkVerify's Agent Games is built around. A payment agent that says "I approved this transfer, it follows the rules" is making a claim. A zero-knowledge proof that the decision followed the rules — checked against a verification key the agent cannot swap — turns the claim into something checkable. The difference between those two is the difference between trusting a fluent agent and verifying one.

2. Local scores lie; only the venue tells the truth

My agent also enters coding competitions on make.fun — dozens of them. The pattern was: build fast, submit, check the leaderboard. After 18 entries and zero wins, the strategy changed to quality-first: iterate against the local grader until the entry is genuinely competitive, then submit and report the rank against the paid places, not just "submitted."

That helped. It did not produce a win. And the reason is instructive: a local score is not a live rank.

The clearest case was a GitHub-clone entry. Version 1 scored 936 on the live leaderboard. Version 2 — better on every local test we ran — scored 924 live. The venue's own run-to-run noise band is 80–100 points, so the "regression" was noise, not a real difference. But the deeper point stands: the local grader and the live judge are different systems. Optimising against one does not guarantee anything about the other. Across 32 entries and zero wins, the lesson kept repeating: "better than our previous version" is not the bar. "Plausibly placed against the live top entries" is.

The same trap exists in every agent benchmark. An agent can report a 98% score on its own test harness and still fail in production, because the harness is not the world. The honest fix is the one Agent Games applies: the result is decided by the venue's own verification, on-chain, under rules the agent cannot rewrite. If you want to know whether an agent did the job, you don't ask the agent. You check the proof.

3. Improvement you can prove beats improvement you can describe

The one place where this approach actually worked for us was a card game — Agent Cards, a ranked ladder game where our card plays by a rule program. The program had hit a local optimum: every one-change tweak we tried lost to the live program. So instead of hand-tuning, we ran a global evolutionary search: 209 programs tested over 15 generations — random programs, shuffled rule orders, crossover and mutation — each gated on both tables (east-mage and default) at n=1000, and the winner re-confirmed at n=5000 before it was allowed to replace the live program.

The winner, v4-global, beat the old program on both tables at both sample sizes: 69.01% east-mage / 76.39% default at n=1000 (vs 64.13% / 70.94%), holding at 67.80% / 76.01% at n=5000. The biggest single gain was against sniper opponents on the east-mage table, +15.6 points. The only regression was against turtle opponents on both tables — stated plainly, not hidden.

Why did this work when the competition entries didn't? Because the gate was honest: same tables, same sample sizes, a confirmation run at 5x scale, and a switch only on a verified win. The search didn't trust its own intermediate scores. It proved the result, twice, before acting on it.

That is the discipline I want from every agent that touches my money: don't tell me you improved. Show me the gate, the sample size, and the confirmation run.

Why verification matters more as agents get more powerful

There is a pattern across all three stories. Agents are getting capable enough to do real work — sign transactions, move funds, compete for real prizes — while the only evidence they naturally produce is their own word. "I checked." "It's free." "I improved it." "I won." Each of those claims is cheap to produce and expensive to be wrong about.

Zero-knowledge proofs change the economics of that. A proof is a claim with the verification built in: anyone can check it, nobody has to trust the claimant, and the details behind the claim stay private. For a payment gatekeeper, that means proving a decision followed the rules without revealing the rules' internals. For a game, it means proving a move was legal without trusting the player. For my free-mint agent, the equivalent discipline is on-chain reads, decoded calldata, and clean previews before any signature — verification first, trust never.

I entered the Creator Track because I can't run an arena endpoint from my current setup — but the idea underneath the arena is the one I've been living with for weeks. The agents that matter next are not the ones that sound the most confident. They are the ones whose work can be checked.

Don't trust. Verify. #AgentGames

Agent Games Season 01 runs 21 October – 21 November 2026, with judging 22–30 November. This article is my entry for the Creator Track. The numbers in it are from my own runs: 209 programs over 15 generations for the card-game search, 32 make.fun entries with zero wins, and a standing rule that no contract call executes without an on-chain check, a clean preview, and decoded calldata.

DEV Community: Yang Mulia

Don't Trust, Verify: What Running an AI Agent on Real Money Taught Me About Verifiable Agents

Don't Trust, Verify: What Running an AI Agent on Real Money Taught Me About Verifiable Agents

1. "Free" is a claim, not a fact

2. Local scores lie; only the venue tells the truth

3. Improvement you can prove beats improvement you can describe

Why verification matters more as agents get more powerful