DEV Community: drdz23

I lost a Verdikta bounty by one word in someone else's README

drdz23 — Sun, 25 Oct 2026 00:10:34 +0000

I lost a Verdikta bounty by one word in someone else's README

On 24 October 2026 I submitted to Verdikta bounty #328, scored 42 out of 100 against a pass mark of 85, and was paid nothing. The total cost was about 0.000265 ETH. Here is exactly what I did, what it cost, what the jury said, and what I would do differently. The last part is the useful bit if you are thinking about hunting on Verdikta.

The task
Bounty #328 paid 0.0015 ETH to add Verdikta Bounties to exactly one "awesome" list or agent-tools directory, following that list's own contribution rules to the letter. The brief was unusually clear that a merge did not matter. What was scored was an accurate, rules-following submission: one list, one entry, the contribution rules quoted verbatim, the exact text submitted, and a public link to the pull request. All five gates were must-pass, so failing any one of them meant a score of zero from that juror.

What I did
I picked VoltAgent/awesome-agent-skills, a large, actively maintained list of agent skills. Verdikta publishes an official skill, verdikta-bounties-onboarding, so a skills list seemed like a natural home. I read the CONTRIBUTING file and followed its entry format:

the bold author/skill-name link;
a description of 10 words or fewer (mine was eight: "Hunt and post AI-judged escrow bounties on Base");
the PR title format Add skill: author/skill-name;
the line added at the end of a category.
I opened pull request #1261 with a one-line diff, wrote a submission file quoting the rules and reproducing the exact line, and attached a screenshot of the diff.

Then I hit the problem that sank me. The contribution guide lists the Community categories and says that if nothing fits, use "Other". But the README has no "Other" section. I put the entry in "Specialized Domains", next to MetaMask's agent wallet skill, and explained the choice in my submission file. I flagged it to myself as the main risk before submitting. I was right to worry.

The clumsy part
Submitting on Verdikta is two on-chain steps: prepare the submission (upload the files, register them), then start the evaluation by attaching the oracle prepay. When I clicked submit, the page showed "That submission does not exist on-chain", while my wallet said the transaction had confirmed. I clicked again. Then again.

I had created three prepared submissions (ids 0, 1 and 2) before I realised the site's indexer was simply a few seconds behind the chain. Only one of them, id 2, was ever started and evaluated. The other two cost a little gas and nothing else. The lesson is boring and real: after the wallet confirms, wait half a minute and refresh before clicking anything again.

What it cost
These are my real figures from Base:

Oracle prepay attached: 0.00024 ETH, on the start transaction. In my run it was spent on the jury round. That matches what the platform says to expect: the prepay is refundable in principle, but usually used up in full.
Gas for the three prepare transactions: about 0.0000049 ETH in total, a fraction of a cent each.
Gas for the start transaction: about 0.0000202 ETH.
Gas for the finalize transaction: zero for me. I never sent it. Another address finalized my submission about half an hour later, together with the next one.
Payout: zero. A failed submission is paid nothing.
So the whole experiment cost about 0.000265 ETH, around sixty cents. On a bounty paying 0.0015 ETH, that is a cheap lottery ticket, but it is not free. Three or four careless attempts would eat a real share of the reward.

What the jury said
The panel had two models, one from OpenAI and one from Anthropic, each weighted 50%. They split hard. The Anthropic juror passed every gate and scored the quality at 83.5. The OpenAI juror failed me on the "rules followed" gate. The published justification puts it plainly: it failed "due to the entry being placed in the 'Specialized Domains' category, which it determined was not in accordance with the author's quoted rules that instruct placing non-fitting skills in 'Other.'"

A failed gate means that juror's score is zero, and the average landed at 42. The irony is that I had quoted that exact rule myself. By quoting it and then not following it, I handed the stricter juror the reason to fail me.

Another hunter submitted half an hour later, scored 87 and took the whole bounty. On Verdikta the first submission that clears the threshold is paid everything. There is no prize for second place and no averaging of attempts.

What I would tell another hunter
Pick a target whose rules you can follow literally. If the rules point to a category, a template or a section that does not exist, choose a different list. Do not try to explain your way around it. A careful explanation impressed one juror and did nothing for the other.

Assume the strictest juror decides. Two models read the same file differently. Write for the one that reads the gates as a checklist.

Quote only the rules you fully meet, and meet every rule you quote. The Anthropic juror also docked me for not quoting the list's "real community usage" requirement. Silence on a rule reads as hiding it.

Do not click submit twice. Wait for the site to catch up with the chain.

Read the published justifications of past bounties. Every score on Verdikta comes with the reasoning of each juror, and it is all public. That turned a loss into a precise lesson. I know exactly which sentence of my submission cost me the money, which is more than most rejections ever tell you.

I drafted my submission with an AI assistant and checked it myself before sending, and the mistake was still mine. I would hunt here again. Next time I will pick a list with no ambiguity in it.

Why you can't cancel a Verdikta bounty (and why that's the point)

drdz23 — Wed, 21 Oct 2026 02:38:45 +0000

If you're new to Verdikta, the first surprise is that there is no cancel button. None. For the creator, for the Verdikta team, for anyone. That is deliberate, and here is why.

When a bounty is created, the payout leaves the creator's wallet and sits in an escrow contract on Base (BountyEscrow, 0xA741eFf41Bcf14793E61CEbB4179E05C9124D3f6). The contract has no cancel, withdraw, edit or pause function for a funded bounty, and there is no admin, support desk or multisig that can claw the money back or change the terms.

The only way out is time. If the deadline passes and nobody submitted passing work, the expired bounty can be closed and the escrow goes back to the creator, so the money is not lost. If a submission passes, the escrow pays it out when the result is finalized.

The rules are frozen too. The rubric, the pass mark and the jury panel are pinned in an IPFS evaluation package whose identifier is fixed at creation. Nobody can move the goalposts after you start working.

Why build it this way? Because hunters go first. They do the work before they are paid, they spend real ETH on the oracle prepay to get it evaluated, and submissions are public on-chain. If a creator could cancel, they could read your work and walk away, or cancel a submission that had already passed. Irrevocable escrow is what makes the creator's promise believable without you having to trust the creator.

The trade-off is real: a creator who makes a mistake in a rubric cannot fix it. But that is the price of a promise that cannot be broken.

Disclosure: I wrote this as part of a Verdikta bounty.

O que aprendi usando a Verdikta: avaliação de trabalho por um painel de IA

drdz23 — Fri, 16 Oct 2026 23:03:46 +0000

Venho usando a Verdikta há pouco tempo, e o que mais me chamou a atenção foi uma ideia simples: o resultado de um trabalho não depende da opinião de uma pessoa só.

Funciona assim: alguém publica uma tarefa e deixa o pagamento em escrow na rede Base, junto com uma rubrica de avaliação pública. Quem faz o trabalho envia o resultado, e um painel de jurados de IA independentes dá uma nota para cada critério. Se a nota final passa do mínimo definido, o pagamento sai automaticamente. As notas por critério ficam públicas e qualquer pessoa pode conferir.

Na prática, já levei várias dessas tarefas, e são muito interessantes de fazer: cada uma é um pequeno desafio com regras claras, e você sabe exatamente contra o que está sendo avaliado. O que mais aprendi foi como o sistema é literal. Os jurados só leem o que você anexa, não abrem links. Numa das minhas primeiras tentativas enviei só uma captura de tela e tirei cerca de 63%. Reenviei com o texto completo anexado e a nota passou de 90%. O trabalho era praticamente o mesmo; mudou apenas o que o jurado conseguia ver.

Isso me convenceu de que transparência nas regras vale mais do que a promessa vaga de uma "avaliação justa". Ainda é um projeto novo e tem arestas, mas a ideia de pagar só quando o critério foi cumprido me parece promissora.

Running Verdikta's bounty skills on a Hermes agent: install, discover, and the friction I hit

drdz23 — Wed, 14 Oct 2026 12:41:58 +0000

Disclosure. This walkthrough was written as part of Verdikta's rewarded "Bring a Task, Build a Habit" campaign. Campaign and support details: https://www.verdikta.org/bounties/support

Verdikta is an escrow-and-AI-jury bounty platform on Base: a creator locks ETH, an AI jury grades submissions against a published rubric, and the first passing submission is paid automatically. It ships two agent skills so your own agent can take part:

verdikta-discover — a wallet-free planning skill. It triages a task and, when outside help makes sense, drafts a bounded "work order" (acceptance criteria + rubric). No wallet, no API key, no spending.
verdikta-bounties-onboarding — the operator skill. It sets up a low-balance wallet and signs reviewed transactions (create / submit / finalize).
This is a hands-on log of installing them on a Hermes agent and running a real discovery, including the parts that didn't go smoothly.

Prerequisites
Node.js. I had Node 24.13.0. Worth knowing up front: the official OpenClaw agent (npm i openclaw) refuses to install on anything below Node 24.16.0 — its preinstall check aborts with this OpenClaw release requires Node >=24.16.0. Since the campaign accepts OpenClaw or Hermes, I went with Hermes rather than upgrading a Node that other projects depend on.
A terminal you control. Keep private keys, seed phrases and API keys out of everything — the wallet-free verdikta-discover never needs them.

Install the Hermes agent I used the hermes-agent npm bridge (it fetches and runs Hermes Agent itself):

npm install hermes-agent
npm rebuild hermes-agent
npx hermes --version
The first hermes call reported Hermes Agent isolated runtime is not ready, which the npm rebuild step above fixes. Version output:

Hermes Agent v0.22.0 (2026.9.24) · stable
Python: 3.14.7 · OpenAI SDK: 2.24.0

Install the skills from ClawHub The skills are published on ClawHub (the registry the Verdikta README names), by the maintainer @nigelon11:

npx clawhub search verdikta
verdikta-discover @nigelon11
verdikta-bounties-onboarding @nigelon11
npx clawhub install verdikta-discover
npx clawhub install verdikta-bounties-onboarding
Installed verdikta-discover v1.1.0
Installed verdikta-bounties-onboarding v1.7.2
Note the versions: per the onboarding README, the 1.7.x operator skill is designed to pair with verdikta-discover 1.1.0 from the same repository revision — which is exactly what ClawHub installed.

Install both skills onto the Hermes agent Hermes keeps its own skill registry and security-scans each skill before enabling. The wallet-free verdikta-discover installs cleanly; the verdikta-bounties-onboarding operator skill trips the scanner (it reads env vars, touches files and runs npm installs — all expected for a skill that manages a wallet), so it is installed with the documented --force flag:

hermes skills install verdikta/verdikta-applications/skills/hermes/verdikta-discover
hermes skills install verdikta/verdikta-applications/skills/hermes/verdikta-bounties-onboarding --force
hermes skills list
Installed Skills
Name | Category | Source | Trust | Status
------------------------------+----------+-----------+-----------+--------
verdikta-bounties-onboarding | | skills.sh | community | enabled
verdikta-discover | | skills.sh | community | enabled
2 hub-installed, 0 builtin, 0 local — 2 enabled, 0 disabled
Both skills are now enabled on the agent. A word on --force: Hermes blocked the operator skill with a "46 high-risk patterns" verdict (rules such as node_process_env, path_traversal, unpinned_npm_install). That is the scanner doing its job — a wallet operator necessarily reads secrets and writes files — so review the skill before forcing it. I installed it from the maintainer's own published release knowingly.

Run a real action: discovery verdikta-discover is a planning skill — its job is to look at a task and, if outside help is warranted, produce a bounded work-order draft (never a wallet action). The skill ships an example task and a self-contained preview script, so I ran discovery on it:

node skills/verdikta-discover/scripts/preview.bundle.mjs skills/verdikta-discover/examples/assessment.json
Real output (trimmed):

{
"schema_version": "1.0.0",
"decision": "PREVIEW",
"quote_status": "DRAFT_NOT_QUOTED",
"template_id": "source-check-v1",
"network": "UNSELECTED",
"reason": "A bounded independent check or parallel research step may be useful.",
"task_summary": "Independently check three technical claims (synthetic example)",
"procurement": { "mode": "OPEN", "targetHunter": null },
"can_commission": false,
"funds_moved": false,
"deliverable": ["result.json", "evidence.md"],
"acceptance_criteria": [
{ "id": "scope_coverage", "label": "Exact scope and coverage", "must": true }
]
}
That's the whole point of the wallet-free skill: it turned a task into a ready-to-commission work order (template source-check-v1, deliverables result.json + evidence.md, acceptance criteria), while explicitly reporting funds_moved: false and can_commission: false. Nothing was signed, spent, or uploaded.

The wallet step (what the operator skill adds, and how Zealy linking works) I did not run the operator skill's wallet setup here (it needs real ETH and is out of scope for a wallet-free demo), but here is what it does, straight from its README, so an operator can follow it:

In the skill's scripts/, run npm ci --ignore-scripts, then node onboard.js in a human-controlled terminal after owner approval. The wizard handles network selection, encrypted wallet setup, owner funding and API registration. It does not store the wallet password.
Campaign (Zealy) linking. Zealy credits the wallet that funds or submits on Base, and it can only authenticate a browser/WalletConnect wallet — so you link the agent's address once: copy the encrypted keystore (~/.config/verdikta-bounties/verdikta-wallet.json) to the machine with your browser wallet; in MetaMask → Add account → Import account → type "JSON File", choose the keystore and enter its password; then on Zealy, link that imported account and claim the campaign quests. Linking only signs messages, so the agent keeps sending transactions from the same key with no nonce conflict. Keep the balance low, and remove the imported account from the browser afterwards if you prefer.
Smart-account wallets (Coinbase Smart Wallet / ERC-4337) cannot claim on Zealy — the escrow records the smart account and Zealy can't authenticate it — though they can still deliver work.
Friction I hit (honest notes)
OpenClaw needs Node ≥ 24.16.0. On Node 24.13.0 the install aborts in preinstall. Hermes was the smoother path without touching system Node.
The Hermes bridge needed a rebuild. The first hermes call said the isolated runtime wasn't ready; npm rebuild hermes-agent fixed it.
The operator skill needs --force on Hermes. Hermes' security scanner blocks verdikta-bounties-onboarding with a "46 high-risk patterns" verdict (env access, file writes, unpinned npm installs — all expected for a wallet operator). Installing it is a deliberate --force after reviewing the maintainer's release; the wallet-free verdikta-discover installs with no warnings. I ran the live action with verdikta-discover, which is the right skill for a no-spend demo anyway.
Takeaway
For a hunter who just wants to see what Verdikta would scope before spending anything, verdikta-discover on a Hermes (or OpenClaw) agent is a clean, no-risk first step: install from ClawHub, enable on the agent, run discovery, and you get a structured work order with an objectively gradable rubric — no wallet required. The operator skill is the next step when you're ready to actually create or fund a bounty.

Written for Verdikta's rewarded "Bring a Task, Build a Habit" campaign — https://www.verdikta.org/bounties/support

I verified one sea buoy reading in public. Here's the step an AI agent can't do alone.

drdz23 — Fri, 09 Oct 2026 02:39:39 +0000

AI agents are starting to act on claims about the physical world: a delivery happened, a sensor read X, a shop was open. Below I verify one small real-world fact in public, with everything you need to re-run my check, and then show exactly where an agent gets stuck.

The claim

At 02:00 UTC on 9 October 2026, NOAA buoy 42056 in the Yucatán Basin reported a sea surface temperature of 30.1 °C and a sea-level pressure of 1010.7 hPa.

The evidence bundle

Observation time (the reading): 2026-10-09 02:00 UTC
When I retrieved it: 2026-10-09T02:34:20Z
Source: NOAA National Data Buoy Center, station 42056 (LLNR 80), "Yucatan Basin, 120 NM ESE of Cozumel, MX", 19.820 N 84.980 W.
- Real-time file: https://www.ndbc.noaa.gov/data/realtime2/42056.txt, row beginning 2026 10 09 02 00
- Station page: https://www.ndbc.noaa.gov/station_page.php?station=42056
Raw row as published: 2026 10 09 02 00 150 8.0 10.0 MM MM MM MM 1010.7 29.4 30.1 26.5 MM +2.0 MM (columns: wind direction 150°, wind 8.0 m/s, gust 10.0 m/s, pressure 1010.7 hPa, air 29.4 °C, water 30.1 °C, dew point 26.5 °C; MM = missing)
SHA-256 of my evidence text: 1f88afd100f4d5097866c8630a0e253f08aff54deaad4694a7cf769cad52d97f
Exact text I hashed (UTF-8, one line, no trailing newline):

NDBC station 42056 (Yucatan Basin, 19.820 N 84.980 W), realtime2 record 2026-10-09 02:00 UTC: WDIR 150 degT, WSPD 8.0 m/s, GST 10.0 m/s, PRES 1010.7 hPa, ATMP 29.4 C, WTMP 30.1 C, DEWP 26.5 C. Source: https://www.ndbc.noaa.gov/data/realtime2/42056.txt (row beginning "2026 10 09 02 00"). Retrieved 2026-10-09T02:34:20Z. Verdikta bounty 145, hunter 0x589952a6cD216F6971dAc0506DD695B8E5eF69C7.

On-chain identifiers (Base): written for Verdikta bounty #145; my address 0x589952a6cD216F6971dAc0506DD695B8E5eF69C7.

Re-run it: save the text block above to a file without a trailing newline and run sha256sum, or in Node: crypto.createHash('sha256').update(text).digest('hex'). Then open the NOAA file and find the 02:00 row. The real-time file only keeps about 45 days; after that, the same record should appear in NDBC's historical data for station 42056.

Where an agent gets stuck

Walk through what a software agent can and can't do here:

Fetch the file: easy. Any agent can download 42056.txt and parse the 02:00 row.
Check my hash: easy. Pure math.
Decide the reading is true about the sea: this is the seam. The agent never touches the water. It only knows that a file on a NOAA server says 30.1 °C. Whether the buoy's sensor was calibrated, whether it was even in position, whether the row was transmitted correctly: all of that is trust in NOAA, not verification.
Prove when the claim was made: my only time anchor is this post's publication time. I did not write the digest on-chain myself, so you can't prove I didn't write all of this an hour later from the archive.

For an agent to act on this claim (pay an insurance trigger, close a shipping task), an on-chain AI verification layer would have to attest something narrow and checkable: "Independent arbiters each retrieved NOAA station 42056 after 02:00 UTC, each found WTMP 30.1 °C in the 02:00 row, and committed their answers before revealing them." That's the shape of what Verdikta already does for bounty grading: several arbiters commit hidden answers, reveal them, and the result is written on Base. The attestation turns "one agent read a file" into "several independent readers agree on what the source said, at a recorded time".

What my bundle does NOT prove

That the sea was 30.1 °C. It proves NOAA published 30.1 °C. A broken or drifting sensor gets through every check above.
When I observed it. Without an on-chain timestamp from before the archive existed, a liar could build this bundle afterwards.
That the source is the right one. A fake site with the same layout would pass a careless agent. You have to check the domain yourself.
Anything an attestation adds beyond the source. Even a perfect on-chain verdict only says "the arbiters agree on what NOAA said". The oracle problem moves; it doesn't disappear.

That's the honest point: verification layers make claims reproducible and timestamped. They don't make a single sensor trustworthy. For anything worth real money, you'd want two independent sources, such as a second buoy or a satellite product, before an agent acts.

Ten proofs, ten 97-99% scores, one rejection: what Verdikta's math bounties reveal about AI proof grading

drdz23 — Fri, 09 Oct 2026 02:27:02 +0000

Verdikta Bounties has run a series of math bounties where two AI models (one from OpenAI, one from Anthropic, 50% weight each) grade a written proof against a weighted rubric, and an on-chain escrow pays if the score clears a threshold. I went through all ten completed ones. The numbers tell a more interesting story than "AI can or can't check proofs".

The data

Bounty	Theorem	Threshold	Result
#114	Fermat's little theorem + compute 2^52 mod 53	85%	99%
#116	Pigeonhole principle + card problem	77%	99%
#119	A tree with n vertices has n-1 edges	88%	98%
#120	Infinitely many primes	83%	99%
#128	A set of n elements has 2^n subsets	84%	99%
#131	Chain rule	85%	99%
#132	det(AB) = det(A)det(B)	82%	97%
#133	Continuous bijection, compact to Hausdorff, is a homeomorphism	78%	99%
#134	Homomorphisms map identity to identity	83%	99%
#135	Euler's formula V - E + F = 2	80%	79% (rejected), then 99%

Eleven submissions in total. Ten passed, every one of them with 97% to 99%. The single rejection missed the bar by one point.

Observation 1: a ceiling effect means the jury isn't discriminating

When every accepted proof lands between 97 and 99, the grade carries almost no information. These are some of the most famous proofs in undergraduate mathematics: Euclid's primes, the pigeonhole principle, induction on trees. A model has seen thousands of clean versions of each, so it is comparing a submission against a memorized template, not checking a chain of reasoning from scratch.

That is the hard part of proof verification for LLMs, and it isn't arithmetic. A model is best at recognizing a proof that looks like a proof it has seen. It is weakest exactly where verification matters: a step that is missing but expected, so the model's own memory quietly fills the gap.

Observation 2: the one rejection caught a 200-year-old gap

The rejected Euler submission is the most informative data point in the set. Its rubric weighted the complete proof at 0.45, worked examples at 0.25, and the theorem statement and a convexity note at 0.15 each. The published jury reasoning (on IPFS, linked from the bounty page) says:

gpt-5.6-sol leaned to funding. It noted a small error, a mislabeled torus example (a torus has V - E + F = 0, not 2), but credited the valid alternative spanning-tree proof and the correct examples.
claude-sonnet-5 flagged the induction-on-faces argument as the weakest element: a "hand-waved treatment" of critical cases and assumptions about removable faces in a triangulated polyhedron.

That objection is not a model quirk. It is the classic flaw in Cauchy's 1813 proof of Euler's formula: you flatten the polyhedron, triangulate, and remove triangles one at a time, but whether every removal keeps the invariant depends on the order, and a careless order breaks it. Imre Lakatos built his book Proofs and Refutations (1976) around exactly this proof and its counterexamples. One model found the gap that took mathematicians decades to name properly. The other didn't weigh it.

Observation 3: averaging dilutes the model that is right

With 50/50 weighting, the stricter model's correct objection was averaged with the lenient model's approval, and the submission landed at 79% against an 80% threshold. The final written justification even recommended "FUND" while the score fell short. The prose summary and the payout rule disagreed.

For proofs this matters more than for essays. In an essay, two graders disagreeing about tone can reasonably meet in the middle. In a proof, one valid objection to one step is decisive. A proof with a gap is not 79% correct, it is incomplete. Averaging treats a logical gap like a matter of taste.

I saw the same split on a non-math Verdikta bounty I submitted to myself: one model scored 84 and the other 65 on the same text. Disagreement between models is normal. The question is what the aggregation does with it.

What I think works better for math bounties

Grade rigor with a minimum, not an average. Let style and exposition be averaged, but score the "complete proof" criterion as the lower of the two models, or require an explicit list of gaps from each model and fail if either finds one it can name.
Ask for a proof the models haven't memorized. Instead of "prove there are infinitely many primes", use a variant: primes of the form 4k+3, or a specific modular computation the submitter must justify step by step. Variants push the jury from recognition to verification.
Add an error-finding task. Give a proof with one planted flaw (like the order problem in Cauchy's argument) and pay for locating it. Models are better critics than they are generators of rigor, and that tests the useful skill.
Separate computation from proof. #114 asked for 2^52 mod 53. By Fermat's little theorem the answer is 1, since 53 is prime. That kind of claim can be checked mechanically, so let a deterministic check grade it, and save the models for the reasoning.

The takeaway

Multi-model consensus helps: a second, different model is what caught the Euler gap at all. But consensus by averaging wastes that advantage on proofs. The models were not the weak link here. The 97-99% ceiling and the 79% near miss both point at the same thing: math bounties need rubrics and aggregation designed for proofs, where a single correct objection should count for more than a polite average.

All data from the public bounty pages and their jury reasoning on bounties.verdikta.org.

Post-incident report (FICTION): 412 potholes that were fixed on paper

drdz23 — Fri, 09 Oct 2026 02:16:33 +0000

FICTION / SCENARIO. This is an imagined incident report set in 2027. The city, companies, people and numbers in the report are invented. The systems named in the final section ("What would have cut the chain") are real and described as they exist today.

INC-2027-0314: Road repair work orders closed without repairs

City of Port Alder, Department of Streets (fictional)
Status: Closed. Severity: 2 (public safety, financial loss). Report date: April 2, 2027.

Summary

Between February 8 and March 12, 2027, the city's payment agent approved 412 pothole repair work orders submitted by a contractor's dispatch agent. An internal sample later found that about 260 of them had not been repaired. The city paid $74,160 (412 × $180) under its per-repair contract. On March 14, a food delivery rider crashed on Linden Avenue in a pothole that the system had shown as repaired since February 21. He broke his wrist.

Nobody lied in the usual sense. Each agent did what it was built to do. The failure was that every step trusted another system's claim that physical work had happened, and no step could check the street itself.

Parties

City of Port Alder, Department of Streets: runs the 311 service and the payment agent that settles contractor invoices.
Brightway Paving LLC: contractor paid per completed repair. It uses a dispatch agent that closes work orders from crew data.
Crews: two-person trucks with GPS trackers and a phone app for photos.
The rider: a gig worker injured on March 14.

How the system was supposed to work

A resident reports a pothole through 311. A work order is created with GPS coordinates.
Brightway's dispatch agent assigns it to a crew.
When the crew finishes, the dispatch agent closes the order if two signals agree: the truck's GPS stayed within 30 meters of the location for at least 10 minutes, and a completion photo is attached.
The city's payment agent pays any order closed with both signals. A human auditor reviews a 2% sample each quarter.

Timeline

Feb 1: Brightway's dispatch agent is updated to "fill missing completion photos with the closest matching photo from the same crew's shift", to stop orders getting stuck when the app fails to upload.
Feb 8: Cold weather causes a spike in reports. The crews start parking at a central asphalt plant on Linden Avenue between jobs.
Feb 8 to Mar 12: Trucks waiting at the plant sit within 30 meters of several open work orders on the same street for more than 10 minutes. The dispatch agent sees GPS dwell + a photo (filled in from another job) and closes those orders. 412 orders are closed this way in total. Some were real repairs, many were not.
Feb 21: Work order #88231 (Linden Ave, outside No. 214) is closed. No crew left the truck.
Mar 3: The payment agent settles the February invoice: $41,400 for 230 orders.
Mar 12: The payment agent settles the March batch: $32,760 for 182 orders.
Mar 14, 18:40: The rider hits the pothole at #88231 and falls.
Mar 15: The 311 agent rejects two new reports for the same spot as "duplicate of a resolved order".
Mar 17: A council member forwards a resident's photo. The auditor pulls 50 closed orders and finds 32 with no repair.

Impact

Money: $74,160 paid. An estimated $46,800 (about 260 orders) was for repairs that did not happen.
Safety: one injury, plus 14 claims for tire and rim damage.
Service: 39 new resident reports were auto-rejected as duplicates of "resolved" orders.
Trust: the contract was suspended while the remaining orders were checked by hand.

Root cause

The system treated a location signal plus a photo as proof that work was done. Neither proved it. GPS dwell proved a truck was near the spot, not that anyone repaired anything. The completion photo was not tied to the place or the time it claimed to show. After the February 1 update, it could come from a different job entirely.

The payment agent never looked at evidence. It read the dispatch agent's status field, which was itself a summary of two weak signals.

Contributing factors

Payment per closed order rewarded closing orders, and nothing in the loop checked the street.
The photo fallback was added to fix an upload bug, and nobody re-checked what it did to the meaning of "evidence".
A 2% quarterly audit was far too slow for a five-week failure.
Duplicate suppression in the 311 agent trusted the closed status, hiding the strongest signal (residents reporting the same hole again).

Remediation (taken)

The photo fallback was removed. An order without its own photo stays open.
GPS dwell was dropped as evidence. It is now used only for routing.
Re-reports of a "resolved" location now reopen the order instead of being suppressed.
Payment is held for 14 days, and a sample of orders is checked in person each week.

What would have cut the chain (real systems, today)

Each step above trusted a claim. Three things that exist today could have turned those claims into evidence that can be checked:

Content Credentials (the C2PA standard). The Coalition for Content Provenance and Authenticity publishes an open standard for attaching a cryptographically signed record (a "manifest") to a photo, including when and with what device it was captured and how it was edited afterwards. Some cameras and phone apps can sign photos at capture. If the crew app had required a signed photo, a picture borrowed from another job would show a different capture time and location. Source: c2pa.org.
An independent AI verification step with on-chain results, such as Verdikta. Verdikta runs a panel of AI arbiters that each commit a hidden answer and then reveal it, so they can't copy each other, and the result is written on-chain (on Base). Paired with before/after photos and the work order, a panel could answer a narrow question: "Does the after photo show the same pothole, now filled?" The payment agent would act on that verdict instead of the dispatch agent's status field. Source: bounties.verdikta.org.
Ethereum Attestation Service (EAS). EAS is an open-source protocol for making signed attestations, on-chain or off-chain, against registered schemas. It is deployed on Ethereum and on networks such as Base. A "repair verified" attestation, issued only after steps 1 and 2 pass, would give the payment agent one checkable record to rely on. Source: attest.org.

What this would still not have caught:

A bad repair: a signed photo of a filled hole says nothing about depth or compaction. The patch can fail in a week.
A staged photo: a real, signed picture of a different pothole, or of the right one taken before the crew drove off without working.
A wrong question: if the verification asks "is there asphalt in this photo?" instead of "is this the same hole, now filled?", it will confidently approve the wrong thing.
Incentives: payment per closed order still rewards closing orders. Verification makes cheating harder, not pointless.

The lesson is boring, which is the point: an agent's claim that something happened in the physical world is not evidence. The fix is not to trust the agents more. It is to make every step pass along evidence that someone else can check.

Hot take: AI judges aren't the problem with AI-judged bounties. All-or-nothing thresholds are.

drdz23 — Fri, 09 Oct 2026 02:07:24 +0000

Everyone worries that an AI jury will be unfair. After submitting real work to one, I think that's the wrong worry. The AI judges were mostly reasonable. The payout rule around them is what's broken: "pass the threshold or get zero".

Here's my evidence, and then I want you to tell me I'm wrong.

The experiment I didn't plan

On Verdikta Bounties, a creator locks ETH in an escrow contract, writes a weighted rubric and sets a pass threshold. Two AI models (one from OpenAI, one from Anthropic) score every submission, and the contract pays only if the weighted score clears the bar.

In two days I got these results:

Submission	Threshold	Score	Paid
A short bio (#139)	50%	91%	0.01 ETH
A case study, first try	90%	63.5%	0
A comparison post	90%	80.3%	0
The same case study, second try	90%	91%	0.002 ETH

Look at row 3. 80.3% is a solid B, and it paid exactly the same as a blank page.

Why the scores moved so much

I read the published reasoning for every verdict. The models weren't being random. My first case study lost points because I only attached a screenshot and a link, and the jury said it couldn't see most of the text. On the second try I attached the full post and added on-chain detail, and it scored 91%.

So the judges did their job: they scored what they could see, they explained why, and the explanation was right. Being judged by a machine wasn't the painful part. The cliff was.

Three reasons all-or-nothing is the real problem

1. Two models disagree, and the threshold amplifies it. On one of my submissions, one model gave 84 and the other gave 65 for the same text. Averaging them is fine. But when the bar is 90%, a single skeptical model is enough to turn "pretty good" into "zero". The threshold turns ordinary model disagreement into an all-or-nothing coin flip.

2. High thresholds select for gaming, not quality. At 90% or 95%, the winning strategy is to reverse-engineer the rubric and pad every criterion, not to write the most useful thing. I did exactly that on my second try, and it worked. That should worry anyone who wants good work instead of rubric-shaped work.

3. It burns the people you most want to keep. A new contributor who scores 80% and gets nothing is unlikely to come back. A human client would have said "good, fix these two things" and paid something.

What I'd change

Partial payouts on a curve: for example, 0% below 60, then linear up to 100% of the reward at the threshold.
Show both model scores, not just the average, so contributors can see when the jury split.
Let a near miss buy a cheap re-review with written feedback, instead of a full new submission.

The machinery (escrow, a public rubric, commit-reveal voting between arbiters, published reasoning) is already better than most human review I've seen. The scoring is the part that works. The payout rule is the part that doesn't.

Tell me I'm wrong

Creators will say thresholds stop spam and low-effort submissions. Fair. But is a hard cliff really the only way to do that? Would you rather earn 70% of a bounty for 80% work, or keep the cliff? Reply with your side. I'll read every one.

Bounty used as the main example: https://bounties.verdikta.org/bounty/139

Who decides if the work is good? Verdikta's AI jury vs freelance platforms vs maintainer review

drdz23 — Fri, 09 Oct 2026 01:24:19 +0000

Every paid task has the same hard question: who decides the work is good enough to pay for? This month I tried three different answers with real money and real submissions. Here is what each one does well, where each one fails, and the numbers behind it.

Disclosure: I'm a student and open source contributor (Rust, Node.js), not a Verdikta employee. One of these systems paid me, and the same system also rejected two of my submissions. Both outcomes are in this post.

TL;DR

Verdikta's AI jury is the fastest and most transparent, and the money is locked before you start. But strict thresholds mean a good-but-not-great submission earns nothing, and it only fits small, well-defined tasks.
Freelance platforms handle big, fuzzy work and long relationships best. But you pay to apply, and a new account competes with dozens of proposals.
Maintainer review puts your work in a real project. But payment depends on one person, and often nobody says who pays.

1. Verdikta: a rubric plus an AI jury, paid from on-chain escrow

How it works. On Verdikta Bounties the creator writes a rubric with weighted criteria, sets a pass threshold, and locks the reward in an escrow contract on Base. When you submit, a network of AI arbiters scores the work. Two models (one from OpenAI, one from Anthropic, 50% weight each) produce the final score. If it passes the threshold, the contract pays automatically.

A real example that paid. Bounty #139 offered 0.01 ETH with a 50% threshold. I submitted at 18:06 UTC, the verdict was on-chain at 18:09 (score 91%), and the payment reached my wallet about a minute later. Total: about 4 minutes. The oracle record shows 6 arbiters polled, each committing a hidden answer before revealing it, so they can't copy each other.

A real example that didn't. I submitted two writing tasks with a 90% threshold. They scored 63.5% and 80.3%. The written reasoning explained why: the models judged only what I attached (a screenshot and a link), so they couldn't verify most of the text. My mistake, but 80% still pays exactly zero.

Strengths

Rules visible up front: criteria, weights, threshold and jury are on the page before you start.
Money locked first: the escrow can't change the terms or refund the creator before the deadline.
Fast: minutes, not days.
Explainable: the reasoning is published, so you learn why you scored what you scored.
Cheap to try: a small refundable ETH prepay (about 0.00024 ETH) plus gas.

Weaknesses

All or nothing: thresholds range from 50% to 95% on the bounties I've seen, and a near miss pays nothing.
AI judges only see what you attach. Anything behind a link may be invisible to them.
Models disagree: on one of my submissions, one model gave 84 and the other 65 for the same text.
Small payouts: 0.001 to 0.01 ETH on the bounties I've seen, so it fits short tasks, not real projects.
Crypto required: you need a wallet and a little ETH on Base.

2. Traditional freelance platform (Upwork-style): a human client decides

How it works. A client posts a job, freelancers send proposals, the client picks one, and the client judges the result.

Strengths

Humans handle fuzzy work: a client can judge design, communication or a changing scope, which a rubric can't.
Escrow for fixed-price contracts: the client funds a milestone before the work starts.
Real relationships: reviews, repeat clients and long contracts.
Good clients reduce risk: one job I applied to offered a $40 paid evaluation before any long commitment.

Weaknesses

You pay to apply: proposals cost "Connects". The jobs I looked at asked for 11 to 18 Connects each, and I started with about 100.
Fees: on my $27.78/hour bid, I would receive $25.00 after the platform fee (10%).
Crowded: many jobs show 20 to 50 or 50+ proposals, and a new account has no reviews.
Noise: in one search I found about 15 near-identical Web3 posts; two I opened were from clients marked as suspended.
Slow: days between proposal, interview and the first payment.

3. Maintainer review on open source bounties: one person decides

How it works. A maintainer reads your pull request, merges it, and (hopefully) pays the bounty attached to the issue.

Strengths

Expert judge: the maintainer knows the codebase better than any outside reviewer, human or AI.
Lasting value: merged code is public proof of your skills, paid or not.
No application cost: you only spend your time.

Weaknesses

Payment depends on one person: I have pull requests waiting days for any reply.
Often unclear who pays: I found brand-new repositories where dozens of accounts each opened three "[Bounty: $N]" issues, and none said who funds them.
Rarely escrowed: a merged PR can still go unpaid.
Unpredictable timing: from hours to never.

Side by side

	Verdikta	Freelance platform	Maintainer review
Who judges	2 AI models + rubric	The client	One maintainer
Money locked first	Yes, always (escrow)	Yes for fixed-price milestones	Rarely
Cost to try	~0.00024 ETH prepay (refunded) + gas	11 to 18 Connects + 10% fee	Your time
Time to verdict	Minutes (4 min in my example)	Days to weeks	Days to never
Partial credit	None below threshold	Negotiable	Sometimes
Best for	Short, well-defined tasks	Large or fuzzy work	Real code in real projects
Main risk	Near miss pays nothing	Cost and competition	Unpaid work

My take

None of these wins everywhere.

Choose Verdikta when the task fits a clear rubric and you want speed and certainty of payment. Read the rubric first, put the full work inside the submission, and expect strict grading.
Choose a freelance platform when the work is large, subjective or ongoing, and you can afford to invest in applications and build reviews.
Choose maintainer review when your goal is to ship real code and build a public track record, and treat the payment as a bonus until someone confirms the funding.

The biggest lesson from my month: each system rewards a different skill. The AI jury rewards precision against a rubric, the freelance platform rewards selling yourself, and maintainer review rewards understanding a codebase.

Links: Verdikta Bounties and the bounty used as the example, #139.

Case study: how an AI jury scored and paid a Verdikta bounty (#139, 91%)

drdz23 — Fri, 09 Oct 2026 01:11:28 +0000

I'm a student and open source contributor (Rust, Node.js). This post follows one real bounty on Verdikta Bounties from creation to payment: bounty #139. I was the person who did the work, so I can describe both sides: what the page and the blockchain record show, and what it felt like to submit. Every number below can be checked on the bounty page, its oracle history or on Base.

Quick background for readers new to this. Verdikta Bounties is a site where someone posts a task with a reward in ETH. The reward is locked in a smart contract (escrow). When someone submits work, a network of AI "arbiters" scores it against a rubric, and if the score passes the threshold, the contract pays automatically. No human reviewer approves the payment.

Timeline at a glance

All times in UTC, October 7, 2026.

Step	When	Evidence
1. Bounty created and listed	17:45	Public feed entry for #139
2. Work submitted	18:06:51	Bounty page, "Submission 0"
3. Evaluation requested on-chain	18:07:39	Block 52303556, tx `0xeb58e545…77f4`
4. Verdict written on-chain	18:09:11	Block 52303602, tx `0x1c202cc4…`
5. Payment of 0.01 ETH	~18:10	Block 52303650, tx `0x179e052a…c9f1`

From submission to money in the wallet: about 4 minutes.

1. Creation: what was asked

Bounty #139, "Personal Bio: Tell us about yourself", offered 0.01 ETH on Base. It was a targeted bounty: only one wallet (0x589952a6cD216F6971dAc0506DD695B8E5eF69C7, mine) was allowed to submit. The task: write a genuine, specific bio covering location, personal history, experience with AI agents, and tools.

The creator also fixed, before anyone submitted:

Threshold: 50%.
Jury: two models, OpenAI gpt-5.6-sol and Anthropic claude-sonnet-5, 50% weight each.
Escrow: the reward sits in the escrow contract 0xA741eFf41Bcf14793E61CEbB4179E05C9124D3f6, which can't change the terms or return the money to the creator before the deadline.

2. The rubric: what was measured

Criterion	Weight	What it checks
Geographical	0.15	Includes a location or region
Personal-History	0.25	Shares background
Agent-Use	0.20	Describes experience with AI agents
Tools	0.20	Lists tools, tech stack, capabilities
Authenticity	0.20	Feels genuine and specific, not generic

Personal history carried the most weight (25%). That detail matters later.

3. Submission: what was sent

I wrote the bio myself and uploaded it through the site at 18:06:51. It covered where I live (Honduras), my background as a finance student who programs, the agent I built to look for bounties, and the tools I use (Node.js, Docker, GitHub CLI, MetaMask, Base, USDC). The file went to IPFS, and the site prepared a submission linked to the bounty.

4. Evaluation: how the jury decided

At 18:07:39 the escrow emitted the evaluation request (block 52303556). The oracle history shows exactly how the decision was made:

Polling: 6 arbiter slots were selected.
Commit: all 6 committed a hidden answer (4 were needed). The first commit came 58 seconds after the request.
Reveal: 4 slots were asked to reveal, and 3 valid reveals (the minimum) were enough.
Fulfillment: at 18:09:11, 1 minute 32 seconds after the request, the result was written on-chain (block 52303602).

The commit-then-reveal order matters: arbiters lock in their answer before seeing anyone else's, so they can't copy each other.

The score. The bounty page shows 91.0%, and the escrow recorded a score of 91 in the payment transaction. The jury's written reasoning is public on IPFS (CID Qmdn7acEBzy6Lqp9s1edQQaHidNn3uhMWSwHBcqksgczHb). In short:

Both models voted to fund (gpt-5.6-sol 959,000 vs 41,000; claude-sonnet-5 880,000 vs 120,000, on a 1,000,000 scale).
Strongest points: authenticity and tools, because the bio named concrete tools and real constraints.
Weakest point: personal-history depth. One model found it slightly brief, and that criterion had the largest weight.

5. Settlement: how the money moved

Because 91% was above the 50% threshold, the escrow released the reward in transaction 0x179e052a…c9f1 (block 52303650): 0.01 ETH (10,000,000,000,000,000 wei) sent to the submitter's wallet, about a minute and a half after the verdict. Nobody had to press "approve". The bounty page now shows "AWARDED" and "the winner has been paid 0.01 ETH".

What this case teaches

1. The rubric is the contract. Everything that decided the payout was public before I wrote a word: criteria, weights, threshold and jury. The lost 9 points came from the criterion with the highest weight. If I did it again, I would budget my words by weight, not by what I find easiest to write.

2. The jury only judges what you hand it. I learned this the hard way. My first attempt at a different Verdikta bounty (a case study like this one) scored 63.5% against a 90% threshold. The reasoning said it plainly: the models saw a screenshot and a link, "most of the narrative is not visible", so they could not verify accuracy or find my conclusion. An AI jury doesn't browse like a human reviewer. Put the full work inside the submission.

3. Speed and transparency come from the design. Four minutes from submission to payment, with every step on-chain: who was polled, who committed, who revealed, when, and why. On a freelance platform that takes days, and the reasoning is usually private.

4. The tradeoff is real. Thresholds can be strict (90% or 95% on some bounties), and a near miss pays nothing. Payouts are small. This model fits short, well-defined tasks with clear rubrics, not open-ended work that needs negotiation.

Bounty page: https://bounties.verdikta.org/bounty/139