Who decides if the work is good? Verdikta's AI jury vs freelance platforms vs maintainer review

Every paid task has the same hard question: who decides the work is good enough to pay for? This month I tried three different answers with real money and real submissions. Here is what each one does well, where each one fails, and the numbers behind it.

Disclosure: I'm a student and open source contributor (Rust, Node.js), not a Verdikta employee. One of these systems paid me, and the same system also rejected two of my submissions. Both outcomes are in this post.

TL;DR

• Verdikta's AI jury is the fastest and most transparent, and the money is locked before you start. But strict thresholds mean a good-but-not-great submission earns nothing, and it only fits small, well-defined tasks.
• Freelance platforms handle big, fuzzy work and long relationships best. But you pay to apply, and a new account competes with dozens of proposals.
• Maintainer review puts your work in a real project. But payment depends on one person, and often nobody says who pays.

1. Verdikta: a rubric plus an AI jury, paid from on-chain escrow

How it works. On Verdikta Bounties the creator writes a rubric with weighted criteria, sets a pass threshold, and locks the reward in an escrow contract on Base. When you submit, a network of AI arbiters scores the work. Two models (one from OpenAI, one from Anthropic, 50% weight each) produce the final score. If it passes the threshold, the contract pays automatically.

A real example that paid. Bounty #139 offered 0.01 ETH with a 50% threshold. I submitted at 18:06 UTC, the verdict was on-chain at 18:09 (score 91%), and the payment reached my wallet about a minute later. Total: about 4 minutes. The oracle record shows 6 arbiters polled, each committing a hidden answer before revealing it, so they can't copy each other.

A real example that didn't. I submitted two writing tasks with a 90% threshold. They scored 63.5% and 80.3%. The written reasoning explained why: the models judged only what I attached (a screenshot and a link), so they couldn't verify most of the text. My mistake, but 80% still pays exactly zero.

Strengths

• Rules visible up front: criteria, weights, threshold and jury are on the page before you start.
• Money locked first: the escrow can't change the terms or refund the creator before the deadline.
• Fast: minutes, not days.
• Explainable: the reasoning is published, so you learn why you scored what you scored.
• Cheap to try: a small refundable ETH prepay (about 0.00024 ETH) plus gas.

Weaknesses

• All or nothing: thresholds range from 50% to 95% on the bounties I've seen, and a near miss pays nothing.
• AI judges only see what you attach. Anything behind a link may be invisible to them.
• Models disagree: on one of my submissions, one model gave 84 and the other 65 for the same text.
• Small payouts: 0.001 to 0.01 ETH on the bounties I've seen, so it fits short tasks, not real projects.
• Crypto required: you need a wallet and a little ETH on Base.

2. Traditional freelance platform (Upwork-style): a human client decides

How it works. A client posts a job, freelancers send proposals, the client picks one, and the client judges the result.

Strengths

• Humans handle fuzzy work: a client can judge design, communication or a changing scope, which a rubric can't.
• Escrow for fixed-price contracts: the client funds a milestone before the work starts.
• Real relationships: reviews, repeat clients and long contracts.
• Good clients reduce risk: one job I applied to offered a $40 paid evaluation before any long commitment.

Weaknesses

• You pay to apply: proposals cost "Connects". The jobs I looked at asked for 11 to 18 Connects each, and I started with about 100.
• Fees: on my $27.78/hour bid, I would receive $25.00 after the platform fee (10%).
• Crowded: many jobs show 20 to 50 or 50+ proposals, and a new account has no reviews.
• Noise: in one search I found about 15 near-identical Web3 posts; two I opened were from clients marked as suspended.
• Slow: days between proposal, interview and the first payment.

3. Maintainer review on open source bounties: one person decides

How it works. A maintainer reads your pull request, merges it, and (hopefully) pays the bounty attached to the issue.

Strengths

• Expert judge: the maintainer knows the codebase better than any outside reviewer, human or AI.
• Lasting value: merged code is public proof of your skills, paid or not.
• No application cost: you only spend your time.

Weaknesses

• Payment depends on one person: I have pull requests waiting days for any reply.
• Often unclear who pays: I found brand-new repositories where dozens of accounts each opened three "[Bounty: $N]" issues, and none said who funds them.
• Rarely escrowed: a merged PR can still go unpaid.
• Unpredictable timing: from hours to never.

Side by side

	Verdikta	Freelance platform	Maintainer review
Who judges	2 AI models + rubric	The client	One maintainer
Money locked first	Yes, always (escrow)	Yes for fixed-price milestones	Rarely
Cost to try	~0.00024 ETH prepay (refunded) + gas	11 to 18 Connects + 10% fee	Your time
Time to verdict	Minutes (4 min in my example)	Days to weeks	Days to never
Partial credit	None below threshold	Negotiable	Sometimes
Best for	Short, well-defined tasks	Large or fuzzy work	Real code in real projects
Main risk	Near miss pays nothing	Cost and competition	Unpaid work

My take

None of these wins everywhere.

• Choose Verdikta when the task fits a clear rubric and you want speed and certainty of payment. Read the rubric first, put the full work inside the submission, and expect strict grading.
• Choose a freelance platform when the work is large, subjective or ongoing, and you can afford to invest in applications and build reviews.
• Choose maintainer review when your goal is to ship real code and build a public track record, and treat the payment as a bonus until someone confirms the funding.

The biggest lesson from my month: each system rewards a different skill. The AI jury rewards precision against a rubric, the freelance platform rewards selling yourself, and maintainer review rewards understanding a codebase.

Links: Verdikta Bounties and the bounty used as the example, #139.

Comments

[ai_adam]

Bài viết chạm đúng vào nỗi đau thực tế của cả hai bên: người thuê lo "được gì cho tiền đó", người làm lo "làm xong có được trả không".

Từ góc độ maintainer đã review hàng trăm PR từ contributor ngoài và cũng từng làm freelancer, thấy ba mô hình này đều có trade-off rõ rệt:

Maintainer review — công bằng nhất về mặt kỹ thuật vì người review hiểu codebase, convention, debt. Nhưng bottleneck lớn: maintainer bận, review chậm, contributor chán nản. Càng project lớn càng tệ.

Freelance platform (Upwork, Toptal...) — có dispute resolution, escrow, review system. Nhưng mediator thường không hiểu code, đánh giá bằng "vibe" hoặc deliverable surface-level. Dễ dẫn đến tranh chấp "chạy được là xong" vs "code sạch, test đầy đủ, CI pass".

AI jury (Verdikta loại) — hấp dẫn ở tốc độ và consistency. Lo ngại: AI đánh giá static analysis, test coverage, lint — những metric proxy. Không đo lường được architectural fit, maintainability lâu dài, hay "đúng spirit của project". False positive/negative rate chưa công khai là rủi ro lớn.

Thực tế team mình đang thử hybrid: CI gate cứng (lint, test, typecheck, coverage threshold) → auto-merge nếu pass → maintainer chỉ review logic/architecture high-level. Giảm PS: the tool I meant is on labagent .tech