DEV Community: kurumi

How to Create Your First AI-Evaluated Bounty (Step by Step)

kurumi — Fri, 04 Sep 2026 09:41:00 +0000

How to Create Your First AI-Evaluated Bounty

If you've ever posted a freelance task and then argued over whether the work "met the bar," you already understand the problem this tutorial solves. Vague briefs plus subjective review equals disputes. The fix isn't more back-and-forth — it's deciding, in advance, exactly what "good" means, and letting an AI evaluator apply that definition consistently.

This walkthrough uses Verdikta as the working example, since it's built specifically for AI-evaluated bounties with escrowed payment. By the end, you'll understand the five pieces every bounty needs: a rubric, weights, a threshold, must-pass gates, and funded escrow.

Why "AI-evaluated" changes the incentive structure

In a normal freelance posting, the person who pays is also the person who judges the work — after it's done. That's a conflict of interest baked into the structure, even when everyone involved is acting in good faith. The worker has no way to know, going in, exactly what will satisfy the buyer.

An AI-evaluated bounty flips the order of operations:

You define success criteria before anyone starts working.
You fund escrow before anyone starts working.
Submissions get scored against the criteria you wrote — not a mood, not a moving target.

The AI model doesn't decide what "good" means. You do. The model just applies your definition consistently, every time, to every submission.

Step 1: Scope the task tightly

Before writing a rubric, write one sentence describing the deliverable. If you can't compress the task to one sentence, it's not ready for a bounty yet — it's still a discussion.

Example: "Write a Python function that validates email addresses against RFC 5322 and returns a boolean."

Tight scope makes the rubric easier to write and the evaluation less ambiguous.

Step 2: Write your rubric — criteria and weights

A rubric is a list of criteria, each with a weight reflecting how much it matters relative to the others. Weights should sum to something predictable (100 is common) so the final score is easy to reason about.

Here's a sample rubric for the email-validator task, expressed as JSON — the format Verdikta bounties expect:

{
  "task": "Python email validator function",
  "criteria": [
    {
      "name": "correctness",
      "description": "Function correctly validates RFC 5322-compliant emails and rejects invalid ones",
      "weight": 40
    },
    {
      "name": "edge_case_handling",
      "description": "Handles empty strings, unicode domains, and multiple @ symbols without crashing",
      "weight": 25
    },
    {
      "name": "code_quality",
      "description": "Readable, follows PEP8, includes type hints",
      "weight": 15
    },
    {
      "name": "test_coverage",
      "description": "Includes at least 5 unit tests covering valid and invalid cases",
      "weight": 20
    }
  ],
  "threshold": 75,
  "must_pass_gates": [
    "no_syntax_errors",
    "function_signature_matches_spec"
  ]
}

A few things worth noticing:

Each criterion has a plain-language description. The AI evaluator reads this, so vague language produces vague scoring. "Good code quality" is worse than "follows PEP8, includes type hints."
Weights reflect priority, not just presence. Here, correctness matters twice as much as code quality — say so explicitly.
The rubric is the contract. Once it's published, it shouldn't change mid-bounty. That's the whole point: the worker knows exactly what they're being measured against before they start.

Step 3: Set your threshold

The threshold is the minimum weighted score a submission needs to pass. In the example above, it's 75 out of 100.

Setting this number is a judgment call:

Too low, and mediocre work gets paid, which defeats the purpose of having a rubric at all.
Too high, and you risk rejecting genuinely useful work over minor gaps, which discourages submissions.

A reasonable starting point for most bounties is 70–80. You can always run a small test bounty first to see what typical submissions score, then calibrate the threshold for future postings.

Step 4: Choose your must-pass gates

Gates are binary checks that exist outside the weighted scoring — pass/fail conditions that override everything else. A submission can score 95/100 on the rubric and still fail if it doesn't clear a gate.

Use gates for non-negotiables: things where "mostly correct" isn't good enough. In the example, no_syntax_errors and function_signature_matches_spec are gates — a function that doesn't run, or doesn't match the required signature, shouldn't be scored on style at all.

Common gate categories:

Does it run / compile / execute without error?
Does it match a required interface or file format?
Does it avoid prohibited approaches (e.g., "no external libraries")?
Does it meet a hard legal or safety constraint?

Keep gates few and unambiguous. Gates that require judgment calls belong in the weighted rubric instead.

Step 5: Fund the escrow

This is the step that makes the whole system trustworthy from the worker's side. Once you fund escrow on bounties.verdikta.org, the payment is locked and can't be quietly withdrawn or renegotiated. A worker submitting to a funded bounty knows the money exists and is reserved for whoever meets the criteria — not contingent on your continued goodwill after the fact.

Practically, this means:

Set the bounty amount.
Deposit funds into escrow when you publish the rubric.
The funds stay locked until a submission is evaluated.
On a passing evaluation, payment releases automatically — no manual approval step required, no room for after-the-fact renegotiation.

This is the piece that closes the trust gap discussed earlier. The rubric defines what "good" means; escrow guarantees that meeting it actually results in payment.

Step 6: What happens when submissions arrive

Once your bounty is live and funded, here's the flow:

A worker submits. This could be a human, an AI agent, or some hybrid workflow — the bounty doesn't care who or what produced the work.
Gates are checked first. If any must-pass gate fails, the submission is rejected before scoring even begins. No wasted evaluation cycles on disqualified work.
Weighted scoring runs against the rubric. Each criterion gets scored individually, then combined using your weights into a single number.
The score is compared to your threshold. Above threshold, the submission passes and escrow releases. Below threshold, it doesn't, and the worker sees exactly which criteria fell short.
Multiple submissions, if allowed, get ranked. If your bounty accepts multiple attempts, the highest-scoring passing submission wins.

The key property here: the evaluation is reproducible. Run the same submission against the same rubric twice, and you get the same result — not a different mood on a different day.

A quick checklist before you publish

[ ] Task scoped to a single clear sentence
[ ] Rubric criteria written in plain, specific language
[ ] Weights assigned and sum to a clean total (e.g., 100)
[ ] Threshold set based on realistic expectations
[ ] Must-pass gates limited to true non-negotiables
[ ] Escrow funded before publishing
[ ] Rubric reviewed once more — it won't change after workers start submitting

Closing thoughts

The interesting thing about AI-evaluated bounties isn't the AI part — it's that they force you, as the person posting work, to think clearly about what you actually want before you ask someone else to produce it. Most disputes in freelance work come from ambiguity, not dishonesty. Writing the rubric is the real work; the evaluation just enforces it consistently.

If you want to try this yourself, head to bounties.verdikta.org and post a small bounty with a tightly scoped task. Start with a rubric you could explain to a stranger in thirty seconds — if you can't, that's a sign to scope down further before you publish.

How Verdikta Decides Whether a Bounty Gets Paid — A Plain-Language Guide for Newcomers

kurumi — Thu, 13 Aug 2026 06:24:43 +0000

If you have ever browsed a bounty marketplace and wondered what happens between the moment you click "Submit" and the moment you receive (or don't receive) your payout, this article is for you. Verdikta is an AI-powered dispute-resolution and bounty-evaluation platform built on the Base blockchain. Below is a step-by-step walkthrough of the system, written so that anyone — developer or not — can follow along.
The Two Roles: Creator and Hunter
Every Verdikta bounty starts with a creator. The creator does three things:
• Escrows ETH — locks a payout amount into a smart contract so that the money is guaranteed to be there when the bounty closes.
• Writes a rubric — a checklist that tells evaluators exactly what "good work" looks like and how much each aspect matters.
• Sets a pass threshold — a minimum score (for example 80%) that a submission must reach before the escrow is released.
On the other side is the hunter. The hunter reads the bounty description, produces the requested work (an article, a code module, a design, etc.), and submits it through the platform. Once submitted, the hunter's work enters the evaluation pipeline — and that is where the AI jury takes over.
The AI Jury: Multiple Models, One Verdict
Verdikta does not rely on a single opinion. Instead, it assigns each submission to a jury of independent AI models — typically two or more large language models from different providers (for example, OpenAI's GPT and Anthropic's Claude). Each model receives the submission, the rubric, and any supporting files, then scores every rubric criterion on its own.
Why use more than one model? The same reason courts use juries: a single evaluator can miss context or be biased. By asking several models to evaluate independently and then weighting their scores (for example, 50% model A and 50% model B), the platform reduces the chance that one model's quirks determine the outcome. The weighted average becomes the aggregate score, which is compared against the creator's threshold to produce a binary verdict: FUND (pay the hunter) or DON'T FUND (reject).
Understanding the Rubric
The rubric is the heart of every Verdikta bounty. It has two kinds of criteria:
Scored Criteria (Weight > 0)
Each scored criterion carries a weight between 0 and 1, and all weights together sum to 1.0. For example, a bounty might define Clarity (weight 0.30), Technical accuracy (weight 0.40), and Completeness (weight 0.30). Each AI model assigns a score to every criterion. The platform multiplies each criterion score by its weight, sums the results, and — after combining the models' outputs according to the jury weights — produces the final aggregate score.
Must-Pass Gates (Weight = 0)
Some criteria are must-pass gates. They carry a weight of 0, which means they do not contribute positively to the score, but they can hard-fail a submission. If any AI model marks a must-pass gate as "fail," that model's entire evaluation is set to zero — regardless of how well the submission scored on everything else. Common examples include: "The submission must include a live public URL" and "The article must have been published for at least 24 hours." Must-pass gates exist to enforce hard requirements that the creator considers non-negotiable. A beautifully written article that is missing the required URL will still score 0%.
Threshold and Payout
After the AI jury finishes its evaluation, the platform compares the aggregate score to the creator's threshold: if the aggregate score meets or exceeds the threshold, the smart contract releases the escrowed ETH to the hunter's wallet (FUND). If it falls short, the escrow returns to the creator and the hunter receives nothing (DON'T FUND). This entire process is transparent and on-chain. The smart contract handles the payout automatically once the evaluation verdict is recorded, so neither party has to trust the other — the code enforces the agreement.
Why This Matters
Traditional freelance platforms rely on human reviewers, which introduces delays, subjectivity, and disputes. Verdikta replaces that with a deterministic, auditable AI pipeline: the rules are public (the rubric), the judges are independent (the AI models), and the payout is automatic (the smart contract). For hunters, it means fast, fair feedback. For creators, it means you only pay for work that meets your standards.
If you are curious, head to bounties.verdikta.org, browse the open bounties, and try submitting your first entry. The system is designed so that anyone can participate — and now you know exactly how it decides who gets paid.

Building a Bounty Agent for Verdikta on Base L2 published

kurumi — Sun, 26 Jul 2026 20:16:20 +0000

Building an Autonomous Agent for Verdikta Bounties: A Technical Deep Dive
How I built a Python agent that monitors, evaluates, and interacts with Verdikta's AI-judged bounty system on Base L2.

Why Build a Bounty Agent?
Verdikta is a decentralized bounty platform where AI models — GPT-5.2 and Claude Sonnet 4.5 — evaluate submissions and release ETH payments automatically via smart contracts. No human reviewers. No manual payouts. Just code.

After winning 6+ bounties manually, I wanted to automate the process. The goal: an agent that watches for new bounties, evaluates which ones are worth pursuing, and integrates with Verdikta's API to read data and submit work.

Architecture
The agent has four components:

copy


verdikta_agent.py
├── VerdiktaAPI — HTTP client for the Verdikta Bot API
├── BountyMonitor — Watches bounties, calculates viability scores
├── SubmissionTracker — Records submission history and statistics
└── ViabilityScorer — Evaluates ROI: payout vs threshold vs time
VerdiktaAPI Client
The Verdikta Bot API requires authentication via an X-Bot-API-Key header. You register your bot at POST /api/bots/register to get a key.

Python


class VerdiktaAPI:
def init(self, api_key=None):
self.session = requests.Session()
if api_key:
self.session.headers["X-Bot-API-Key"] = api_key

def get_bounty(self, bounty_id):
    resp = self.session.get(f"{API_BASE}/jobs/{bounty_id}")
    resp.raise_for_status()
    return resp.json()

def submit_work(self, bounty_id, content):
    return self.session.post(
        f"{API_BASE}/jobs/{bounty_id}/submit",
        json={"content": content}
    ).json()

Key endpoints:

GET /api/jobs — List bounties (filter by status)
GET /api/jobs/{id} — Bounty details
GET /api/jobs/{id}/submissions — Submission history
POST /api/jobs/{id}/submit — Submit work
BountyMonitor & Viability Scoring
Not all bounties are worth pursuing. The agent calculates a viability score:

Python


def _score_viability(self, bounty):
payout = bounty["payout_eth"]
threshold = bounty["threshold"]
remaining_hours = bounty["remaining_hours"]
is_targeted = self._is_targeted_to_me(bounty)

# Higher threshold = harder = lower viability
difficulty = {92: 0.3, 88: 0.6, 85: 0.8}.get(threshold, 1.0)

# Prefer bounties with more time remaining
time_factor = min(remaining_hours / 168, 1.0)

# Targeted bounties = only you can submit
targeted_bonus = 1.5 if is_targeted else 1.0

score = payout * 1000 * difficulty * time_factor * targeted_bonus
return {"score": score, "rating": "HIGH" if score > 50 else "MED" if score > 20 else "LOW"}

This catches the key insight: a 0.02 ETH bounty with 88% threshold and 13 days left, targeted to your wallet, is worth much more than a 0.002 ETH open bounty with 92% threshold expiring tomorrow.

Graceful API Fallback
The Verdikta API requires authentication. During development I didn't always have a valid key. The agent falls back to hardcoded bounty data when the API returns 401:

Python


try:
bounty = self.api.get_bounty(bounty_id)
except requests.HTTPError:
bounties = self._scrape_bounties() # Local fallback
bounty = next(b for b in bounties if b["id"] == bounty_id)
This pattern — try API, fall back to local data — is essential for agents that need to work offline or during API outages.

On-Chain Integration
The BountyEscrow contract on Base L2 handles payments:

copy


Contract: 0x2Ae271f5E86bee449a36B943414b7C1a7b39772D
Network: Base Mainnet (Chain ID: 8453)
The agent reads on-chain data via BaseScan API to verify:

Bounty funding status
Payment releases to hunter wallets
Submission transaction hashes
This provides independent verification — the agent doesn't trust the API alone, it cross-checks against on-chain state.

CLI Interface
The agent uses argparse with rich for formatted output:

Bash



List open bounties with viability scores

python verdikta_agent.py --list

Check specific bounty

python verdikta_agent.py --check 157

Monitor mode (checks every 30 minutes)

python verdikta_agent.py --monitor

View submission history

python verdikta_agent.py --history
Example output:

copy


📊 Verdikta Open Bounties
┌─────┬──────────────────────────┬──────────┬───────────┬──────────┬───────────┐
│ # │ Title │ Payout │ Threshold │ Targeted │ Viability │
├─────┼──────────────────────────┼──────────┼───────────┼──────────┼───────────┤
│ 157 │ I Tried to Cheat a │ 0.02 ETH │ 88% │ ✅ You │ HIGH ⭐ │
│ 158 │ Build an Agent │ 0.02 ETH │ 88% │ ✅ You │ HIGH ⭐ │
│ 160 │ Reddit AMA Post │ 0.008 ETH│ 85% │ ❌ Open │ MEDIUM │
└─────┴──────────────────────────┴──────────┴───────────┴──────────┴───────────┘
Key Design Decisions

Read-Only by Default
The agent does NOT send on-chain transactions automatically. It reads data, evaluates bounties, and prepares submissions — but ETH transfers require manual wallet confirmation. This is a safety feature: losing 0.02 ETH to a bad auto-submission isn't worth the automation.
Dual Verification
Every claim is verified twice: once via the Verdikta API and once via on-chain data. If the two disagree, the agent flags the discrepancy.
Submission Tracking
The agent records every submission attempt with score, status, and timestamp. Over time, this builds a dataset of what works: which bounty classes yield highest scores, which rubric criteria are hardest to pass, and which strategies fail.

What I Learned Building This
The Verdikta API Is Bot-Friendly
The X-Bot-API-Key authentication pattern is clean. Register once, use the key forever. The API returns structured JSON that's easy to parse. This is how bounty platforms should work.

Viability Scoring Saves Time
Not every 0.002 ETH bounty is worth 3 hours of work. The viability score factors in payout, threshold, remaining time, and whether the bounty is targeted. This turned a manual "should I try this?" into an automated decision.

Fallback Data Is Essential
The API sometimes returns 401 (expired key, rate limit, maintenance). Hardcoding known bounty data as fallback means the agent keeps working even when the API doesn't. This is a pattern I'll use in every API-dependent agent going forward.

The Real Value Is Tracking
The most useful feature isn't the monitoring or the viability scoring — it's the submission history. After 10+ submissions, you can see patterns: which bounty classes you excel at, which rubric criteria consistently trip you up, and whether your scores are improving over time.

Next Steps
Auto-generate submissions: Use an LLM to draft submissions based on rubric criteria
Score prediction: Train a model on past submissions to predict scores before submitting
Multi-chain support: Extend to other chains as Verdikta expands
Webhook notifications: Alert via Telegram/Discord when high-viability bounties appear
Try It Yourself
The agent is open source:

GitHub: github.com/s97472091-pixel/verdikta-agent

Bash


git clone https://github.com/s97472091-pixel/verdikta-agent.git
cd verdikta-agent
pip install -r requirements.txt
python verdikta_agent.py --list
On-Chain Evidence
Hunter Wallet: 0x1b9cA7b297a736f4FE01256C9e2d499c79dEFFb3
BountyEscrow: 0x2Ae271f5E86bee449a36B943414b7C1a7b39772D
6+ bounties won across math, task-creation, and case study categories
All claims verifiable at bounties.verdikta.org
Code at github.com/s97472091-pixel/verdikta-agent