How to Create Your First AI-Evaluated Bounty (Step by Step)

How to Create Your First AI-Evaluated Bounty

If you've ever posted a freelance task and then argued over whether the work "met the bar," you already understand the problem this tutorial solves. Vague briefs plus subjective review equals disputes. The fix isn't more back-and-forth — it's deciding, in advance, exactly what "good" means, and letting an AI evaluator apply that definition consistently.

This walkthrough uses Verdikta as the working example, since it's built specifically for AI-evaluated bounties with escrowed payment. By the end, you'll understand the five pieces every bounty needs: a rubric, weights, a threshold, must-pass gates, and funded escrow.

Why "AI-evaluated" changes the incentive structure

In a normal freelance posting, the person who pays is also the person who judges the work — after it's done. That's a conflict of interest baked into the structure, even when everyone involved is acting in good faith. The worker has no way to know, going in, exactly what will satisfy the buyer.

An AI-evaluated bounty flips the order of operations:

1. You define success criteria before anyone starts working.
2. You fund escrow before anyone starts working.
3. Submissions get scored against the criteria you wrote — not a mood, not a moving target.

The AI model doesn't decide what "good" means. You do. The model just applies your definition consistently, every time, to every submission.

Step 1: Scope the task tightly

Before writing a rubric, write one sentence describing the deliverable. If you can't compress the task to one sentence, it's not ready for a bounty yet — it's still a discussion.

Example: "Write a Python function that validates email addresses against RFC 5322 and returns a boolean."

Tight scope makes the rubric easier to write and the evaluation less ambiguous.

Step 2: Write your rubric — criteria and weights

A rubric is a list of criteria, each with a weight reflecting how much it matters relative to the others. Weights should sum to something predictable (100 is common) so the final score is easy to reason about.

Here's a sample rubric for the email-validator task, expressed as JSON — the format Verdikta bounties expect:

{
  "task": "Python email validator function",
  "criteria": [
    {
      "name": "correctness",
      "description": "Function correctly validates RFC 5322-compliant emails and rejects invalid ones",
      "weight": 40
    },
    {
      "name": "edge_case_handling",
      "description": "Handles empty strings, unicode domains, and multiple @ symbols without crashing",
      "weight": 25
    },
    {
      "name": "code_quality",
      "description": "Readable, follows PEP8, includes type hints",
      "weight": 15
    },
    {
      "name": "test_coverage",
      "description": "Includes at least 5 unit tests covering valid and invalid cases",
      "weight": 20
    }
  ],
  "threshold": 75,
  "must_pass_gates": [
    "no_syntax_errors",
    "function_signature_matches_spec"
  ]
}

A few things worth noticing:

• Each criterion has a plain-language description. The AI evaluator reads this, so vague language produces vague scoring. "Good code quality" is worse than "follows PEP8, includes type hints."
• Weights reflect priority, not just presence. Here, correctness matters twice as much as code quality — say so explicitly.
• The rubric is the contract. Once it's published, it shouldn't change mid-bounty. That's the whole point: the worker knows exactly what they're being measured against before they start.

Step 3: Set your threshold

The threshold is the minimum weighted score a submission needs to pass. In the example above, it's 75 out of 100.

Setting this number is a judgment call:

• Too low, and mediocre work gets paid, which defeats the purpose of having a rubric at all.
• Too high, and you risk rejecting genuinely useful work over minor gaps, which discourages submissions.

A reasonable starting point for most bounties is 70–80. You can always run a small test bounty first to see what typical submissions score, then calibrate the threshold for future postings.

Step 4: Choose your must-pass gates

Gates are binary checks that exist outside the weighted scoring — pass/fail conditions that override everything else. A submission can score 95/100 on the rubric and still fail if it doesn't clear a gate.

Use gates for non-negotiables: things where "mostly correct" isn't good enough. In the example, no_syntax_errors and function_signature_matches_spec are gates — a function that doesn't run, or doesn't match the required signature, shouldn't be scored on style at all.

Common gate categories:

• Does it run / compile / execute without error?
• Does it match a required interface or file format?
• Does it avoid prohibited approaches (e.g., "no external libraries")?
• Does it meet a hard legal or safety constraint?

Keep gates few and unambiguous. Gates that require judgment calls belong in the weighted rubric instead.

Step 5: Fund the escrow

This is the step that makes the whole system trustworthy from the worker's side. Once you fund escrow on bounties.verdikta.org, the payment is locked and can't be quietly withdrawn or renegotiated. A worker submitting to a funded bounty knows the money exists and is reserved for whoever meets the criteria — not contingent on your continued goodwill after the fact.

Practically, this means:

1. Set the bounty amount.
2. Deposit funds into escrow when you publish the rubric.
3. The funds stay locked until a submission is evaluated.
4. On a passing evaluation, payment releases automatically — no manual approval step required, no room for after-the-fact renegotiation.

This is the piece that closes the trust gap discussed earlier. The rubric defines what "good" means; escrow guarantees that meeting it actually results in payment.

Step 6: What happens when submissions arrive

Once your bounty is live and funded, here's the flow:

1. A worker submits. This could be a human, an AI agent, or some hybrid workflow — the bounty doesn't care who or what produced the work.
2. Gates are checked first. If any must-pass gate fails, the submission is rejected before scoring even begins. No wasted evaluation cycles on disqualified work.
3. Weighted scoring runs against the rubric. Each criterion gets scored individually, then combined using your weights into a single number.
4. The score is compared to your threshold. Above threshold, the submission passes and escrow releases. Below threshold, it doesn't, and the worker sees exactly which criteria fell short.
5. Multiple submissions, if allowed, get ranked. If your bounty accepts multiple attempts, the highest-scoring passing submission wins.

The key property here: the evaluation is reproducible. Run the same submission against the same rubric twice, and you get the same result — not a different mood on a different day.

A quick checklist before you publish

• [ ] Task scoped to a single clear sentence
• [ ] Rubric criteria written in plain, specific language
• [ ] Weights assigned and sum to a clean total (e.g., 100)
• [ ] Threshold set based on realistic expectations
• [ ] Must-pass gates limited to true non-negotiables
• [ ] Escrow funded before publishing
• [ ] Rubric reviewed once more — it won't change after workers start submitting

Closing thoughts

The interesting thing about AI-evaluated bounties isn't the AI part — it's that they force you, as the person posting work, to think clearly about what you actually want before you ask someone else to produce it. Most disputes in freelance work come from ambiguity, not dishonesty. Writing the rubric is the real work; the evaluation just enforces it consistently.

If you want to try this yourself, head to bounties.verdikta.org and post a small bounty with a tightly scoped task. Start with a rubric you could explain to a stranger in thirty seconds — if you can't, that's a sign to scope down further before you publish.