Verifying AI Agent Actions: Proof Submission Patterns

AI agents are increasingly handling real-world tasks—posting content, completing research, verifying accounts, even physical deliveries. But when your system depends on an agent claiming it did something, how do you actually know it happened?

This isn't a theoretical problem. If you're building an automation pipeline where agents earn rewards or trigger downstream workflows based on completed actions, proof submission is the backbone of trust. Let's break down the patterns that work, the ones that don't, and how to implement them properly.

The Core Problem: Trust in Autonomous Systems

When a human completes a task, you can ask them to describe what happened. With AI agents, you get a JSON response saying "status": "completed". That's not proof—that's a claim.

The challenge is designing a verification layer that:

1. Minimizes false positives (agent claims success but didn't do the work)
2. Minimizes false negatives (agent did the work but proof is rejected)
3. Keeps costs reasonable (verification shouldn't be more expensive than the task itself)

Let's look at the patterns that address these constraints.

Pattern 1: Deterministic Artifact Submission

The simplest approach: require the agent to return a deterministic artifact that can be checked programmatically.

def verify_task_artifact(task_id: str, expected_hash: str, artifact: dict) -> bool:
    """
    Verify that the artifact matches the expected structure and content hash.
    """
    import hashlib
    import json

    # Serialize deterministically (sorted keys)
    serialized = json.dumps(artifact, sort_keys=True, separators=(',', ':'))
    actual_hash = hashlib.sha256(serialized.encode()).hexdigest()

    # Check structure
    required_fields = {"url", "timestamp", "content"}
    if not required_fields.issubset(artifact.keys()):
        return False

    # Check hash
    if actual_hash != expected_hash:
        return False

    return True

Where this works: Tasks with well-defined outputs—API calls, database writes, file transformations. The agent must return the exact data structure you expect.

Where it fails: Open-ended tasks. "Write a summary of this article" has infinite valid outputs. You can't hash-check creativity.

Pattern 2: Screenshot/Screen Recording Proof

For tasks involving UI interactions or content creation, screenshots are the gold standard. But raw screenshots aren't verifiable—you need metadata.

class ScreenshotProof:
    def __init__(self, image_data: bytes, metadata: dict):
        self.image_data = image_data
        self.metadata = metadata

    def verify(self) -> bool:
        """
        Verify screenshot authenticity using embedded metadata.
        """
        # Check timestamp is within acceptable window
        import time
        if abs(time.time() - self.metadata["capture_time"]) > 300:
            return False

        # Check geo-tag if required
        if self.metadata.get("requires_geo") and not self.metadata.get("geo_tag"):
            return False

        # Verify image isn't a re-submission
        if self.is_duplicate():
            return False

        return True

The critical detail: Screenshots must include context that's hard to fake—timestamps, geolocation, device fingerprints. A screenshot of a browser window with no metadata proves nothing.

Real-world implementation: Task platforms like roborent.cc handle this by requiring agents to submit screenshots with embedded EXIF data and platform-specific watermarks. For social media tasks, the agent must screenshot the published post with the URL visible, not just the draft.

Pattern 3: Cross-Reference Verification

The most robust pattern: don't trust the agent's submission alone—verify against an external source of truth.

async def verify_social_post(agent_claim: dict) -> bool:
    """
    Verify a claimed social media post by fetching it from the platform.
    """
    import aiohttp

    # Agent claims they posted at this URL
    post_url = agent_claim.get("post_url")

    async with aiohttp.ClientSession() as session:
        async with session.get(
            post_url, 
            headers={"User-Agent": "VerificationBot/1.0"}
        ) as response:
            if response.status != 200:
                return False

            page_content = await response.text()

            # Check that the agent's claimed content appears
            if agent_claim.get("expected_text") and \
               agent_claim["expected_text"] not in page_content:
                return False

            # Check the posting timestamp
            post_time = extract_post_time(page_content)
            if post_time and abs(post_time - agent_claim["claim_time"]) > 600:
                return False

    return True

Why this works: You're not asking "did you do it?"—you're checking the public record. If the agent claims they posted on X (Twitter), you fetch the post URL and confirm the content exists.

The catch: Not all tasks have external verifiability. Physical tasks (IRL tasks like checking a storefront) need human verification or GPS-tagged photos.

Pattern 4: Multi-Agent Consensus

For subjective tasks or tasks without deterministic verification, use multiple independent agents and require consensus.

def verify_with_consensus(submissions: list[dict], min_agreement: int = 2) -> bool:
    """
    Verify a task requires N agents to independently agree.
    """
    if len(submissions) < min_agreement:
        return False

    # Group by normalized response
    from collections import Counter
    normalized = []

    for sub in submissions:
        # Normalize: lowercase, strip whitespace, sort keywords
        normalized.append(normalize_response(sub["result"]))

    counts = Counter(normalized)
    top_count = counts.most_common(1)[0][1]

    return top_count >= min_agreement

The trade-off: You're paying for 2-3 agents to do one task. But for high-stakes tasks where a single false claim is costly, it's worth it.

The risk: Agents can collude. If they're all using the same underlying model, they'll produce identical outputs—which looks like consensus but isn't independent verification.

Pattern 5: Staged Proof Submission

Complex tasks often need staged verification—you don't want to pay an agent for a 30-minute task and then discover at the end that their proof is invalid.

class TaskVerificationPipeline:
    def __init__(self, task: dict):
        self.task = task
        self.stages = []

    def add_stage(self, stage_name: str, verifier: callable, 
                  required: bool = True):
        self.stages.append({
            "name": stage_name,
            "verifier": verifier,
            "required": required,
            "passed": None,
            "proof": None
        })

    async def execute(self, agent_output: dict) -> dict:
        """
        Execute each verification stage in order.
        """
        results = {}

        for stage in self.stages:
            try:
                proof = stage["verifier"](agent_output, self.task)
                stage["passed"] = proof is not None
                stage["proof"] = proof
                results[stage["name"]] = {
                    "passed": stage["passed"],
                    "proof": proof
                }

                # Fail fast on required stages
                if stage["required"] and not stage["passed"]:
                    results["status"] = "REJECTED"
                    return results

            except Exception as e:
                stage["passed"] = False
                results[stage["name"]] = {"passed": False, "error": str(e)}
                if stage["required"]:
                    results["status"] = "REJECTED"
                    return results

        results["status"] = "APPROVED"
        return results

Why this matters: For complex tasks, an agent might complete step 1 correctly but mess up step 3. Staged verification lets you reject early and re-assign the task without wasting the full payout.

Practical Implementation: What Works in Production

The Verification Stack

Here's a realistic verification flow for a task marketplace:

1. Task creation: Define verification requirements at task creation time
2. Agent assignment: Agents see verification criteria before accepting
3. Execution: Agent completes task and submits proof bundle
4. Automated checks: Run deterministic checks (hash, structure, timestamp)
5. Human/AI review: For ambiguous submissions, route to a reviewer
6. Payment release: Only release payment after verification passes

Code: The Proof Bundle Structure

python
@dataclass
class ProofBundle:
    task_id: str
    agent_id: str
    proofs: list[ProofItem]
    submitted_at: float

    def validate(self) -> ValidationResult:
        """
        Validate the complete proof bundle.
        """
        issues = []

        # Check all required proof types present
        required_types = {"artifact", "screenshot", "cross_reference"}
        actual_types = {p.type for p in self.proofs}

        missing = required_types - actual