Engineering Verifiable Systems: From Zero-Knowledge Proofs to AI Agent Trust

Originally published on tamiz.pro.

The fundamental unit of trust in software has historically been the authority of the entity that built the system. We trust sha256 because NIST says it’s secure; we trust our payment processor because it has a brand; we trust our AI model because the vendor claims it aligns with safety guidelines. This "Trusted Execution Environment" (TEE) model, whether physical or logical, is breaking. As we move into an era of autonomous AI agents, decentralized financial protocols, and distributed edge computing, we can no longer rely on a central party to attest to the integrity of an operation. We must shift from asking "Do I trust this system?" to "Can I verify this system?"

This article explores the engineering of Verifiable Systems. It dissects how Zero-Knowledge (ZK) proofs, cryptographic receipts, and attestation frameworks allow us to prove the correctness of computation without revealing the data, or without exposing the internals of the process. We will bridge the gap between high-level blockchain anti-cheat mechanisms and the emerging challenge of verifying the behavior of Large Language Model (LLM) agents.

The Crisis of Black-Box Autonomy

To understand why verifiable systems are critical now, we must look at the two primary domains where trust is failing:

1. Financial & Gaming Integrity: In decentralized applications (dApps), users interact with smart contracts that execute complex logic. If a vulnerability exists, or if an oracle is manipulated, users lose funds. The "code is law" paradigm fails when the code is opaque or the inputs are manipulated.
2. AI Agent Accountability: We are building agents that browse the web, execute code, and make purchases. If an agent claims, "I checked the inventory and purchased the item," how do we know it didn't hallucinate the purchase? How do we know it didn't execute malicious code in the background? Currently, we have no cryptographic receipt for the actions of an AI.

The solution lies in replacing "trust" with "proof." A verifiable system generates a cryptographic artifact that allows a third party to verify a claim (e.g., "computation X was performed correctly") in milliseconds, without needing to re-execute the entire computation or inspect the private inputs.

Core Primitives: The Crypto-Engineering Toolbox

Before we can engineer a verifiable system, we must understand the three pillars that support it.

1. Zero-Knowledge Proofs (ZKPs)

ZKPs allow a prover to convince a verifier that a statement is true without conveying any information aside from the truth of the statement. In software engineering terms, this is the difference between saying "I know the password" (verifying by showing the password) and saying "Here is a hash of the password that matches the known hash" (verifying without revealing it).

There are two main families of ZKPs relevant to production systems:

• zk-SNARKs (Zero-Knowledge Succinct Non-Interactive Arguments of Knowledge): These are compact (often <200 bytes) and fast to verify, but require a "trusted setup" (a one-time generation of common reference strings) and rely on pairing-based cryptography. They are ideal for high-frequency transactions where bandwidth is a constraint, such as in Layer-2 blockchains.
• zk-STARKs (Zero-Knowledge Succinct Arguments of Knowledge): These are based on hash functions and do not require a trusted setup. They are more robust against quantum attacks but result in larger proofs (typically 10-50 KB) and longer verification times. They are ideal for decentralization where avoiding trust assumptions is paramount.

2. Trusted Execution Environments (TEEs)

TEEs are hardware-isolated enclaves within a CPU (like Intel SGX or ARM TrustZone). They allow code to run in a protected memory region that is isolated from the Operating System and other processes. The hardware generates an "attestation"—a cryptographic signature that proves the code running inside the enclave has a specific hash. While TEEs provide strong security, they rely on the CPU vendor's integrity. ZKPs, by contrast, are mathematical and verifiable by anyone.

3. Merkle Trees and Hash Chaining

While not "zero-knowledge" in themselves, Merkle trees are essential for batching proofs. In a game or ledger, every state change is a leaf in a tree. To prove a specific user's balance, you don't need to publish the whole database; you just publish the Merkle path (a logarithmic number of hashes). This optimizes the data size of ZKP inputs.

Architecture of a Verifiable Pipeline

A modern verifiable system usually follows a Prove-Verify pattern. Let's visualize this in a generic pipeline, independent of the specific domain (gaming or AI).

[ Data / Logic ] -> [ ZK Circuit ] -> [ Witness Generation ] -> [ ZK Proof ] -> [ On-Chain / Client Verification ]

1. Circuit Definition: You translate your logic into a Circuit Representation Language (R1CS, AIR, or Boolean formulas). For example, if you are verifying an AI agent's action, your circuit might check: IF action == "purchase" AND balance >= cost AND item_stock > 0 THEN output = "success".
2. Witness Generation: The "secret" data (the private balance, the specific item ID, the agent's internal state) is combined with public data to create a "witness" that satisfies the circuit.
3. Proof Generation: A specialized compiler (e.g., Groth16, PLONK, or STARKd) generates the cryptographic proof from the witness and the circuit.
4. Verification: A lightweight verifier (often a smart contract or a client-side library) checks the proof against the public inputs. If valid, the state is accepted.

Case Study 1: Zero-Knowledge Anti-Cheat in Gaming

In traditional online games, anti-cheat is a client-side process. Cheaters can easily bypass it by modifying the game client to report false coordinates or health values. The server cannot distinguish between a legitimate move and a spoofed input without re-simulating the entire game state, which is too expensive.

The ZK Solution:
Instead of sending raw inputs, the client runs the game logic inside a local, tamper-resistant environment (often a TEE) or compiles the movement logic into a ZK circuit.

• Constraint: Prove that new_position is valid given old_position, input_vector, and physics_constants, without revealing the specific input_vector (to prevent prediction by opponents) or the internal state variables.
• Engineering Challenge: Compiling complex physics engines into ZK circuits is computationally intensive. The circuit size grows with the complexity of the simulation.
• Result: The server receives a small proof. It verifies the proof in O(1) time. If the proof is valid, the move is accepted. Cheats are rejected because the math doesn't add up, and the prover cannot forge the proof without the secret keys or the valid state.

This shifts the burden of integrity from the network to the mathematics.

Case Study 2: Engineering Trust for AI Agents

This is the cutting edge. LLMs are non-deterministic, probabilistic, and opaque. An agent that controls a browser or a financial account introduces a "Principal-Agent" problem in reverse: The human (principal) cannot easily monitor the AI (agent).

How do we verify that an AI agent did what it claimed?

The "Receipt of Action" Pattern

We can engineer a system where every significant action taken by an AI agent is accompanied by a cryptographic receipt. This receipt is not just a log entry; it is a ZK proof that validates the agent's intent and execution constraints.

The Workflow:

1. Policy Compilation: Before the agent runs, the human user defines a policy: "You may spend up to $500 on coffee beans, and you must only use HTTPS endpoints."
2. Circuit Encoding: This policy is encoded into a ZK circuit. The circuit checks:
   • action_type == "purchase"
   • amount < 500
   • domain == "https://secure-coffee.com"
   • timestamp < expiry_date
3. Agent Execution: The agent performs the action. It records the public inputs (amount, domain, timestamp) and the private state (the specific transaction ID, the agent's internal confidence score).
4. Proof Generation: The agent generates a ZK proof that it satisfied the policy circuit.
5. Verification: The human's wallet or dashboard verifies the proof.
   • Crucially: The human does not need to read the agent's internal reasoning (which might be too large or complex). They just need to verify that the constraints were met.
   • Zero-Knowledge Aspect: The proof can be structured so that the specific internal tokens or API keys used are kept private, while proving they were valid.

Handling Non-Determinism

ZK proofs are deterministic. LLMs are not. How do we prove a probabilistic output?

We don't prove the output directly; we prove the bounds of the output.

• Quantile Proofs: The circuit can check that the token probabilities for forbidden words are below a certain threshold, rather than checking for exact string matches.
• Sampling Verification: For large-context reasoning, we can use statistical sampling. The agent provides a sample of its reasoning steps. The verifier checks that the sample is consistent with the distribution of valid reasoning, using cryptographic commitments.

This is similar to how we trust a random number generator: we don't check every bit; we check that the sequence passes statistical tests. For AI, we check that the behavior passes constraint tests.

Comparison of Verification Models

Feature	Trusted Oracles	TEE Attestation	ZK Proofs	Hashed State Checks
Trust Assumption	Centralized Party	CPU Vendor	Mathematics	Previous State
Data Privacy	Low (Oracles see data)	High (Enclave)	High (Inputs hidden)	Low (State public)
Verification Cost	O(1) (Trust)	O(1) (Crypto Check)	O(1) (ZK Verify)	O(log N)
Setup Requirements	None	Attestation Keys	Trusted Setup (SNARK)	None
Quantum Resistance	No	No	Yes (STARKs)	No (SHA-256)
Applicability to AI	Poor (Hallucination risk)	Medium (Black box)	High (Constraint Verification)	Poor

Implementation Hurdles for Engineers

Building these systems is not just about plugging in a library. It requires deep consideration of performance and security.

1. Circuit Complexity and Compilation

Writing ZK circuits is painful. High-level languages like Circom (for Solidity/Blockchain) or Ligature (for Rust) help, but translating general-purpose logic (like a Python AI agent's decision tree) into arithmetic constraints is non-trivial.

• Tip: Keep circuits simple. Offload complex logic to the host and only prove the critical invariant (e.g., "spend limit") on-chain or in the verifier. Do not try to ZK-proof the entire LLM inference pass; that is computationally infeasible. ZK-proof the policy compliance of the result.

2. Trusted Setup Risks

For zk-SNARKs, the trusted setup is a single point of failure. If the

certifying party colludes with malicious participants, the entire proof system collapses. The generated toxic parameters can be used to generate fake proofs for false statements.

Mitigation Strategies:

1. Transition to zk-SNARGs: The modern standard (used by Ethereum’s Groth16 successor, Groth16, and eventually the move toward KZG commitments) allows the trusted setup to be distributed. Multiple parties contribute to the setup, and even if one party is malicious, they must be caught to break the system. If all parties in the setup ceremony are dishonest, the system is broken. In practice, public ceremonies with open-source code and public auditing make a global conspiracy nearly impossible.
2. Use Transparent Systems (zk-STARKs): STARKs (Scalable Transparent ARITHMETIC Proofs of Knowledge) eliminate the trusted setup entirely. They rely on publicly verifiable cryptographic hashes and error-correcting codes. The trade-off is larger proof sizes and higher verification costs compared to SNARKs, but for large-scale AI agent interactions where trust is paramount, the security gain outweighs the overhead.

3. The "Oracle" Problem in AI Contexts

A zero-knowledge proof verifies the computation was performed correctly according to the circuit. It does not verify that the input data was truthful or that the AI model’s weights were not maliciously altered to produce "poisoned" outputs.

Solution: Compositional Trust

You cannot ZK-proof the entire AI reasoning process. Instead, you must decompose the trust model:

1. Input Integrity: Use cryptographic hashes to anchor the input prompt and context to a trusted source (e.g., a blockchain ledger or a signed API response).
2. Model Provenance: The AI operator publishes a hash of the model weights and the inference engine version to a public registry.
3. Policy Verification: The ZK circuit only proves that given these inputs and given this specific model configuration, the output satisfies a specific policy (e.g., "does not contain PII," "does not exceed budget limits").

Phase 3: Implementing a Hybrid Verification Stack

This section provides a practical, runnable example of a lightweight verification layer. While full ZK implementations are complex, we can simulate the cryptographic integrity checks and logic verification using Python and the eth-hash library to represent on-chain anchors.

Step 1: Define the Policy Circuit (Conceptual)

We are verifying that an AI Agent has adhered to a "No-PII" policy. The circuit takes the output text and checks for regex patterns associated with emails and phone numbers.

import hashlib
import json
import re

class PolicyVerifier:
    def __init__(self, policy_hash: str):
        """
        policy_hash: The Merkle root or hash of the policy definition.
        In a ZK context, this is part of the circuit constraints.
        """
        self.policy_hash = policy_hash
        self.regex_pii = [
            r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b', # Email
            r'\b\d{3}-\d{3}-\d{4}\b' # Phone
        ]

    def generate_zk_input_commitment(self, output_text: str) -> str:
        """
        Simulates the commitment of the output to the ZK circuit.
        """
        return hashlib.sha256(output_text.encode('utf-8')).hexdigest()

    def verify_policy_compliance(self, output_text: str) -> dict:
        """
        Simulates the ZK proof verification.
        In production, this would call a ZKVM (like RISC Zero or Lepton) 
        or a smart contract verifier.
        """
        violation_found = False
        violations = []

        for pattern in self.regex_pii:
            matches = re.findall(pattern, output_text)
            if matches:
                violation_found = True
                violations.extend(matches)

        # In a ZK proof, this boolean is the result of the arithmetic circuit
        # executed inside the zkVM.
        proof_valid = not violation_found

        return {
            "proof_valid": proof_valid,
            "commitment": self.generate_zk_input_commitment(output_text),
            "violations_detected": violations,
            "policy_ref": self.policy_hash
        }

Step 2: The Agent Interaction Loop

Here is how an AI agent interacts with a "Trust Layer" before submitting results to a verifier.

class TrustAwareAgent:
    def __init__(self, verifier: PolicyVerifier):
        self.verifier = verifier
        self.audit_log = []

    def execute_task(self, prompt: str) -> dict:
        """
        Simulates the LLM generation and subsequent verification.
        """
        # 1. Simulate LLM Generation
        raw_output = self._generate_response(prompt)

        # 2. Verify Policy Compliance
        verification_result = self.verifier.verify_policy_compliance(raw_output)

        # 3. Log and Submit
        self.audit_log.append({
            "prompt_hash": hashlib.sha256(prompt.encode()).hexdigest(),
            "output_hash": verification_result["commitment"],
            "proof_status": "PASSED" if verification_result["proof_valid"] else "FAILED",
            "timestamp": __import__('time').time()
        })

        if not verification_result["proof_valid"]:
            return {
                "status": "REJECTED",
                "reason": "Policy violation detected",
                "violations": verification_result["violations_detected"]
            }

        return {
            "status": "ACCEPTED",
            "output": raw_output,
            "proof_commitment": verification_result["commitment"]
        }

    def _generate_response(self, prompt: str) -> str:
        """
        Mock LLM response generator for demonstration.
        """
        if "email" in prompt:
            return "Your email is [e-mail]. Let us know if that's correct."
        else:
            return "The server is running optimally. No issues detected."

# --- Execution Demo ---
if __name__ == "__main__":
    # 1. Setup: Operator defines the policy and generates a hash
    policy_obj = {"rules": ["no_pii", "max_length_1000"]}
    policy_hash = hashlib.sha256(json.dumps(policy_obj, sort_keys=True).encode()).hexdigest()

    verifier = PolicyVerifier(policy_hash)
    agent = TrustAwareAgent(verifier)

    # 2. Test Case 1: Safe Output
    print("--- Test 1: Safe Query ---")
    result1 = agent.execute_task("Check server health")
    print(json.dumps(result1, indent=2))

    # 3. Test Case 2: PII Leak
    print("\n--- Test 2: PII Leak ---")
    result2 = agent.execute_task("Who is the admin?")
    print(json.dumps(result2, indent=2))

    # 4. Audit Trail
    print("\n--- Audit Log ---")
    for entry in agent.audit_log:
        print(entry)

Phase 4: Scaling Trust with AI Agent Swarms

As systems evolve from single agents to multi-agent swarms, trust becomes a graph problem. Agent A might trust Agent B, but not Agent C. ZK proofs allow for transitive trust.

The Trust Graph Model

1. Root of Trust: A central, audited identity registry (e.g., Ethereum Name Service or a decentralized verifier network).
2. Agent Proofs: Each agent generates a ZK proof that it executed its code correctly and signed its output.
3. Composition: If Agent A’s output is fed into Agent B, Agent B’s ZK proof can include a hash of Agent A’s proof. Agent B thereby inherits Agent A’s trust level without needing to re-verify Agent A’s entire logic, only its cryptographic signature.

Challenge: Latency.
ZK proof generation for complex agent logic can take seconds. For real-time interaction, you must use recursive proofs or proof aggregation.

• Recursive Proofs: Agent A generates a proof. Agent B generates a proof that includes the verification of Agent A’s proof. The final verifier only needs to check one top-level proof.
• Aggregation: A trusted relayer collects multiple agent proofs and bundles them into a single ZK proof, reducing the number of on-chain verifications.

Conclusion: The New Engineering Paradigm

We are moving from an era of implicit trust (we assume the API works as documented) to cryptographic verifiability (we prove the API acted as intended).

For AI engineers, this means:

1. Policy as Code: Your business rules must be formalized into arithmetic circuits. If you cannot formalize a rule, you cannot prove it.
2. Verification is Cheap, Generation is Expensive: Design your agent interactions to minimize the complexity of the ZK circuit. Prove the exception, not the rule. For example, instead of proving every token of a 10,000-word essay is compliant, prove that a specific "risk score" calculated by a lightweight heuristic is below a threshold.
3. Transparency is a Feature: Publish your ZK circuits. Open-source your verification code. The security of a verifiable system lies in the ability of anyone to audit the constraints.

The future of AI agents is not one where we blindly hope the model is aligned. It is one where we know it is aligned, because we hold the cryptographic key to that certainty. Build for verification. Build for trust.