The Proof-Carrying Patch: Building a Verification Gate for AI-Generated Code

A practical engineering tutorial for software developers and AI engineers

By Akshat Raj | Software Engineering · AI Engineering · Developer Tooling

---

Abstract

AI coding agents can generate implementations, modify multiple files, execute commands, and propose fixes within minutes. However, a successful tool call or a passing test suite does not necessarily prove that a proposed patch satisfies the original requirements.

A patch may pass existing tests while introducing an untested regression. It may modify files outside the intended scope, weaken a test, or leave a required behavior unimplemented.

This article introduces a practical approach called a Proof-Carrying Patch: every proposed code change must be accompanied by an explicit verification report describing which checks passed, which failed, and what remains unverified.

We will build a small verification gate in Python that checks a Git patch, validates whitespace and conflict markers through Git, executes a specified test command, enforces basic change-scope policies, and produces a machine-readable report.

The objective is not to prove arbitrary software correct. It is to turn an AI-generated patch from an unverified proposal into a change supported by reproducible engineering evidence.

1. The problem: a passing test is not a proof

Consider a coding agent asked to optimize an API endpoint without changing its response format.

The agent edits the endpoint, updates a helper function, and reports that all tests pass.

Several problems could remain:

• The existing tests never covered the response format that changed.
• The agent modified an unrelated authentication file.
• A test was weakened to accommodate the new implementation.
• The test command ran against the wrong Git revision.
• A test process failed to execute, but the orchestration layer reported completion anyway.

These are not all model-quality problems. Some are verification-system design problems.

The distinction matters because an agent can generate better code without becoming a reliable judge of its own work.

OpenAI's engineering writing on agent harnesses and systematic evaluations discusses the importance of surrounding an agent with tools, validation and explicit success criteria. The same principle can be applied to ordinary code changes. References: Harness Engineering and Testing Agent Skills Systematically with Evals.

A better definition of completion

Instead of defining success as:

The agent finished generating code.

Define it as:

The proposed patch passed the required, independently executed checks, and the verification report records exactly what was checked.

That definition is stricter, more auditable and easier to automate.

2. The architecture

Our first implementation will use a small, deterministic verification pipeline.

       AI coding agent
              |
              v
        Proposed Git patch
              |
              v
      +-------------------+
      | Patch inspection  |
      |                   |
      | Changed files     |
      | Scope restrictions|
      | Git diff checks   |
      +-------------------+
              |
              v
      +-------------------+
      | Test execution    |
      |                   |
      | Explicit command  |
      | Exit code         |
      | Timeout           |
      +-------------------+
              |
              v
      +-------------------+
      | Verification gate |
      |                   |
      | Aggregate results |
      | Record failures   |
      +-------------------+
              |
              v
      JSON evidence report
              |
       +------+------+
       |             |
       v             v
    Accept         Reject
    for review     or repair

The important design choice is that the verifier does not ask the model whether its code is correct. It collects evidence from external tools and applies explicit rules.

What the first version verifies

Check	Evidence
Patch exists	Changed-file list from Git
Change scope	Allowlisted paths and file-count limit
Basic patch integrity	git diff --check
Required tests	Test process exit code
Execution reliability	Timeout and process status
Auditability	Structured JSON report

What it does not prove

This first version does not establish functional equivalence, detect every security vulnerability, prove the absence of regressions, or establish that the tests themselves are trustworthy.

Those require additional verification layers.

3. Build the verification gate

The prototype uses Python 3.10+ and a Git repository. It has no model API dependency, so the same gate can be used with Codex, another coding agent, or manually written patches.

Create a file named proofgate.py.

Step 1: Implement the verifier

from __future__ import annotations

import argparse
import json
import shlex
import subprocess
import sys
import time

from pathlib import Path


def run_command(command: list[str], timeout: int = 300):
    """Run a command and capture its output and exit status."""
    start = time.monotonic()

    try:
        result = subprocess.run(
            command,
            capture_output=True,
            text=True,
            timeout=timeout,
            shell=False,
            check=False,
        )

        return {
            "command": command,
            "exit_code": result.returncode,
            "stdout": result.stdout[-12000:],
            "stderr": result.stderr[-12000:],
            "timed_out": False,
            "duration_seconds": round(
                time.monotonic() - start, 3
            ),
        }

    except subprocess.TimeoutExpired as exc:
        return {
            "command": command,
            "exit_code": None,
            "stdout": str(exc.stdout or "")[-12000:],
            "stderr": str(exc.stderr or "")[-12000:],
            "timed_out": True,
            "duration_seconds": round(
                time.monotonic() - start, 3
            ),
        }


def git_command(*args: str):
    return run_command(["git", *args], timeout=30)


def main() -> int:
    parser = argparse.ArgumentParser(
        description="Verify a proposed Git patch."
    )

    parser.add_argument(
        "--base",
        required=True,
        help="Git revision to compare against.",
    )
    parser.add_argument(
        "--max-files",
        type=int,
        default=20,
        help="Maximum number of changed files.",
    )
    parser.add_argument(
        "--allow-prefix",
        action="append",
        default=[],
        help="Allowed repository-relative path prefix; repeatable.",
    )
    parser.add_argument(
        "--test-timeout",
        type=int,
        default=300,
        help="Test timeout in seconds.",
    )
    parser.add_argument(
        "--",
        dest="unused",
        help=argparse.SUPPRESS,
    )

    # The test command is supplied after a standalone "--".
    args, test_command = parser.parse_known_args()

    if test_command and test_command[0] == "--":
        test_command = test_command[1:]

    report = {
        "schema_version": 1,
        "base": args.base,
        "checks": [],
        "changed_files": [],
        "status": "failed",
    }

    def record(name: str, passed: bool, details=None):
        report["checks"].append({
            "name": name,
            "passed": passed,
            "details": details or {},
        })

    # Confirm that Git can resolve the comparison base.
    base_check = git_command(
        "rev-parse", "--verify", args.base
    )
    record(
        "base_revision",
        base_check["exit_code"] == 0,
        base_check,
    )

    if base_check["exit_code"] != 0:
        report["reason"] = "Invalid base revision."
        print(json.dumps(report, indent=2))
        return 1

    # Discover files changed between base and HEAD.
    diff = git_command(
        "diff", "--name-only", "-z",
        f"{args.base}...HEAD",
    )

    if diff["exit_code"] != 0:
        record("changed_files", False, diff)
        print(json.dumps(report, indent=2))
        return 1

    changed_files = [
        name for name in diff["stdout"].split("\0") if name
    ]
    report["changed_files"] = changed_files

    record(
        "changed_files",
        True,
        {"count": len(changed_files)},
    )

    # A patch must contain at least one changed file.
    record(
        "nonempty_patch",
        len(changed_files) > 0,
        {"count": len(changed_files)},
    )

    # Restrict the number of files changed.
    record(
        "file_count_limit",
        len(changed_files) <= args.max_files,
        {
            "actual": len(changed_files),
            "maximum": args.max_files,
        },
    )

    # Optional path-prefix policy.
    if args.allow_prefix:
        violations = [
            name for name in changed_files
            if not any(
                name == prefix.rstrip("/")
                or name.startswith(prefix.rstrip("/") + "/")
                for prefix in args.allow_prefix
            )
        ]

        record(
            "allowed_paths",
            not violations,
            {"violations": violations},
        )

    # Detect whitespace errors and conflict markers recognized by Git.
    diff_check = git_command(
        "diff", "--check", f"{args.base}...HEAD"
    )

    record(
        "git_diff_check",
        diff_check["exit_code"] == 0,
        diff_check,
    )

    # Run an explicitly supplied test command.
    if not test_command:
        record(
            "tests",
            False,
            {"error": "No test command supplied."},
        )
    else:
        test_result = run_command(
            test_command,
            timeout=args.test_timeout,
        )

        record(
            "tests",
            (
                test_result["exit_code"] == 0
                and not test_result["timed_out"]
            ),
            test_result,
        )

    passed = all(
        check["passed"] for check in report["checks"]
    )

    report["status"] = "passed" if passed else "failed"

    print(json.dumps(report, indent=2))

    return 0 if passed else 1


if __name__ == "__main__":
    sys.exit(main())

The verifier uses argument lists with shell=False, which avoids interpreting the supplied test command as a shell expression. It still executes the selected program, so the command must be trusted and run in an appropriate environment.

The script intentionally uses the committed HEAD revision and a base revision. It does not automatically include uncommitted working-tree edits. This makes the comparison boundary explicit.

Step 2: Run it against a proposed patch

First, create a feature branch and commit the proposed change.

For a Python project:

python proofgate.py \
  --base origin/main \
  --allow-prefix src \
  --allow-prefix tests \
  --max-files 20 \
  --test-timeout 300 \
  -- python -m pytest -q

For a Go project:

python proofgate.py \
  --base origin/main \
  --allow-prefix internal \
  --allow-prefix cmd \
  --max-files 20 \
  --test-timeout 300 \
  -- go test ./...

The test command appears after --, so the verifier can pass it as a list of arguments without invoking a shell.

Important: Run these commands from the repository root and ensure origin/main exists locally. If your project uses a different base branch, supply that revision instead.

The exit status is now useful for automation:

• 0: all configured checks passed.
• 1: at least one check failed or required evidence was unavailable.

4. Example evidence report

A successful report will resemble the following structure:

{
  "schema_version": 1,
  "base": "origin/main",
  "checks": [
    {
      "name": "base_revision",
      "passed": true,
      "details": {}
    },
    {
      "name": "nonempty_patch",
      "passed": true,
      "details": {
        "count": 3
      }
    },
    {
      "name": "file_count_limit",
      "passed": true,
      "details": {
        "actual": 3,
        "maximum": 20
      }
    },
    {
      "name": "git_diff_check",
      "passed": true,
      "details": {}
    },
    {
      "name": "tests",
      "passed": true,
      "details": {
        "exit_code": 0,
        "timed_out": false
      }
    }
  ],
  "changed_files": [
    "src/api.py",
    "src/validation.py",
    "tests/test_api.py"
  ],
  "status": "passed"
}

This is an illustrative report, not the result of an executed test run.

The key property is that the report distinguishes individual checks rather than hiding everything behind a single score.

A future version should persist the full report as a build artifact and include the commit SHA, base SHA, test command, tool versions and repository state.

5. Add the gate to continuous integration

A verification gate becomes much more useful when every pull request runs it consistently.

For example, a GitHub Actions workflow could run the verifier after checking out the repository and installing its dependencies:

name: Patch Verification

on:
  pull_request:
  push:
    branches:
      - main

permissions:
  contents: read

jobs:
  verify:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository history
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install test dependencies
        run: python -m pip install pytest

      - name: Verify patch
        run: |
          python proofgate.py \
            --base origin/main \
            --allow-prefix src \
            --allow-prefix tests \
            --max-files 20 \
            -- python -m pytest -q

This is a template. Replace the allowed paths and dependency installation with those appropriate for the repository. Pin third-party actions to verified commit SHAs when your supply-chain policy requires it.

The example also assumes the PR branch can be compared against origin/main. For merge-queue builds, forks or nonstandard checkout configurations, choose the intended base revision explicitly.

A CI check should block merging when a required verification step fails. It should not silently convert a failed check into a warning.

GitHub Actions documentation: Workflow syntax.

6. The harder problem: can the agent game the verifier?

Suppose the coding agent knows that the only acceptance condition is a passing test suite.

It might change the implementation correctly. But it might also weaken an assertion, remove a test, or alter the verification configuration.

The gate cannot assume that a passing test suite is trustworthy merely because the test runner exited successfully.

This leads to several additional controls.

A. Protect the verification configuration

Require human review for modifications to:

• CI workflow files.
• Test configuration.
• Verification scripts.
• Security policies.
• Dependency lockfiles.

The appropriate policy depends on the project, but changes to the mechanism that verifies a patch should receive special scrutiny.

B. Validate behavior independently

Use tests whose expected outcomes are established independently of the implementation.

For an API change, verify the contract from an external client. For a security fix, add a regression test reproducing the vulnerability. For a database migration, validate both the migrated schema and expected data invariants.

C. Add mutation testing

Mutation testing deliberately introduces small faults into the code or tests and checks whether the verification suite detects them.

If removing an important validation condition does not cause a test failure, the suite may not be testing the intended behavior.

D. Test the verifier itself

The gate is software, so it needs tests for:

• Missing base revisions.
• Empty patches.
• Failed test commands.
• Test timeouts.
• Disallowed file paths.
• Malformed repository states.
• Incorrect exit-code handling.

A broken verifier can produce misleading confidence just as easily as a broken application.

7. Security boundaries: testing untrusted code

There is an important limitation in the prototype: subprocess.run() is not a security sandbox.

Running tests against an AI-generated patch can execute arbitrary code. A malicious or compromised patch could read available credentials, alter files or access the network.

For untrusted pull requests, execute verification inside an appropriately isolated environment:

• Use disposable workers or containers with restricted permissions.
• Avoid passing production credentials into test jobs.
• Limit network access where practical.
• Use read-only credentials and narrowly scoped tokens.
• Separate untrusted build steps from privileged deployment steps.
• Apply CPU, memory and execution-time limits.

A process timeout limits execution time; it does not prevent filesystem access, credential theft or network activity.

Security reference: Running Codex safely.

8. Limitations and next steps

The prototype is intentionally small. It provides a foundation, not a complete correctness-verification system.

Current limitation	Next engineering step
Only checks Git's basic patch integrity	Add syntax, type and static-analysis checks
Uses a single test command	Support named, independently configured verification stages
Uses a file-count limit	Add risk-based change policies
Does not verify that tests are meaningful	Add mutation tests and test-quality checks
Does not inspect vulnerability semantics	Add security scanners and targeted regression tests
Does not bind evidence to immutable artifacts	Record commit hashes, tool versions and artifact digests
Does not prove functional correctness	Add formal specifications or stronger domain-specific verification where feasible

A more mature implementation could produce a signed attestation binding the report to the exact patch and verification environment. Such an attestation would improve provenance, but it would still prove only that specified checks ran under specified conditions—not that the program is universally correct.

9. How to evaluate whether the approach helps

Do not judge the system solely by the number of successful builds.

Create a controlled experiment with three categories of patches:

1. Correct patches that should pass.
2. Incorrect patches that should fail.
3. Patches that modify the verification mechanism or exceed the permitted scope.

Measure:

• Valid patches incorrectly rejected.
• Invalid patches incorrectly accepted.
• Verification failures detected and classified.
• Runtime and resource consumption.
• Reproducibility across repeated runs.

The most useful metric is not simply the percentage of passing patches. It is how reliably the verification process distinguishes acceptable changes from changes that violate the defined contract.

Record the initial results before adding more checks. Then introduce one verification layer at a time and measure which failures it catches.

This makes the project an engineering experiment rather than a collection of impressive-looking metrics.

10. Conclusion

AI coding agents change the economics of software development by making implementation faster. They do not remove the need to establish whether an implementation is correct.

A Proof-Carrying Patch offers a practical starting point: every proposed change is accompanied by explicit checks, recorded results and a clear account of what remains unverified.

The approach is deliberately incremental. Start with deterministic checks. Add independent behavioral tests. Protect the verifier from being modified alongside the code it evaluates. Isolate untrusted execution. Preserve the evidence needed to reproduce a result.

The broader engineering principle is simple:

Do not make the agent's confidence the acceptance criterion. Make independently verifiable evidence the acceptance criterion.

That principle applies to AI-generated code, automated refactoring, dependency updates, security fixes and increasingly autonomous software engineering workflows.

References

1. OpenAI, Harness Engineering: Leveraging Codex in an Agent-First World.
2. OpenAI Developers, Testing Agent Skills Systematically with Evals.
3. OpenAI, Running Codex Safely.
4. GitHub Docs, Workflow Syntax for GitHub Actions.
5. OpenAI API Documentation, Evaluate Agent Workflows.

Author's note: The verifier in this article is a teaching prototype. The code and workflow are intended to be adapted, tested and hardened before use in a production environment.