A verification evidence bundle is the portable record behind a verdict. It should let a person, a CI system, or a later evaluation understand what was checked without depending on a screenshot or a model summary.
When I cannot answer which task, revision, environment, and checks produced a result, I do not consider that result durable evidence.
Bind the required identities
The bundle should identify the task, repository, base revision, agent change, verifier version, environment, and attempt.
Without those identities, passing output can be attached to the wrong patch or repeated under different conditions. The result may be real and still fail to prove the change under review.
Preserve execution records
For every check, I want the normalized command or runner identity, start and finish state, exit result, timeout or cancellation state, and bounded output.
Screenshots, reports, and other artifacts should be recorded by path and digest when they contribute to the verdict. Environment failures should remain environment failures rather than being hidden inside a generic test failure.
Keep uncertainty in the verdict
A useful verdict separates:
- passed requirements
- reproduced failures
- regressions
- pre-existing failures
- infrastructure failures
- requirements that remain unverified
If an authoritative check is missing, the bundle should fail closed. A confidence score cannot fill an evidence gap.
Redact without destroying portability
Evidence needs to be useful without leaking secrets or entire private repositories. I retain the necessary excerpts and metadata, enforce output bounds, redact sensitive values, and make truncation explicit.
JSON works well for automation. Markdown and self-contained HTML are useful when a person needs to inspect the result offline.
Link the re-check
When a failure is fixed, the new bundle should reference the earlier attempt and preserve the failing evidence. That gives the work a closure trail:
failed behavior -> corrective change -> passing check -> regression result
CodeVetter treats this machine-readable bundle as a primary product surface. Completed reviews can export fail-closed JSON, Markdown, and self-contained HTML while keeping missing proof visible.
The full bundle model is at https://codevetter.com/verification-evidence-bundle.
Top comments (1)
One field I would add is the exact files the verifier read, not just the repo revision and command. Agent reviews often pass because the test hit the right command while the reviewer silently missed the generated file or migration. A small read-set in the bundle makes that failure visible.