TL;DR: A report tells you what a session says happened. The artifact on disk and a fresh rerun tell you what you can check. The parent slice log is where the distinction was recorded.
Your agent says the gate passed. Before you approve it, can you point to the artifact and rerun the command without the agent in the room? If not, you have a description of evidence, not evidence you can independently inspect.
In this note
- A report is self-description
- The artifact is where the claim meets reality
- Approval does not create an observation
- Make your pipeline answerable
- Rerun before you rely on it
A report is self-description
A session report is useful for navigation. It is not the same thing as the files, command, and output it describes.
That sounds obvious until the report is green and the review queue is long. A worker says it changed a fixture, ran a suite, and saw a pass. A reviewer reads that sentence and gives it a second approval. The words become more official, but nobody has created another observation of the work.
Ranex is built around a smaller rule: read the diff on disk; discard the worker summary; run the checks. The rule is not distrust for its own sake. It gives a later reviewer something that exists outside the session which made the claim. A summary can be wrong, stale, incomplete, or written before the final artifact existed. The artifact gives you a place to ask again.
SLICE-011 says the supervisor reran every gate against the worktree on disk and calls the session summary discarded self-report. That wording is worth borrowing for your own pipeline. A report is a claim about a run. It does not get promoted to evidence just because it is formatted cleanly.
The artifact is where the claim meets reality
The artifact lets you test the claim again, using the command that is supposed to support it.
That is the transferable rule. Check out the exact worktree or commit. Read the actual diff. Run the declared command. Preserve its output where another person can retrieve it. When the behavior has a failure mode, force that path too. Each step turns a statement into something a later person can challenge.
A report can still be retained. It can name the command, commit, environment, artifact digest, and outcome so you know what to reproduce. Its proper job is a map, not a substitute for the terrain.
This matters most for concurrency, timing, process cleanup, or shared state. Those failures can disappear in a clean session and return when the same artifact is run again. It also matters for ordinary build work: a report that describes a test without preserving the exact subject has asked you to trust memory at the moment you need measurement.
Approval does not create an observation
Two people approving one report have read one observation twice. They have not independently verified the artifact.
That is not a criticism of review. Review can catch a bad instruction, a mistaken assumption, or a missing case. But no amount of careful reading can make an unrerun report answer a command it never ran.
The incident narrative belongs in How a 12% Flaky Test Suite Got Approved Twice: the suite was reported stable, approved by two reviewers, and then a rerun against disk found two failures in sixteen full-suite runs. This note keeps the principle: approvals are readers; the artifact is the thing that can be observed again.
Use that distinction when someone proposes “another sign-off” as the fix for uncertain evidence. Ask what new observation the sign-off adds. If the answer is none, schedule a rerun or a fresh measurement instead.
Make your pipeline answerable
You do not need a governance kernel to make reports earn their authority. Give every important result a route back to its artifact.
- Record the immutable subject: a commit, build digest, or other exact artifact identity.
- Record the exact command and the inputs it used, not a paraphrase such as “tests passed.”
- Make the artifact available to the person who approves the result.
- Rerun blocking checks outside the producing session before using them to merge, deploy, or certify a control.
- Force a relevant refusal or negative path and confirm the check can distinguish it from success.
- Keep stderr and other diagnostics that explain a failure instead of retaining only a green summary.
These are not ceremonies to add after the real work. They are how you discover whether the work was real before your pipeline gives it teeth.
Questions people actually ask
These questions keep a report in its proper role: a map back to a checkable artifact.
Why is a session report not evidence by itself?
A session report is self-description from the run that produced it. Ranex treats the worktree and a rerun of the recorded command as evidence that can contradict that description.
What should a reviewer verify instead of trusting a report?
A reviewer should inspect the artifact on disk, rerun the exact gate command, force the relevant failure path, and preserve the result outside the producing session.
Can two approvals make one report independently verified?
No. Ranex learned that two reviewers can approve the same self-description; approval adds readers, not a new observation of the artifact.
What did SLICE-011 prove about reports and artifacts?
SLICE-011 proved that rerunning every gate against the worktree on disk found two failures in sixteen full-suite runs after the suite had been reported stable and approved.
Rerun before you rely on it
Pick one report that can block a release or approve an agent change. Locate the artifact it claims to describe. Rerun its exact command against that artifact. Then make the command fail on purpose and verify that the report would not hide it.
Ranex is pre-release. The lesson does not depend on waiting for it: a report is a useful map, and the artifact is still the ground.
Try it. Break it. Tell me what broke.
Disclosure: this post was drafted with AI assistance. Every factual claim traces to the repository’s README or slice records — the same fact gate the product enforces on code. It ships only after Anthony’s own review.
Top comments (0)