We Built PR Auto-Review as an Auditable AI Agent, Not a Faster Code Reviewer

White Fang — Mon, 01 Jun 2026 14:51:42 +0000

Most AI code review demos focus on speed.

How fast can the agent read a pull request? How many comments can it generate? How many small issues can it catch before a human reviewer opens the diff?

Those questions matter. But for enterprise engineering teams, they are not the hardest questions.

The harder question is this:

If an AI agent reviews code and recommends an action, can the team reconstruct what it saw, why it made that recommendation, and when it decided to hand control back to a human?

That is the part we have been working on at Numbers Protocol / Omni AI.

Our first concrete demo is PR Auto-Review. The agent reads a pull request, evaluates the change, records a verdict, and stores the reasoning in an append-only audit trail. In our current internal test suite, the reviewer records 13 verdict types. The 7 tests in the merger module all pass in 0.38 seconds locally.

Those are small numbers. That is intentional.

The point is not to claim that the agent has solved code review. The point is to show the shape of an auditable AI workflow.

Why Logging Is Becoming a Product Requirement
This is not only a software architecture preference.

The EU AI Act already treats logging and traceability as part of the control design for high-risk AI systems. The European Commission describes high-risk systems as subject to obligations including logging of activity, technical documentation, human oversight, robustness, cybersecurity, and accuracy. The AI Act Service Desk's Article 12 summary says high-risk AI systems must technically allow automatic recording of events over the lifetime of the system, and that those logs support traceability, post-market monitoring, and oversight.

The timeline is also worth treating carefully. The AI Act becomes broadly applicable on 2 August 2026, while later high-risk AI implementation dates depend on system category and the EU's simplification package. I would not use that date as a generic panic deadline for every product. But I would use it as a practical signal: auditability is moving from a "nice to have" demo feature into the vocabulary of procurement, legal, and engineering review.

That matters for developers.

If an agent takes action inside a workflow, the record behind that action has to be designed before the workflow becomes business-critical. It is much harder to bolt on an audit trail after teams already depend on the automation.

The Review Is Only Half the Product
When an AI reviewer comments on a pull request, the visible output is the comment.

But in a business workflow, the visible comment is only half the product. The other half is the record behind it:

What files changed?
What risk did the agent identify?
Which rule or policy did it apply?
Did it recommend merge, block, request changes, or human review?
Was the final action automated or handed back to a person?
Without that record, the team can still move quickly, but it cannot explain itself later.

That becomes a problem the moment AI agents move from experiments into operational systems. Engineering managers need to know why a change was approved. Security teams need to know why a risk was ignored or escalated. Compliance teams need evidence that automation is not bypassing the normal control process.

In other words, the value is not only automation.

The value is automation that can be inspected.

Human Handoff Is a Feature
One of the easiest traps in agent design is to over-optimize for autonomy.

It is tempting to describe every workflow as if the best agent is the one that never stops. In real engineering systems, that is rarely true.

Sometimes the best decision an agent can make is to stop and ask for a human.

In our PR Auto-Review demo, that means the agent can block auto-merge when the risk is unclear, when tests are missing, or when the change touches sensitive areas. The handoff is not a failure of the agent. It is part of the control design.

A good enterprise AI agent should not only know how to act. It should know when not to act.

What We Learned
At first, we assumed that detailed audit logging would slow the workflow down.

That felt intuitive. More records, more structure, more metadata. Surely that creates overhead.

But after building the first version, the more interesting effect was different. The record reduced repeated discussion.

When a pull request was flagged, the team could inspect the reason instead of reconstructing the context from memory. When a decision needed to be reviewed later, the verdict and evidence were already there.

The audit trail did not make the agent faster in the narrow sense. It made the next human decision easier.

That distinction matters.

Why This Matters for AI Agents
AI agents are moving from chat interfaces into operational systems: code review, support triage, sales operations, finance workflows, and internal approvals.

Once agents touch those workflows, the product question changes.

It is no longer enough to ask:

Can the model produce a useful answer?

We also have to ask:

Can the organization explain what happened after the agent took action?

That is the problem space we call TAEA: Transparent, Auditable, Explainable AI.

For us, TAEA is not a slogan. It is a design constraint:

The agent must leave a decision record.
The system must preserve the evidence behind that decision.
The workflow must support human handoff.
The team must be able to inspect the process later.
The PR Auto-Review demo is a small starting point, but it gives us a concrete surface to test these ideas.

What We Are Looking For
We are exploring how this framing resonates with teams that are already experimenting with AI agents in engineering workflows.

If your team has tried AI code review, CI automation, or internal agents, I would be interested in one question:

Which agent decisions do you actually need logged?

Not every decision needs a heavy audit trail. Some actions are low risk. Some records are noise. Some workflows should stay lightweight.

The useful line is the one we want to find.

If you have seen this problem in practice, I would appreciate your perspective.

Links
Numbers Protocol: https://www.numbersprotocol.io/
Omni / TAEA context: https://www.numbersprotocol.io/solutions/auditable-ai
European Commission — AI Act overview: https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
AI Act Service Desk — Article 12 record-keeping: https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-12

DEV Community: White Fang

We Built PR Auto-Review as an Auditable AI Agent, Not a Faster Code Reviewer