DEV Community

Adela for BetterToken.ai

Posted on Originally published at bettertoken.ai

Claude Opus 5.5 Code Audit Checklist: Verify Every Finding Before Merge

You let Claude Opus 5.5 audit a repository for several hours. It returns dozens of “high-risk” findings and may even prepare a large patch. The hard part is no longer getting suggestions; it is deciding which findings are real, which changes respect the system’s architecture, and which fixes are safe to merge.

This workflow treats model output as audit hypotheses that require evidence, not as conclusions. You establish a reproducible baseline, separate discovery from editing, and make every accepted finding pass reproduction, risk, architecture, fix, and regression gates.

Use Opus 5.5 to widen the audit, not to approve the merge

Opus 5.5 is a reasonable choice for broad, long-running audits, but it is not an independent reviewer. Anthropic’s September 22, 2026 announcement specifically describes codebase-wide migrations and audits as a strength and reports results from internal tests and early testers. Those vendor and tester results do not prove the same outcome on your repository. Read Anthropic’s Opus 5.5 announcement.

Kent C. Dodds also published a broad audit prompt covering security, performance, accessibility, maintainability, scalability, architecture, documentation, testing, and automation. He said Opus 5.5 found a significant security issue that other models missed, but the post does not disclose the flaw, reproduction steps, or a controlled comparison. It supports trying the workflow, not waiving verification. Read the public report.

Your goal should therefore be: produce as many reproducible, graded, independently reviewable findings as possible—not simply maximize the number of warnings.

Define success as commands before the audit starts

Without a clean baseline, you cannot tell whether a later failure was pre-existing or introduced by the model. Before the model scans the full repository, record the current commit, environment, dependency versions, and the exit code from every baseline command.

Replace the placeholders with the commands your project actually uses. Mark a gate “not applicable” rather than inventing one.

git status --short
<install-command>
<lint-command>
<type-check-command>
<unit-test-command>
<integration-test-command>
<build-command>
Enter fullscreen mode Exit fullscreen mode

Record at least:

  • the commit SHA, runtime, package manager, and material dependency versions;
  • the exact command, working directory, exit code, and failure summary for each check;
  • known failures, flaky tests, and temporary exemptions;
  • directories that are in scope and files the model must not edit, such as generated code, historical migrations, lockfiles, or vendored sources;
  • critical paths such as authentication, authorization, billing, data migration, and external API contracts.

If the baseline already fails, decide whether to fix, isolate, or register the failure before the audit. Do not allow an old failure to be presented as a new discovery.

Ask for an audit first and no code changes

Discovery and remediation must be separate phases. If the model edits while it investigates, a new failure may come from the original code, the first patch, or a later patch that interacts with it.

Use this as a first-pass prompt and add repository-specific commands and boundaries:

Perform a long-running code audit of this repository. In this phase, investigate and report only; do not modify files.

Scope: <directories, services, languages, critical business flows>
Exclusions: <generated files, third-party code, historical migrations, inaccessible systems>
Baseline: <commands already run, exit codes, known failures>

Audit areas:
1. Security and authorization boundaries
2. Correctness, concurrency, transactions, and error handling
3. Performance and resource use
4. Accessibility where applicable
5. Maintainability and scalability
6. Architecture and module boundaries
7. Documentation, testing, and automation gaps

For every finding, provide:
- a unique ID and short title
- severity and impact rationale
- affected files, symbols, and exact lines
- trigger, expected behavior, and actual behavior
- a reproducible command or minimal test
- output summary and exit code
- a plausible false-positive explanation
- the smallest remediation direction
- verification commands required after a fix
- confidence: high, medium, or low

Rules:
- Mark any command you did not run as NOT RUN.
- If you cannot reproduce a finding, label it UNVERIFIED, not confirmed.
- Never delete, skip, or weaken tests to make a result pass.
- Stop and list missing credentials, services, or dependencies.
- Update the audit status table at the end of each phase before continuing.
Enter fullscreen mode Exit fullscreen mode

The prompt does not guarantee compliance. You still need to inspect terminal records, file diffs, and test output. Its purpose is to make acceptance fields explicit so that “this looks risky” cannot move directly into the fix queue.

Split the long task into four controlled phases

A long autonomous run should not mean unlimited scope or permissions. Pause at each phase boundary and review the evidence before allowing the model to continue.

Phase 1: Build a system map

The model reads code, configuration, tests, and architecture documents. It identifies entry points, trust boundaries, data flows, external dependencies, and high-impact paths. It should not chase a target number of bugs or change files.

Phase 2: Produce candidate findings

The model groups candidate issues by audit area and points each one to concrete code. Advice without a location or trigger belongs in an “improvement ideas” list, not the defect count.

Phase 3: Reproduce one finding at a time

Start with high-impact findings that are cheap to test. Add a minimal test or run an existing check, retain the raw output, and record the environment. Do not bundle several hypotheses into one experiment.

Phase 4: Prepare a remediation plan

Only reproduced findings enter the fix plan. The plan must state the smallest change, compatibility impact, migration risk, rollback path, and required gates. Architecture disputes go to the code owner before implementation.

Track state in audit-plan.md or a ticket table:

ID Status Risk Reproduction evidence Architecture decision Fix branch Approver
AUD-001 Awaiting reproduction High None yet Not reviewed — —

Allow only an explicit progression: candidate → awaiting reproduction → reproduced → architecture reviewed → fixed → accepted. A finding that cannot be reproduced returns to candidate status or closes; confident wording is not a reason to advance it.

Risk gate: keep severity separate from confidence

Severity measures impact; confidence measures evidence quality. A possible authorization bypass can be high severity and low confidence. A reproducible typo in a log message can be low severity and high confidence.

Severity Use when Minimum evidence before remediation
Critical Broad privilege escalation, sensitive-data exposure, irreversible corruption, or core-service failure is plausible Controlled reproduction, clear blast radius, immediate owner review
High A critical business flow is affected or realistic input triggers the failure reliably Minimal reproduction, failing test or command output, code-owner confirmation
Medium Impact is bounded, a workaround exists, or unusual conditions are required Repeatable evidence, impact note, prioritization decision
Low Local quality, documentation, maintainability, or non-critical performance issue Concrete code evidence and a case that benefit exceeds regression risk

Do not let the model assign business impact alone. It can trace code paths, but data sensitivity, customer commitments, acceptable downtime, and compatibility policy usually require an owner’s judgment.

Reproduction gate: turn each finding into a failing check

Suspicious code is not enough; acceptance needs a check that fails before the fix and passes after it. For every finding, answer:

  1. Which commit and environment exhibit the behavior?
  2. What is the smallest triggering input?
  3. Is expected behavior defined by a test, specification, interface contract, or business rule?
  4. What exactly happened, and where is the raw output?
  5. Why did existing tests miss it?
  6. Could a valid design choice or environment difference explain it?

The best evidence is a minimal regression test. If automation is impractical, provide deterministic manual steps, expected observations, and cleanup instructions. Reproduce security findings only in systems you own or are authorized to test, preferably local, isolated, or pre-production environments.

When the model says it ran a command, look for the full command, working directory, exit code, and relevant output. A prose summary by itself is not execution evidence.

Architecture gate: learn why the old code exists before changing it

A cleaner-looking implementation can break compatibility, deployment ordering, or an intentional boundary. Before accepting an architecture finding or broad refactor, review ADRs, design documents, API contracts, migration constraints, and the relevant history.

When the repository has usable Git history, the model can help inspect it:

git log -- <path>
git blame -L <start>,<end> <file>
git show <commit> -- <path>
Enter fullscreen mode Exit fullscreen mode

Then require answers to these questions:

  • What constraint is the current design preserving?
  • Which callers, data formats, or deployment steps depend on it?
  • Is the proposed change correcting a defect or changing product behavior?
  • Can a smaller local change solve the reproduced issue?
  • Does rollback require restoring code, configuration, or data?

History is evidence, not a perfect record of intent. If no rationale is available, mark the decision “architecture intent unknown” and ask a maintainer. Do not let the model silently redraw module boundaries.

Fix gate: one reproduced finding per small patch

Do not accept a mega-patch that claims to fix a dozen unrelated findings. A safer pattern is one reproduced issue per branch or independently reviewable commit: add the failing regression test first, then make the smallest change that passes it.

Each patch should clear these gates:

Gate Requirement If it fails
Scope The diff addresses only the approved finding Split unrelated changes and review again
Regression test Fails before the fix and passes after it Repair the test or reconsider the finding
Static checks Format, lint, and type checks pass Do not hide new errors with broad exemptions
Project tests Relevant unit, integration, and build checks pass Investigate the first new failure before stacking patches
Architecture The owner confirms boundaries and compatibility Reduce scope or open a design review
Human diff review Error handling, permissions, data changes, and deletions are inspected Explain every suspicious change

Explicitly reject fake passes: deleting assertions, skipping tests, swallowing exceptions, weakening validation, increasing retries to hide races, or performing a large refactor that makes the original defect hard to trace.

Regression gate: run project-owned checks, not only model-selected tests

The model will naturally focus on checks closest to its change; merge acceptance needs an independent project-level gate. Use existing repository scripts or CI definitions for the final run rather than letting the model substitute an easier command.

Compare:

  • the full baseline before the audit with the post-fix result;
  • whether the new regression test fails on the unpatched revision;
  • lockfiles, database migrations, public APIs, and configuration defaults for unintended changes;
  • performance claims using the same environment and input before and after;
  • security fixes for new authorization or availability regressions;
  • documentation and operational steps against the final behavior.

For flaky tests, do not rerun until green. Record the failure pattern, isolate the source of nondeterminism, and determine whether the patch made instability worse.

Reject the audit result when any of these conditions appear

Do not move a finding toward merge when:

  • it lacks an exact code location, trigger, or inspectable evidence;
  • an unexecuted command is reported as passing;
  • severity is just an adjective with no impact path;
  • the patch exceeds approved scope or casually redesigns architecture;
  • tests are deleted, skipped, weakened, or errors are swallowed;
  • the change conflicts with an ADR, contract, or migration policy without owner approval;
  • you receive only a final summary, with no reviewable commands or diff;
  • a security claim is supported only by another model agreeing with it.

A second model can search for counterexamples, but model consensus is not independent evidence. Independent validation comes from tests, runtime output, history, specifications, and accountable human judgment.

Final pre-merge acceptance checklist

Treat the model-assisted fix as mergeable only when all ten items are true:

  • [ ] Scope, exclusions, and the baseline commit are frozen.
  • [ ] Baseline commands and exit codes are retained.
  • [ ] Every accepted finding has a unique ID and exact code location.
  • [ ] Severity and confidence are recorded separately.
  • [ ] The defect is reproduced by a test or deterministic procedure.
  • [ ] Architecture intent, compatibility, and rollback are reviewed.
  • [ ] Each finding maps to a small, reviewable patch.
  • [ ] The regression test fails before the fix and passes after it.
  • [ ] Full project gates run through existing scripts or CI.
  • [ ] A code owner reviews the final diff and explicitly approves it.

The available public reports make Opus 5.5 a credible candidate for broad, long-running audits and suggest it can surface issues missed elsewhere. They do not remove the need for verification. The minimum next step is not to let it edit the repository: save a clean baseline and issue the first “audit only, no modifications” task.


Originally published on the BetterToken blog.

BetterToken provides pay-as-you-go access to AI model APIs through
OpenAI-compatible and Anthropic-compatible endpoints — useful if you are wiring
Claude Code, Codex, or your own tooling to a custom base URL.
See the docs to get started.

Top comments (0)