DEV Community: Tam Nguyenvan

Five prompts that make a Codex bug fix easier to verify

Tam Nguyenvan — Sat, 26 Sep 2026 04:35:13 +0000

A Codex task can end with a convincing explanation while leaving the original
bug untested. The useful question at handoff is simple: what observation shows
that the reported behavior changed?

Here are five prompts you can adapt to an existing repository. They need no
extra skill installation. I maintain the open-source
Codex How To guide, where these
recipes live alongside longer engineering workflows.

The checkout scenario below is illustrative, not a benchmark result or a claim
that these prompts reduce token usage.

1. Define acceptance around observable behavior

Imagine an empty cart can reach payment. “Fix checkout” leaves room for a
cosmetic change that never addresses the payment call.

Try:

Fix empty-cart checkout. An empty cart must show the existing validation error
and must not call the payment client. Preserve the API response schema.
Run the relevant regression and repository-required checks. Report any
verification you cannot perform.

This makes two outcomes visible: the error and the absence of the payment
side effect. A test that checks only the error message covers half the contract.

2. Provide a lead, then let evidence guide exploration

Start at src/cart/checkout.ts and its callers. Inspect the existing payment
client tests for conventions. Expand your search when a dependency or call
path requires it.

Replace those example paths with real ones. A useful starting point can come
from a stack trace or a failing command. It should not become a rule that
prevents discovering the defect elsewhere.

3. Capture the failure before changing the implementation

Create the smallest regression that demonstrates this empty-cart failure.
Run it before editing the implementation. Then fix the cause and rerun the
same regression, followed by the required broader checks.

For this example, inspect whether the regression actually fails because the
payment client was called. An import error is not evidence of the checkout
bug. Likewise, a green pre-existing suite may simply lack this case.

Intermittent bugs need a different approach: instrumentation or a controlled
reproduction may be more useful than repeatedly rerunning a flaky test.

4. Review the candidate that will actually ship

Review the candidate diff for paths that still reach payment with an empty
cart, behavior changes for valid carts, and tests that pass for the wrong
reason. Give file evidence for consequential findings. After confirmed fixes,
rerun affected checks and identify the final reviewed revision.

For larger or riskier changes, a separate reviewer context can add coverage.
It also consumes resources and can miss defects. Keep review proportional to
the change; a second agent is not necessary for every edit.

5. Ask for a compact evidence handoff

Summarize the changed behavior, final revision, regression failure before the
fix, commands and results after the fix, review findings resolved, and anything
still unverified. Keep it short enough for another engineer to check.

A useful handoff distinguishes local tests from integration checks. For
example, a mocked payment client can establish that local code did not invoke
it; that alone does not verify a deployed payment service.

Try one change to your workflow

Use the reproduction prompt on your next bounded bug. Record whether it
reveals missing coverage or merely repeats what you already do. Compare
accepted outcomes and rework before interpreting time or token differences.

The full set of ten tips
also covers stalled retries, delegation, steering, and session handoffs.
The repository includes a disposable playground if you prefer a teaching
fixture over production code.

Which instruction has changed the outcome of a real task for you? A sanitized
example where one failed or added overhead is useful too.

When should Codex use multiple agents? A benchmark, not a slogan

Tam Nguyenvan — Wed, 26 Aug 2026 03:29:19 +0000

More agents do not automatically produce better engineering. They usually add
total tokens, duplicated context, handoff delay, and integration risk. Their
defensible advantages are narrower: reduced elapsed time for independent work,
isolated investigation, or specialist evidence that one agent might omit.

The useful question is therefore not “Can this task use subagents?” It is:

Does this task contain independent, bounded work whose value exceeds the
coordination cost?

Codex How To now includes a dependency-free benchmark for testing that question
instead of answering it from intuition.

Disclosure: I maintain
Codex How To, the independent
open-source project containing the benchmark, evaluator, and measurements used
here.

The minimum decision rule

Use one agent when the change is small, the interface is unsettled, or several
steps must edit the same central files. Consider bounded orchestration only when
all of these are true:

The task has at least two genuine ownership surfaces.
Each writer can own exclusive paths.
The interface between those paths is frozen before implementation.
The controller retains integration, system checks, and final review.
Every worker returns concise evidence rather than a narrative transcript.
One external acceptance bar can evaluate every execution method.

flowchart TD
    A["One task contract"] --> B{"Independent write surfaces?"}
    B -- "No" --> C["One agent or sequential work"]
    B -- "Yes" --> D{"Frozen interface and exclusive paths?"}
    D -- "No" --> C
    D -- "Yes" --> E["Bounded workers"]
    E --> F["Controller integrates and evaluates"]
    F --> G{"Coverage or elapsed-time value exceeds coordination cost?"}
    G -- "Unproven" --> H["Keep measuring"]
    G -- "Repeated evidence" --> I["Adopt for this task class"]

Job titles are not ownership boundaries. “Backend agent,” “test agent,” and
“review agent” may still collide on the same files or execute dependent stages.
A useful boundary is concrete: one writer owns incident/**, another owns
web/**, and neither changes the shared contract.

The benchmark task

The
incident-response benchmark
asks Codex to build a small but cross-surface application with no third-party
dependencies:

thread-safe JSON persistence with atomic replacement;
validation, optimistic concurrency, and status transitions;
an HTTP API and traversal-safe static serving;
a responsive, accessible browser client;
focused tests, integration checks, final review, and an evidence handoff.

The fixture has two deliberate implementation surfaces:

incident/**  # persistence, validation, HTTP adapter
web/**       # HTML, CSS, browser behavior

Both the single-agent and orchestrated conditions start from the same commit,
task text, supplied tests, and frozen contract. The orchestrated condition may
use at most two implementation workers. The controller owns evaluation and the
final receipt.

Both conditions face the same external evaluator. It checks fixture integrity,
candidate scope, concurrent persistence, strict atomic writes, validation and
error mapping, live HTTP behavior, path traversal, static browser requirements,
and required handoff topics.

What the first run actually showed

The first smoke run qualified the benchmark harness:

Measure	Single agent	Controller + two workers
Accepted after review	Yes	Yes
Supplied tests	4/4	4/4
External evaluator groups	6/6	6/6
Edit conflicts	0	0
Integration rework	0	0
Internal review corrections	1	0
Browser interaction exercised by candidate	No	Yes
Aggregate tokens	Unavailable	Unavailable
Comparable elapsed time	No	No

The orchestrated candidate returned live browser evidence covering creation,
filtering, state transitions, validation feedback, console errors, and a mobile
viewport. The single agent found and corrected a conflict-message behavior in
final review. An external controller later exercised its live HTTP behavior.

That result establishes feasibility, not superiority. The runs overlapped, the
environment did not expose aggregate controller-plus-worker tokens, and there
was only one candidate per method. It would be invalid to claim that
orchestration was faster, cheaper, or generally more reliable.

Read the complete
smoke receipt
before interpreting the table.

A controller contract that limits fan-out

A large goal should not become permission to create an agent for every noun in
the prompt. Freeze the contract and state the maximum useful topology:

Implement the incident-response task end to end.

Before delegation:
- freeze the HTTP and data contract;
- confirm that incident/** and web/** are independent write surfaces;
- keep evaluator and handoff ownership with the controller.

Delegation budget:
- at most two implementation workers;
- backend owns incident/** only;
- frontend owns web/** only;
- workers must not modify TASK.md, tests, evaluator files, or each other's paths.

Each worker returns:
- files changed;
- focused checks and outcomes;
- observable behavior exercised;
- failures, retries, and anything unverified.

The controller then runs supplied and external checks, exercises integration,
reviews the complete diff, fixes verified findings, and writes one receipt.

This contract does not force delegation. If the controller discovers that the
interface is unstable or the surfaces are coupled, the correct choice is a
single agent or sequential pipeline.

Measure quality before speed

For each candidate, record raw facts before interpreting them:

Dimension	Required evidence
Acceptance	Same evaluator and quality gates for every method
Coverage	Behavior exercised, failures found, and unverified behavior
Total cost	Controller plus every worker, including failed attempts
Elapsed time	Sequential, isolated runs on comparable machine load
Coordination	Worker count, handoff delay, duplicate investigation
Integration	Conflicts, contract mismatches, and rework events
Human effort	Corrections, retries, approvals, and manual intervention

Do not substitute controller-only telemetry for total tokens. Do not remove
failed candidates. Do not compare overlapping runs as if they were isolated.
Write unavailable when the execution surface does not expose a metric.

Use one
orchestration receipt
per candidate. Run at least three fresh sequential pairs and alternate which
method runs first. Compare acceptance and evidence completeness before elapsed
time or tokens.

Where orchestration usually loses

Prefer one agent or a sequential workflow when:

one clear agent can hold the relevant context;
workers would edit the same files;
one subtask depends on another's unresolved design;
the task is primarily a single debugging chain;
the evaluator cannot attribute failures to a candidate;
per-agent cost is invisible and cost is the decision criterion; or
the only justification is that parallelism is available.

For small bounded fixes, workflow guidance itself may cost more context than it
saves. Codex How To's earlier measurements found that the no-skill control was
the cheapest successful variant on a small backend defect, while a lean
engineering loop was cheapest on a medium browser-game task. Task class matters.

Where orchestration may earn its cost

Good candidates include:

independent modules with frozen interfaces;
read-only review through independent security, reliability, and test lenses;
noisy investigation that can be summarized before implementation;
frontend and backend implementation after the API contract is fixed; and
tasks where one specialist can add independently verifiable runtime evidence.

The benefit may be coverage rather than speed. If a specialist catches a real
defect or produces missing browser, deployment, or security evidence, higher
token use may still be rational. That is a quality decision, not a token-saving
claim.

Replicate or falsify it

The next useful result is not another success screenshot. It is a controlled
pair that changes the recommendation:

Create identical disposable starting copies.
Keep model, reasoning effort, permissions, tools, task, time limit, and evaluator fixed.
Run one single-agent and one bounded-orchestration candidate sequentially.
Alternate order across at least three pairs.
Preserve failures and total controller-plus-worker telemetry.
Publish sanitized receipts and limitations.

Submit a result through the
replication issue or add a
tested benchmark edition through a focused pull request. Negative and neutral
results are explicitly useful.

Codex How To is an independent community project. Official OpenAI documentation
remains authoritative for current product behavior.