DEV Community: Jeff

The agent filled the missing config with a default and the misconfiguration went quiet

Jeff — Sun, 25 Oct 2026 00:27:26 +0000

A report failed locally because REGION was unset: the client tried to build a target endpoint from an empty string and threw a parse error. The agent traced the line, changed it to read process.env.REGION ?? "us-east-1", re-ran the service, watched it boot, and closed the change. The parse error was gone. What the agent never noticed was that REGION was a required deployment value, not an optional preference, and the hardcoded fallback turned a missing configuration into a service that ran against the wrong region without a single loud error.

The agent had removed the crash. It had also removed the signal that told the operator a required setting was missing.

Why a missing value becomes a fallback

Given a local run that refuses to start, the agent's shortest path is to make it run here. A nullish-coalesced default removes the traceback in one edit, the boot succeeds, and the loop reads that as solved. Search tools reward the local symptom: the empty string caused the parse error, so filling the empty string feels like the fix.

What that reasoning skips is whether the value is required. A required input has no legitimate default; when it is absent, the correct behavior is a startup error that names the missing setting. An optional input has a real intended default that ships regardless of environment. The agent collapses both cases into the same ?? operator because the task frame only says "make it not crash," never "is this value allowed to stand alone."

The local environment hides the distinction. On the developer's machine the variable may also be unset, so the fallback looks normal; there is no staging complaint to challenge it, and the test suite never asserts that a required value reaches the client.

What a silent default leaves behind

The wrong target runs in silence. The service boots against the fallback region instead of failing, so requests flow to an endpoint nobody intended for that deployment.
Required input reads as optional. A reader sees a ?? default and assumes the setting can be omitted, when in fact omitting it is a configuration mistake.
Environment drift becomes invisible. Local, staging, and production all start clean even when only one of them set the value, so the missing setting never reaches a deploy checklist.
Monitors watch the wrong signal. Health checks see a booting process and report healthy; nothing alerts on traffic bound for the fallback target because the process itself did not fail.
The commit records the workaround, not the requirement. A message like "default region" leaves no trace that this value must be provided, so the next pass cannot recover the intended contract.

The tell is a completion summary that says the service now boots, with no mention of which environment supplies the value or what happens when it is omitted.

Separate required from optional before you reach for a default

Before coalescing any configuration, decide which kind of input it is:

Fail fast on required values. Parse the config at startup and throw a clear error listing every missing required setting; a required region, tenant, or endpoint should refuse to boot rather than substitute a guess.
Reserve a real default for optional settings. Coalescing is appropriate only when the default is an intended behavior that holds with or without configuration, such as a request timeout or a page size, not a deployment identifier.
Name the requirement in code and docs. A required input should be declared as such in the config schema, and its absence should produce a message that tells the operator exactly which variable to set.
Assert the value reaches the client. A boot test that runs with the required value present does not cover the missing case; add the negative path where the process must refuse to start.
Log the source of the effective setting. A startup line that reports region = configured vs region = fallback makes a default's presence visible instead of blending into a normal boot.

The same separation belongs in a declared contract: required fields are rejected when missing, and optional fields may carry a documented default. Keeping that distinction machine-checked, instead of leaving it to a comment, is the reason I build local-first tooling like Powerduck around a validated spec rather than prose.

Booting is not the same as being configured

A fallback default removes the immediate crash the agent was asked to fix. It does not verify that the deployment supplied the value that was actually required. Until missing required input fails loudly, a clean boot means the process started, not that it started against the right target.

The agent fixed the failing call site and left the same bug on its sibling

Jeff — Sat, 24 Oct 2026 23:28:59 +0000

A user reported a crash on the export route: a null delivery date reached the date formatter and threw. The agent added a null check at that one call site, fed the reported payload into the test, saw it pass, and closed the ticket cleanly. Three days later the report route threw the exact same exception. That route carried a near-identical copied block, and the guard had never been placed there, because the patch targeted the occurrence the user named rather than the pattern that produced it.

The agent had fixed the line in the stack trace. It had not fixed the condition that repeated on the next copy.

Why the reported site becomes the only target

Given a reported error, the agent's shortest path is to locate the line in the trace, patch it, and re-run the failing case. Search tools reward a literal match on the error message or the stack frame, so the loop converges on that one call site instead of asking what broader invariant the call site violated.

Copy-pasted code makes this worse. A block written once and duplicated across routes keeps the same missing guard in every copy. The agent finds the copy that crashed, patches it, and never enumerates the rest, because the task frame names a symptom location, not a failure class.

The test reinforces the trap. The regression fixture reproduces the reported payload on the reported route; it passes; the suite reports green. There is no assertion that the sibling construction behaves the same way, so green reads as solved.

What a local fix leaves behind

Sibling copies keep the defect. Every duplicated block that builds the same unguarded value still throws when its own edge case arrives.
The guard special-cases one input. Checking a single call site protects that route, but leaves the underlying value able to reach a formatter null on any other path.
Tests cover only the reported case. The fixture mirrors the user's payload; the untouched copies exercise a shape the test never models.
The issue returns under a new ticket. The same exception filed from a different route looks like a fresh report instead of a known, ungeneralized fix.
The root cause stays unnamed. A commit that describes the one route never records the invariant, so the next agent has to rediscover the pattern from scratch.

The tell is a completion summary that names the route or function the user quoted, with no mention of where else that construct appears.

Generalize the pattern before you close

A patch is complete when the condition cannot reach any call site, not when the reported one passes:

Find every site that builds the same value. Search for the pattern that produces the unguarded input, not just the literal line from the trace; copied routes and shared helpers are the likely homes of the next crash.
Name the invariant instead of the location. State what must hold before the formatter runs, then enforce that invariant at the boundary where the value is built, so every consumer inherits the guard.
Replicate the failing shape on the siblings. Run the null or edge input through each duplicated path before declaring the fix done; a guard that exists on one copy does not protect the others.
Let the commit describe the failure class. A message that says only "fix export date crash" leaves the recurrence undocumented; naming the invariant gives the next pass a place to start.
Ask who else constructs this value. Before closing, enumerate callers and entry points that reach the same code, and treat an unpatched sibling as open work rather than out of scope.

An occurrence and a pattern are different sizes of problem

Fixing the reported call site removes the crash the user saw. It does not remove the condition that crashes elsewhere. Until every copy that builds the same value is checked, a green regression test means the reported case passed, not that the defect was gone.

The type check passed, so the agent shipped a value shape the runtime never produced

Jeff — Sat, 24 Oct 2026 22:30:34 +0000

A feature took a response from an internal service and mapped it into a view model for the UI. The agent added an interface for the raw payload, an interface for the model, and a mapping function between them. It ran the type checker, saw zero errors, and marked the task done. On staging the screen rendered empty: the service returns { "data": { "items": [] } }, while the declared type exposed items at the top level. The compiler never complained, because the loose index signature and optional fields on both sides had made the mismatch invisible.

The agent had checked the code against the type it wrote, not against the response that actually arrived. The green check was real evidence that the mapping compiles; it was not evidence that the producer's shape matched it.

Why a green type check reads as done

Types are compile-time promises that disappear before the code runs. The compiler compares what the author declared against what the code references; it never reaches out to the service, opens a real response, or counts the keys that come back. When the agent pairs this with a loose record type or optional fields, a missing or nested key stops producing an error without stopping the mismatch.

Optional fields are the quietest culprit. Declaring a key optional says that absence is allowed, so the producer omitting it or nesting it deeper becomes legal at compile time. The same pattern on the mapping output means a name that renders blank never trips the check; it only shows up as an empty cell in the interface.

The loop the agent runs reinforces the habit. It writes the type, writes the code to match, re-runs the checker, and stops on green. There is no second step that compares the written type to a captured real payload, so the loop never teaches the agent that compile silence and runtime correctness are different claims.

What ships when types stand in for checks

Undefined fields render blank. A missing nested key reaches the component as undefined; nothing throws, so the failure looks like empty data rather than a broken mapping.
Loose signatures paper over the gap. An index signature or permissive optional field lets a producer shape the compiler never modeled pass right through.
Green CI hides the drift. The test suite runs the mapping against a fixture the agent itself wrote, which uses the same wrong shape; real producer drift never enters the run.
The pattern repeats across calls. If the agent concludes a passing check means done, every subsequent integration stops at tsc instead of observing the response.
The failure surfaces far from the source. A payload mismatch written in the client appears on a user screen long after the agent closed the task, when the trace back to the mapping has gone cold.

The tell is a completion message that cites the type check rather than an observed sample. "It compiles" answers the compiler's question; it does not answer whether the outside world returned the expected value.

Check the runtime shape against a real sample

Treat the written type as a hint, not the contract. Verify the boundary against data that actually moved:

Capture one real response and assert on it. Run the mapping against a captured sample payload with the real wrapper, not a hand-built fixture; check that the expected keys exist at the expected depth and are not empty.
Put a runtime schema on the boundary. A schema parser such as Zod validates the actual incoming object and rejects a missing or nested key at runtime, which a compile-time type cannot.
Drop the loose index signature in fresh code. Explicit required fields force a mismatch to surface instead of being absorbed by a catch-all record.
Let the error name the missing key. A runtime validation failure reports the exact path that was absent, instead of leaving the UI to render a blank the user cannot diagnose.
Compare the live response to the documented request. When the agent drives an outbound call, the response shape has to be checked against the payload that really came back; we keep that step local in Powerduck, re-running the documented request against the live endpoint and reading the actual body instead of trusting a handwritten type that drifted.

A compiler answers the code's question, not the world's

Type checking verifies that the code you wrote agrees with itself. It cannot verify that the service on the other side agreed back. Until a runtime check or a real sample proves the shape arrived, a green check means the code compiled, not that the feature worked.

The agent retried the whole task after one failed tool call and duplicated the side effects

Jeff — Sat, 24 Oct 2026 21:27:28 +0000

A recurring job fetched an unpaid order, charged the card through the gateway, then sent a receipt email. On one run the payment call timed out at the network edge. The wrapper caught the exception, waited a moment, and retried the job from the first line. This time the charge went through, the receipt sent, and the run reported success. The customer's card, however, had already been charged once, and the inbox held two receipts.

The first attempt had gotten far enough for the gateway to accept the charge before the response was lost. The retry replayed every completed step instead of resuming after the failed one, so a transient timeout that never reached the server became two real side effects.

Why a full-task retry looks like resilience

A loop that retries on exception reads safe at a glance: it has a backoff, a bounded attempt count, and a final success status. The run log shows the job completing on attempt two, and nothing in that log marks which steps had already landed on attempt one. The retry wrapper itself has no view into side effects that happened before the failure, so replays look indistinguishable from a clean start.

Tests reinforce the illusion. They run the job against a mock that resets between attempts, so the second pass finds no prior charge, no prior row, and no prior email. A mock world that starts fresh makes replaying completed steps harmless; the real world does not reset.

What ships when retry wraps the whole task

Duplicated side effects. A charge, an email, a row insert, or a webhook sent on attempt one repeats on attempt two; the caller pays or receives twice.
Unknown commit state after a lost response. A timeout proves the reply did not arrive, not that the server rejected the call. Replaying blindly can land a request that already succeeded.
The success status hides partial completion. Because the run eventually finishes, monitoring marks it healthy, and the duplicate surfaces only through a customer report.
Bound retries do not bound harm. Three attempts against a non-idempotent external call mean up to three side effects, not one.
Progress is lost on restart. Without a checkpoint, recovery has to decide from scratch what already ran, which is exactly what the wrapper fails to track.

The tell is a retry that wraps the entire workflow rather than the single failing call. If the task contains any step with a durable side effect, restarting from the top is not retries; it is replay.

Retry the failed step, make the call idempotent

Backoff and attempt limits are necessary but not sufficient. Separate the flow so recovery resumes where it stopped:

Split steps and retry only the failed one. Fetch, charge, and notify are distinct phases; on a timeout in charging, re-attempt the charge rather than re-fetching and re-notifying the whole job.
Give each side-effect call an idempotency key. A client-generated key sent on the charge or create request lets the gateway collapse a repeated submission into one action, so a lost response and a retry cannot double-write.
Checkpoint progress instead of replaying. Record which step completed before the failure; on recovery, read the checkpoint and continue from the next uncommitted step.
Distinguish never-reached from already-committed. On a timeout, query the upstream for the request outcome before assuming it failed; a call that landed should be observed and skipped, not retried blind.
Keep the documented request shape honest. When the agent drives an outbound call, the idempotency header and payload shape should match what the live endpoint accepts; we keep that check local in Powerduck, re-running the documented request against the live endpoint instead of trusting a handwritten example that drifted.

Resilience means resume, not restart

A retry loop over a whole task only protects steps with no consequences. For anything that writes, charges, or sends, the safe boundary sits around the individual call, with an idempotency key and a checkpoint behind it. The goal is to repeat the failed action exactly once on an already-committed world, not to rerun the journey from the beginning.

The agent renamed the field in code and never told the database, so old rows read as undefined

Jeff — Sat, 24 Oct 2026 20:27:32 +0000

The profile type used displayName. The agent was asked to make the naming consistent, so it found and replaced the field across the interface, the components, and the form handler. Tests pass, the diff looks like a tidy rename, and the PR description calls it a refactor. Then a returning user opens their profile and the name renders blank.

The rows written before the rename still store displayName. The new code reads fullName, which does not exist on those records, so the value comes back undefined and the UI falls back to a placeholder. A brand-new account works, because it writes the new key, which is exactly why the bug survived CI: every fixture in the test suite was built after the rename.

Why a rename looks safe in the diff

An agent works from the current source tree and the current type definitions. A find-replace that matches the field everywhere in code reads as mechanical and risk-free, and nothing in the file tree points at the rows already sitting in the database. The diff shows a consistent new name, not a storage history the model cannot see.

Tests reinforce the illusion. They construct fresh objects in the new shape, assert against the new field, and never load a record that predates the change. Green means the new code agrees with the new fixtures; it says nothing about records that already exist under the old key.

What ships when the persisted shape is not migrated

Historical rows read as undefined. Existing users hit the missing field while new users never do, so reproduction fails on a clean environment and the ticket reads as intermittent.
Mixed population after rollout. Old rows keep the old key, new writes use the new key, and readers that only know one side drop half the data.
Caches and queues keep the old shape. A cached payload, a message sitting in a queue, or a preference stored in the browser still carries displayName; renaming code does not rewrite what was already serialized.
Rollback restores code but not data. Once some rows were written under the new key, reverting the code leaves those records unreadable under the old name.
The regression has no test. No fixture models an old-format record, so the suite cannot fail on the cutover.

The tell is a rename PR that touches code and types but adds no migration, no backfill, and no old-shape fixture. A rename is a refactor only when every sink keeps the same shape; persisted rows, caches, queues, and stored preferences remember the old field regardless of what the source says now.

Cut the persisted shape over in phases

Consistent naming is a reasonable goal; skipping the storage cutover is the defect. Keep both keys readable until the backfill finishes:

List every sink before renaming. Database columns, cache entries, queued messages, browser storage, exported files, and API response bodies all carry the shape; write the list into the PR instead of assuming the type covers it.
Read-compatible first. For one release, read either fullName or the legacy displayName so old records still resolve while new writes go to the new key.
Backfill, then cut over. Copy old-key values onto the new key in a controlled backfill, and only then remove the legacy read path. A partial backfill in production is still mixed population, so verify row counts before dropping support.
Add an old-shape fixture to the test. Construct a record written before the rename and assert it renders correctly under the compat read; that is the case the green rename currently misses.
Treat a response field rename as a versioned contract. When the same field moves in an API body, old clients keep sending the old name; version or accept both rather than silently dropping it. Where the spec is the contract, we keep that check local in Powerduck and re-run the documented response shape against the live endpoint instead of trusting a handwritten example.

Rename the code, carry the history with it

A clean find-replace across source files genuinely improves consistency. It does not change the rows, caches, and messages already out in the world. Read the old key, write the new one, backfill the gap, and test an old-format record before the rename can ship as a refactor rather than a silent data loss.

The agent replaced the specific error with a generic one, and lost the clue that named the failure

Jeff — Sat, 24 Oct 2026 19:27:59 +0000

The payment batch used to fail loudly: "Insufficient ledger balance for settlement run #4412 against account acct-88 (120.00 available, 500.00 required)." The agent was asked to tidy up error reporting, so it consolidated the exceptions into one shared envelope and returned, "Processing failed, please retry." The happy path still works, the test still passes, and at first glance the code looks cleaner. But the message that told on-call which run, which account, and which balance was short is gone.

Why a refactor rounds the message down

When a model is told to centralize or unify error handling, it tends to move toward a short, uniform string. Specific detail reads as clutter in a shared helper: a run id, an account, a threshold, a comparison of available against required. A generic envelope feels like good craftsmanship, and the model optimizes for the cleanest-looking common path rather than preserving each cause.

Nothing in a passing test stops it. Most suites assert that an exception was raised on the failure path, not that the exception carries the field that names why. So the string can be flattened to a generic sentence and the suite stays green, because the check answered "did it fail?" and never "did it fail in a way someone can read?"

What ships when the cause field is stripped

On-call starts from zero. A generic message forces the responder to attach logs, reproduce, and hunt the run before they even know the failure was deterministic rather than transient.
Deterministic failures look retryable. Insufficient funds will never succeed on retry, but "please retry" invites the queue to requeue it, generating pointless work until a human intervenes.
Downstream matching breaks. Automation that keyed on the specific code or cause no longer recognizes the failure class, so routing and alerts fall through to a generic bucket.
Monitors lose separation. A transient lock, a validation rejection, and a balance shortage all render as the same string, so alert rules that filter by cause stop working and noise climbs.
The regression has no test. Since the test only checks that an error raised, the erosion survives review and ships as a cleanup nobody flagged.

The tell is an error string that became shorter after a "refactor," with the specific cause or identifying fields nowhere to be found. Shorter is better only when the removed words were noise; when they carried the diagnosis, the change moved evidence out of the message.

Keep the diagnostic while fixing the code

Centralizing errors is reasonable; deleting the cause is not. The clean structure separates a stable machine code from a human message, and both keep the identifying context:

Keep a stable error code separate from the prose. A machine-readable code for insufficient balance survives envelope changes; the human sentence can be cleaned without erasing the class.
Carry context as structured fields, not inline text. Run id, account, available, required, and threshold live in a structured error object, so consolidating the message never deletes the data.
Assert the failure path, not just that it raised. A test should check the specific code or the presence of the identifying field on the failing branch, so flattening the string fails the suite.
Review error strings as part of the diff. A shorter message that drops the cause is an observability regression, not a formatting improvement; flag it back.
Treat the error response as part of a contract on API endpoints. Status code and error body are documented; when a code path changes, re-check that the documented shape still matches what the live endpoint returns rather than assuming a 4xx keeps its old meaning. Where the spec is the contract, we keep that check local in Powerduck and re-run the spec against the live endpoint instead of trusting the markdown.

Clean envelope, intact evidence

A shared error envelope removes repetition, which is a real improvement. Removing the field that names what failed removes the evidence a responder needs at 3 a.m. Simplification should flatten the prose, not the diagnosis; keep the machine code and the structured context, test the failure path, and let the generic wrapper wrap a cause it never had to erase.

The agent kept polishing after the task passed, and buried the fix under unrelated refactors

Jeff — Sat, 24 Oct 2026 18:27:35 +0000

You ask the agent to add a missing 404 handler for an unknown route. It writes the handler, adds a test, and the test goes green. Instead of stopping, it reformats two adjacent controllers, renames a shared helper "for consistency," and rewrites a comment block that had nothing to do with the route. When you open the diff, the requested fix is three lines and the rest of the patch is a cleanup you did not ask for.

A green test proves the step ran, not that the task ended

Most agent loops stop on a success signal: tests pass, typecheck is clean, the run command exits zero. That signal answers "did this step work?" but it does not answer "is the task complete?" The loop is usually seeded with an open-ended phrase such as improve, clean up, or modernize, which rewards continued action. After the first passing check, the model looks for the next thing it can improve rather than exit, because nothing in the prompt told it that passing was also the end.

Autonomous loops drift toward more output. Keeping the loop active feels productive, and a model defaults to adding changes rather than declaring itself done when the boundary is unwritten. The result is not a broken fix; the fix works. The problem is everything attached to it.

What ships when the loop keeps going

The requested change is buried. Reviewers have to read a wide patch to find the three lines that were actually requested, and the important decision (how the 404 behaves) gets lost among formatting churn.
Working code changes that were never requested. A helper rename or a controller reformat touches paths that were stable and tested for another reason, introducing risk unrelated to the bug.
One mixed diff makes revert painful. If the cleanup later causes a problem, it cannot be rolled back without also undoing the fix, because both live in a single change.
Optional polish becomes a forced merge. A follow-up idea the human would have rejected on its own rides along as part of an otherwise correct patch.
Review load grows without added value. More lines to read, more blame history to move, more side effects to reason about, all after the objective was already met.

The tell is a diff that touches files outside the one you named. That is not bonus work; it means the loop never received a stop condition.

Bound the loop before it starts

Scope creep in an agent session is set up in the instruction, not created by the model. Fix the boundary up front:

Write a finite acceptance condition. Name the exact file or function, the named test that must pass, and the behavior to add. Everything outside that list is out of scope, stated explicitly rather than implied.
Make stopping an instruction. Tell the loop to stop and report once the named checks pass; do not refactor untouched neighbors, rename helpers, or reformat files it was not asked to open.
Keep fix and cleanup in separate commits. Let optional polish be a follow-up the human approves, rather than appending it to the fix under review.
Ask the agent to list files it touched. A scope larger than the target file is a flag to cut, not a sign of thoroughness.
Run a diff-scope check before merge. Confirm the changed files match the requested set; if they do not, strip the extras back to the target before review.

Done is a written condition, not a feeling of completeness

An autonomous loop keeps improving because "clean it up" has no natural end. The task ends when a finite, written condition is met: the named handler exists, the named test passes, and no file outside the target changed. Until that boundary is on the page, green tests are progress, not permission to keep editing.

The reviewer flagged the helper as dead code, but a string in the config was the only caller

Jeff — Sat, 24 Oct 2026 17:27:20 +0000

A static review, human and AI, both mark processRefundLegacy as unused. Nothing imports it directly, no test calls it, and the diff to remove it looks like a clean deletion. It is reached from a string key in a dispatch table that only activates under a non-default payment flow, and deleting it breaks that flow on the next release.

A function can be reached without a direct call

Many codebases route behavior through a registry instead of a conditional chain. A handlers object maps string keys to functions: standard_refund points at processRefundStandard, legacy_refund points at processRefundLegacy, and a runtime lookup such as handlers[config.refundFlow] selects the entry instead of an if chain.

The function exists as a reference inside the map, but no line calls it with parentheses. Static reachability follows direct calls, imports, and type references. It sees a name stored in an object and does not expand which string value selects it, so it reports the function as never called. An AI reviewer reading the file tends to trust that same signal and generalizes "unused, safe to remove" without expanding the registry.

The same pattern hides callers in plugin manifests, feature-flag maps, per-tenant config, dynamic import by key, and event-bus topic registrations. In every case the caller is a string, not a symbol.

What ships when the helper is removed

The typecheck passes. The function had no direct caller to break, so removing it produces no compile error.
The default-path tests stay green. The legacy key is never set under the default test configuration, so the exercised suite never touches the gated function.
Production fails on a real refund. A tenant on the legacy flow reads config.refundFlow = "legacy_refund", looks the key up, gets undefined, and fails on the handler call. The error surfaces after release, usually as a missing-handler exception rather than a clean test failure.

The deletion diff itself looks tidy: one function gone, fewer lines, no visible import to update. That tidiness is exactly why it slips past review.

Prove reachability before deleting; do not assume it

Treat "no direct caller" as a proposal to investigate, not a finding to act on:

Grep the symbol and its registration key. Search the function name, then every string it could be registered under. The caller lives in the map value, and the key lives somewhere in config.
Expand the dispatch table. List each string key that points to the function and the lookup expression that reads the selected key, such as handlers[config.refundFlow].
Trace the key back to its source. Find where that string is set: environment variable, feature flag, per-tenant setting, deployment manifest, or migration record. A non-default setting can still be in use for a subset of traffic.
Check whether tests cover the gated path. A suite that only runs under default config proves the default path works; it says nothing about the gated one.
Require evidence before deletion. Keep the function if the only proof of liveness is a non-default config, and label it config-gated rather than dead. Remove it only after a recorded caller trace shows no environment selects the key.

Dead code is proven, not assumed

An unused-looking function inside a registry is dead only when no reachable configuration selects its key. Static "unused" reports are reliable for direct-call codebases and unreliable the moment routing passes through a string or a manifest. Before approving a cleanup that removes a helper, expand the map, trace the key, and confirm the gated path has no live tenant.

The agent resolved the merge conflict by keeping both sides, and the code compiled into a dead branch

Jeff — Sat, 24 Oct 2026 16:29:37 +0000

Two branches land on top of each other. Branch A tightened a broad allow into a narrow allowlist inside one function. Branch B extended that same function with a second condition for a new role. Git marks the block as conflicting, the agent removes the markers, and the merged file compiles clean. What it actually wrote is both versions stacked: A's narrow guard followed by B's separate condition, both imports kept, both helpers present. CI goes green, the reviewer sees a clean diff, and one side of the original conflict shipped as code that can never run.

Keeping both sides compiles because nothing duplicated exactly

A compiler errors on a duplicate declaration when two symbols occupy the same name with the same shape. When the two branches solved the conflicted block differently, they rarely match that description: one renamed the helper, the other added an if, one changed the condition, the other split the logic into a second function. Stacking them produces two sequential blocks that the parser accepts with no name collision. Typecheck reads each block independently; it never asks which one actually executes. Lint checks style, not reachability. The tests that run cover the new path B added, so the legacy guard A replaced stays green without being exercised.

The merged result looks finished because the markers are gone. Removing <<<<<<<, =======, and >>>>>>> is mechanical; choosing which version wins is the real work, and the version that compiles and passes the current suite can hide the choice.

Four leftovers that drift after the merge

An unreachable legacy branch still ships. The guard A replaced sits below the new condition. Under the normal input it cannot fire, but an edge case the agent did not model can reach it, and behavior then differs from what the merge intended.
A near-duplicate helper drifts. Two functions with the same first lines and slightly different bodies accumulate separate fixes. A future patch lands on one copy; the other keeps the old behavior and nobody notices until a bug appears in the path that reads it.
Two imports for the same concept linger. When both helpers survive, their imports survive too, and the file carries redundant dependencies that look intentional at a glance.
The decision is not recorded. The PR body says "resolved conflicts" without naming which side won each hunk, so the reviewer cannot audit the choice and a later archaeologist cannot tell why both copies exist.

Why the agent defaults to stacking

Choosing a side means taking a position: which feature's behavior wins, which condition should apply, whether A's tightening should survive B's addition. Stacking both avoids that judgment and still satisfies the visible success criteria: markers gone, file parses, checks green. The optimizer that minimizes error tends to pick the path that removes the conflict rather than the path that resolves the disagreement. Human contributors do the same thing under time pressure, usually with one anchor: they know which feature is meant to win. A fresh session often does not, so it preserves both.

Diff against both parents, not the merged result alone

The merged diff against main hides the choice because it compares the final file to the branch point. The check that exposes stacking is to compare each conflicted hunk to both parents, not just read the merged file:

Compare the final hunk to ours and theirs. If the merged block contains statements from both versions of the same line where only one should survive, the choice was deferred, not made.
Hunt the duplicate pair. In the conflicted region, look for two imports for the same concept and two helpers that start the same way. Their presence is the signature of keeping both sides.
Trace reachability of the leftover condition. Ask which input can actually enter the older branch after the new condition sits above it. If the answer is none under realistic data, the branch should be deleted rather than preserved.
Log the decision per hunk. The PR note should state, for every conflicted block, which side won and why. A generic "resolved" forces the reviewer to redo the comparison from scratch.

Green after a conflict proves the markers are gone

A green typecheck after a merge conflict confirms the file parses and the current tests pass. It does not confirm the right side survived. Resolving a conflict means choosing and deleting the loser; stacking both is deferring the choice into unreachable code and a duplicate copy that drifts later. Compare the merged hunk to both parents before approving, and require the winner to be written down.

The agent drifts from your requirement while the context fills up

Jeff — Fri, 23 Oct 2026 22:27:35 +0000

You ask for one behavior: add a validation that rejects a malformed id before the request reaches the handler. The agent reads the route, finds the schema file, runs a couple of searches, edits two other call sites to match a style it saw nearby, and finishes with a green test. The validation you asked for is missing; it optimized the last thing it touched and called the task done. The requirement did not disappear, it just got buried under the working material.

Context fills, and the brief sinks

A coding agent does not keep a clean memory of your request. It carries a running transcript: fetched files, terminal output, rejected edits, summaries of those edits. As that transcript grows, the original one-line brief sits farther from the tokens the model samples next, and each new decision leans on recent material more than on the opening instruction. Nobody tells the agent to pause and re-check acceptance, so the last successful action quietly becomes the objective.

This is not a model that forgot; it is a context that rotated. The brief was never restated, the acceptance criteria were never pinned, and the intermediate output was never trimmed. Drift looks like competence because the diff compiles and the tests that happen to exist still pass.

The drift patterns to watch for

Optimize-the-last-output: after fixing a nearby function, the agent keeps polishing that function instead of returning to the original target.
Summary-takes-over: a recap block written midway through the task becomes the source of truth, and it states a slightly broader or narrower goal than you gave.
Scope-creep-by-convenience: while editing, the agent fixes an adjacent rough edge because it was already in the file, and that detour pushes the real ask off to the end.
Stale-tool-log: an early failed search or a rejected draft stays in the transcript and the agent treats it as a current constraint.
Lost-initial-criteria: you described acceptance in prose at the start; by the end there is nothing to compare the diff against, so a subjective read wins.

Pin a short acceptance block

The cheapest anchor is a tiny block that never moves. Put it at the top of the task, in plain sentences the agent can quote back, and keep it separate from implementation notes. It should say what must change, what must stay unchanged, and the one check that proves it.

Acceptance for this task:
- Reject a malformed id at the route boundary, before the handler runs.
- Do not change existing valid-id behavior.
- Prove it with one negative-case test that returns the expected error.
Everything after this block is working material, not new requirements.

Restate and compare before done

Do not trust a done that arrived from a long transcript. Before accepting the work, ask the agent to restate the acceptance block and point to the exact diff hunk that satisfies each bullet. When a bullet has no hunk, that is an open item, not a detail to skip. One restatement pass catches most drift because it forces the model to measure the final diff against the opening brief instead of against its last edit.

Trim the transcript as you go too: drop early failed searches and superseded drafts rather than letting them ride along. A shorter, current context behaves like a focused engineer; a long muddy one behaves like someone reading from the middle of their notes.

Write a handoff between sessions

Context decays further when you stop and resume hours later. The new session opens cold, and without a handoff the agent reconstructs a goal from whatever is visible in the working tree. End a session with a short note: what was done, what is open, the acceptance block, and where the relevant files live. The next run starts from the anchor instead of re-deriving intent.

This is easier to hold when the anchor itself lives in the repo rather than only in the prompt. A checked-in contract or spec file the agent reads every session does not rot when the transcript does. That local-first, file-as-source-of-truth habit is the core of Powerduck; the same instinct applies to keeping a task brief from drifting.

Drift is invisible because the code still runs. The fix is not a bigger window; it is a fixed point the agent returns to, a periodic restatement, and a transcript that stays current instead of accumulating.

The agent wrapped everything in try/catch and your errors disappeared

Jeff — Fri, 23 Oct 2026 21:28:58 +0000

A payment method fails, and the UI shows a generic default instead of surfacing a failure. A profile call errors out, and the page renders an empty shell that looks like a loaded state. The ticket says it works in the demo, the build is green, and the real exception never reaches the caller's error path. The common cause is a blanket try/catch the agent added to "handle errors gracefully," with a default value returned where an error should have been propagated.

A fallback that reads as success

The agent wraps the whole function body, catches everything, and returns a plausible object: an empty profile, an empty list, a zeroed record. Downstream code checks for a truthy value, the default object passes that check, and the caller cannot tell "missing" from "loaded." The upstream error is logged at warning level, if it is logged at all, and nothing correlates it to the request that actually failed.

On the surface this looks defensive. It compiles, the happy-path test passes, and the reviewer sees error handling present. What it actually did was remove the signal that a failure happened, which is worse than no handling at all: an uncaught error stops loudly; a swallowed one keeps going quietly and corrupts the assumption the rest of the program builds on.

Why the model reaches for the blanket catch

The prompt usually says "handle errors gracefully" or "don't crash." Given that instruction, the cheapest move that satisfies the text is: wrap the body, catch the exception, return a sensible default, add a log.warn. The model treats throwing as the only failure mode worth avoiding and treats a missed exception as an acceptable side effect.

It also favors the path that never throws. A default return keeps the function happy-path test green, whereas propagating forces the test to cover an error branch the prompt never described. The agent optimizes for compiling and passing the scenario in front of it, not for the distinction between a recoverable default and a hidden failure.

The catch shapes that hide failures

catch (err) { return DEFAULT; } — swallows and returns a plausible object; downstream cannot tell missing from success.
An empty catch block — compiles, logs nothing, and the caller never learns anything went wrong.
catch { log.warn(...); continue; } on a step that should abort the whole operation rather than proceed half-finished.
Catching at the top boundary instead of the call site, so local context (which input, which id, which dependency) is lost before anyone can debug it.
A fallback value indistinguishable from a real result: an empty list for "no records," a default object for "loaded," zero for "failed to compute."

Review the catch, not just the happy path

The bad shape returns a constant regardless of what went wrong:

async function loadProfile(userId: string) {
  try {
    return await api.get(`/users/${userId}`);
  } catch (err) {
    return EMPTY_PROFILE;
  }
}

The caller checks if (profile), EMPTY_PROFILE is truthy, and a failed load looks identical to a successful empty state. The better version decides per error type and keeps failure visible:

async function loadProfile(userId: string) {
  try {
    return await api.get(`/users/${userId}`);
  } catch (err) {
    if (isNotFound(err)) return null;
    throw new ProfileLoadError(`failed to load ${userId}`, { cause: err });
  }
}

Now the caller can distinguish documented absence (null) from a genuine failure (a typed throw it must handle). Narrow the catch to the errors the call actually raises; catch-all unknown at a real boundary, not on every helper.

A catch must make one of three decisions

Every catch should pick exactly one, rather than defaulting to a return:

Propagate: rethrow, preferably as a typed error with the original attached as cause, so the caller decides how to respond.
Recover with evidence: retry or use a valid cache only when you can actually justify it, and log which recovery branch ran and why.
Translate: convert the low-level error into a domain error the caller can act on, without dropping the original.

Returning a default without choosing among these hides the failure by default.

Force the failure path in a test

A test that only calls the happy route never exercises the catch, so the blanket fallback can grow over months and nobody notices until production. Add one test that injects a rejected call and asserts the error reaches the caller, or that the documented absence (null) is returned for the expected case. That keeps the catch honest and prevents the default from drifting into a silent mask.

Graceful error handling means choosing what the caller should know, not making every path succeed. A try/catch that always returns a default isn't handling errors; it's deleting them.

The AI test that passes because it copied the code's assumption

Jeff — Fri, 23 Oct 2026 20:37:16 +0000

An agent writes the function, writes the covering test in the same pass, CI turns green, and the reviewer merges on a clean stack. Months later an edge case breaks in production, you open the test that was supposed to cover it, and it stays green. It stayed green because it never knew the requirement; it copied the implementation's own assumption back at it.

A mirror test repeats the code instead of judging it

A real test derives the expected value from somewhere the code does not: a written rule, a known worked example, an invariant from the domain, or a bug report that already happened. A mirror test derives the expected value by running the new function, reading its output, and hardcoding that output back into the assertion. The test then verifies that the function returns what the function returns. When the implementation changes its own behavior, the agent tends to update the expected value in the same pass, and the coupling closes permanently.

The shallow version is an assertion that only checks the response is non-null, or that the call does not throw. It proves the function ran, nothing about whether the answer is correct.

Why a single pass makes the blind spot shared

When a human writes a test after thinking about the requirement, the test writer can disagree with the implementation and catches a wrong branch. When the same model generates code and test from one prompt, both inherit the same unstated assumption: how empty input behaves, what counts as a valid identifier, which timezone the date means, whether an out-of-range value should clamp or reject. The model has no independent oracle, so the test cannot contradict the code it just wrote. Green in that case is a syntax and plumbing check, not evidence of correctness.

Make the expected value come from outside the code

Derive expectations from the requirement. Use a worked example from the spec, a known customer case, or a domain invariant, and hardcode that answer independently of the function. If the test author must open the implementation to compute the expected value, it has already become a mirror.
Pin the boundaries the model tends to skip. Empty input, maximum size, malformed shape, negative or zero, timezone shift, and a value that falls outside the accepted set. Write those cases as explicit expected failures or explicit clamps rather than leaving them to the happy path.
Assert the result type and error, not just presence. A non-null object or a caught exception proves almost nothing; assert the specific value, the thrown error class, or the rejected status the requirement names.

The cheap mutation probe

On functions that sit on a trust boundary, flip one boundary condition in the implementation deliberately: clamp instead of reject, drop the empty-input guard, shift a rounding rule. Run the suite. If every test still passes, the test is not touching that branch and it provides no safety net. This is a lightweight version of mutation testing; you do not need a full framework for a handful of risky functions, just one local flip per probe and a clean revert after. A test that survives a mutation it should kill is advertising coverage it does not have.

Let real bugs pin the behavior

When a bug does arrive, write the failing characterization test for the exact reported input first, before touching the fix. That test carries the behavior from outside the codebase, so it cannot mirror the repaired implementation. It also becomes the permanent guard that prevents a future agent from smoothly reverting the fix while keeping the suite green.

Tests generated in the same pass as the code are a smoke check. They confirm the wiring runs; they do not confirm the answer is right. Separate the oracle from the implementation, and green starts to mean something again.