DEV Community: alphanumericentity

The agent did exactly what I asked. That was the bug.

alphanumericentity — Tue, 06 Oct 2026 02:21:18 +0000

Three times this month my agent executed a flawed request perfectly. Hooks can't catch this one, because the mistake isn't in any single tool call, it's in the plan. Here's what I changed instead.

I wrote a while back about moving rules out of my instructions file and into hooks, because a rule in prose gets followed most of the time and a rule in a tripwire gets followed every time. I still believe that. Someone in the comments pushed on where determinism stops being the right tool, and I didn't have a good answer then.

I do now, because three times in the past month my agent did precisely what I asked, did it well, and the result was wrong anyway. Not a hallucination, not a bad diff. Correct execution of a flawed request. That class of failure has no tool call to intercept, which makes it the exact thing a hook cannot save you from.

One: I asked for a demo GIF and got a security bug

I have a side project that redacts PDFs in the browser. I wanted an animated demo for the landing page, so I asked the agent to drive the app with Playwright and record the whole flow: open a file, drag a box over a name, export, done.

It built the script. The script worked. I watched the GIF and the black box was sitting next to the name instead of on top of it.

The redaction code was placing boxes by splitting a text run's total width by its character count, so a box over characters 4 to 9 was a flat fraction of the run. That's correct for a monospaced font and wrong for every other font, and the drift grows with the length of the run. Sometimes it covered the word. Sometimes it covered the word and a bit of the next one. Sometimes it sat just to the right of the thing I was trying to hide.

Here is the part that stings. The app already verified its own output. After exporting, it re-opens the PDF it just produced, extracts the text layer and checks that the redacted string is gone. That check passed every single time, and it was right to pass: the text really was destroyed. It just wasn't the text under the box the user drew. My verifier and my renderer were computing position the same wrong way, so the verifier could not see the error even in principle.

I asked for a marketing asset. What found the bug was that a GIF is looked at by a human with eyes, and every assertion I had written shared an assumption with the code it was testing.

Two: a perfect entry in the wrong section

Different day. I wanted my project listed in one of the big awesome-privacy lists, so I asked the agent to add it to the Office section and open the PR.

It did the work properly. Read the contributing guide, matched the entry format down to the trailing source-link convention the other entries use, filled in the seven-point checklist, picked an insertion point that wouldn't conflict with other open PRs, wrote a disclosure paragraph. Genuinely good work.

I caught it at the last second, with the commit dialog open: that section is titled "Avoid Microsoft Office and Google Docs, instead use", and everything in it is a suite, an editor or a spreadsheet. A PDF utility in there is a category error. Any maintainer would have closed it, and fairly.

The agent never had a reason to question the section. I named it. It optimized for "is this entry well-formed and well-argued", which it was, and never for "is Office the right section", because I'd already answered that.

Three: a confident click path to a setting that lives somewhere else

Smallest one, same shape. I asked where a particular GitHub repo setting lives. I got "Settings, then General", stated plainly, with the surrounding steps.

It isn't there. That field is behind a gear icon next to the About panel on the repo's front page, which is not Settings at all. The answer was assembled from what a settings page plausibly contains. When I made it go read the documentation page first, it came back with the right path in one shot, and with the character limits and naming rules I hadn't thought to ask about.

What these have in common

The agent is a premise-taker. It will interrogate its own implementation all day and will not touch the frame of the request, because the frame came from you and you're the one it's trying to satisfy. "Did I do what was asked" is a question it can answer. "Was the ask right" isn't even in scope.

Four things I changed, in rough order of how much they've paid off.

Ask for the decision, not the destination. "Find where this belongs in the list, then add it" instead of "add it to the Office section". One extra clause moves the category judgement inside the task, where the agent will actually spend reasoning on it. When I name the destination, I've silently converted a decision into a constraint.

Primary source, or say it's unverified. For anything about a UI, a config field, an API shape, a provider's semantics: read it this session and cite it, or label the answer as a guess. Plausible-sounding click paths are the most confident wrong answers I get, because nothing in the model's experience punishes a well-formed guess about a settings page.

Make it tell me what it assumed. Every non-trivial task ends with what it had to assume and what it could not check. This is cheap and it has caught more than I expected, because the premise shows up in writing where I can see it, instead of staying implicit in a diff that looks fine.

Build the artifact that can disagree with me. A screenshot, a recording, a pixel diff, a second measurement taken a different way. The GIF found the redaction bug for one reason: it didn't share any code with the thing it was checking. Now that project has a test that pins box positions against independently computed glyph metrics and then samples the burned pixels, which is the same idea with the human taken out of the loop. It's here if you want the shape of it, AGPL like the rest of it. If your check is built from the same assumption as the code it checks, you don't have a check, you have a second copy of your belief.

Why a hook can't help here

I tried to find the tripwire for this and there isn't one, which took me a while to accept.

A hook fires on a single tool call and decides from that call's input alone. That works when the mistake is local: a dangerous shell command, a comment block, an edit to a file that should never change. A wrong premise doesn't look like that. It produces a sequence of individually reasonable calls, each one defensible on its own, all of them marching toward a thing that shouldn't have been built. There's no string to match, no file path to guard, no moment where the input looks wrong.

Determinism needs a decidable question. "Is the Office section the right home for a PDF tool" is not one.

So the fix isn't mechanical. It's that the review has to happen earlier than I was doing it, before the work rather than on the diff. For agent work I'd now go further: reviewing the plan is worth more than reviewing the code, because the code is usually right. The expensive errors arrive fully formed, well-tested and pointed in the wrong direction.

That's the part I'd like to be argued with on. Do you actually review plans before the agent starts, and does it hold up, or does it just move the rubber stamp one step earlier? Because I notice I'm much better at approving a plan I half-read than at approving a diff I half-read, and I don't think that's a good sign.

Your agent's instructions file is a suggestion. A hook is a contract.

alphanumericentity — Fri, 02 Oct 2026 09:12:06 +0000

Every coding agent now reads an instructions file. CLAUDE.md, AGENTS.md, .cursorrules, whatever yours is called. Mine is long. It has been long for months, because every time something went wrong I did the obvious thing and wrote a sentence about it.

Here is what I eventually noticed. The agent follows those sentences most of the time. Not always. And the times it doesn't are not random: they're the moments when the file is long, the task is interesting, and the rule is the least salient thing in a 30k-token context. Which is exactly when you need the rule.

A rule in prose competes with every other sentence in the file for attention. A rule in a hook does not compete with anything. It just runs.

So over a few months I moved the rules that actually mattered out of the instructions file and into PreToolUse hooks. There are about a dozen now. The agent is noticeably less annoying to supervise, and I stopped repeating myself.

The examples below are Claude Code hooks, because that's what I use. The idea is the part worth stealing.

The mechanics are almost insultingly simple

A PreToolUse hook is a program. The agent is about to call a tool, your program gets the call as JSON on stdin, and you decide.

#!/usr/bin/env python3
import sys, json

data = json.load(sys.stdin)
cmd = (data.get("tool_input") or {}).get("command", "")

if "psql" in cmd and "DELETE" in cmd.upper():
    print(json.dumps({
        "decision": "block",
        "reason": "No DELETE against this database from an agent session. "
                  "Write the statement into a migration and let a human run it.",
    }))
sys.exit(0)

Two details make this better than a code review or a nag in the prompt:

The call never happens. Not "the model is discouraged from it". The tool returns your refusal instead of running.

And the reason goes back to the model, not to you. That's the part I underrated. A good reason makes the agent correct itself and carry on, so a blocked call costs a few seconds instead of a round trip through your attention. A bad reason ("blocked") makes it guess, flail, or ask you. The reason string is the whole product.

The tripwires

Grouped by what they're actually protecting, because the categories turned out to matter more than the specifics.

Things that can't be undone. A guard that makes writes to the production database impossible from any agent session: mass deletes, truncates, cache flushes, a session-level SET through a connection pooler. A guard that blocks merge commands, because merging is a human's decision and an agent that can merge will eventually merge something at 2am. A guard that refuses sed -i into source files, which is how an agent rewrites a file without the edit ever passing through the checks on the edit tools.

Things that are expensive. A guard that blocks local type-checking entirely. That one sounds mad until you know that a cold run across this monorepo costs about fifty seconds of CPU and one process peaked at 5.6 GB, and that several agents in several worktrees will happily do it at once on a 24 GB laptop. CI type-checks every push anyway. The hook's reason says so, so the agent commits instead of grinding.

Things that are silently wrong. My favourite hook, and the shortest. Our analytics query language reads a bare timestamp literal in the project's timezone, not UTC. Write timestamp > '2026-07-03 00:00' and you have quietly shifted your window by ten hours. It once made a feature flag look like it was ramping when it wasn't, which produced a confident wrong diagnosis and a needless experiment reset. Now:

for m in re.finditer(r"(timestamp|created_at|_at)\s*[<>=]+\s*'(\d{4}-\d{2}-\d{2}[^']*)'", text, re.I):
    if re.search(r"([+-]\d{2}:?\d{2}|Z)\s*$", m.group(2)):
        continue  # explicit offset, fine
    print(json.dumps({"decision": "block", "reason": TZ_TRAP_EXPLANATION}))

Forty lines, including the explanation. That class of bug cannot reach me any more, from any session, in any file or shell command, whether or not anybody remembered the rule.

Things that are about taste, and therefore hopeless in prose. A guard that blocks any edit adding a multi-line comment block. I had written "do not write comments" in the instructions file in capital letters, with reasons, twice. It did not work reliably, because the model's prior for "explain this clever bit" is strong and my sentence was one sentence. The hook works every time, and its reason tells the agent to rewrite the block as one line rather than just deleting the thought.

Things about process. A guard on new UI that requires a stable test id, because the e2e suite selects by test id and a renamed element silently stops testing anything. And a guard that refuses to push a branch until an adversarial review receipt exists for the current head. The second one is the most aggressive thing in my setup, and I'd defend it: an agent that is sure its work is ready is not evidence that the work is ready.

What it cost

I don't want to sell this as free.

False positives are real and they're the price. My shell guards block patterns that are usually footguns and occasionally the right thing, and then I'm reading a refusal about a command I meant to run. The honest fix is to go edit the hook, which I do, and which is still cheaper than the alternative because the fix is a diff somebody can review rather than another sentence in a file nobody re-reads.

They're also per-repo and they rot. A hook that encodes "CI runs this check" is wrong the day CI stops. Mine have tests, which felt absurd when I wrote them and does not any more.

And they don't replace the instructions file. Taste, context, architecture, the reason a weird thing is weird: that's all still prose, and prose is still the right shape for it. Hooks are for the small number of rules where "usually" isn't good enough.

The thing I actually changed my mind about

I started doing this because I didn't trust the model. That was the wrong frame, and it made me write mean little guards with bad reasons.

The better frame is that an agent has no way to know which of your forty rules is the one that costs money. You do. A hook is how you say "this one is not a preference", and a good reason string is how you teach instead of scold. The agent reads it, understands why, fixes its own command, and moves on. That's a better loop than any amount of bold text in a markdown file.

So: what's on your deny list? I'm especially curious about the opposite view, because one rule I keep going back and forth on is whether blocking git push crosses from guardrail into busywork. Tell me I'm wrong about that one.

A SELECT that returned more rows than its LIMIT

alphanumericentity — Wed, 30 Sep 2026 10:11:54 +0000

A scheduled job in one of our services started crashing every few days with this:

TypeError: Cannot read properties of undefined (reading 'id')

Inside a for..of over the rows of a query. Not on the first row, not on every run, and never reproducible locally.

The other clue turned up in a log line from the same job, which prints how many pending records it picked up. The query has LIMIT 500. The log said pendingTotal: 534. Another run said 813.

A SELECT with a LIMIT of 500 returning 813 rows is the kind of thing that makes you go back and read your own SQL four times. The SQL was fine. So was the ORM. The bug was one line below the driver's surface, and it has been open upstream for about a year.

What actually happens

We use postgres-js (the postgres package) with a connection pool, like most people do. Here's the shape of the bug, straight out of src/connection.js in the current release:

A connection keeps a single write cursor for the result it's filling in:

let ...
  , rows = 0

Every DataRow message the server sends gets written at that cursor:

function DataRow(x) {
  // ...
  : (result[rows++] = transform.row.from ? transform.row.from(row) : row)
}

rows goes back to zero in exactly two places: CommandComplete, which is the server saying the query finished, and PortalSuspended, which is the cursor case. That's it.

Now look at what happens when a query doesn't finish. ErrorResponse records the error and returns. Then ReadyForQuery cleans up:

function ReadyForQuery(x) {
  // ... resolve or reject the query ...
  query = results = errorResponse = null
  result = new Result()
  connectTimer.cancel()
}

It allocates a fresh result array. One line later than where the cursor should have been reset, and the cursor isn't in that list.

So: a query streams 97 rows, then the server kills it. rows is left at 97. The connection goes back in the pool. The next query that lands on it gets a brand new empty array, writes its two rows at index 97 and 98, and resolves with an array whose length is 99 and whose first 97 slots are holes.

Why this one is nastier than a normal wrong answer

A sparse array is a bad failure mode because nothing about it throws:

JSON.stringify(rows) gives you a list starting with 97 nulls. That goes straight into a log line or an API response.
for..of and .entries() hand you undefined, which is where our TypeError came from.
.length is 99, which is how a LIMIT 500 query reports 534.
.map() preserves the holes, so an ORM mapping layer passes them through.
.filter(Boolean) hides it completely, so whether you ever find out depends on which array method you happened to use.

And the quiet one, which is the reason I'm writing this down:

const [row] = await db.select().from(users).where(eq(users.id, id));
if (!row) return notFound();

row is undefined, the record exists, and you take the "not found" branch. No error, no log, just a wrong decision. We had a version of that on a write path.

The trigger is almost always a statement timeout

To hit this you need a query that errors after it has streamed at least one row. The common way is statement_timeout, Postgres error 57014, firing mid-scan on a query that's already returning rows.

The thing to check is where your timeout is set. Ours is on the database role, not in application code, which means any slow query in any service arms this for the next query on that connection. One heavy scan in a cron job, and an unrelated request handler sharing the pool gets the holes. That's also why it never reproduced locally: on a laptop nothing takes long enough to get cancelled.

If you want to confirm it in your own logs, look for these two together:

A count in a log line that exceeds the LIMIT of the query it came from.
A TypeError on undefined inside a loop over query results, within a few seconds of a canceling statement due to statement timeout in the same process.

Same process matters, since the pool lives in the process. Different pod, different pool, unrelated.

Upgrading doesn't fix it

I checked the current release and master while writing this. postgres@3.4.9 is the latest on npm, and both it and master reset the cursor in the same two places and no others. The fix is a one-line PR, #1120, open since October 2025. The mechanism is written up in issue #1181, which is open and still getting comments this week.

So until it lands, patch it. With pnpm:

pnpm patch postgres

Add the one line to ReadyForQuery:

   query = results = errorResponse = null
   result = new Result()
+  rows = 0
   connectTimer.cancel()

Then pnpm patch-commit <dir>, which writes the patch file and the patchedDependencies entry into your workspace. npm and yarn both have their own equivalents.

If you'd rather not patch a dependency, the other mitigations are all worse but worth knowing: set max: 1 so a poisoned connection only hurts the code that poisoned it (it doesn't make the problem go away), drop your statement_timeout low enough that queries die before streaming anything (good luck), or check rows.length against your own LIMIT on every read and throw. We patched.

The general lesson I took

I had a rule in my head that an error on a connection is contained: the transaction rolls back, the pool hands the connection to someone else, life continues. This was a reminder that the client library also has per-connection state, and an error path is exactly where that state stops being maintained.

So when a client hands you a fresh object for each query, the question worth asking is whether anything else is still pointing into the old one.

Give every coding agent its own git worktree

alphanumericentity — Wed, 16 Sep 2026 02:58:28 +0000

Quick scenario. You've got two coding agents going in two terminals. Agent one is halfway through a refactor. Agent two gets asked to fix a bug that lives on another branch, so it does the obvious thing and runs git checkout. Agent one's next tool call reads files that aren't there anymore. Neither of them notices. Both keep going.

I've had some version of this happen more than once before I stopped and fixed it properly. The problem is that a git checkout was designed for one person with one intention at a time. You'd never switch branches while a coworker was typing in the same directory, but an agent has no concept of "someone else is using this". It assumes the tree it saw a minute ago is the tree it has now. Often that's just false.

The fix turned out to be a git feature I'd basically ignored for ten years: worktrees. They've been in git since 2.5 (2015). One repo, multiple working directories, each on a seperate branch, all sharing the same object store. One human rarely needs two checkouts of the same repo so nobody bothered. A handful of agents need exactly that.

Ok so what's a worktree

git fetch origin main
git worktree add -b fix/login-redirect ../myrepo-wt-login-redirect origin/main

That makes a branch off the current origin/main, checks it out in a sibling directory, and registers it with the repo. Leave off the -b if the branch already exists. git worktree list shows you all of them, git worktree remove ../myrepo-wt-login-redirect gets rid of one.

Same .git, many directories. A branch can only be checked out in one worktree at a time, which sounds like a limitation but is actually the whole point. The second agent that tries to grab a branch gets an error instead of silently pulling the rug out.

One thing: put the worktree next to the repo, not inside it. If it's inside, the main checkout sees a big new untracked directory and sooner or later someone's git add . swallows it, and that's a fun one to untangle.

That's the mechanism. The mechanism alone doesn't save you though, the rules around it do.

The four rules

These live in the agent's instruction file (CLAUDE.md, AGENTS.md, whatever your tool reads). Not in my head, because I forget, and the agent definitely doesn't read my head.

Isolate. Every edit happens in a dedicated worktree created off the exact base you mean. New change? Fetch and base it off origin/main. Fixing an open PR? Base it off that PR's current remote head, not whatever stale local copy of the branch happens to be lying around. The main checkout is read only for agents. Grep in it, git log in it, don't edit in it.
Stage by path. git add -A and git add . are banned. The tree might have another agent's half done work in it, or a stray .env, or some debug script. An agent that stages everything commits everything. It names the files it touched and those go in, nothing else.
Check the diff. Twice. Before pushing, git diff origin/main...HEAD (three dots, not two, this matters) has to be exactly the change you intended, nothing extra pulled in and nothing missing. Then again before merging, because main moved while the agent was working and the diff that got reviewed at push time is not the diff that actually lands. If anything unexpected shows up the agent stops and says so. It does not merge and hope.
Clean up. git worktree remove once the PR is merged, then delete the branch. Old worktrees that outlive their PR are exactly where the next agent finds a stale branch and builds on top of it.

Here's roughly what's in my instructions file:

## Changing code
- Never edit in the main checkout. Work in a worktree off the intended base:
  git -C /abs/path/to/repo fetch origin main
  git -C /abs/path/to/repo worktree add -b <branch> /abs/path/to/repo-wt-<slug> origin/main
- Stage by path. `git add -A` and `git add .` are forbidden.
- Before pushing, and again before merging, `git diff origin/main...HEAD`
  must be exactly the intended change. Anything else: stop and report.
- Update a pushed branch by merging main into it, never by rebasing.
- Remove the worktree once the PR is merged.

Two things in there are on purpose.

Absolute paths and git -C everywhere. An agent's working directory is not something you want to depend on. Some harnesses reset it between tool calls, some (Claude Code, in my setup) prompt you every time a command starts with cd, and an agent that thinks it's in the worktree but is actually sitting in the main checkout will do precisely the thing the worktree was supposed to prevent. Absolute paths make that whole class of bug impossible.

And merge, don't rebase, once a branch has been pushed. A rebase rewrites commits that already exist on the remote, then needs a force push, and the force push throws away anything a human added to the branch in the meantime. An "update branch" click, a review fix, a commit from a coworker who had no idea an agent was on that branch. I know rebasing gives you a prettier history. I don't care anymore.

Why not just use branches?

Fair question, git already has branches.

Because switching a branch is a write to a shared mutable global. Every agent in that checkout sees it and none of them get told. It also trashes everything in the tree that's keyed to its contents: build output, the bundler cache, tsc's incremental state, node_modules if the branches have different deps. Two agents ping ponging between two branches in one directory spend most of their time rebuilding the world and very little of it doing the task.

Also you can't run two dev servers from two branches out of one directory. With worktrees you can, and you'll want to, because "does the fix actually work" means running the thing.

Stuff that will bite you

The mechanism is simple. The edges are where the afternoon goes.

Untracked files don't come along. A new worktree is a clean checkout of tracked files only. Your .env, local config, that cert you generated months ago, none of it is there. Copy what the tree needs over explicitly. And don't let the agent "fix" a missing .env by committing one.

node_modules is per worktree. Every tree needs its own install. With pnpm this is nearly free, packages get hardlinked out of the global content addressable store so a second install is mostly bookkeeping. With npm it's a full copy every time, which adds up fast. If your monorepo has a build cache, the local part of it is per tree as well, so a remote cache is what makes worktree number two start fast rather than rebuild everything.

Ports collide. Two worktrees, same app, same port. Derive the port from the worktree name or pass it explicitly, and make the agent print the URL it actually ended up on.

The database is the next shared global. Two agents, two worktrees, one local Postgres. The moment one of them runs a migration the other's app is talking to a schema it doesn't expect. Per branch databases (a database or schema named after the branch, created on first boot) close this hole. It's the same move as the worktree, one level down.

Hooks and config are shared. Every worktree reads the same .git/hooks and the same repo config. Good, the guard rails apply everywhere. But a hook that assumes it's running from the main checkout path will do weird things in a worktree. Use git rev-parse --show-toplevel in hooks, never a hardcoded path.

Stashes are shared too. The stash is a ref on the repo, not the worktree. An agent that stashes in one tree leaves a lovely surprise for whoever runs git stash pop in another. Honestly just don't stash. Commit the WIP on the branch and squash it later, or don't, nobody cares.

Lockfiles diverge. Two worktrees each add a dependency and now the lockfile conflicts at merge time. Never hand merge a lockfile. Take either side, run install, commit whatever it produces.

Strays pile up. If you rm -rf a worktree directory git still thinks it's there. git worktree prune clears the bookkeeping. git worktree list is the first thing I run when anything git related is being weird. A branch that's checked out in some worktree also can't be deleted or checked out anywhere else until that worktree is gone, which is the first thing to check when an agent complains that a branch is "already checked out".

What actually changed

Before, the checkout was a global. Everything in my agent setup had to reason about its state and none of it could, because none of them could see the others.

Now the checkout is a parameter. Each agent gets its own, does its work there, proves what it did with a diff against the base, and the directory goes away when the PR merges. The main checkout just sits on main, clean, and I mostly use it to read code.

None of this is clever. It's one git command that's been sitting there for a decade plus four rules written down somewhere the agent will actually read them. Most of the damage agents were doing to each other's work just stopped, and whatever's left at least shows up in the diff.