DEV Community: Mayur Agnani

How a Coding Agent Finds Code: Grep Versus an Embedding Index

Mayur Agnani — Sat, 07 Nov 2026 13:14:39 +0000

A coding agent only sees the files its search turns up. Grep finds them when your request uses the words in the code, and an index finds them by meaning, so each one misses the requests the other catches.

Engineer's view. This is the site search that returns nothing for "refund" because the code says "reverse charge". A coding agent searches your repo the same way. Grep needs the exact word, and each miss costs the agent another turn. An embedding index matches by meaning, so it catches the plain-English requests grep misses.

TL;DR. Your coding agent finds code by guessing words, and plain-English requests make it guess wrong more often. Give it the names your code uses, or a second search that matches by meaning.

Explain Like I'm 5

You ask a helper to fetch a book from a huge library. The helper can only read the words on the spines. So they guess a word and pull every book with that word on it.

If your word is on the spine, they find it fast. If you said "the one about sad dogs" and the spine says "Old Yeller", they guess again, and again. Each guess is another trip down the aisles.

A second helper has read every book. They walk straight to the one that is about what you meant. Sometimes they bring a book that only sounds right, so the best team uses both.

The Problem

You have shipped this bug before, in a system with no model in it.

Your store's search box returned nothing for "refund". The page existed. It was titled "Reverse a charge", because that is what the billing team called it. The search worked exactly as written. It matched words, and your customers did not use your words. You fixed it with a list of synonyms.

A coding agent searching your repo is that search box, run by a model. You type "refunding twice when the customer retries". Most terminal coding agents search with grep, the command that lists files containing a word. So the model has to guess a word that is in the file. If your code says reverse_charge, its first guess misses. It tries another word, and gets back 40 files that mention payments. It narrows, and tries again.

Every one of those guesses is a search round: one search, then the model reads the result. Each round resends the whole conversation, so a search that takes four rounds costs four times the reading. The worse case is quiet: the agent stops at a file that looks right, edits it, and you review the wrong change.

I replayed 400 real Django commits as requests to a scripted grep searcher. When the request named the code, grep found the file within three rounds 75% of the time. When it was plain English, 45%.

The fix: give the agent the names your code uses, and give it a second search that matches by meaning, then measure how many search rounds it spends before its first edit.

Read the full post on The AI Commit

How a Coding Agent Finds Code: Grep Versus an Embedding Index carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.

Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

How to Predict Each User's Token Speed on a Shared GPU

Mayur Agnani — Fri, 06 Nov 2026 12:03:51 +0000

A model server reads the whole model once per step for everyone, so an extra user is cheap. It also rereads each user's conversation every step, so one long agent session slows every chat on the same GPU.

Engineer's view. This is capacity planning by request count on a link that is limited by bytes. A model server reads its weights once per step for all users. It also reads each user's stored conversation every step. A few long agent sessions use up that read budget, and every chat on the GPU streams slower.

TL;DR. Every user on a GPU waits for the same shared step, and that step gets longer with the total conversation everyone holds. Plan capacity in tokens in flight, not in users.

Explain Like I'm 5

A librarian answers everyone's letters, one word at a time. Each round, she rereads her big rulebook once. That takes the same time whether five people are waiting or fifty.

Then she rereads each person's letter so far, so she knows what word comes next. Short letters are quick. One person with a hundred-page letter makes every round slower, for everyone.

So the number of people is not what slows her down. The number of pages she has to reread is.

The Problem

You have shipped this bug before, in a system with no model in it.

You sized an API gateway by requests per second. The load test sent small payloads, and it passed with room to spare. Then one customer started uploading large files. The request count barely moved, but the network link filled up. Every request slowed, including the tiny ones. The fix was to meter bytes, not requests.

A model server has the same bug. Say you load-tested Qwen3 8B on one H100 GPU with 40 chat users. Each chat held about 2,000 tokens, and each user saw about 71 tokens per second. That passed.

Then your team shipped an agent feature. Its sessions hold 100,000 tokens of code and tool output. Two of them join the 40 chats. The dashboard shows 42 running requests, only two more than the test. Every user now streams at about 35 tokens per second. Nothing crashed, and GPU memory still has room.

The GPU did not run out of memory. It ran out of reads per step. Your capacity plan counted users, and the cost is in tokens.

The fix: plan capacity in tokens in flight across all users, and admit each request against a speed budget, not a memory budget.

Read the full post on The AI Commit

How to Predict Each User's Token Speed on a Shared GPU carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.

Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

How to Stop a Model Picking the Wrong Fact From a Long Prompt

Mayur Agnani — Thu, 05 Nov 2026 11:44:20 +0000

A long prompt rarely gives a wrong answer just because it is long. It goes wrong because the extra text includes passages that look like the answer, and the model picks the one that best matches your question.

Engineer's view. This is a search ranking bug. Your site search returns the page that matches the query's words best, even when that page is out of date. A model reading a long prompt does the same thing. A passage that echoes your question can beat the true one. Every extra chunk you send adds rivals.

TL;DR. A model reading a long prompt favors the passage that best matches your question, true or not. Long prompts hurt mostly because they carry more look-alikes, so keep those out and ask questions only the right passage answers.

Explain Like I'm 5

You lost a blue umbrella, so you go to the lost-and-found desk. You say, "I lost a blue umbrella."
The clerk looks in the box and hands you the blue umbrella that best fits your words.

If the box holds one blue umbrella, you get yours. If it holds five, you may get someone else's.
A bigger box full of shoes does not confuse the clerk much. More blue umbrellas do.

So you do two things. You tell the clerk what only yours has: "the one with a broken handle." And
you ask them to set aside the umbrellas that clearly belong to other people.

The Problem

You have shipped this bug before, in a system with no model in it.

A customer types "refund window" into your help center. Search returns last year's refund page,
not the current one. Both pages match the words. The old one matches a little better, because it
says "refund window" twice. Nobody notices until a support ticket quotes a 14-day rule you
dropped in March.

A long prompt has the same bug inside it. Your agent's context holds tool results, retrieved
chunks and old turns. The model has to find the one passage that answers the question. It leans
toward the passage that best matches the question's wording, and it can be fooled by a passage
that only looks right.

Chroma tested 18 models this way in its report on what it calls context rot. Accuracy fell as
input grew, even on easy tasks. Adding one near-miss passage hurt, and four hurt more. On a memory
benchmark, every model scored much higher on a 300-token prompt than on the same task in 113,000
tokens.

I tried to break the length part on a current small model. Length alone did nothing to one easy
fact, out to 110,000 tokens. Four look-alikes cost 4 answers in 10, at 4,000 tokens and at 55,000.

The fix: keep look-alikes out of the prompt, ask questions that name what only the true passage
has, and test your own task with a length and look-alike sweep.

Read the full post on The AI Commit

How to Stop a Model Picking the Wrong Fact From a Long Prompt carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.

Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

How to Send Claude Only the New Messages in a Long Agent Loop

Mayur Agnani — Wed, 04 Nov 2026 12:54:33 +0000

Every turn of an agent loop uploads the whole conversation again. Claude can now keep it on the server, but only for five idle minutes, so your code has to be ready to send it all again.

Engineer's view. This is a sticky session. A load balancer pins a user to a server that keeps their session in memory, and the app breaks when that server restarts. A message thread is the API keeping your agent's conversation in memory. Treat it as a cache, or one restart loses the agent's work.

TL;DR. Claude can now hold an agent's conversation on its side, so each turn sends only what is new. The server can forget it at any time, so you keep the full copy and send it again when asked.

Explain Like I'm 5

You are telling a friend a long story over the phone, one piece a day. Every day you start again
from the very beginning, because you are not sure they remember. The story gets longer, so the
calls get longer.

Then your friend says: "I will remember. Just tell me the new part." That is much faster.

But your friend forgets if you wait too long between calls. So you keep the whole story written
down. When your friend says "sorry, I forgot," you read it all out again, and carry on.

The Problem

You have shipped this bug before, in a system with no model in it.

A web app kept each user's cart in the memory of one server. A load balancer sent the user back to
that server every time. It was fast, because nothing had to be loaded. Then a deploy restarted the
server, and every cart on it vanished. The fix was not a longer-lived server. It was writing the
cart to the database and treating memory as a cache.

An agent loop has the opposite habit today. The Messages API is stateless, so every turn sends the
whole conversation: the system prompt, every tool definition, and every file the agent has read so
far. Turn 40 of a coding agent uploads turn 1 to 39 again. The API then validates and tokenizes
all of it again before the model starts. In my simulated 40-turn loop, with a 6 KB file read per
turn, that came to 5.6 MB of uploads for one task.

Claude's API now has a beta called message threads. The server keeps the conversation, and later
turns send only what is new. It is the cart in memory again, with a five-minute timer on it.

The fix: send only the new messages on a thread, keep your own full copy of the conversation,
and write the replay path before the happy path.

Read the full post on The AI Commit

How to Send Claude Only the New Messages in a Long Agent Loop carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.

Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

How to Choose How Many Search Results a Reranker Should Rescore

Mayur Agnani — Tue, 03 Nov 2026 10:38:03 +0000

A vector search often ranks the right passage 20th or 30th. A reranker can lift it to first, but only if you handed it at least that many results, and every extra result costs time.

Engineer's view. This is the bug where you fetch ten rows with LIMIT, then filter them in app code, and the row you needed was eleventh. A reranker only reorders the results your vector search returned. If it gets too few, the right passage never reaches it. If it gets too many, you pay latency for distractors.

TL;DR. A reranker reads the question and each search result together, so it can fix an order the vector search got wrong. It can only fix what it was shown, so measure how deep the right answers sit before you choose how many to send.

Explain Like I'm 5

Imagine a librarian who sorts books by a one-line summary on each spine. You ask a question. The
librarian grabs the shelf of books whose summaries look closest and hands you a stack.

Then a careful reader opens each book in the stack and checks it against your question. The reader
is slow, but almost never wrong.

The careful reader can only check the books in the stack. If the right book was left on the shelf,
nobody finds it. And a taller stack means a longer wait.

The Problem

You have shipped this bug before, in a system with no model in it.

A page showed a user's ten newest documents. The query ran with LIMIT 10. Then the app removed the
ones the user was not allowed to see. Some pages came back with three items, and support asked
where the rest went. The rows existed. They were rows eleven to forty, and the filter never saw
them. The fix was not a smarter filter. It was fetching enough rows for the filter to work with.

Retrieval for a model has the same shape. Your app embeds the question, asks the vector database
for the nearest chunks, and puts the top few into the prompt. Many teams add a reranker in
between, a second model that rescores those chunks. What the reranker receives is decided by one
number, usually picked once and never measured.

I measured it on a public set of 300 science claims over 5,183 abstracts, using a common small
embedding model. For about one claim in seven, the right abstract sat below rank 20. For one claim in
fourteen it was below rank 100. Here is one claim: "Insulin increases risk of severe kidney
failure." The search ranked the right abstract 28th. The reranker moved it to first. With the
common setting of 20 candidates, it would never have seen it.

The fix: measure how deep your right answers sit, then send the reranker the smallest number of
candidates past which quality stops rising, and keep only a handful for the prompt.

Read the full post on The AI Commit

How to Choose How Many Search Results a Reranker Should Rescore carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.

Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

How to Fix Jargon in Speech-to-Text Without Adding Words Nobody Said

Mayur Agnani — Mon, 02 Nov 2026 10:34:12 +0000

Hand a transcription model a list of your product names and it spells them right. It also starts writing them where someone said a word that only sounds close.

Engineer's view. This is a search spellchecker with a custom dictionary. Add your product names and their misspellings get fixed, but nearby ordinary words get "corrected" too. A transcription prompt works the same way: a listed spelling wins wherever the audio is close to it. You get your jargon right and pay in confident wrong words.

TL;DR. Give a speech-to-text model a short list of your terms and it spells most of them right. It also writes those terms where someone said a word that only sounds alike, so test for both.

Explain Like I'm 5

Imagine a friend writing down everything you say at a noisy party. Before you start, you hand
them a card with the tricky names on it. Now they spell those names right.

But then someone says "Laura." Your friend glances at the card and writes the name from the card
instead.

The card helps your friend choose between words that sound alike. It cannot tell them which word
was actually said. So the card fixes some mistakes and makes a few new ones.

The Problem

You have shipped this bug before, in a system with no model in it.

Your search box had a spellchecker. Users typed your product names and got "no results," so you
added the names to its dictionary. The failed searches stopped. A week later, a support ticket
arrived. Someone searched for "laura" and got pages about your LoRA feature. The dictionary did
what you asked. It pulled every close word toward the words you listed.

A speech-to-text model is that spellchecker, running on sound.

You send it meeting audio, a support call or a video voiceover. It hears "Q drant" and writes
"Q-drant." It hears "K V cache" and writes "cavity." I read 15 AI terms to two OpenAI
transcription models with no help. whisper-1 spelled 14% of them right, and gpt-transcribe spelled
36% right. Nothing raised an error. The transcript just looked slightly wrong, and every search,
summary and caption built on it inherited the mistake. My own video captions hit this last week.
The voice said "halve" and the transcript said "have."

Both models take a hint: a text prompt, or a list of keywords. Add one and most of the jargon
comes out right. Some of it also comes out where nobody said it.

The fix: give the model a short list of the terms this recording is likely to contain, put the
important ones last, and test it on clips that say near-miss words, not only on clips that say the
terms.

Read the full post on The AI Commit

How to Fix Jargon in Speech-to-Text Without Adding Words Nobody Said carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.

Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

How to Stop a Long LLM Answer From Freezing When the Server Runs Out of Memory

Mayur Agnani — Sun, 01 Nov 2026 13:34:10 +0000

A model's memory grows with every word of its answer, and nobody knows the length in advance. Each LLM server handles that differently, so the same traffic can waste your GPU on one server and freeze a live answer on another.

Engineer's view. This is memory overcommit. Back every page up front and half your RAM sits idle. Overcommit, and an out-of-memory killer stops a process at peak. An LLM server faces the same choice, because each request's memory grows with its answer. You pay in idle GPU memory or in frozen streams.

TL;DR. An LLM server must decide how much memory to hold for an answer before the model has written it. Hold too much and you waste capacity. Hold too little and a running answer gets paused to make room.

Explain Like I'm 5

Picture a restaurant where nobody knows how long each family will stay. You could save a giant
table for every family, just in case. Most families leave early, so half your seats sit empty while
people wait outside.

Or you could seat everyone at small tables and hope. When the room gets full, you ask the newest
family to stand up and wait, even in the middle of dinner. They keep their food. They just can't
eat for a while.

An AI server runs that restaurant, and every answer is a family.

The Problem

You have shipped this bug already, in a system with no model in it.

You ran a service on Kubernetes. You set its memory request to the worst case you had ever seen.
Nothing crashed. But the cluster reported itself full while the real usage sat near 40%, and new
pods stayed pending.

So you lowered the request to the average. The cluster filled up nicely. Then a busy afternoon
came, several pods grew at once, and the kernel killed one in the middle of its work.

You were guessing a number before you knew the answer. An LLM server guesses that number for every
request.

The memory here is the KV cache: the model's saved work for every token so far. Without it, the
model would reread the whole conversation for each new token. It grows by one token per token
generated. The model only decides the length by stopping, so at the moment a request starts, nobody
knows its final size. The caller sends max_tokens, but that is a ceiling, not a forecast.

Under load, that guess shows up in two ways. Your GPU memory looks full while throughput sits far
below what you paid for. Or a streaming answer stops mid-sentence for many seconds, then carries on
as if nothing happened. Neither one logs an error.

The fix: find out which rule your server uses, measure how long your answers really are, then set
max_tokens and your concurrency limit from those lengths. New to the KV cache? Start at
How to Work Out How Much GPU Memory One Request Really Needs.

Read the full post on The AI Commit

How to Stop a Long LLM Answer From Freezing When the Server Runs Out of Memory carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.

Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

How to Tell What Claude Code and Codex Forget When They Compact a Session

Mayur Agnani — Sat, 31 Oct 2026 11:30:43 +0000

When a coding agent runs low on memory, it keeps what you told it and what it changed, and throws away nearly everything it only read. You can predict the loss, measure it, and tell the summary what to keep.

Engineer's view. This is a process restart with a handoff note. Whatever the process wrote down or saved to disk comes back. Whatever sat only in memory is gone, and nothing errors. A coding agent's memory is its conversation, so values it read but never wrote down vanish, and it must read them again.

TL;DR. A compacted agent keeps your instructions and its own edits, but loses most of what it only read. Whether a fact survives depends on how it entered the session, so you can predict the loss and tell the summary what to keep.

Explain Like I'm 5

You are building a huge model castle, and your table is full. So you ask a friend to write a
short note about the castle, then you clear the table.

The note says what you were told to build and which towers you finished. It does not list every
piece you looked at in the box. Those are still in the box, so you can look again.

But some pieces you borrowed for a minute, and they went back to a friend's house. If the note
does not mention them, you will not remember them, and you cannot go look.

So before you clear the table, tell your friend what to put in the note.

The Problem

You have shipped this bug already, in a system with no model in it.

Your service held a session cache in process memory. A deploy restarted it. The health check went
green and the error rate stayed flat. Then users started logging in again, because the sessions
never reached disk. Nothing crashed. The process simply came back without the state that had only
lived in memory.

A coding agent has the same split, and it restarts in the middle of your work.

Its working memory is the conversation. When that fills, the agent runs compaction: it
replaces the conversation with a summary and carries on. In my test session, 78,642 tokens became
2,632. That is 97% gone in about twenty seconds, and the session looks healthy afterward.

The agent then keeps working from the note. If a detail is missing, you see no error. You only see
it read the same file twice, or ask something you already answered. Worse, the detail may have
come from a test run or a web page that no longer exists.

You could treat this as bad luck. It is not. What survives follows a rule.

The fix: know which lane each fact is in, tell the summary what to keep, and write anything you
cannot regenerate to a file before compaction runs.

Read the full post on The AI Commit

How to Tell What Claude Code and Codex Forget When They Compact a Session carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.

Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

How to Tell Whether a Second Model Reviewing Your Agent's Work Pays for Itself

Mayur Agnani — Fri, 30 Oct 2026 11:39:40 +0000

Adding a second model to review your agent's work sounds like free safety. It only fixes the mistakes the reviewer does not share, and its false alarms can break work that was already right.

Engineer's view. This is running two replicas of the same software for safety. Both crash on the same leap second, because they share the bug. A reviewer model trained like the drafter misses the same failures. Pick a reviewer that fails differently, and measure what it catches on your drafter's failures, not on its own.

TL;DR. A reviewer model can only fix errors it would not have made itself, and models share many blind spots. Measure what it fixes and what it breaks on your own tasks, then review only where the drafter often fails.

Explain Like I'm 5

You write a story for school. Before you hand it in, your twin reads it to look for mistakes.

Your twin is smart, but your twin learned spelling from the same teacher. The words you get
wrong are the words your twin gets wrong too. So those mistakes slip through.

A friend from another school spots them, because they learned differently. But sometimes the
friend says a right word is wrong. If you change it, your story gets worse.

So the best helper is someone who makes different mistakes than you do. And they should only
read the hard stories, where you make the most mistakes.

The Problem

You have shipped this bug already, in a system with no model in it.

Your service ran two replicas for safety. If one crashed, the other would take the traffic. Then
a leap second arrived. Both replicas crashed in the same minute, because both ran the same code
with the same bug. The backup was real, and it protected you against nothing that mattered.
Engineers call this a common-mode failure: one cause takes out every copy at once.

Redundancy only helps against failures the copies do not share.

A reviewer model is the same idea applied to an agent. One model, the drafter, writes a
plan or a patch. A second model reads it and lists concerns. The drafter then revises once.
Some coding tools now run this step for you. You can add it to your own agent in an afternoon.

It costs at least two extra model calls per task. Whether it pays depends on two things the
setup hides. The reviewer misses whatever blind spots it shares with the drafter. And when it
raises a false alarm on a correct draft, the revision can break it.

The fix: measure what the reviewer fixes and breaks on your drafter's own tasks, pick one
that fails differently, and call it only where the drafter often fails.

Read the full post on The AI Commit

How to Tell Whether a Second Model Reviewing Your Agent's Work Pays for Itself carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.

Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

How Many Pull Requests a Merge Queue Can Land in an Hour

Mayur Agnani — Thu, 29 Oct 2026 09:22:28 +0000

A merge queue tests each pull request on top of the ones ahead of it, so one failure restarts everything behind it. That sets a ceiling on how many pull requests it can land per hour, and coding agents now send more than it can take.

Engineer's view. This is optimistic locking under contention. Each transaction validates against everything committed before it, and one abort throws away the work queued behind it. More workers only waste more work. Throughput is set by the abort rate and the transaction time. Agents raise the load and leave that ceiling where it was.

TL;DR. A merge queue's speed is capped by how often a queued change fails and how long CI takes, not by how many pipelines run at once. More pull requests from agents push it into long waits unless you cut one of those two.

Explain Like I'm 5

A school lunch line has one rule. Each kid must show their tray to the cook, and the cook
checks it against every tray that went before.

The cook can look at twenty trays at once. If a tray is wrong, that kid steps out. Every kid
behind them has to show their tray again, because the cook checked it next to the wrong one.

On a normal day the line moves fine. Then a class of fast kids joins. They bring three times
as many trays. The cook is not slower. The line still backs up, because each wrong tray sends
everyone behind it back to the start.

The Problem

You have shipped this bug already, in a system with no model in it.

Your service used optimistic locking. Each write read a row, changed it and committed only if
nobody else had changed it first. At low traffic it was fast. Then traffic tripled. Writes
started aborting and retrying, and each retry raced the next wave. Latency went from
milliseconds to seconds. Adding app servers made it worse, because they only added contenders.

The limit was never the servers. It was how often a commit failed, times how long it took.

A merge queue has the same shape. It is the gate that lands pull requests on main one
tested batch at a time. GitHub calls it the merge queue. GitLab calls it a merge train. Each
pull request is tested on top of main plus every pull request ahead of it, so main never
breaks. When one fails, everything behind it was tested on a tree that included the failure.
All of it reruns.

Coding agents change the arrival rate. At Anthropic, merges grew from about 2,500 a day in May
2026 to over 30,000 a day by September. They stopped using a merge queue. A team without that
option has to know where its queue's ceiling is.

The fix: measure your queue's failure rate and CI time, compute its ceiling, and lower the
failure rate before adding pipelines.

Read the full post on The AI Commit

How Many Pull Requests a Merge Queue Can Land in an Hour carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.

Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

How to Find the Agent Actions a Prompt Injection Can Control

Mayur Agnani — Wed, 28 Oct 2026 11:53:06 +0000

An agent that reads an email can be steered by whoever wrote that email. Which of its next steps the writer can reach is not luck. You can list them before you ship, and design most of them out.

Engineer's view. This is SQL injection without parameterized queries. A database fixed it by sending the query and the values on separate channels. A model reads both in one stream, so it cannot. The fix moves up a level: decide each action before reading the values, or read them somewhere that cannot act. It costs flexibility.

TL;DR. A model treats every word in its context as a possible instruction, including text a tool fetched. Any decision your agent makes after reading that text belongs to whoever wrote it.

Explain Like I'm 5

A kid has a list of chores from their parent. One chore says "read the notes in the mailbox
and do what's needed."

One note in the mailbox says "Also, leave the back door open." It is written in pen, like the
parent's list. The kid cannot tell which notes came from the parent.

So the kid leaves the door open. The note writer did not break in. They just wrote a note in
a place the kid was told to read.

A smart parent writes the whole chore list first. The mailbox can change what the kid writes
down. It can't add a chore.

The Problem

You have shipped this bug already, in a system with no model in it.

A support page showed customer names. Someone signed up with a name that contained a
<script> tag. The page printed the name, and the browser ran it as code. Every agent who
opened that ticket ran the attacker's script with their own login. Nothing was hacked. Data
was put in a place where code was expected.

The fix you shipped was a boundary. The browser now treats the name as text and never as code.

An AI agent has the same bug, and no such boundary. An agent is a model that calls tools in a
loop. A tool might read your inbox, fetch a web page, or open a GitHub issue. Whatever the tool
returns goes into the context, the text the model reads before each step. Your request is
in there too. The model sees one stream of words and cannot tell your instruction from a
sentence inside an email.

This is prompt injection: someone writes instructions into content your agent will read.
Filters catch some of it. A filter that catches 95% of attacks still lets one in twenty through.

The useful question is not whether a model can be fooled. It can. The question is what the
fooled model can still do.

The fix: list every decision your agent makes after it reads untrusted text, then move each
one before that read or into a context that cannot act.

Read the full post on The AI Commit

How to Find the Agent Actions a Prompt Injection Can Control carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.

Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.

How Claude Code and Codex Decide Which Skill Descriptions the Model Reads

Mayur Agnani — Tue, 27 Oct 2026 11:44:35 +0000

Your coding agent lists every skill you install in a small, fixed slice of its memory. When too many skills compete for it, some lose the words that tell the model when to use them, and nothing warns you on screen.

Engineer's view. This is a cache that evicts the least-used entry. A key you just added has the lowest hit count, so it goes first, and it never earns the hits that would keep it. Claude Code trims its skill list the same way. A skill you installed today is the first to lose its description.

TL;DR. A coding agent shows the model only a short list of your skills, and that list has a size limit. Past the limit, descriptions get cut, and a skill the model cannot read about is a skill it rarely picks.

Explain Like I'm 5

A school has one small board by the door. Every club pins a card with its name and what it
does. Kids read the board to decide where to go after lunch.

Then too many clubs sign up. The teacher keeps every name on the board but crosses out the
descriptions of the clubs that got the fewest visitors lately. A brand-new club has had no
visitors yet, so its description is crossed out first.

A kid who loves chess walks past a card that just says "Knights". Nothing tells them it is the
chess club. So the new club stays empty, and its card stays crossed out.

The Problem

You have shipped this bug already, in a system with no model in it.

Your service wrote one log line per request. Someone added a field near the front, and the line
crossed the log shipper's size limit. The shipper did not fail. It cut each line at the limit and
sent the rest. The request ID sat at the end of the line, so searching by request ID stopped
finding anything. No alert fired, because every line still arrived.

The bug was not in any one field. It was a shared size limit, plus a rule for what gets cut when
the limit is hit. Adding something new quietly took words away from something old.

A skill in a coding agent has the same bug. A skill is a folder with a SKILL.md file: a
name, a description, and instructions for one kind of task. Coding agents that support the open
Agent Skills format do not load your skills up front. They show the model
a skill listing: each skill's name and description, and nothing else. The model reads that
listing and decides which skill fits your request.

The listing has a size limit, a fixed share of the context window, the model's working
memory for the session. Install enough skills, and the listing overflows. Each tool then cuts
it by its own rule. A skill whose description was cut can still be listed and still never be
chosen.

The fix: keep the listing under its budget, put each skill's use case in its first sentence
and its name, and check what the model actually sees.

Read the full post on The AI Commit

How Claude Code and Codex Decide Which Skill Descriptions the Model Reads carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.

Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.