DEV Community: Nabbil Khan

What a Right Answer Costs

Nabbil Khan — Mon, 26 Oct 2026 14:01:51 +0000

My AI agent could call an insurance company for about $0.98. An offshore biller cost about $5 for the same call. I had the spreadsheet open and thought I was done. Then I added one more column: how many calls came back with an answer I would actually trust. That column made the agent the more expensive option.

What you divide by

Every AI cost model I've seen divides by calls. Cost per call, cost per minute, cost per token. Those numbers are easy to get because the meter gives them to you.

But nobody in medical billing wants a call. They want to know if a claim was received, why it was denied, and what it will take to get it paid. The call is only how you find out. So what does finding out cost?

In July my agent got 5 to 10 right answers per 100 calls. At $0.98 a call, each right answer cost somewhere between $10 and $20. Say it got 7 right out of 100. That's $14 an answer. Now take the offshore biller at $5 a call, and assume they only get the answer half the time. That still works out to $10 an answer. The person was cheaper. I just couldn't see it, because I was dividing by the wrong thing.

$0.98 AI cost per call
$5 offshore biller per call
5 to 10 right answers per 100 calls in July
$10 to $20 cost per right answer

Key insight: The real unit cost of a voice agent is what it pays per right answer. Cost per call doesn't tell you that.

Wrong is worse than nothing

July was bad. October 21 was worse.

That day the agent didn't just fail to get answers. It got wrong ones and saved them. On 8 of its last 11 calls, it wrote results into the record that weren't true.

What broke: On October 21 the agent saved wrong results on 8 of its last 11 calls. Each one looked like a finished task, so nothing downstream knew to check it.

Why is that worse than a dead call? A dead call is honest. It shows up as a failure, someone sees it, and someone calls again. A wrong answer in the record looks like success. The claim moves forward on a false fact. Maybe a biller acts on it. More likely nobody acts at all, because the system says it's handled. By the time the money fails to show up, weeks have gone by and nobody remembers why.

So a wrong answer doesn't cost zero. If you're honest about the math, it costs less than zero.

A wrong answer in the record costs more than no answer at all.

It isn't only billing

I build systems in a few industries that have nothing to do with each other. This same problem shows up in every one.

I build a quoting engine for water-feature contractors. A fast quote that comes out wrong is a disaster. If you under-quote a pond, you pay the difference out of your own pocket on the job. So the engine has a check that refuses to under-quote. Nobody asks how fast it quotes. They ask if they can sign the number.

The denial agent works the same way. Drafting a hundred appeals for pennies means nothing if the few that matter cite the wrong policy. What you're paying for is the appeal that gets the claim paid.

Training local models taught me the same thing again. A run that finishes fast and scores well is worth nothing if the grader is wrong. I've been fooled by a bad grader more than once.

In each case the meter counts raw output, and raw output is cheap. What matters is the one output you'd stake money on. Those two numbers can be ten times apart. You can go a long time without noticing, because the first one is on the invoice and you have to dig for the second.

What I do now

Divide by answers you'd bet a claim on. If you can't count those, you don't know your cost yet.
Count wrong answers as negative. A saved false result has to cost more than a blank, or your system will learn that confident guessing pays.
Make it easy for the agent to say "I don't know." An honest blank is cheaper than a confident mistake.
Put the human through the same math. The offshore biller doesn't cost $5. They cost $5 divided by how often they're right.

The result: Once I priced by right answer, I stopped asking how to make calls cheaper and started asking how to make answers right. Only the second question lowers the cost.

The agent is still cheaper per call than any person I could hire. But that was never what I needed to know. I needed to know what a right answer costs. For a while, it cost more than I was paying before.

If you run voice agents or offshore teams in revenue cycle, how do you count a right answer, and who checks it?

Originally published at nabbilkhan.com.

What Not to Hear

Nabbil Khan — Mon, 19 Oct 2026 14:06:50 +0000

Five minutes. That is how long the recruiting system waits after anything in a candidate's file changes before it rewrites the summary at the top. A recruiter adds a note, a resume comes in, a call gets logged, and five minutes later the summary catches up. It is a good number. It is also a trap, and I did not see the trap until I asked a dumb question: what counts as a change? The summary lives in the file. Rewriting the summary changes the file. So five minutes after the summary rewrites itself, it rewrites itself again. Forever.

The loop

There is nothing special about this loop. Any system that reacts to a store and also writes to that store has it. The recruiting system reads the candidate file and writes a summary into it. The pipeline that publishes my worklog reads my commits and writes a journal entry, which is itself a commit. The denial agent in the billing operation reads a claim and writes a note back onto it. Three different businesses, and the shape is the same in all of them. The machine reads a thing, says something about it, and puts what it said in the same place it was reading from.

The wasted tokens are the obvious cost. The worse cost is quieter. Every signal built on top of "did this change" goes bad. The hashes in a candidate file exist to answer one question: has the evidence for this person moved since the last pass? If the summary is in the hash, the answer is always yes. The hash stops meaning the candidate changed and starts meaning the machine did something. And the machine always did something.

What broke: My first instinct was to hash the whole file, summary included. That hash would have flipped every five minutes on every candidate. A recruiter looking for what actually moved would have seen everything flagged, and would have stopped trusting the flag within a day.

Fix the write, not the trigger

The fix is not a smarter trigger. I spent a while thinking about it that way. Filter on who made the change. Check a timestamp. Compare the new summary to the old one and skip if they match. Each of those works until the next write path shows up, and there is always a next write path. The real fix lives where the write happens. When the system writes something, it decides right there whether that write is something the system is allowed to hear later.

There are three of these silent writes in the recruiting system now.

The dossier write-back emits no audit event. When the machine updates a candidate's dossier, nothing goes into the stream the trigger listens to. A recruiter's edit goes in. An inbound resume goes in. The machine's own update does not.

The AI summary notes are kept out of the evidence and out of the hashes. The note still exists. You can open it and read what changed and why. But it is not evidence, so it does not count toward what we know about this person. And it is not hashed, so it does not count toward whether this person changed.

A bulk item-create emits nothing. Extraction pulls a resume apart into dozens of items. If each item fired a create event, extraction would wake up extraction. So the bulk path is silent on purpose. One resume, one pass, done.

Key insight: A system that reads and writes the same store will loop on its own output unless you decide, write by write, which of its own changes it is deaf to.

5 min wait after a change before the summary rewrites
3 kinds of writes the system cannot hear
0 audit events from a dossier write-back
159 journal entries the site pipeline reads and writes

The same shape everywhere

Once I had the list, I started seeing it in every system I run. The site pipeline is the clearest case. It reads what I shipped and writes a worklog entry, refreshes the operation graph, and adds to an insight journal that is now 159 entries long. Every one of those outputs is a commit. If the pipeline treated its own commits as work, it would publish an entry about publishing an entry. So its own commits are the writes it is deaf to.

The billing side has the same shape and higher stakes. A voice agent calls a payer and writes the outcome onto the claim. The denial agent watches claims. If the call note counted as a claim change, the denial agent would re-evaluate the claim and write its own note, and the voice agent's scheduler would see a changed claim and call the payer again. Nobody designs that loop on purpose. It falls out of two reasonable agents sharing one table. So the same question gets asked there before anything ships: which of these writes does the other agent get to hear?

Anyone can wire a trigger. The hard part is choosing which of its own writes the machine is not allowed to see.

Silence leaves no trace

Here is the part that still bothers me. A write that emits no event is a write the audit log does not show. I chose that on purpose, and I would choose it again. But it means the silence is a fact I have to carry somewhere, because it does not carry itself. Logs show what happened. They never show what was deliberately not heard. So the list of deaf writes is written down next to the trigger, and it is the first thing I read when the summary does something I do not expect.

The result: The summary now rewrites within five minutes of a real change and stays quiet after its own. The audit stream shows recruiter edits and intake events, not the machine talking to itself.

Write down every write the system makes before you wire the trigger. The trigger takes an afternoon. The list is the design.
Make your own output silent by default. If the machine wrote it, the machine does not get to hear it, unless you can say why it should.
Keep your output out of the hash. A hash that includes the summary says "changed" every time the machine runs.
Bulk writes emit nothing. If a step creates a hundred items, it must not wake the step that creates items.
Keep the deaf list where a person will read it. Silence leaves nothing in the logs, so it needs to leave something somewhere else.

The trigger took me an afternoon, and it would take you one too. The week went into sitting with a list of every write the machine makes and asking, for each one, whether the machine should be allowed to hear it. A system that listens to everything ends up mostly hearing itself.

If your agent writes to the same table it watches, how do you mark its own writes so it skips them?

Originally published at nabbilkhan.com.

Don't Ask the Model What It Did

Nabbil Khan — Mon, 28 Sep 2026 14:02:24 +0000

My denial engine kept citing appeal deadlines that did not exist. The claim was real. The denial was real. The deadline was made up. And right next to it, in a neat little field, the model had written down which lookups it ran and what they returned. It said it had checked. It hadn't. So I had a system writing its own alibi, and the alibi was wrong.

The model was a witness, not a log

Here is how the engine worked. A denied claim comes in. The model reads it, calls tools to pull payer rules and filing timelines, and writes an appeal plan. Along with the plan, it returned a field describing its own run: which tools it called, which sources it touched, where each fact came from.

That seemed reasonable when I built it. Who knows better what the model did than the model?

It turns out almost anyone does. A model does not remember its run the way a log does. It rebuilds it after the fact. When it wants a deadline and the data is thin, it fills the gap. Then it tells you, in good faith, that there was never a gap. In most software that would be odd. In medical billing, a wrong appeal deadline can cost you the claim.

An audit trail the model writes about itself is testimony. You would not let the defendant write the court record.

What broke: For a while I stored the model's account of its own run as fact. The reconciliation pass kept flagging deadlines the data could not back up, and every one of them came with a confident provenance line saying it had been checked.

Let the server keep the record

The fix was boring, which is usually a good sign. I put a callback on the tool layer. Every time a tool runs, the server writes down what was called, with what inputs, and what came back. The model is never asked. It can't edit that record, and it can't improve on it.

Now when the appeal plan names a deadline, I check it against what the tools actually returned. If no tool returned a date, there is no date. The plan goes back for review instead of out the door.

Key insight: If the system can see something for itself, never ask the model to report it.

The second bug was the same bug

Then something surprised me. I had a separate problem on a separate list. The small model I used for this step kept returning JSON that would not parse. I had it filed under flaky output, and I was treating it the usual way: retries, stricter prompts, more examples.

When I looked at where the parse failures happened, most of them were in the audit trail field. That was the longest and most nested part of the output. It was also the part the model had no business writing. The hardest thing to get right was the least worth having.

So the field is coming out of the output contract entirely, and the server builds the whole trail. Retries only make a broken field fail less often. A field that isn't there can't fail at all.

0 model self-reports stored as fact
1 field leaving the output contract
2 bugs traced to that one field

The result: Once the server kept the record, every deadline in an appeal plan could be traced to a real tool result. The fabricated provenance stopped, and the biggest source of parse failures now has a date to disappear.

Why it travels

I build and run AI systems in a lot of unrelated fields, and this pattern keeps coming back. People ask the model to do the task, and then to describe the task. Those are two jobs. The second one is almost always something the code around the model already knows.

The payer-calling voice agents are the clearest case. An agent can say it confirmed a claim status. The call recording and the transcript can say whether it did. In the quoting engine, the model can explain how it got to a price, but the pricing code already knows every line it added. In a local training loop, the model can tell you it improved, but only the eval harness gets to say so. Different industries, same mistake. Each time you hand the second job to the model, you add a way to be lied to and a way for the output to break.

So the rule I use now is simple. If only the model can produce something, it stays in the contract. If the system can see it, it comes out.

Don't ask the model what it did. Record it where it happens.
Treat every field in the output contract as a place to fail, and cut the ones you can compute.
When two bugs keep turning up in the same spot, check whether they are one bug.
Get provenance from your logs, never from something that gains by sounding sure.

The model is good at the work. It is just not a reliable witness to the work. Once I stopped asking it to be one, the system got more honest and more stable at the same time, which makes sense, since it had been one problem all along.

Which field in your agent's output could your own system compute instead?

Originally published at nabbilkhan.com.

Prefer Loud Bugs

Nabbil Khan — Mon, 14 Sep 2026 14:03:17 +0000

The rule took four minutes to write. If a contact already has an open opportunity, do not make a second one, leave a note instead. Nobody argues with a rule like that. Then I opened the CRM it writes into and counted twenty open opportunities, all created in the last ten weeks, and not one of them marked won or lost in the entire history of the account.

The field had never moved

Sit with that for a second, because it took me a while. My rule says skip while the opportunity is open. The data says opportunities here do not close. Nothing has ever gone from open to anything else. So open is not a stage. It is a permanent property of every record anyone has ever made.

Which means I did not write a dedupe rule. I wrote a rule that mutes a person for life the first time they fill in the form. After that, the only thing the agent ever does for them is leave a note, and notes get read by nobody. I built a machine whose failure mode is silence.

Key insight: A rule that keys on a status field is a bet that somebody will eventually change that field, and when nobody does, your agent goes quiet instead of loud.

That is the part worth keeping. Loudly wrong is fine. Loudly wrong gets caught in a day, because somebody gets two emails and tells you. Quietly wrong gets caught never. No one writes in to complain that the agent did not contact them.

20 open opportunities in the account
10 weeks since the oldest one opened
0 ever marked won or lost, all time

Same bug, different industry

I am writing this down instead of just fixing it because I have hit the same thing in businesses that have nothing to do with sales.

In the reimbursement engine I run for remote therapeutic monitoring, each patient gets a green dot for the month once a live call happens. The check is an EXISTS against the billable ledger. Same shape as the dedupe rule: look at a field, decide whether to act. It works, and it works for a boring reason. Somebody logs a call every month, the row appears, the dot flips. The state moves, so the check means something.

Now the other side. In the denial engine we queue batch jobs and hand the caller a URL to poll. For a while the local worker only woke up when something read the jobs collection. A client doing exactly what our docs said (post a batch, poll the URL you were given) never touched that collection. So the job sat there. No error. No 500. Just a status field that would have changed, if anything had ever come along to change it.

The bug was never in the rule. It was in assuming somebody keeps the field honest.

Three industries, one mistake. B2B sales, clinical billing, claims automation. Every time I wrote a condition against a field and quietly assumed a person or a cron job would keep that field true.

What broke: I shipped a dedupe rule against a status field that had never once changed value, and the cost was invisible: people who wrote in, got a note, and never heard from us again.

Go look at the field before you key on it

The fix is not clever. Before you write a condition against a state field, go read the history of that field. Not the schema. Not what the sales rep says the process is. The actual spread of values across the actual rows.

One query. How many records sit in each state, and when did any of them last move? If everything is in one bucket and nothing has moved in ninety days, that field is not a state machine. It is a label somebody applied once and forgot. You cannot hang a decision on it.

That question saved me again last month in the quoting engine, where I almost gated follow up on a quote status nobody updates after they send the PDF.

The result: The monthly check in the billing engine holds up for one boring reason: a human logs a live call every month, so the ledger really changes, and an EXISTS against it means something.

Look at the distribution before you key on the state. If one value holds every row, it is not a state.
Ask when the field last changed, not what it is supposed to mean.
Prefer time to status. No outreach in 30 days fails softly. No open opportunity can hang forever.
Give every skip a loud path. If the agent has said nothing about a contact for a month, that is an alert, not a note.
Make silent failures expensive to ignore. What you cannot see, you will not fix.

Loud is a feature

I have spent years making agents careful. Guard the submit. Do not double up. Check before you act. That instinct is right, and it is also how you end up with a system that never does anything wrong and never does anything at all.

A duplicate email costs you a bad afternoon. A muted prospect costs you a quarter you never find out about. When you are picking between the two, pick the noisy one. You can always turn the volume down later.

Go look at your state fields. Not the code. The data.

What state field in your stack does everyone trust, that nobody actually updates?

Originally published at nabbilkhan.com.

The Quota Picked the Model

Nabbil Khan — Mon, 07 Sep 2026 14:04:24 +0000

A line in my denial engine said the model was Sonnet. Every assessment that engine has ever served ran on Haiku. Nobody edited the config, nobody shipped the wrong build, and the line was true the day I wrote it. It just was not true any time the thing ran.

The number was real, the name was not

The engine takes a denial, reads the codes and the payer language, and scores the odds of winning the appeal. It has a primary model and a fallback. Primary was Sonnet. When the primary comes back with a 429, the code drops to Haiku and keeps going, which is what you want at 2 AM CT with four hundred claims to clear before the morning.

On my key, Sonnet 429s. Not sometimes. Every time. So the fallback was never a fallback. It was the road. The primary was a door that had been locked since before I started measuring anything.

I found it while chasing something else, and then I had to go back and re-label the whole eval set.

94 eval cases scored
100% of them answered by the fallback model
400 claims in a single overnight batch
0 lines of code that had to change to swap models

The numbers were fine. They came from real runs on real denials, and I still trust them. What was wrong was the sentence I kept saying out loud about them.

What broke: For weeks I called the engine's accuracy a Sonnet number in rooms where the difference mattered, and every one of those numbers came from a model I had never once said the name of.

The upgrade is the regression

Here is the part that should bother you more than the wrong name.

Think about who actually gets Sonnet. Not me. A client on a paid key with real headroom, whose requests never hit a limit, gets the primary on every call. The person with the better account is the only one running the model I never tested. Same commit, same container, same config file. The code did not change. The quota did.

We are trained to worry about the other direction. You picture the slow machine, the bad network, the throttled key, and you make the thing degrade gracefully. Nobody writes an eval for degrading upward.

A rate limit is a config value, so an eval score belongs to the account that produced it.

Key insight: An eval score is a fact about the account that ran it and not about the code, so the moment you hand that code to someone else the score quietly stops being true.

I pinned it. claude-haiku-4-5-20251001, spelled out with the date, because Haiku is what I actually validated. If I want Sonnet numbers I can go get Sonnet numbers, on a key where Sonnet answers.

The result: The engine now runs the model its numbers came from on every account, and each assessment records which model replied instead of which one was configured.

The same bug in other rooms

Once you see this shape you cannot stop seeing it. I run systems in a few different industries, and it turned up in all of them inside one week.

An appeal deadline stored as 2026-10-13 rendered as October 12 on a laptop in Chicago. The date in the database was right. The reader's clock decided what it said. Anyone west of UTC lost a day on a deadline with money attached to it.

On the Medicare remote-monitoring billing platform, invite and password-reset mail never reached the testers. The code was correct. The sending account sat in a sandbox that only delivers to verified addresses, and mine was verified, so it worked perfectly for the one person checking whether it worked.

In the same denial engine, the local batch worker only woke up when something asked for the jobs list. A caller following the documented flow, post a batch and poll the URL you are handed, never touched that route. The job sat queued forever while the code was, technically, doing its job.

On the local box that serves open source models, which model answers depends on what fits in memory that night. The config has an opinion. Memory has the last word.

Different systems, one shape. The code is identical on every machine. The behavior gets picked by things nobody wrote down: a quota, a sandbox flag, a clock, a route somebody happened to hit.

What I do now

Pin the model. A fallback that fires every time is not a fallback, it is your default in a costume.
Log what answered, not what was configured. The record should say Haiku because Haiku replied, not because a variable claimed so.
Treat the account as an input. Quota, verified senders, region, sandbox state, clock. All of it is config you never wrote and will never see in a diff.
Re-run the evals where the code is going to run. A score from my key is a fact about my key.
Ask which way a difference fails. If the better environment is the untested one, the upgrade is the regression.

I used to think the code was the product and the environment was setup. It is closer to the other way around. The commit says what the system is supposed to do. The account it runs on decides what it does.

If you have a number you say out loud, go look at what actually answered. You can read every line and still not know. Some of it is written somewhere else.

Which environment fact, a quota, a region, a sandbox flag, has changed your system's behavior without a line of code changing?

Originally published at nabbilkhan.com.

Most of It Says No

Nabbil Khan — Mon, 17 Aug 2026 14:10:55 +0000

I went looking for the model call in our denial agent last month and it took me four minutes to find it. One function, about forty lines. Everything around it, eleven files, exists to decide whether that function is allowed to run at all. I had built what I thought was an AI system, and most of it was a system for not using AI.

The same shape in three places

I run software in industries that have nothing to do with each other. One system reads insurance denials and drafts appeals. One prices water features for contractors, the pump, the liner, the rock, the labor. One calls insurers and waits on hold for forty minutes so a person does not have to. Different buyers, different rules, different words for everything.

They came out the same shape. Each one has a small part that produces an answer and a large part that decides whether the answer is allowed to leave. I did not design that. I noticed it the third time.

It goes further down than I expected. On my own hardware I run open models, Qwen and DeepSeek, and their main job is not to answer anything. They read the input and decide whether the expensive model ever sees it. A gate in front of a gate.

What guessing costs

Early on I let the model do arithmetic in the quoting engine. It read the job, picked the parts, added up the bill of materials, and produced a number that looked completely reasonable. It was low by about nine percent. In construction, low is the direction that takes money out of the contractor's pocket, and nobody notices until the hole is already dug.

What broke: A quote went out under cost because I let a language model add. The number looked right, which is exactly why nobody checked it.

Pricing moved into plain code that afternoon. The model picks parts and writes the description. Multiplication belongs to a function that cannot be talked into anything.

The denial agent taught me the same lesson in a different accent. It once wrote a careful, well argued appeal that cited a document we did not have. Nothing was wrong with the writing. The rule now is boring: no source in hand, no sentence on the page. I did a Master's in Language and Theology before I wrote a line of production code, and the useful thing that training gives you is a nose for the gap between what a text says and what you wish it said. That turns out to be most of the job.

Abstention is a dial

Everyone tunes accuracy. Almost nobody publishes the other number, which is how often the system declines to answer. The two move together, and the second one is the one you actually control. Our denial classifier runs at ninety percent on the cases it keeps, and it keeps about seven out of eight. I can push coverage to everything and watch accuracy fall, or refuse more and look useless. The work is choosing the point and being able to defend it.

90.0% accuracy on the denials it keeps
1 in 8 cases handed to a person instead
3 independent checks before a quote can leave

The dial only matters if the refusal has somewhere to go. This is the part I got wrong for a year. A system that stops is not automatically a safe system.

If the machine stops and nobody gets the work, you have not built caution. You have built a hole.

So every refusal now has an address. The denial agent hands the case to a biller with the reason attached. The voice agent drops the call into a human queue with the recording and the time in Central. The quoting engine flags the line item and will not render a PDF until someone signs off on it.

The result: Since pricing left the model, no quote has gone out under cost, and the hard jobs land in a review queue instead of a customer's inbox.

Key insight: The part of an AI system that decides not to answer is what makes the rest of it safe to put in front of anyone.

What to build first

Write the refusal path before the happy path. It is harder, and you will not go back and do it later.
Give every refusal an address: a queue, a person, a fallback. Silence is not a destination.
Take arithmetic away from the model. Anything with a right answer belongs in code.
Report coverage next to accuracy. One number without the other is marketing.
Watch the gate, not the model. The gate is the part that fails quietly.

None of this is clever. It is the least interesting code I write, and it is the reason the interesting code is allowed near a customer at all. The people shipping demos are tuning the model. The people running things are mostly building the part that says no, and they are mostly doing it alone, which is a shame, because the shape is the same in every industry I have worked in. If you are building that part, I would like to compare notes.

Originally published at nabbilkhan.com.

Nobody Asked the Model

Nabbil Khan — Mon, 10 Aug 2026 14:06:55 +0000

For months my eval dashboard said my skills scored 0 percent, and I believed it. Zero is a believable number for a young system. So I rewrote prompts. I restructured skills. I tightened trigger phrases, reran the suite, and watched the zero not move. The whole time, no model was ever asked a single question. The harness had quietly swapped credentials on me. Every request died at the front door with a 401, and every 401 was scored as a wrong answer. I spent months improving the answers of a model nobody asked.

One line of fallback

The bug was one line. Line 90 of run_eval.py had a fallback: if no API key was set, use the OAuth token from my Claude Code session as an API key. It looks harmless. It is the kind of line you write at midnight so a script will run on a fresh machine. But an OAuth token is not an API key. The API rejected every request with a 401, the harness caught the failure, and it had nowhere to put it except the answer column. Wrong credential became wrong answer. Wrong answer became 0 percent. And 0 percent became a fact I planned around.

What should have tipped me off was that the number never moved. Real evals are noisy. Skills improve a little, regress a little, flake on one case out of twenty. A metric that sits at exactly zero for months is not describing a model. It is describing something upstream that fails the same way every time. Flat numbers are plumbing numbers.

0% reported skill accuracy, for months
401 what every eval case actually returned
1 line of fallback code behind the whole number
0 questions any model was asked

What broke: I treated 0 percent as a fact about my skills for months. I rewrote prompts and restructured workflows to move a number that was actually about a credential. The fix was deleting one line, and the score I had been chasing had never once been about the work.

Two kinds of fallback

I build and run AI systems in several industries: a medical billing operation with a denial engine, an autonomous ad operator named Mia, a quoting engine for water-feature contractors, and the harness that evals my own agents. The same shapes keep turning up in all of them, and this bug taught me to sort fallbacks into two kinds.

A fallback in a compute path costs you quality. The big model times out, so you route to a smaller one. The cache misses, so you recompute. You get a worse answer or a slower one, but it is still an answer about the right subject. Degrading there is fine. Degrading there is the point.

A fallback in an identity path costs you the truth. Identity is the part of the system that says who is talking: which credential, which user, which account. When that part fails and something helpfully substitutes a different identity, nothing crashes. The run finishes and hands you a confident number about the wrong subject. My 0 percent was not a measurement of my skills. It was a measurement of a token, laundered through an eval harness until it looked like a measurement of my skills.

Key insight: A fallback in a compute path costs you quality. A fallback in an identity path costs you the truth, because the run still finishes and hands you a confident number about the wrong subject.

The same shape in an approvals dashboard

A while later the same shape showed up in my ad operation. Mia is my autonomous advertising operator. Mia drafts the moves, and a human approves the risky ones through a dashboard behind Cloudflare Access, which stamps every request with a JWT that says who you are. The tempting fallback was obvious: if the JWT is missing, fall back to a session cookie so the dashboard keeps working.

I said no, and the eval bug is why. An approval attributed to a person who did not approve is worse than a login screen. A login screen costs you a minute. A false approval is a record that lies, and it lies in the one place you will later go to find out what happened. When identity fails, the only honest state is blocked. So the dashboard fails closed. No JWT, no page.

The result: The Mia approvals dashboard fails closed behind Cloudflare Access: no JWT, no page. Every approval in its history is attached to a person who actually clicked it.

The rule under both systems is the same one.

A 401 is not a wrong answer, and a missing JWT is not an approval.

An infrastructure failure is not a verdict about the work. My denial engine draws the line in the same place: its audit trail records which tools actually ran, never which tools the model claims it ran. It is the same line, drawn in three different industries.

Ask who answered

The fix took five minutes. I deleted the fallback and made the script refuse to run without a real key. Auth errors now kill the run loudly instead of flowing into the score. The first honest eval was strange to watch, because the skills were fine. They had always been fine. Nobody had asked them anything.

What I took from it:

Before you ask what a number means, ask who answered. Sometimes nobody did.
Fall back on compute. Fail closed on identity.
Keep error channels out of the answer column. A 401 is infrastructure, not a verdict.
Distrust flat metrics. Real systems are noisy, and a number that never moves is usually measuring the plumbing.

If you run agents in production, go read your fallbacks. Most of them are fine. You are looking for the ones that swap who is talking, because those will hand you a confident number about the wrong subject, and you will believe it, like I did. The skills were never the problem. Nobody asked the model.

Where in your stack does a failed credential quietly turn into a different answer?

Originally published at nabbilkhan.com.

The Card Said AI

Nabbil Khan — Mon, 27 Jul 2026 14:03:12 +0000

The card said to use AI to build a quarterly compliance report out of a client's event platform. Before I wrote a line I cloned the app. Laravel, about 22,000 files. I spent a day reading it instead of building. There was already a scheduled tracker running. There were attendee exports nobody opened. There were seventeen compliance sheets sitting in the repo that nobody had connected to the ask. The report was most of the way built. It just had no name and no owner, so from the outside it looked like it did not exist.

What the card actually meant

The card said AI. What the client meant was that the quarter closes and somebody sits down for three days with spreadsheets and dreads it. Nobody wants a model. They want the dread to stop. Those are different deliverables, and if you take the first one at face value you will build a beautiful thing that solves nothing.

So I did the boring part. I took the compliance template, all 147 columns of it, and went column by column against what the platform already held. Forty seven mapped straight across, same field, different name. Sixty one were derivable, a join or some date math or a rollup away. Thirty nine were real gaps, things the platform never collected at all.

147 template columns triaged
47 mapped directly
61 derivable from existing data
39 real gaps in the data

The triage took longer than the code. And when it was done there was nothing left for a model to do. The mappings were fixed. The derivations were arithmetic. The gaps were gaps, and no model can invent a field nobody collected. It can only write a confident sentence where the field should be.

The mapping was not ambiguous. It was just unknown, and those two feel identical until somebody does the work.

Why I took the model out

I had already wired one in. Pulling it back out felt like losing. Then I asked the question that decides these things: does this report have to come out the same way twice?

Yes. Somebody pulls a compliance report a year later, somebody who was not in the room, and it has to match the one that was filed. Not roughly. Exactly. A model that is right 97 percent of the time is a model that will quietly contradict last quarter's filing, and you find out in an audit.

So it became a fixed alias table and a hash chained log. Every row traces back to the record it came from. Every run leaves a chain you can check. Boring. Replayable. When the auditor asks where a number came from, the answer is a row, not a probability.

Key insight: If the same input has to give the same answer twice, you do not want a model, you want a table.

What broke: I built the model path first because the card said AI, and I threw that day away. I should have read the codebase before I read the requirements.

The same shape keeps showing up

I keep running into this in industries that have nothing to do with each other. In healthcare the sharpest thing I shipped this month was not a model at all. It was a QR code that opens the patient app with no download and no password, plus one slide that says plainly which decisions we still need from the doctor. In pharma compliance it was a lookup table. In the medical billing work the model earns its keep in exactly one place, reading the messy denial letter, and everything downstream of that is deterministic on purpose.

The rule underneath is simple. Use a model where the input is genuinely unstructured and the output is allowed to vary. Everywhere else it is an expensive way to build a system you can no longer explain.

The result: Shipped the quarterly report with no model in it, fully replayable, and handed over the 39 real gaps as a list the client can go fix.

What I would do next time

Read the codebase before you read the requirements. Half of what you are asked to build is already there under a different name.
Do the triage by hand, once, column by column. It is slow, and it is the only thing that tells you whether the problem is hard or just unmapped.
Ask whether the output has to replay. If it does, make it deterministic.
Take the model out when the triage kills it, even though you built it and it feels like waste.
Hand over the gaps as a list. The 39 things the data does not contain are worth more than the 108 you automated.

Sometimes the whole contribution is proving the AI was not needed. You cannot bill for that easily and it does not demo well. But the client stops dreading the quarter, and the report replays the same way in an audit two years out. That is what they were asking for the whole time. They just used the only word anyone gives them for it.

What is your test for when a workflow is genuinely unstructured enough to deserve a model?

Originally published at nabbilkhan.com.

Tools an Agent Can't Use

Nabbil Khan — Mon, 20 Jul 2026 14:05:19 +0000

Last month I tried to fire Webflow. Not because it was slow, and not because it was expensive. The nonprofit site it runs, muslimscholars.us, was doing fine. I tried to fire it because its Designer is a canvas, and a canvas is a thing you drive with your hands. My workforce does not have hands. I run my companies with AI agents now, and any tool an agent cannot operate quietly turns back into my job.

A canvas with no door

Here is what I mean by cannot operate. Webflow has an API, but the API stops at content. It will update a blog post all day. It cannot move a div. The design itself, the layout, the classes, the breakpoints, lives inside the Designer, and the Designer is a rendered canvas. There is no API into it. So I tried browser automation next, the way you would drive any stubborn web app, and it bounced off. The canvas is a picture of the design, not the design. There was nothing underneath to click.

What broke: I spent days pointing browser automation at that canvas before I admitted defeat. Every selector I could reach was part of the frame, not the picture. The tool was not broken. It was sealed.

So the site is moving to code. Not because code is a better site builder. In most ways it is a worse one. But code is the one interface every agent already speaks. A change becomes a diff, and a diff can be reviewed, tested, and reverted by something that never sleeps.

Teaching Unity to work blind

A few days later I did the opposite. I build a VR app that trains pharmacy technicians. It runs on Quest, which means it is built in Unity, and Unity is a giant GUI editor. By the Webflow test it should have been next out the door. But I kept it, because what my agents actually needed from Unity was never the editor. It was one answer, over and over: does this C# still compile against this project? The editor knows, because the editor holds the real reference assemblies. So I pointed Roslyn at those assemblies directly, on a server.

The result: A headless server with no screen, no GPU, and no Unity license now typechecks the Quest app in half a second. Agents edit the C#, get compiler truth back instantly, and never open the editor at all.

One rule, two directions

Two opposite moves in the same week, in two businesses that share nothing: a nonprofit's website and a pharmacy training app in VR. That is the part that surprised me. The rule was the same both times. The difference is where the value lives. Webflow's value is the canvas; take that away and there is nothing left to wrap. Unity's value sits under the canvas: the compiler, the assemblies, the build pipeline. The GUI is just the lid. When the value is under the lid, you pry the lid off. When the value is the lid, you leave.

A tool only I can drive is not a tool anymore. It is a queue with one worker, and the worker is me.

Key insight: When your workforce is agents, the question that decides whether a tool stays is not what it can do. It is whether something that is not you can operate it.

0.5s Unity typecheck on a headless server
0 APIs into the Webflow Designer canvas
2 opposite fixes, one rule, one week

I used to judge software the way everyone does: features, price, polish. That list still matters. But it now has a gate in front of it, and most tools I look at do not get past the gate.

What I ask now

Ask who, or what, can operate it. If the answer is only you, it is a bottleneck wearing a nice interface.
Find where the value lives. Under the GUI, wrap it. In the GUI, replace it.
Do not settle for "has an API." Check that the API reaches the part the tool is actually for. Webflow has one. It just does not touch the thing Webflow is for.
Treat headless as a requirement, not a feature. It is the difference between a tool and a job.

The strange part is that neither tool changed. Webflow is the same product it was a year ago, and so is Unity. What changed is who shows up for work. Software has always competed on what it can do. It is starting to compete on what can do it.

Originally published at nabbilkhan.com.

Zero Is Not a Score

Nabbil Khan — Sun, 19 Jul 2026 00:20:18 +0000

The evals for my agent skills scored 0% for as long as I had records. Not low. Not noisy. Exactly zero, every skill, every run. And I believed it. For months I thought my skills were bad, because the number said so and the number never wavered. Then one night I actually read the harness. It had fallen back to the wrong auth token. Every call it made came back 401, and it quietly graded each one a failure. The skills never got a chance to fail on their own. I was not measuring them at all. I was reading a broken thermometer.

Real weakness is jagged

Here is what took me too long to see. When a system is genuinely bad, it scores 40% one week and 60% the next. It passes the easy cases and trips over the hard ones. It has good days. Incompetence has texture, because an incompetent system is still in contact with the world, and the world varies.

A flat number has no texture. A flat number means the measurement stopped touching the thing being measured somewhere upstream, and what you are reading is the instrument's resting state. Doctors know this. A heart monitor drawing a perfectly straight line does not mean the patient is calm.

Only a broken thermometer writes the same number every time.

Key insight: A performance number with no variance is a reading of the instrument, not of the thing being measured.

The same bug in three industries

I run systems in advertising, in healthcare billing, and in agent operations, and the same shape shows up in all of them. In advertising I found a dashboard figure that had been hardcoded for two years. Nobody questioned it, because it looked right, and it looked right because it never moved. In agent operations, an account-rotation bug in one of my pipelines overwrote every real error with the same generic message, "no active accounts," so for a while every distinct failure in that system looked identical. And in the denial-assessment engine I run for a medical-billing operation, an agreement metric came back at 44.7%, alarmingly low, until I noticed the two sides of the comparison were scored in different units. One side used an equivalence map; the other compared raw strings. Twenty-seven of the "disagreements" were the same operational decision written two ways. Rescored in units that meant something, the engine sat at 92.0% on classification and 90.7% on action against the golden set.

Three industries, one pattern. When a number is perfectly flat, perfectly round, or perfectly terrible, the odds you are looking at the system go down, and the odds you are looking at the pipe go way up.

0% skill eval score, every run on record
401 the status code behind every one of those zeros
44.7% an agreement score that was really a units mismatch
90.7% action accuracy once the eval scored meaning

What a healthy number looks like

The week after I fixed that eval, the numbers got interesting. Interesting is the point. Mapped-action accuracy on the golden set went from 36.7% to 76.7% across four runs in one afternoon of prompt work. Classification went from 80% to 93.3%. Errors went from one to zero. The line wobbled, jumped, and climbed, and every point on it told me something I could act on.

That is what contact with reality looks like. A healthy metric breathes. So I have inverted my instincts. Steady numbers used to feel comforting and volatile ones used to feel alarming. Now the number that never moves is the one that keeps me up at night. I judge the autonomous operators I run the same way. When one of them turns in the same figure day after day, I do not ask what it is doing wrong. I ask what stopped being measured.

What broke: I believed a broken harness over my own work for as long as I had records. Zero felt like an answer, so I never read the grader. The fix was one auth token. The cost was every decision I made off that number.

Grade the grader

The rule I run now is simple. Before I trust an eval, I make it prove it can pass. Feed it a case where the right answer is known and pinned, and watch it score a success. A harness that has never emitted a passing grade is not strict. It is dead. And dead harnesses grade perfectly, forever.

The result: The denial engine's eval now gates every ship at 90% against a golden set, and I trust that gate because I have watched it pass, fail, and change its mind as the system changed. It moves. That is how I know it is alive.

Distrust any metric with no variance. Jagged is what real looks like.
A flat zero is a flatline. Check the instrument's pulse before the patient's.
Grade the grader. Run a known-good case through every eval and watch it pass.
Never let a fallback swallow an error. One wrong token cost me months of belief.
Perfectly round and perfectly stable deserve the same suspicion as perfectly bad.

The zero on my dashboard was never a grade. It was an instrument talking to itself. The numbers I trust now are the ones that move, because a number that moves is still touching the world. Most of us grade everything except the grader. Start there.

Originally published at nabbilkhan.com.

Write Your Exceptions Down

Nabbil Khan — Sat, 18 Jul 2026 23:50:17 +0000

I had a rule with no exceptions. RETSBAN, my primary agent, runs local inference only: open models on my own hardware, no cloud, no fallback. If the GPU box is down, the agent is down. Then Mia, the agent that runs my marketing, needed a frontier model to do her job well. And I did something that felt strangely formal for a one-person company. I amended my own policy, in writing, with the date attached.

No one was in the room

Here is what took me a while to see. A human employee remembers the day you made an exception. They were in the room when you said fine, just this once. An agent was never in the room. There is no room. Every session starts cold from the files, and the files are the only memory the company has.

So when the policy says local only, no exceptions, and reality contains an exception, one of two things happens. Either the agent obeys the file and blocks work I actually want done, or it notices the contradiction and starts guessing which side to trust. The guessing is the dangerous case. A rule that has been contradicted once, silently, is not a rule anymore. Every agent that loads it gets to decide, in every session, whether to believe it. You never see the deciding. You just see the drift.

An exception that lives in your head does not bend a rule. It erases it.

The amendment

On 2026-04-23 I opened the policy file, a file literally named local_only_no_anthropic, and narrowed it instead of breaking it. The amendment names who is exempt: Mia, and only Mia. It says why: marketing work that needs capability the local stack does not have. It carries the date, and it points to the full model-topology doc for anyone, human or agent, who wants the whole picture. RETSBAN's constraint did not move an inch. Still local only. Still no fallback. Still down when the GPU box is down.

That is the difference between an amendment and a repeal. An unwritten exception repeals the rule and hides the repeal. A written amendment narrows the rule and makes it stronger, because now the rule has visibly survived contact with a real exception. The next agent to read the file does not have to wonder if the local-only line is stale. It can see the line was tested, on a specific date, for a specific reason, and held.

What broke: Before this I kept preferences in my head and re-explained them session after session. Every correction died with the session that heard it. I was giving the same review, over and over, to a workforce with perfect skill and no memory.

The same move in medical billing

This is not an agent-fleet quirk. Last week, in my medical-billing business, I retired a ship gate on the denial-assessment engine. The old gate required the tuned API to agree with a weaker agent rater. A 150-case run showed the gate measured the wrong thing: on genuine divergences the API matched the golden answers about 78 percent of the time, and the agent rater about 38. The gate could never pass, and passing it would have meant nothing.

I did not quietly stop running it. The change record says the gate was retired, says why, and says when the change was ratified: 2026-07-08. It says what replaced it: score the API against golden answers directly, with the floor at 90 percent. Whoever touches that system next quarter, human or agent, inherits the reasoning and not just the residue.

0 cloud calls RETSBAN has made, before and after the amendment
1 agent named in the exception
150 cases in the run that killed the old ship gate
90% the floor the replacement gate has to clear

The result: Nearly three months after the amendment, both rules hold exactly as written. RETSBAN has made zero cloud calls, Mia has exactly the access the file grants, and no session has argued with either fact.

Your judgment is production config

You version your code. You probably version your infrastructure. The strange part of running an AI agent workforce is that your own judgment joins the list. What you want, what you forbid, what you decided to allow anyway: the agents execute whatever the file says. So the file is the policy, and an edit to the file is a deploy.

Key insight: An exception you keep in your head deletes the rule. Amend the file instead, with the date, the scope, and the reason attached.

So this is what I do now, and what I would tell anyone running agents:

Put the rule where the agent reads, not where you think. A rule outside the files does not exist.
Amend, do not repeal. Narrow the rule and keep it.
Attach the date, the scope, and the reason. A future session cannot ask you follow-up questions.
The moment you catch yourself carrying an exception in your head, the rule is already gone. Writing it down is the undo.

A rule you never amended is not a rule. It is a rumor about a rule, and every session of every agent decides whether to believe it. Write your exceptions down. A rule is only as real as its history.

Originally published at nabbilkhan.com.

Who Owns the Row

Nabbil Khan — Sat, 18 Jul 2026 23:20:15 +0000

One of my systems runs a nightly job with two sets of manners. On 154 rows it overwrites whatever it finds and asks nobody. On 15 rows it cannot change a single field; the best it can do is file a request and wait. Same job, same feed, same table. The only difference is whether a human being ever touched the row.

Two doors

The system is a command center for a workforce development operation. The State of Illinois publishes a directory of workforce centers, and every night my sync pulls that feed and compares it against what I have. Most of those sites, 154 right now, are anonymous directory entries. Nobody on my side has ever opened one. For those, the feed writes straight through. New address, new phone, new hours, done. Arguing with the state about a row nobody looks at would be silly. The feed is the only party that knows anything about that row, so the feed decides.

Fifteen sites are different. Those are partnered centers, rows people on my side have filled with staff notes and visit history. Someone drove out there. Someone wrote down which door to use and who to ask for. When the feed disagrees with one of those rows, the sync does not write. It queues the change in a table called WorkNetPendingChange and waits for a person to approve it or reject it.

And nothing gets deleted, ever. A site that vanishes from the state's export just gets a missingSince date. Feeds hiccup. Exports get truncated. A date is reversible. A delete that cascades through visit history is not.

154 directory sites the feed overwrites freely
15 partnered centers where changes wait for review
0 rows the sync is allowed to delete

The row, not the feed

The usual way to design a sync is to ask whether the feed is the source of truth. That question hides an assumption: that authority belongs to the feed. It does not. It belongs to the row.

Think about what each side actually knows. The state knows what the state published. It knows nothing about the visit last Tuesday, or the note saying the listed phone number rings a fax machine. On a row nobody here has touched, the feed's knowledge is all the knowledge there is, so it should win every conflict. On a row people have worked on, the feed holds the smaller share, and letting it overwrite means destroying the larger one.

Authority is not a property of the feed. It is a property of the row.

Key insight: Give a sync write authority row by row. It overwrites freely where no person has done any work, and it only gets to propose changes where someone has.

What broke: Every sync I have ever regretted failed the same way. An upstream export had a bad night, and the job faithfully copied the bad night over months of human work. The overwrite took milliseconds. Getting back what a coordinator knew about a site took weeks, when it happened at all.

The same rule keeps showing up

Once I had the rule, I started seeing it all over my stack. The same command center runs an inbox agent that triages a shared mailbox. Routine messages it handles by itself. Anything that touches a real relationship goes to a review queue. Two doors again.

My publishing pipeline works the same way. Work-log entries publish to the enterprise timeline on their own, because nobody's judgment rides on a changelog line. Public essays, the ones that carry my name, wait for approval. And in a different industry entirely, the denial-assessment engine I run in medical billing acts directly on the mechanical cases and calls escalate_to_human on the few where judgment actually lives.

Three corners of my work, one rule. Automation gets full authority over work no person has touched. The moment a person touches it, the automation drops from writer to proposer.

The result: The sync keeps 154 rows fresh with zero human effort, and every drift against the 15 partnered centers gets human eyes before it lands. Fresh and safe at the same time, without trading one for the other.

What to build

None of this is exotic. It is a WHERE clause and a pending-changes table. The hard part is deciding to build it, because "the feed is the source of truth" sounds so clean. If you are wiring an external feed into a system people actually work in, here is what I would do:

Give write authority per row, not per feed. Source of truth is a row-level question.
Let human work demote the feed. The first staff note on a row turns the feed from a writer into an advisor.
Never let a sync delete. Record the absence and let a person decide what it means.
Make every change pick a door: direct write or review queue. No third path.
Keep the queue short enough that people actually read it. A review queue nobody reads is just a delete with extra steps.

Machines should own what nobody cares about, so people can own what they do. Go look at which rows in your database have fingerprints on them. Those are the ones the feed should have to ask about.

Originally published at nabbilkhan.com.