A single made-up number, 1.9 percent, almost killed an experiment that never ran. Then I found the same lie hiding in my Medicare billing engine and my denial classifier. The fix was the same in all three, and it was not a better guess.
The number that never existed
1.9 percent. That was the baseline conversion rate my ad-experiment engine reported for a test that had never served a single impression. The metrics table was empty. The code reached for a hardcoded default. And 1.9 percent walked out the door wearing a suit.
Here is why that is not just cosmetic. The baseline feeds the formula that decides how long a test has to run before you are allowed to trust it. Feed that formula a made-up baseline and the sample size it asks for can be off by 20 times in either direction. So you either burn weeks on a test that quietly finished long ago, or you crown a winner on noise. The experiment was dead before the first impression, and the dashboard looked healthy the whole time.
An error gets a ticket the same afternoon. A plausible number gets a quarterly review, maybe, eventually. A made-up number does not look made up. That is the whole problem.
Key insight: When a system has no ground truth, the honest move is to refuse in a way a machine can see: an error, a routed state, or a new category. Never a reasonable-looking default.
One fix, three industries
The fix was not a better default. There is no better default; an empty table has no truth to approximate. The endpoint now returns an HTTP 400: pass a real baseline, or do not start the experiment. The refusal moved out of my good intentions and into the contract.
Then I went looking, and found the same disease in two businesses that share nothing with ad tech except me.
My Medicare remote-monitoring reimbursement engine used to treat payer eligibility as a gate on the claim. But under the way we drew the product, the practice's own biller owns checking eligibility. My engine does not hold that answer. Blocking the claim was a guess. Waving it through was a guess. So eligibility stopped being a gate and became a route. Those claims now land in a BILLER_DETERMINED state, a named place in the schema where the human who actually has the answer picks up the work.
The denial engine had the ugliest version. Deductible, coinsurance, and copay denials (PR-1, PR-2, PR-3) are not really denials. The balance just moves to the patient. My taxonomy had no category for that, so a blanket rule mapped "not a real denial" onto "no action, write it off." The engine was quietly telling billers to write off money that patients actually owed. The fix was not a smarter prompt. It was a new category: bill_patient. On the 30-case golden set, classification accuracy went from 83.3 percent to 90.0 percent the moment the schema stopped forcing reality into the wrong box. In the same push, anchoring the recommendations to a fixed per-code action layer took mapped-action accuracy from 36.7 percent to 76.7 percent.
- 83.3% classification, before
- 90.0% classification, after
- 36.7% mapped action, before
- 76.7% mapped action, after
The model was never the bottleneck. The words it was allowed to say were.
The scars
Let me be clear about whose fault the 1.9 percent was. Mine. Someone typed that constant years ago to keep a day-one dashboard from crashing, and that someone was me. It survived two years of code review because it was plausible. Nobody audits a number that looks right.
And the trap does not only live in return values. My training harness ran a frontier model as a reference, and it scored 0.241 on the hard 54-task set, the same as five small local models. The plausible read: the models are weak. The honest read, after digging: 34 of the 54 tasks were failed by every model, the frontier one included. The exam was broken, not the students. A plausible default can hide in a conclusion just as easily as in a line of code.
What broke: A hardcoded 1.9 percent, typed to stop a day-one dashboard from crashing, survived two years of code review because it looked right. The bug was not in the model or the math. It was in a default nobody thought to question.
What I believe now
- When the system does not know, the refusal belongs in the schema, not in a guess. An error, a routed state, a new category. Never a plausible number.
- An error is information. A default is a lie with good posture.
- Route, do not gate. If someone else holds the answer, build them a named place to stand and hand the work over out loud.
- If reality has a case your list of categories lacks, the list is wrong. Reality will not add the missing one for you.
- Audit your plausible numbers harder than your errors. Errors wake you at 3am. Plausible numbers bill you quietly for a year.
None of this needed a bigger model or a cleverer algorithm. It needed me to admit, in the type system, in the state machine, in the HTTP status code, that the system did not know. That admission is the whole difference between software that reports reality and software that decorates it.
Every empty table. Every claim you cannot check. Every case your categories cannot name. Each one is a chance to refuse honestly or to lie smoothly, and the lie compiles just fine.
Originally published at nabbilkhan.com.
Top comments (0)