If you use an LLM to turn messy input into structured fields (an email into a lead, a voice note into an intake form, a PDF into a row), it will eventually hand you a value nobody said. Not often, and usually with high confidence, which is the problem.
Prompting doesn't fix this. "Only use information from the input" helps, but a prompt is a request, not a check. What fixed it for us was three deterministic checks that run after the model answers, on every extraction. None of them uses a second model, and all three together cost a few microseconds.
The setup
Each field the model extracts comes back with three things, not one:
{
"budget": {
"value": "90000 to 100000",
"confidence": 0.85,
"source_snippet": "around ninety to a hundred"
}
}
source_snippet is the exact text in the input that the value came from. Asking for it costs a few tokens per field. It's what makes the checks possible.
Check 1: the snippet has to exist
If the model says a value came from some text, that text has to be in the input. Compare after lowercasing and collapsing whitespace, because models re-space things:
function normalize(s: string): string {
return s.toLowerCase().replace(/\s+/g, " ").trim();
}
function snippetGrounded(snippet: string | null | undefined, input: string): boolean {
if (!snippet) return false;
const s = normalize(snippet);
return s !== "" && normalize(input).includes(s);
}
When this fails, we throw the value away: null, confidence 0. A missing snippet means the model can't point at where the value came from, so none of it can be trusted.
Note what this deliberately does not check: the value itself. "eight people" should become 8, and "12 LPA" should become 1200000. Requiring the value to appear verbatim would break every useful transformation. Only the evidence has to be verbatim.
That's also its blind spot, which is where check 2 comes in.
Check 2: every number has to come from somewhere
A respondent said their budget was "around ninety to a hundred". The model returned the snippet around ninety to a hundred (grounded, passes check 1) and the value 90000 to 95000. Nobody said thousands, and nobody said 95.
So the second check asks: can every number in the value be derived from the input? A number counts if it was written as digits, spelled as a word, or written as a stated number times a stated scale ("20k", "12 LPA", "100 thousand"). Simplified:
const WORDS: Record<string, number> = {
one: 1, two: 2, three: 3, four: 4, five: 5, six: 6, seven: 7, eight: 8, nine: 9, ten: 10,
twenty: 20, thirty: 30, forty: 40, fifty: 50, sixty: 60, seventy: 70, eighty: 80, ninety: 90,
hundred: 100, thousand: 1000,
};
const SCALES: Record<string, number> = { k: 1e3, thousand: 1e3, lakh: 1e5, lpa: 1e5, m: 1e6, million: 1e6 };
function statedNumbers(input: string): Set<number> {
const out = new Set<number>();
for (const m of input.matchAll(/(\d[\d,]*(?:\.\d+)?)\s*(k|thousand|lakh|lpa|m|million)?\b/gi)) {
const n = Number(m[1].replace(/,/g, ""));
out.add(n);
if (m[2]) out.add(n * SCALES[m[2].toLowerCase()]);
}
for (const word of input.toLowerCase().match(/[a-z]+/g) ?? []) {
if (word in WORDS) out.add(WORDS[word]);
}
return out;
}
function numbersGrounded(value: unknown, input: string): boolean {
const stated = statedNumbers(input);
const numbers = String(value).match(/\d[\d,]*(?:\.\d+)?/g) ?? [];
return numbers.every((n) => stated.has(Number(n.replace(/,/g, ""))));
}
For "around ninety to a hundred", the stated set is {90, 100}. 90000 isn't in it, so the check fails.
The real version also handles compound words ("twenty-five"), "a hundred", month names and decimals, but the idea fits in 25 lines.
When this one fails, we do not null the value. It's usually partly right (the 90 was said), and deleting it hides the mistake from the person who could fix it. Instead we cap the confidence at 0.5, which sends it to a human.
Check 3: a hedged number is an uncertain number
The third failure was honest. The input said "around ninety to a hundred, depending", the model returned exactly that, and reported confidence 0.7. In our tests it did this in six runs out of six, because the prompt said to go "below 0.7" for uncertain answers and the model read that as "0.7".
Our review threshold is confidence < 0.7, so 0.7 sailed through as approved. The prompt asked for the right thing. Only a check could make sure it happened:
const HEDGE =
/\b(around|about|approx(?:imately)?|roughly|maybe|probably|-?ish|or so|ballpark|depending|not sure|i think)\b/i;
function hedged(snippet: string, value: unknown, input: string): boolean {
if (!/\d/.test(String(value))) return false; // "probably remote" is a choice, not a measurement
if (HEDGE.test(snippet)) return true;
// The model sometimes leaves the hedge just outside its snippet: "around [90 to 100]".
const at = normalize(input).indexOf(normalize(snippet));
return at > 0 && HEDGE.test(normalize(input).slice(Math.max(0, at - 24), at));
}
Two details came from real misses:
- It only applies to numeric values. Flagging every "I think" would turn the review queue into noise, and people learn to ignore noise.
-
It looks a few words before the snippet. When someone dictated without punctuation, the model snipped
90 to 100and left "around" outside it. Three words of lookbehind catches the hedge on this number without reaching a hedge on something else in the sentence.
A failure caps confidence at 0.55.
What each check does when it fires
| Check | On failure | Why |
|---|---|---|
| Snippet not in input | value removed, confidence 0 | Nothing supports the value |
| Number never stated | value kept, confidence capped at 0.5 | Usually partly right, so a human should look |
| Number hedged | value kept, confidence capped at 0.55 | Honest, but uncertain |
Two smaller rules sit underneath all three:
- Placeholder answers ("n/a", "unknown", "not provided") are treated as empty. Models love to fill a required field with a polite non-answer.
- The model's self-reported confidence is capped at 0.98, so 1.0 stays reserved for values a person typed into a structured field.
Anything under the threshold goes to a review queue instead of auto-approving. A false positive costs one person one look. A false negative costs a wrong number in someone's CRM. For a check that runs on every extraction, that's the right trade.
The general rule
Give the model a way to show its work (source_snippet), then check the work with code. Checks that are cheap, deterministic and dumb catch the failures that a smarter prompt keeps letting through, and they fail the same way every time, so you can test them.
These checks run on every extraction in DodoForm, a form builder where people can answer by typing, talking or pasting something messy. You can try the extractor without an account on the homepage.
Top comments (0)