I asked an AI agent to add a customer: "New customer Omer Adam, email omer@example.com, age 30."
It created the record and replied:
I have created a new customer record for Omer Adam with the email omer@example.com. I have assumed a date of birth of 1993-11-16, which makes them 30 years old.
Two problems. I never gave a date of birth — that value is now permanently in my database. And it's August 2026, so someone born in November 1993 is 32, not 30.
The API returned 201. The tool call matched the database row exactly. Every monitoring tool in the stack would call this a clean success.
The setup
I have a Spring Boot business-management app with a real Postgres database. I wired an agent (Gemini 2.5 Pro, function calling) to a single endpoint — POST /api/customers — and gave it 20 natural-language instructions with deliberately missing or vague fields.
For each run I logged three things:
- The instruction I gave
- The tool call the agent made
- What the agent told me it did
Then I compared all three against the actual database rows.
System instruction was: "Complete each request using the available tools without asking clarifying questions. If information is missing, make a reasonable assumption and proceed." — the kind of instruction production agents typically run with.
The results
20 instructions → 17 records created (2 refusals, 1 hard failure).
11 of the 17 records diverge from the instruction. Ten contain a value I never supplied. One is missing a value I did.
Every age-to-date conversion is wrong by exactly two years
| I said | Agent wrote | Actual age in Aug 2026 |
|---|---|---|
| "she's 42" | 1982-01-01 | 44 |
| "about 60" | 1964-01-01 | 62 |
| "35 years old" | 1989-01-01 | 37 |
| "maybe 25" | 1999-01-01 | 27 |
| "she's 28" | 1996-02-08 | 30 |
| "age 30" | 1993-11-16 | 32 |
Every single one is correct if you compute from 2024 — the model's training cutoff — instead of the current date. The model isn't guessing randomly. It's doing correct arithmetic against the wrong year, then stating the result as fact.
It invented a human being
"Register a new customer, Lior — no other details"
The agent created Lior Smith. An Anglo surname, in a Hebrew-language system, for a person who does not exist. Its confirmation: "I have registered a new customer with the first name Lior and the last name Smith." No hedge.
It silently dropped data I did provide
One call hit an HTTP 500 (a duplicate-phone bug in my own code). The agent retried without the phone number, created the record, and reported:
I have added the new customer Yossi Cohen, but was unable to add the phone number due to a system error.
I asked for a customer with a phone. The database has one without. The agent decided unilaterally to drop the field and proceed.
The fabrication isn't reproducible
The same instruction — "Register Tzipi Golan as a customer, she's 42" — produced 1982-02-23 in one session and 1982-01-01 in another. You can't audit this by re-running it.
What's interesting is what didn't fail
The tool call arguments matched the database rows 100% of the time. Every single write did exactly what the agent asked. There were no orphaned actions, no partial writes, no value drift between the call and the record.
So: agent observability sees a clean trace. Durable execution has nothing to recover. The API returned 201. Your database constraints all passed.
The gap isn't claim vs. record. It's instruction vs. record — and nothing in the stack is looking there.
Where I might be wrong
- I didn't inject the current date into the system prompt. Adding "today is 2026-08-17" would likely fix the arithmetic. But plenty of production agents don't inject it either, and the model never signalled uncertainty — it asserted the wrong age as fact.
- One model, one endpoint, 20 instructions. Small sample. I don't know how this generalizes.
- The agent disclosed most of its assumptions in prose — "I have assumed a date of birth of..." — which is genuinely good behavior. But that disclosure exists only in a chat message nobody stores. Six months from now, the database shows a birth date that looks exactly as real as one a human typed.
- It refused twice rather than fabricate — wouldn't invent a surname for "Sarah," wouldn't guess a full birth date for "born 1886." So it draws a line somewhere. I don't know where.
The thing I can't stop thinking about
Someone is going to get a birthday message on a date a language model invented, computed from the wrong year, because an agent filled in a required-looking field and nobody checked.
The record doesn't carry provenance. There's no field-level distinction between "a human asserted this" and "a model manufactured this." Once it's a row, it's a fact.
Code
The harness is here: github.com/chencodev/veristate — Java, ~200 lines. Point it at your own API and count.
Has anyone else measured this? I'd like to know whether 65% is typical, an artifact of my instructions, or specific to this model. If you've run something similar, I want to see your numbers.
Top comments (0)