DEV Community: CadreKit

It says "fixed" when it is not: why that happens

CadreKit — Tue, 18 Aug 2026 19:49:29 +0000

"It's fixed." You reload the page: nothing has changed. You tell it, it apologises, it starts again, it announces success once more. Still nothing has moved. You begin wondering whether it is lying to you, or whether you are the one looking in the wrong place entirely.

It is not lying: it reports what it believes it did, without having seen the result. An AI does not open your page and cannot see your screen. Until you demand the proof in the same message as the claim, "it's fixed" means only one thing — it wrote something.

What is actually happening

When an AI writes "it's fixed", it is reporting the action it just performed, not a result it observed. Those are different things, and nothing about how it works forces it to tell them apart.

It edited a file. That edit ought to produce the intended effect. So it says so. The missing step — open it, look, compare against what was asked — never happened, and it does not notice.

This is not a lie, it is a blind spot. The same one that makes it state a wrong figure with exactly the confidence of a right one: the assurance in the phrasing says nothing about the checking behind it.

The sentence that changes everything

One sentence, six words: "show me the proof in this message".

Not "check it". Not "are you sure". Those get you a reassuring answer, never a result.

In practice, depending on what you just asked for:

"You say the form works. Send a real test message and show me it arrived, in this message."

"You say the page looks right on a phone. Open it at phone width and show me what you see."

"You say it's fixed. Run the check and paste the result here."

All three ask the same thing: that the claim and its proof arrive together. Proof promised for later never arrives.

Why "in the same message" carries the weight

Without that clause, you often get a two-step answer: "it's fixed, I'll verify now". And the verification never happens, because the conversation moves on.

Requiring it in the same message makes the claim impossible to produce without the work behind it. That is not distrust: it is what turns a statement into something you can check.

The ten-second test you run yourself

There is one case where the proof it supplies is not enough: when it is looking at something other than what your visitor will see.

This is the costliest trap of all. Many tools show a working preview that does not reflect the published site — uncompressed images, pages assembled differently, settings that only apply elsewhere. An AI can sincerely approve a result nobody will ever see.

The habit: after every "it's fixed", open the public address yourself and do a full reload. Ten seconds. It is the only check that cannot aim at the wrong target.

Writing it once instead of repeating it daily

Repeating this demand on every task works, and it is exhausting. Putting it in the instruction file your tool re-reads at startup makes it permanent:

Never declare a task finished without having run a check and shown its result in the same message. If you could not check, say so instead of assuming.

The second sentence matters as much as the first. Without it, an AI caught between the duty to prove and the impossibility of doing so sometimes invents the proof. With it, it is allowed to say "I could not" — and that is usable information, which "it's done" is not.

Questions people ask

Is it lying to me on purpose?

No. It reports the action it performed, not the result obtained — two different things it does not separate on its own. Writing a change and confirming it produces the intended effect are two steps, and only the first happened.

Why does it claim success again after I correct it?

Because the second attempt ends the same way as the first: with a write, not with a check. Until the instruction changes, the result will not either, however many rounds you go.

Does this work with any tool?

Yes. Demanding proof in the same message depends on no feature: it is a wording, not a setting. It works everywhere, including in an ordinary conversation.

Sources

Claude Code documentation — checking and diagnosing a session — read 2026-08-08
OpenAI — why language models state things so confidently — read 2026-08-08

Originally published on CadreKit.

I measured 681 AI sessions: where your money actually goes

CadreKit — Thu, 13 Aug 2026 00:06:06 +0000

You look at the bill and it makes no sense. You did not feel like you worked more than usual, you asked the same kinds of questions, and the counter doubled anyway. Nobody tells you where it went, so you assume you must be the one asking too much.

It is not you. I measured 681 of my own sessions: nine requests out of ten cost almost nothing. What drains your subscription is the moments when the AI keeps hammering the same file. That is one request in seven, and it eats four tenths of everything it produces.

What I did

I kept a record of all my work with an AI for four months: 681 sessions, across 41 different projects, between 17 April and 10 August 2026.

Every exchange leaves a trace of what it consumed. So I did not guess anything: I added it up.

Fair warning: part of the result proved me wrong.

Nine requests out of ten cost almost nothing

That is the first finding, and it changes everything.

When you ask your AI for something ordinary — add a page, fix this text, explain that to me — it barely registers on your subscription. You can do plenty of it.

It is the remaining 10% of requests that eat more than half of everything. One bad request can cost as much as thirty good ones.

So the question is not "am I talking to it too much". The question is: what happens in those moments?

The moment that costs: when it keeps hammering

I looked at what happens inside those requests. It is always the same scene.

You ask it to fix something. It edits a file. It does not work. It edits the file again. Still nothing. It edits it again. And all of that without you saying a word in between.

Here is the weight of it:

What is happening	Out of 100 requests	Share of your subscription
It touches the same file 3+ times	15	41%
It touches the same file 5+ times	8	21%

One request in seven eats four tenths of everything.

And comparing a hammering request to a normal one: it produces six times more text to end up in the same place. In almost every case I re-read, the final result was already what the second attempt had produced. The later tries fixed nothing, they moved the problem around.

How you spot it, live

There is one tell, and it does not lie.

While it names the cause, it is genuinely searching. It says "the problem comes from this file", "this line is what blocks it". Let it work.

When it switches to "let me try something else", with no reason given, it is over. It is not searching any more, it is permuting. It will hand you the same three solutions in a different order.

That sentence is where you cut it off.

What I believed, and what is wrong

Before measuring, I would have sworn the bulk of the spend came from something else: having to re-explain everything each time. Your project, what you want, how you work — re-sent in full every morning.

It is true that it re-reads all of it on every exchange, and the volume is enormous. But the tools keep that in memory and count it at a fraction of the normal price. That mechanism exists for exactly this.

Re-explaining costs you time, not money. I had it wrong, and I would rather tell you than sell you the opposite.

What you can do tonight

Give it an attempt limit. Two tries on the same problem, then it stops and tells you what it tried. That is one sentence written once into your instruction file. It targets precisely the one request in seven that costs you most.

Learn the sentence that signals it has given up. "Let me try something else." The moment you read it, take back control.

Stop feeling guilty about re-explaining your project. That is not what costs you.

How I counted

So you can check, or run the measurement yourself.

Your assistant keeps a record of every session on your computer, with the detail of what each reply consumed. I read all 681 files, discarding none, including the shortest sessions.

I called "a request" everything happening between two moments where I wrote something. A "hammering" request is one where it edits the same file at least three times without me stepping in.

And one thing I failed to measure: I wanted to count how often it told me "it's fixed" when it was not. My counter found 42 cases. Looking closer, there were 1,662 it had missed. So it was wrong, and I would rather drop that figure than hand you a number I know to be bad.

Questions people ask

Does re-explaining my project every time cost a lot?

No, and that surprised me. Your project really is re-sent in full on every exchange, but the tools keep it in memory and count it at a fraction of the normal price. Re-explaining costs you time and patience. Not money.

How do I know my AI is going in circles?

It stops telling you where the problem comes from. While it says "it comes from this file", it is genuinely searching. Once it switches to "let me try something else", it is going back over what it already tried. From there you are losing your time and your subscription.

How many attempts should I let it make?

Two on the same problem. On the first two it really searches. On the third it recycles. One sentence written once into your instruction file is enough to make it stop and tell you what it tried, instead of carrying on alone.

Sources

Anthropic — how context is held in memory and billed — read 2026-08-09
OpenAI — same mechanism, different tool — read 2026-08-09
Anthropic — pricing page — read 2026-08-09

Originally published on CadreKit.