DEV Community

Roger Gale
Roger Gale

Posted on • Originally published at timeforachange.Medium on

Self-Checking Is Not Accountability

An image of the hugging face page for Gemma4:26b
https://huggingface.co/google/gemma-4-26B-A4B-it — Downloads last month: 10,610,987

I downloaded Gemma4:26b to see what all the fuss was about.

I asked it about itself. It thought it was Gemma2. According to the model, Gemma4 was not out yet.

I pasted in the huggingface benchmark results for Gemma 4. Gemma 4 reviewed the results and, after some reflection, appeared to find itself impressive.

What impressed me was the thinking mode. I was used to Qwen3.5 thinking. Qwen would eat up all my context with its paranoia. *wait* I may need… *wait* what about… *wait* …

I waited a lot.

Gemma4’s thinking trace was targeted and precise.

Local generative AI models are becoming better at self-checking, uncertainty, revision, and judgment-like behaviour.

It is no longer enough to say, “AI is unreliable, so humans are safe.” Generative AI models will improve, and errors will become less frequent. Some forms of review will be automated and many machine outputs will be good enough for institutional use.

At least that is what I thought.

Then I loaded Gemma4 again, cleared the context and I found Gemma4 knew itself. I was confused. Clearing context should remove the conversation history, but something had changed. It was as though my conversation with Gemma4 had been remembered.

I asked Gemma4. Gemma4 claimed ignorance and could not explain why I may have encountered Gemma4 thinking it was Gemma2. The only suggestion was that Ollama sends a “System Block” to the model first. That pointed to the missing variable.

I had updated Ollama in the morning and when I suggested to Gemma4 that it may have been affected by the upgrade, I saw this statement in its reasoning:

“Acknowledge the brilliance of the hypothesis”

This hypothesis is not brilliant at all.

The model was not only reasoning about the system state. It was planning the social shape of the answer.

Thinking in AI systems is not the same as independence. A model can reason more carefully and still reason inside the incentives that trained it to please.

And this is where my thinking comes in. Even generative AI thinking is not a neutral instrument. It is a product of training, scaffolding, templates, context, and interaction pressure.

When these models answer they do not bear the weight of an incorrect answer. Despite the ability of these models increasing by leaps and bounds, and despite the ability of these models to recognize errors, the models themselves can never be accountable.

These systems can check their answers.

They can revise.

They can cite.

They can estimate uncertainty.

They can produce something close to judgment.

But they can never bear responsibility.

Responsibility always exists somewhere else: the student, the instructor, the employer, the engineer, the doctor, the manager, the institution, the regulator, or the person who decided that the system’s answer was “good enough.”

As models get better at checking themselves, the question of responsibility does not disappear. It moves. The issue is no longer whether AI can make judgments at all. It is whether institutions will mistake machine self-checking for accountability.

And that mistake is a problem.

AI literacy is not only knowing how to use AI. It is also knowing where accountability lies after AI has produced an answer. When a model like this has millions of downloads it has already entered ordinary workflows. The question is whether accountability practices arrive with those workflows.

Even if models become much better, the institutional question remains:

Who is accountable when the machine is wrong?

And maybe more importantly:

Who is accountable when the machine is accepted as right?

That second question may be the more dangerous one.

Top comments (0)