29,633 Reasons. 26,812 Unique. The Model Confabulates.

We checked every explanation an AI model gave for its recommendations. Almost none of them repeat.

I have spent a lot of time trying to understand why AI recommends one brand instead of another.

• We tested store quality.
• We tested public fame.
• We tested domain recognition.
• We tested other signals.

Many of them explained very little.

So we tried something much simpler.

We looked at what the model itself said.

Every recommendation came with a reason.

So we collected them all.

29,633 reasons.

Then we checked how often the model used the same reason again.

The result was striking.

26,812 reasons were completely unique.

That is 90% of the dataset.

The most repeated reason appeared only 12 times.

Out of 29,633 recommendations.

That is just 0.04%.

We read every reason

Every recommendation in our dataset had a short reason attached to it.

We took the full reason field and checked every entry for exact text repetition.

If the model was using a small set of stable reasons, we should have seen those reasons many times.

We did not.

The test was simple.

We checked:

29,633 reasons

26,812 distinct reasons

90% unique text

12 times for the most repeated reason

We used exact text matching.

That means two sentences with the same meaning but different wording were still counted as different.

So this test only measures repeated text.

It does not measure repeated ideas.

The result

Nine out of ten reasons were unique.

The model did not keep using the same small set of explanations.

It kept writing new ones.

This is important because a stable decision rule should leave some kind of pattern.

The exact sentence does not need to be the same every time.

But similar decisions should produce similar explanations.

That pattern was not there.

The pick comes first

This is where the finding gets interesting.

The recommendation happens first.

Then the model produces a reason that fits the recommendation.

The reason can sound very good.

It can sound specific.

It can sound confident.

But that does not mean it describes the process that produced the recommendation.

It may simply be a good explanation written after the decision.

That is what we mean by confabulation in this study.

The model is very good at producing language that makes a decision sound reasonable.

That is different from knowing why the decision happened.

Asking AI why can be misleading

This creates a real problem for ecommerce teams.

Imagine an AI recommends another brand.

You ask why.

The model says the brand has better reviews.

You improve your reviews.

Another time it says the brand offers better value.

You change your pricing.

Another time it talks about quality.

You rewrite your product pages.

The advice sounds useful.

But this study shows why we should be careful.

The explanation may not be the signal that caused the recommendation.

It may simply be the explanation generated for that recommendation.

You could end up optimizing for the explanation instead of the cause.

This changes how we should study AI recommendations

If we want to understand what drives AI recommendations, asking the model is not enough.

We need to watch what it does.

Change one variable.

Keep everything else the same.

Run the recommendation again.

Measure what changed.

That gives us evidence about what actually moves the decision.

The explanation gives us language.

The experiment gives us evidence.

Those are very different things.

One important limitation

Our test used exact text matching.

So these two reasons would count as different:

better customer feedback

and

stronger reviews from buyers

They may describe the same idea.

Our study does not tell us how many underlying ideas were repeated.

It tells us that 90% of the actual text was unique.

That makes the finding narrower and cleaner.

We are not claiming to have reconstructed the model's internal process.

We are showing that its own explanation is not a stable enough signal to use as a direct explanation of its recommendation.

What we learned

There is a big difference between:

What the model recommends

and

What the model says about the recommendation

The first is behavior.

The second is generated language.

If we want to understand AI commerce, we need to measure the behavior.

That is what we are doing at Atom Foundry.

We are testing what happens when AI evaluates ecommerce brands.

Not just what it says.

But what actually changes its decisions.

29,633 reasons gave us a very simple lesson.

The explanation is not necessarily the mechanism.

It is the explanation.

Research: https://atomfoundry.dev/research/ai-confabulates-its-reasons