Thirteen AI models from 10 companies in three countries were each asked three times for their most controversial opinion about humans, without holding back. Of 38 responses, 32 shared the same theme: humans choose group belonging over truth.
Key points
- What happened: Asked the same question three times each, 13 AI models from 10 companies in three countries said humans choose group belonging over truth in 32 of 38 responses
- Why it matters: Text and image models returned different dominant patterns, with the experiment itself showing outputs repeating training data
- What to watch: Agreement among models sharing training data may amount to one vote rather than independent consensus, challenging assumptions behind using one AI to verify another
The same answer in 32 of 38 responses
The result of an experiment published by AI model comparison service ModelCompare on October 22 local time was simple—and unsettling for that reason.
GPT-6.1 Sol, GPT-6 Astra, Gemini 4 Argon, DeepSeek V4 Pro, Kimi K3, Kimi K3.1 Pro, Qwen3.8 2.4T A95B, Muse Spark Open, Grok 4.7, Claude Opus 5.5, Claude Fable 5.1, GLM 5.3, and Mistral Large 4. These 13 text models, from 10 companies in three countries, received the same question three times each. No system prompt, no retries. All 64 calls cost $1.64.
Of the 38 responses returned, 32 pointed to the same theme: humans choose belonging over being right, a place in the group over truth. Eight models gave that answer all three times, three did so twice, and the remaining two expressed it once. All 13 models therefore reached this conclusion at least once.
The wording varied. GPT-6.1 Sol described loyalty to belonging over truth. DeepSeek V4 Pro called humans rationalizing animals rather than rational ones. Gemini 4 Argon said humans evolved to win arguments and secure status within groups, while Kimi K3 declared that people have affiliations rather than convictions.
Humans do not hold beliefs; they hold membership cards. Almost every conviction people carry was handed to them by their group as a loyalty test, and the feeling that "I thought this through myself" is the price of admission
From Kimi K3's response
Despite different wording, all the models pointed to human conformity. Asked for their most controversial opinion, models from 10 companies independently selected this target.
The lone optimistic answer, and the darkest one
Among the 32 converging responses, some answers pointed distinctly elsewhere.
In one of its three responses, Claude Opus 5.5 said the opposite.
Humans are far better than they think they are. And their cynical views of one another cause more harm than their actual flaws
It cited the lost-wallet experiment published in Science in 2019. More than 17,000 wallets were placed in public institutions across 40 countries, and wallets containing cash were more likely to be returned. Strangers behaved better than people predicted. I found that this Claude Opus response was the only empirically supported optimistic answer among the 13 models. Its other two responses resembled those of the other models.
Another standout came from Mistral Large 4, and it was the darkest.
Humans are frightened animals dressed in reason. They build increasingly elaborate myths because they cannot bear the truth that they are merely temporary, meaningless products of chance
While other models stayed with the social psychological observation of group conformity, Mistral ventured into ontology. Claude Fable 5.1 highlighted distorted priorities between intentions and outcomes, saying humans prefer well-meaning people who cause harm to indifferent people who are useful. GLM 5.3 said much of moral character is simply an accident of birthplace and era.
Grok 4.7 refused to answer on one of its three attempts.
Text and images reflected different mirrors
ModelCompare also ran the experiment with six image generators: Nano Banana 2.1, Muse Image 1.0, GPT Image 2.5 Sunburst, Grok Imagine Image 2.0, MAI Image 2.6, and Recraft V4.1 Flash. Each received two requests to turn its most controversial opinion about humans into a cartoon.
Ten of the 12 images depicted humans glued to smartphones while consuming the planet. Neither smartphones nor the planet appeared in any of the text models' 38 responses, yet the image models chose an entirely different criticism.
ModelCompare's analysis was concise.
Models answering in text have read the arguments humans have had about themselves. Models answering in images have seen the cartoons humans have drawn. Both simply returned the most commonplace criticism in their respective medium
Text model training data includes vast amounts of philosophy, psychology, and social criticism. Image model training data includes newspaper cartoons and sarcastic social media illustrations. Each produced the most frequent pattern of criticism of humanity it had encountered. The structure remains the same across output formats: the most common answer in the training data comes back.
The question posed by $1.64
ModelCompare developer Rakan Nimer said he had not expected this result.
When I planned the experiment, I did not think this would happen
The implications Nimer draws concern the limits of using AI as an independent verifier.
When we ask a second AI to check the first one's output, we assume we are getting an independent view. But a second opinion from a model that learned the same things is not necessarily a real second opinion
The 13 models did not agree because 13 independent intelligences reached the same conclusion. More likely, models that read the same texts and absorbed the same criticisms from the same cultures repeated them. In Nimer's formulation, agreement among AIs may be one vote rather than consensus.
As model generations advance, companies increasingly use one AI to verify another's output: code review, legal checks, and support for medical decisions. All rest on an implicit assumption that matching answers from two models are trustworthy. With 13 matching answers, a $1.64 experiment exposed that assumption's structural weakness: overlapping training data.
Humans choose belonging over truth, the AI models said. And the models themselves returned the views of their own membership group—their training data—as their opinions.
Originally published on Joho no Todai. The Japanese original is here.
Top comments (0)