This is why you shouldn’t ask ChatGPT for a random number. Six chatbots out of seven, asked for something between 1 and 50, gave the same answer: 27. ChatGPT, Gemini, Claude, DeepSeek, Llama and Qwen all said it. Only Grok broke ranks, with 42 — the joke answer to the meaning of life.
That was one person’s test — a data scientist called Mohd Faraaz, in June 2025 — and it would be easy to assume he got unlucky. He didn’t. Javier Coronado-Blázquez, in Madrid, asked for a random number 75,600 times: six models, seven languages, three ranges, six settings. Every model had its favourites. The big ones were no better at this than the small ones. If anything, he found, they were worse.
The reason is simple, and a little disarming. To a model a number is not a quantity. It is a piece of text like any other — “a number such as ‘2’ has no further meaning for a LLM than ‘3’, ‘+’ or the word ‘horse’”, as the study puts it. A chatbot writes its answer one small piece at a time, working out a chance for every possible next piece and drawing one. The draw is real. What it cannot do is even the chances out.
Before we feel smug: we are no better. In a study from 1976, people asked for the first digit that came to mind chose 7 more than any other, 28.4% of the time. When the YouTube channel Veritasium asked 200,000 people for a number from 1 to 100, the favourites were 7, 73, 77 and 37. Asked which number they thought nobody else would choose, their top answers were 73 and 37 — two of the most popular picks in the same survey.
None of this is a flaw to be fixed. It is what these things are. Just don’t use one to draw the raffle.
Read the full note in AI Garden
Top comments (0)