42 Is Not a Random Number

Go ask any frontier model for a random number between 1 and 100.

Did you get 42? Maybe 73? Possibly 47?

Try it again. And again. You'll notice something: the "random" numbers aren't random. They cluster around the same small handful of values.

42 shows up a disproportionate amount. So do 47, 57, 37, and 73. Most of the range never comes up at all.

Better yet, don't ask the model anything. Open your own codebase and search for 42. Has something been writing your tests for you?

This probably doesn't surprise you if you've spent any time thinking about how these models work. But it surprises a lot of people. I've watched colleagues, smart engineers who use Claude and GPT every day, genuinely believe the model was generating random numbers. It wasn't. It was doing what it always does: predicting the most likely next token based on its training data.

And since Douglas Adams burned 42 into the collective consciousness of the internet as "The Answer to the Ultimate Question of Life, the Universe, and Everything," that's the number that surfaces when you ask a language model to pick one at random. It's not random. It's popular.

We are eighteen months and four model generations past when people first noticed this, and it still happens. Every frontier model I have access to does it. The newest releases do it. We made the models staggeringly smarter and the dice still read 42.

The Illusion of Competence

The 42 problem isn't a quirky party trick. It's a symptom of something that scaling never fixed and never will. LLMs are really good at looking like they can do things they cannot do. They're so fluent, so confident, that we forget they're pattern matchers operating on token probabilities. They don't compute, measure, or observe. They predict.

Most of the time the prediction is good enough that we don't notice the difference. When you ask a model to write a Python function, it isn't "thinking through" the logic. It's generating the most probable sequence of tokens given the context, and for code that works shockingly well because there are millions of examples in the training data.

But "good enough most of the time" breaks down fast when you need things to be correct all of the time. And the gap doesn't close as models get bigger, because it isn't a knowledge gap. It's architectural. Research keeps finding the same thing: even simple arithmetic like 1 + 1 is retrieved as a memorized pattern rather than executed as an algorithm. Make the numbers bigger or the format unfamiliar and the apparent certainty evaporates. You can't train your way out of the wrong mechanism.

Things LLMs Still Can't Do

Math. Ask a model to multiply 1,847 by 923 and it'll probably get it right. Probably. Ask for 184,739 times 92,341 and you're rolling the dice, except it can't roll dice either. There's no calculator in there. It's predicting which digits look right, and for simple arithmetic the patterns hold. Past that you get a confident wrong answer.

Randomness. We covered this. No dice, no coin flips, no shuffling. The model will perform randomness for you and it looks convincing, but it's theater.

Time What time is it? What's today's date? What's 14:05UTC in PST?

Encoding and hashing. Base64 encode this string. Generate an MD5. These have exactly one correct answer, and models produce wrong ones confidently, because they're predicting what encoded text looks like rather than encoding anything.

Unit conversions. Kilometers to miles, bytes to gigabytes, how many seconds in 47 days and 3 hours. It'll nail the common ones it has seen a thousand times. Step off that path and accuracy drops.

Counting characters. The r's in "strawberry" is the meme version, but tokenization is the real limit. The model works in chunks of text rather than letters, so anything depending on exact character positions is a guess.

I could go on and on. Here's a whole list of things we shouldn't be letting LLMs vibe out.

Agents Aren't Coming. They're Here.

Agentic behavior is the default now, not a feature. Long-running multi-step execution, computer use, parallel sub-agents. Models run autonomously for hours, and the trendlines point at day-long runs.

That changes the stakes of everything above. When the model was an autocomplete in your editor, these limitations were annoying but contained. You'd catch the bad math in code review. You'd notice the wrong hex color when you previewed the UI. A human sat between every mistake and the world.

Agents remove the human from the loop. An agent that can't reliably do math shouldn't be reconciling invoices. One that doesn't know the current time shouldn't be scheduling anything. One that can't generate real randomness shouldn't be minting security tokens. But hand it those tasks without tools and it'll try anyway, confidently, and because the answers look right, nobody catches it until something downstream breaks.

We Won the Tool War

The fix has never been a secret: stop asking the model to be a calculator and give it access to one.

User: Give me a random number between 1 and 100

Without tools → "42" (pattern matching, not random)

With tools    → calls random_int(1, 100) → 67 (actually random)

And the industry went and did it. The Model Context Protocol went from a promising spec to genuine infrastructure in about a year. SDK downloads run into the tens of millions a month and keep climbing. Anthropic handed the protocol to the Linux Foundation. Every major model speaks it, every serious IDE embeds it, and the public server registry passed the point where anyone bothers counting. The tool war is over. Tools won.

So why are agents still failing?

Not for lack of tools. Because a tool only helps if it actually returns the right answer, and a huge share of what agents reach for still isn't wired to anything deterministic. The model is left guessing at the hash, the conversion, the timezone, the random number. We gave it hands. We didn't always make sure the hands were reliable.

And in an agent, unreliable is contagious.

Why Agents Actually Fail

Say each step in an agent's chain is 90% reliable. That sounds great. Chain ten steps together and you're at 0.9^10, which is about 35%. Drop to a more honest 85% per step across a twenty-step run, the kind of run real agents actually do, and you land below 4%. The agent doesn't fail because one step is terrible. It fails because small uncertainties compound, and a long enough chain of "probably" multiplies down to "almost certainly not."

Every curve on that chart bends toward zero. Only the flat one is a choice you can make.

This isn't about getting a single base64 string right. Every probabilistic step you leave in the chain is a coin you flip again, and agents flip a lot of coins.

I've written before about which LLM features actually survive production, and this is the same argument from the other end: the ones that hold up are the ones whose steps aren't negotiable.

So the lever everyone reaches for is "make the model more reliable per step." Push 90% to 95%. Worth doing, but you're fighting an exponent. The far cheaper move is to take steps out of the probability calculation entirely.

That's what a deterministic tool does. It doesn't make a step 99.9% reliable. It makes it 100%, the same way the + operator is 100%. Base64 encoding through a real encoder isn't usually right. It's right, every time, by construction. Every step you can convert from "probably" to "guaranteed" is a step that can no longer compound into failure. You're not improving the odds. You're deleting a die from the table.

Your Agent Did the Math. Your Model Didn't.

Here's the objection I get every time I bring this up now: "This is solved. I asked Claude to convert 47°F to Celsius, it wrote a Python script, ran it, and gave me 8.33. It does math fine."

It does not do math fine. Python does math fine.

That distinction is the entire point, and it gets waved away constantly. When an agent converts a temperature by writing and executing code, the model contributed exactly one thing: the decision to delegate. The arithmetic happened somewhere else, in an interpreter, deterministically, exactly as it would have if you'd typed it yourself. Swap in a weaker model and the conversion is still correct. Take the interpreter away and the best model on earth goes back to guessing. That tells you precisely where the competence lives, and it isn't in the weights.

This matters because people watch an agent succeed and then update their beliefs about the model. Wrong entity. The tool got it right, the model got the credit, and the next thing that happens is somebody ships a pipeline that assumes arithmetic is handled, with nothing wired up to handle it.

The upgrade here is delegation, not arithmetic. Only one of those is load-bearing. (The mirror image of this, where you credit the model for something your own instructions caused, is why I refuse to maintain a CLAUDE.md.)

The Fix Is Deterministic Tools

So the job was never to make the model smarter. It's to make sure every step with exactly one correct answer gets handled by something that can't get it wrong, and that the model reaches for that something instead of its own intuition. Stop asking it to fake the parts it was never built for and the chain stops rotting from the inside.

A tool that's only probably correct just relocates the coin flip without removing it. You want the ones whose answer is fixed the moment you ask the question, not negotiated token by token.

The big categories already work this way and are basically solved: search, code execution, database access, vision. The gap is the long tail. Base64 and hashing. Timezone math that respects DST. CIDR ranges, unit conversions, business days between two dates, compound interest. Each one trivial alone, and collectively hundreds of operations every agent eventually reaches for.

Right now most teams either let the model guess and hope, or rebuild the same utility functions from scratch and maintain them forever. Neither scales. One ships unreliable agents. The other has every team in the industry independently reinventing a base64 encoder.

"But Claude Already Does This"

Whenever I make this argument, someone opens a chat window, asks Claude to convert a temperature, watches it write and run Python, and shows me the correct answer like it settles the matter.

It doesn't, because the chat window isn't where agents run.

Claude.ai is a product. It ships with an interpreter, a search tool, a file sandbox, and a system prompt that people at Anthropic tuned carefully, and all of that is working on your behalf before the model ever sees your question. Build against the API and none of it comes with you. You get a model and whatever tools you wire up yourself. There is no interpreter in the box.

This is the same gap I hit with pgvector, where the thing that works in a blog post and the thing that survives production turn out to be different systems.

Standing one up in production is a project, not a checkbox. You need a sandbox, because you are executing generated code. You need to decide what it can reach on the network, how long it runs before you kill it, which packages exist inside it, what happens when it throws, and how any of that gets logged. Teams do this, and it's worth doing for genuinely open-ended computation. But reaching for a container to convert 47°F to Celsius is a remarkable amount of machinery to perform arithmetic, and it brings its own failure modes: wrong library, wrong flag, an exception the model reads as success.

The other half of it is that the chat window has a human in it. You're sitting right there. You read the answer. If the conversion came back wrong you would probably notice, and if you didn't, the damage is one bad number in one conversation.

Production agents run for hours with nobody watching. That's the whole reason to run them. The same wrong conversion you would have caught in chat instead feeds step nine, and step fourteen, and a record somebody trusts next quarter.

So yes, Claude can convert a temperature. Inside a product built by people who thought carefully about tools, with you sitting there to check its work. That is not the thing you are shipping.

This Is Why I Built TinyFn

This is the wall I kept hitting. Every agent I built needed a handful of these, and each time I was either bolting on another heavyweight MCP server that ate my context budget or writing the thing myself for the tenth time.

So I built TinyFn.

It's 500+ deterministic functions for exactly that long tail, each returning the single correct answer every time. They're split into focused toolsets you connect to individually through MCP, so you wire up the two or three a given agent actually needs and skip the rest. It plugs into Claude, Cursor, or anything that speaks MCP.

The premise underneath all of it is one line: 0 hallucinations for operations that have exactly one correct answer.

A model can approximate a timezone conversion. The datetime server gets it right, DST edge cases and all. Every one of those is a die off the table.

None of this replaces the model. The model is genuinely brilliant at what it's built for. TinyFn handles what it wasn't.

The Lever Isn't the Model Anymore

We spent years with one main lever: make the model bigger and smarter. More parameters, more data, better post-training. It worked, and the jump from GPT-3 to what shipped this year is staggering.

But that lever is hitting diminishing returns on exactly the problems that matter most for agents. A model that's 10% better at predicting tokens is still 0% better at SHA256. The gains left aren't in the model. They're in what you wire around it: the right tools, sharp and deterministic, called at the right moment, so the model never has to fake the parts it can't do.

Nobody judges a physicist for not doing long division in their head. We judge them on their reasoning. Agents are the same.

42 is not a random number. It never was. But give the model a real random number generator, and it doesn't have to pretend anymore. Do that for every deterministic step in the chain, and your agent stops failing for reasons that were never the model's fault.