If you've built a system with integrated AI (or are building one now) you've probably felt the pain of a climbing token bill.
When costs spike, the instinct is often to look for a cheaper model. But what if the problem isn't the model but the "task" you gave it?
People usually bring models into a system because they need some "thinking" done. But what if the task you handed it isn't actually a “thinking” task? what if it's just basic math? adding up a column of numbers? sorting a list alphabetically? checking if a string matches a regex pattern? scraping data off a page?
For example, does a task like 2 + 2 need any “real” thinking? Nope! The answer is always going to be 4!
Whether you're in 1978 or 2025, India or Japan the answer is always going to be 4.
When a task returns the same output for the same input, EVERY time, it's deterministic.
There's ZERO creativity, judgment, or opinion needed. A calculator, a basic script, or an existing library can handle a deterministic task perfectly, instantly, and for fraction of a penny.
Now you may ask, “Shloka WHY are you whining and being so dramatic? Why the heck is it a problem to hand a deterministic task to an AI model when it can do both deterministic and non-deterministic tasks?”
Wellll……
There are a bunch of problems, but for this blog, let's stick to two of em:
Tokens (i.e some sweet Moolah). by now we've learned that more tokens means more money. When you ask a model to do something deterministic, like "add these 50 numbers", it has to generate every step of that work. That's slow AND expensive compared to just running actual code, which does the same task in a fraction of a second for basically nothing.
Reliability. Fun fact, models CAN get deterministic tasks wrong. Now this is a dumb example, cause we have come a long way, but to keep things simple... a calculator never miscounts. Whereas A language model, doing arithmetic by "reasoning" in text, sometimes does.
It's like asking a toddler "what's 2+2?", they'll usually say 4, but every so often they'll confidently get it wrong, and you won't catch it until it's already caused a problem downstream.
Now, in a real codebase, at scale, the line between "this needs a model" and "this doesn't" gets blurry FAST since AI ends up woven into a dozen different steps of a pipeline. So how do you actually draw that line?
Well, the general fix isn't asking the model to do "less" thinking. It's asking it to do "different" thinking.
A good system doesn't ask the model to do everything itself, instead, it gives the model the ability to call actual tools: a code interpreter, a calculator, a database query, a search function, for anything deterministic. The model's real job becomes:
- Understand what's being asked
- Decide which tool solves the deterministic part
- Hand that part off
- Use its own judgment only for the fuzzy, non-deterministic parts (writing, reasoning, interpreting, deciding)
This is often called "tool use" or "function calling."
So how do you actually decide what gets delegated and what doesn't?
Think of it as a priority list, checked top to bottom, cheapest and most reliable first, most expensive and flexible last. Only move down if the step above can't do the job.
- Can plain math or logic solve it? Write a script. Always check this first.
- Can an existing tool or lookup solve it? Call a database, API, or search index directly instead of guessing.
- Can simple fixed rules solve it? Even a long list of "if this, then that" rules can usually be handled with plain code.
- Does it need real understanding, language, or judgment? Now bring in AI.
A quick gut-check, if you forget the order is to ask yourself: would 10 different careful people all land on the exact same answer? If yes, you're probably still in script territory. If no, that's AI's job.
This keeps your system fast and cheap by default, and saves the AI's "thinking" for the one thing it's actually needed for.
PS: In the next blog I want to build on this and talk about how even handing AI control of deterministic tasks is not the end game.
Top comments (1)
Noice