It is amazing how many problems can be solved by agents now but getting them right is still a pain as they are pretty bleeding edge right now. This article is some of the learnings that I had with agents since past year or so. These may or may not work for you, but it's good to share these insights so that rational choices can be made while building agents. Idea is that you:
1. Use agents only when needed
Main idea: not needed for orchestration tasks where no need to decide next steps. Eg if task flows like step1 → step2 → LLM → step3, no need of agent, use foundation models directly.
Remember fundamental rule of decision (applies to humans too). Do not decide when you don't have to. Why? Making decision is tedious, mental energy is limited. Every decision has cognitive load, takes mental resources. Time spent on trivial decisions (opportunity cost) is time not spent on important ones. Too many decisions leads to fatigue, you are spending more time on frivolous stuff rather than one which really need attention. Plus context switching overhead, switching between decisions has cost.
For agents, same problems exist. Decision fatigue = token budget depletion. Cognitive load = context window filling up with decision logic (mostly useless), attention mechanism may not work out in our favour. Each decision point is also potential failure point where agent might pick wrong option.
Also on same line, if it's possible to do job X without agents, just do it. Deterministic systems (plain old code) > uncertain systems (agent).
2. Make decisions simple
From decision theory, decision under uncertainty is tricky as you don't know probabilities of outcomes, just that multiple outcomes exist. In agents, this happens when tool returns vague error, external API might be down, or you don't know if user input is valid: agent has to decide next step without knowing what will work (how can it recover from unusual error? should it ask user for clarification or make assumptions?). You have bunch of strategies:
minimax/maximin: minimize worst loss or maximize worst gain (pick option where worst-case scenario is least bad, like choosing restaurant where even their worst dish is edible, how agents are really supposed to decide that? This might be somewhat alleviated by good defaults, eg if user's intent is not clear, do X)
min regret: minimize maximum regret (pick option you'll regret least if wrong, like
limit = 10in API GET calls or some SQL query)
Problem is, agents can't compute these properly, unless we set some sensible default and getting that 'sensible' default requires domain expertise or at least good knowledge of business context in which agent is operating.
Instead try to convert decision under uncertainty to decision under risk (where probabilities are known) or even better, decision under certainty. For example, by having tools return structured options with clear outcomes. Eg if any API call fails, instead of returning vague error, tool may capture it and simply return 'Service unavailable, redirect to fallback' (risk with known-ish outcomes). This transforms unbounded search space into finite choice set.
3. Always set defaults (reduce choices to make)
Do not decide where you don't have to, eg setting batch size or limit in API. Why? Every parameter is additional entropy. If 90% of time limit = 100 works, making agent decide this every time is waste of time, cost and context window. Remember each token in context window should be for solving problem.
4. Offload work to plain old code
Whether it's fixed sequences or bookkeeping tasks, if you know what needs to happen, don't make agent figure it out.
Eg if something requires step1 → step3 in fixed order, just write code that does it. If agent always needs init stuff, cleanup, or reporting, do it yourself (before/after calling agent). Why? Faster, less chance of error, less decisions for LLM, easier to use dumber models.
From complexity perspective, you're reducing from O(n) decisions to O(1). Each decision costs latency + tokens + error probability. Fixed sequences and deterministic operations are just state machines, using agent for them is like using ML to add 2+2.
This is about separation: agents handle policy (what to do), code handles mechanism (how to do it). Running mechanical, deterministic operations through probabilistic system is wasteful. It's like using random number generator to compute pi (which is as useful as mining bitcoin…wait, I take my words back).
5. Keep track of context length
Instead of returning raw values from tools, parse it and return only relevant. Eg there may be sql_query tool, it maybe have query like select * from xyz and that might return 1000 rows, you can just limit output to say 10 rows and say more and let LLM decide if they need any specific filtering.
Why matters: LLMs perform worse as context fills up. Eg some interesting papers like "Lost in the Middle" shows models struggle to retrieve info from middle of long contexts, degrading to <60% accuracy at full context. NoLiMa benchmark shows even GPT-4.1 drops from 95.6% at 1K tokens to 64.7% at 128K (effective length only ~16K), GPT-4o even worse: 98.1% at 1K to 56.0% at 128K (effective length ~8K). More irrelevant tokens = more noise = worse decisions.
Tool outputs should do progressive disclosure returning summary + samples, not full dumps. Let agent request specifics if needed (lazy evaluation).
6. Agent is only as good as its tools
Keep tools as dumb as possible (minimum arguments, pydantic types instead of dict, ambiguity next to none etc). Eg once I've seen a tool call function get_projects(…)-> None | list[Projects], name itself doesn't make sense to me, will it get me one project or multiple? I need to handle None too, blah blah. Keeping it simple and dumb like get_project, list_project makes it pretty obvious what tool does, and helps agent to decide quickly. Also thoroughly test tools and add all retry in tool rather than leave it up to agent. Eg if tool is summarize_text(text)-> str, whether to retry and not should be up to summarize_text, no need to bother agent with internal details (how can agent even decide that? eg say token is expired, agent has no way to recover that).
Overall
Make decision as dumb as possible, make it easy to decide, do as much as possible yourself and leave only things that depend on dynamic conditions to agent.
This is my favorite approach in solving any agentic problem: put yourself in agent's shoes. If for given conditions/environments, if I have trouble solving the problem and can't decide what to do next (next toolcall or reply back etc), no way agent can do that. Then my way out is, how can I make my life easier? Once I do that, I arrive at pretty dumb and simple solution.
Here's the mental model: imagine you're the agent, you're lazy (as you should be), you don't want to make unnecessary decisions, and you'd rather use existing solutions than figure things out from scratch. What would make your life easiest? That's your design principle.
Many cases once you make decision dumb and simple, it actually becomes simple orchestration call and you just need to make llm call somewhere (and use smaller models like gpt-5-mini instead of sonnet 4.5 which you had planned). You're minimizing entropy of system: less uncertainty = more reliability, lower costs, better performance.
Make the work dumb, stupid, and simple. Simplicity is hard to achieve but it's the only thing that reliably works.
Top comments (0)