Most developers start their AI journey by building a wrapper around a prompt. You send a user query, the LLM gives an answer, and the interaction ends. That is a chatbot. It is a linear process.
Agentic workflows are different. Instead of a straight line, you build a loop. An agent does not just answer; it plans, executes a tool, observes the output, and corrects its path until the goal is reached.
The mental shift: From Prompting to Orchestration
When you build a standard chatbot, your main concern is the prompt. You spend hours tweaking the system message to get the right tone. With agents, the prompt is just the starting point. The real engineering happens in the orchestration.
An agentic system usually consists of three things: a core LLM, a set of tools (functions), and a reasoning loop.
The loop looks like this:
- Plan: The LLM decides which tool to use based on the goal.
- Act: The system executes that tool (e.g., a database query or an API call).
- Observe: The result of the tool is fed back into the LLM.
- Refine: The LLM decides if the result is sufficient or if it needs another step.
The trap of the 'Black Box' agent
One mistake I see often is giving an LLM too much autonomy without constraints. If you tell an agent to "fix the bug in the repo," it might start deleting files or creating infinite loops of commits.
To avoid this, you need to implement guardrails. I recommend two specific patterns:
1. The Human-in-the-Loop (HITL) checkpoint
Do not let the agent execute "write" operations (POST, DELETE, PUT) without a manual approval step. Your agent should propose a plan, and a human should click "Approve" before the code actually hits the server. This turns the agent from a risky autonomous entity into a powerful assistant.
2. Tool Narrowing
Do not give your agent a general "execute_python" tool. That is a security nightmare. Instead, give it specific, atomic functions. Instead of run_sql(), give it get_user_by_email() or update_order_status(). This limits the blast radius and makes the LLM more reliable because the tool definitions are clearer.
Dealing with the 'Infinite Loop' problem
Agents can get stuck. They might try the same failing tool call over and over because the LLM thinks a slight change in the arguments will fix the error.
I handle this with a hard iteration limit. If the agent has not reached a "Final Answer" within 5 or 10 loops, the process must kill itself and return an error. You should also log the "thought process" (the internal monologue of the LLM) to a database. When an agent fails, you do not debug the code; you debug the reasoning trace.
Practical implementation tips
If you are starting this today, do not jump straight into heavy frameworks. Start with a simple while loop and a JSON schema for tool calls.
Keep your tool descriptions extremely literal. Instead of saying "This tool fetches data," say "This tool takes a user_id as a string and returns the last five transactions from the SQL database." The LLM relies on these descriptions to decide which tool to pick. If the description is vague, the agent will hallucinate arguments.
The Takeaway
Agentic behavior is not about using a better model. It is about moving the complexity from the prompt into the system architecture. Stop trying to write the "perfect prompt" that handles every edge case. Instead, build a loop that allows the model to fail, see the error, and try a different approach.
Build small, atomic tools. Implement a hard loop limit. Always keep a human in the loop for destructive actions. That is how you move from a toy demo to a production-ready agent.
Top comments (0)