The Builder’s Letter — Edition 6
From Chatbots to AI Agents: When Should Software Take Action?
For the past few years, much of the conversation around Artificial Intelligence has focused on chatbots.
We ask questions, receive answers, generate content, summarize documents, and interact with AI through conversational interfaces. These capabilities have already changed how people work with software.
But a new shift is taking place.
AI is moving beyond simply answering questions. It is increasingly being used to perform tasks, interact with systems, execute workflows, and take actions on behalf of users.
This is where AI agents enter the picture.
However, there is an important engineering question that every software builder should consider:
Just because AI can take an action, does it mean we should allow it to?
1. The Difference Between an AI Chatbot and an AI Agent
A traditional chatbot primarily responds to user input.
For example, a user might ask:
"What were my total expenses last month?"
The chatbot processes the question, retrieves or interprets relevant information, and provides an answer.
An AI agent can potentially go further.
It might:
- Understand the user's request.
- Identify the required information.
- Call the appropriate API.
- Retrieve relevant records.
- Perform calculations.
- Prepare a report.
- Ask for confirmation before sharing or submitting it.
The difference is not simply that one uses a better AI model.
The real difference is the ability to interact with external tools and systems.
A chatbot communicates.
An AI agent can communicate, reason through a workflow, and take permitted actions.
2. An AI Agent Is More Than an LLM
One common misconception is that building an AI agent simply means connecting an application to an LLM and giving it a prompt.
That is not enough for production software.
A reliable agent typically requires several components:
LLM or reasoning model
Interprets user intent and determines the next step.
Tools and APIs
Allow the agent to interact with external systems, databases, services, and application functionality.
Business logic
Enforces the actual rules of the application. Financial calculations, permissions, validation, and transaction rules should not depend entirely on AI-generated decisions.
Context and state
Help the agent understand what has already happened in a workflow and what information is still required.
Permission and approval layer
Controls which actions the agent can perform independently and which require user confirmation.
Monitoring and logging
Make it possible to understand what the agent did, which tools it called, and where something went wrong.
An agent is therefore not just a model. It is an engineered system built around a model.
3. The Most Important Question: What Should AI Be Allowed to Do?
Imagine an AI-powered business application.
A user asks:
"Show me customers who have outstanding payments."
This is a relatively low-risk request. The system can retrieve customer records and display the results.
Now consider another request:
"Send payment reminders to all customers who owe me money."
This request involves an external action.
The system needs to identify the correct customers, prepare appropriate messages, verify the communication channel, and potentially send messages on the user's behalf.
Should the AI send everything automatically?
Not necessarily.
A better workflow could be:
- AI identifies customers with outstanding balances.
- The application retrieves verified account information.
- AI prepares reminder messages.
- The user reviews the customer list and message content.
- The user approves the action.
- The application sends the approved reminders.
- The system records the outcome.
This approach combines AI flexibility with deterministic software control.
The goal is not to eliminate human involvement. The goal is to reduce unnecessary work while keeping important decisions under control.
4. Why Human Approval Still Matters
Automation is valuable, but uncontrolled automation can create serious problems.
Consider these scenarios:
- An agent sends a payment reminder to the wrong customer.
- An AI-generated financial summary contains an incorrect figure.
- An agent updates a record without sufficient authorization.
- A workflow executes twice because the system retries an operation.
- An agent interprets an ambiguous instruction incorrectly.
These are not merely AI problems. They are software engineering, security, and product-design problems.
For this reason, production AI systems should follow clear permission boundaries.
A practical classification is:
Low-risk actions: Reading records, searching, filtering, summarizing, and generating draft reports.
Medium-risk actions: Preparing reminders, creating draft invoices, or proposing changes for review.
High-risk actions: Sending external communications, modifying financial records, deleting data, or initiating payments.
The higher the impact of an action, the stronger the validation and approval requirements should be.
5. Deterministic Software + AI: A Better Architecture
One of the most important architectural decisions is knowing which responsibilities belong to AI and which belong to traditional software.
AI is useful for:
- Understanding natural-language requests.
- Extracting information from unstructured input.
- Classifying and categorizing content.
- Suggesting actions.
- Coordinating multi-step workflows.
Traditional application logic should handle:
- Financial calculations.
- Data validation.
- Authentication and authorization.
- Database transactions.
- Business rules.
- Audit records.
- Final execution of sensitive operations.
For example, an AI agent may understand that a user wants to record a payment of PKR 5,000.
But the actual balance calculation, transaction validation, database update, and audit trail should be handled by the application's trusted business logic.
AI should help decide what needs to happen. The application should control how it happens.
This separation makes systems more predictable, testable, secure, and maintainable.
6. Building Practical AI Agents for Small Businesses
AI agents are not limited to large enterprises.
Small businesses can benefit from carefully designed automation too.
Consider a shopkeeper managing customers, expenses, receivables, and daily transactions.
A future AI-powered business assistant could potentially help with workflows such as:
Business insights
"Which customers have the highest outstanding balances?"
Transaction assistance
"Prepare a transaction entry for the payment I just received."
Daily reporting
"Summarize today's income and expenses."
Customer follow-ups
"Prepare reminders for customers whose payments are overdue."
Business questions
"Compare this month's expenses with the previous month."
These are practical use cases because they address real business problems rather than introducing AI simply for marketing purposes.
At HisabDo, the broader product direction is focused on making everyday financial management easier for individuals and small businesses. AI-assisted workflows could become a useful extension of that direction, provided they are introduced with proper validation, permissions, and user control.
The objective should always be to solve a real problem.
7. Start Small Instead of Automating Everything
When building an AI agent, it is tempting to connect every available API and allow the system to perform multiple operations.
That is rarely the best starting point.
A more reliable approach is incremental:
Step 1: Read-only assistance
Allow the agent to search, retrieve, and summarize information.
Step 2: Recommendations
Let it identify patterns and suggest possible actions.
Step 3: Draft creation
Allow it to prepare records, messages, and reports without submitting them.
Step 4: Human-approved execution
Introduce actions that require explicit user confirmation.
Step 5: Limited autonomous workflows
Only automate well-tested, low-risk tasks with clear boundaries, monitoring, and recovery mechanisms.
Each stage should be evaluated before moving to the next.
Measure accuracy, execution success, latency, cost, failure rates, and user trust.
A useful AI agent is not the one that performs the most actions. It is the one that performs the right actions reliably.
8. The Engineering Challenges We Should Not Ignore
AI agents introduce new challenges that software teams must address.
Reliability: Models can misunderstand instructions or select inappropriate tools.
Security: Tool access must follow least-privilege principles. An agent should never receive broader permissions than its task requires.
Data privacy: Sensitive business and customer information must be protected throughout the workflow.
Cost and latency: Multiple model calls and API interactions can make agent workflows expensive or slow.
Observability: Developers need visibility into decisions, tool calls, failures, and execution history.
Idempotency: Repeated requests or retries should not accidentally create duplicate transactions or trigger the same external action multiple times.
These challenges remind us that building an AI agent is not just an AI development exercise.
It is a complete software engineering responsibility.
9. The Future Is Not Fully Autonomous Software
I believe the next generation of useful business applications will combine three capabilities:
- Traditional software for accuracy, consistency, and control.
- AI for understanding, reasoning, and assistance.
- Human judgment for decisions that require responsibility and trust.
We should not measure progress only by how much work AI can perform independently.
We should measure it by how effectively technology helps people complete their work.
For businesses, the real opportunity is not replacing every human decision with AI. It is removing repetitive work, making information easier to understand, and helping people make better decisions.
That is where AI agents can create meaningful value.
Final Thoughts
Moving from chatbots to AI agents is an important step in the evolution of software.
But the transition requires more than connecting an LLM to APIs.
It requires thoughtful architecture, secure integrations, reliable business logic, clear permissions, monitoring, and human oversight.
As software engineers and product builders, we must remember:
The future of AI is not just about building systems that can take action. It is about building systems that know when, how, and under whose authority to take action.
That is the difference between an impressive AI demo and a dependable AI-powered product.
What's Next?
In Edition 7 of The Builder’s Letter, we will explore:
"The Data Problem in AI: Why Better Data Often Matters More Than a Bigger Model."
Because even the most capable AI model cannot consistently produce reliable results when the information behind it is incomplete, inaccurate, or poorly structured.
Mian Usman Khalid
Founder & CEO, XICTEK Systems
The Builder’s Letter — Building practical technology for real-world problems.
Top comments (0)