A startup asked us for "an AI agent". That request is a trap, and the way you
scope the first two weeks decides whether the thing ships. I led two builds
that made it to production, a travel agent with a team of 10 and an
e-commerce concierge that finalizes purchases inside user constraints, also
- Here is the sequence that worked, and the mistakes I would skip next time.
1. Scope the decision, not the demo
"AI agent" is not a product. Write down the single decision the agent is
allowed to make. The travel agent could suggest and prepare an itinerary. The
e-commerce concierge could complete a purchase. One decision each. If you
cannot name the decision, you do not have a scope, you have a wish.
The rule I use: the agent may act when the action is reversible and cheap to
verify. Finalizing a purchase is neither, so the concierge shows the exact
cart and asks for confirmation. That constraint killed a hundred design
arguments in one meeting.
2. Architecture: boring beats clever
The stack that shipped both agents:
- One orchestrator with a tool interface. Not a swarm, not a framework of the week. A loop, a set of tools, and a budget.
- Retrieval over structured data, not free-text dumps. The travel agent queried flight, weather, and geolocation APIs directly. Tools returned typed results, so the model reasoned over clean data.
- A state store that survives restarts. Users abandon sessions and come back. If your agent forgets, it is a toy.
The concierge ran on AWS with the architecture mapped before the first sprint,
and governance guardrails defined up front. Boring, reviewable, and each
piece testable without the model.
3. Evaluate before you trust
The question is not "does it feel smart". It is "how often does it pick the
right action". We built a small eval set of real scenarios and measured
against it after every prompt or tool change. When the number went down, we
stopped and fixed it.
Two findings that surprised me:
- Most failures were tool failures, not reasoning failures. Bad error messages from an API caused more wrong answers than the model itself.
- The model followed system prompts worse as we added more rules. Splitting rules into the tool layer (the concierge physically cannot order outside constraints) beat any prompt engineering.
4. Production is a different project
Latency, retries, cost per session, and abuse. Log every tool call and its
arguments, because the first incident report will ask. Set a spend cap per
session. Version the prompts like code.
I now run the same sequence as a fractional AI tech lead for teams that have
the product but not the AI leadership: scoped architecture first, working
slice second, the person who designed the system stays in code reviews until
it ships. If you want the details, they are here:
https://dotrungnghia.com/fractional-ai-tech-lead
Full case studies: https://dotrungnghia.com/ai-consultant
Do Trung Nghia is an AI consultant and tech lead in Vietnam. 40+ AI/ML
solutions shipped across telecom, finance, blockchain, and cloud. Remote for
global teams. More: https://dotrungnghia.com
Top comments (0)