A demo proves that something can work. Production proves that it keeps working when everything gets messy.
There is a sentence I have started to distrust in AI projects:
“It worked in the demo.”
It sounds positive.
Usually everybody smiles.
The founder is happy.
The client is impressed.
Someone records a Loom video.
Maybe there is even a little celebration 😂.
Then real users arrive.
Different documents.
Missing fields.
Strange requests.
Old data.
Long conversations.
Network failures.
Permissions nobody thought about.
Suddenly the magical AI product becomes significantly less magical.
This is why I think “it worked in the demo” is one of the most expensive lies in AI engineering right now.
Not because the demo was fake.
The demo probably did work.
The lie is believing that demo success tells you how close you are to production.
It usually doesn't.
There is a huge difference between:
One clean prompt
+
One controlled workflow
+
One happy-path result
and:
Thousands of users
+
Bad data
+
Long context
+
Retries
+
Tool failures
+
Permissions
+
Model variability
+
Real money
That second system is where the engineering starts.
1. The 0 → 60 Trap 🏎️
AI creates a very strange feeling of velocity.
You can go from:
“We should build this.”
to:
“Look! It works!”
in a weekend.
That is amazing.
It is also dangerous.
Because teams start assuming that the first 80% of the product represents 80% of the engineering work.
It doesn't.
Getting an AI workflow to work sometimes is increasingly cheap.
Getting it to work consistently is where things become painful.
This is the March of 9s.
Maybe your first prototype works:
80%
of the time.
Pretty good.
Then you try to reach:
90%
Then:
99%
Then:
99.9%
Each extra 9 starts demanding more architecture.
More evaluations.
More guardrails.
More monitoring.
More fallback logic.
More boring software engineering 😂.
The demo gives you false velocity because the first part of the journey is ridiculously fast.
The last mile isn't.
2. Your Demo Has Almost No Entropy 🧹
Most demos are clean.
Very clean.
You know exactly what document is going in.
You know exactly what question will be asked.
You probably tested the prompt five minutes earlier.
Sometimes you even know what the model is going to say 😂.
Production users do not behave like this.
They upload:
- Terrible PDFs
- Half-completed forms
- Scanned documents
- Duplicate information
- Contradictory information
- Extremely long files
- Completely irrelevant files
Then they ask:
Can you just figure it out?
Welcome to production.
This is where context rot becomes real.
Longer context doesn't automatically mean better intelligence.
Every extra piece of information competes for attention.
Important instructions.
Retrieved documents.
Tool responses.
Conversation history.
System rules.
User requests.
All sitting inside the same context.
The model has to decide what matters.
And sometimes it decides badly.
3. AI Doesn't Always Fail Loudly 🚨
This is probably one of the biggest differences between normal software and AI systems.
Traditional software often fails like this:
500 INTERNAL SERVER ERROR
Beautiful.
Ugly for the user.
Great for the engineer.
We know something broke.
AI can fail like this:
200 OK
Then give you a perfectly formatted, grammatically correct, completely wrong answer.
That is much scarier.
I call these plausible-but-wrong failures.
The answer looks good.
The reasoning sounds intelligent.
The JSON validates.
The UI looks beautiful.
And the conclusion is wrong.
A scripted demo is almost perfectly designed to hide this problem.
You show three successful generations.
Nobody sees the 17 strange ones you generated before recording 😂.
That is why visual polish can create a dangerous illusion of system quality.
A beautiful UI does not make a probabilistic backend reliable.
4. “I Tested It” Usually Means “I Eyeballed It” 👀
Ask a team how they evaluated their AI application.
Sometimes the answer is basically:
We tried a bunch of prompts and it looked pretty good.
That is not an evaluation system.
That is a vibe check 😂.
And vibe checks are useful during exploration.
They are terrible production gates.
You need to move toward Evaluation-Driven Development.
Before changing a prompt, model or retrieval strategy, you should know:
What exactly are we trying to improve?
Maybe:
- Factual accuracy
- Citation correctness
- Tool selection
- Extraction quality
- Retrieval precision
- Task completion
- Refusal behaviour
Then create examples.
Not five examples.
A proper regression set.
For example:
Input
Expected Behaviour
Actual Behaviour
Pass/Fail
Reason
Now when somebody says:
I changed the system prompt and it seems much better.
You can actually test that statement.
This changes everything.
Because AI development stops becoming:
I think version B feels smarter.
And becomes:
Version B improved task completion from X to Y on our reference set but caused a regression in citation accuracy.
Much more useful.
5. Agent Demos Hide Excessive Agency 🤖
Agents look incredible in demonstrations.
You ask:
Research these companies and update our CRM.
Then you watch the agent:
Search.
Reason.
Call a tool.
Update something.
Done.
Magic.
But every new tool you give an agent expands the amount of damage it can potentially do.
Read database?
Fine.
Write database?
Different risk.
Delete database?
Now I am awake 😂.
The demo usually proves:
The agent knows how to call the tool.
Production requires proving:
The agent knows when it should NOT call the tool.
Those are very different engineering problems.
This is why production agents need deterministic boundaries.
For example:
READ
↓
Allowed automatically
But:
DELETE
↓
Human Approval
↓
Execute
Or:
PAYMENT > $1,000
↓
Human Approval
The model can recommend an action.
It shouldn't automatically have permission to perform every action it can imagine.
6. The Happy Path Hides Retry Problems 🔁
Here is a boring production problem that can become very expensive.
Your agent calls:
charge_customer($50)
The payment succeeds.
But the response times out.
The agent thinks the tool failed.
So it retries.
charge_customer($50)
Now the customer paid twice.
Nothing about your model intelligence caused this.
This is just distributed systems meeting autonomous agents 😂.
Your demo probably never tested it.
Because your Wi-Fi worked.
Production doesn't care.
This is why write operations need things like:
Idempotency keys.
The agent can retry ten times.
The underlying system understands:
This is still transaction ABC123.
And processes it once.
These boring controls are what separate experiments from systems.
7. Real Users Discover Your Architecture For You 🕵️
During a risk review, one question matters a lot:
What happens when somebody uses this differently from how the builder imagined?
Because they will.
Developers think:
User → Question → Agent → Answer
Users think:
Let me upload 43 PDFs, ask about something unrelated, change the goal halfway through and then tell the agent to email everything to my colleague.
😂.
Real users explore every edge of the architecture.
Sometimes accidentally.
Sometimes intentionally.
That is why production testing cannot only cover:
Does the intended workflow work?
You also need:
What happens when the workflow becomes weird?
This includes things like:
- Ambiguous tool parameters
- Missing information
- Repeated actions
- Invalid files
- Prompt injection
- Long context
- Tool outages
- Partial failures
- User contradictions
- Permission boundaries
The sandbox protects you from entropy.
Production introduces it.
8. Context Hygiene Is Architecture 🧠
There is another failure mode I see creeping into AI systems.
The system prompt starts small.
Then something goes wrong.
So somebody adds another instruction.
Never do X.
Then another bug.
Always remember Y.
Then another.
Unless Z happens.
Six months later your system prompt looks like pothole repair 😂.
Layer after layer of instructions covering previous failures.
Eventually the model is trying to reconcile dozens of overlapping rules.
This is not prompt engineering anymore.
This is architectural debt.
Your prompts, repository instructions and agent context need hygiene.
Delete redundant instructions.
Separate policy from task instructions.
Separate retrieved data from trusted instructions.
Keep context relevant.
More context is not automatically better context.
9. Observability Cannot Stop at Server Metrics 📊
Your Grafana dashboard is green.
CPU normal.
Memory normal.
API latency normal.
Application uptime:
99.99%
Fantastic.
Your AI agent might still be terrible 😂.
Traditional monitoring tells us whether the infrastructure is healthy.
AI systems also need semantic observability.
We need to measure things like:
- Hallucination rates
- Retrieval failures
- Task abandonment
- Agent retries
- Human corrections
- Tool-call accuracy
- Evaluation scores
- Cost per successful task
Because your model provider can change.
Your users can change.
Your data can change.
Your code can stay exactly the same while the quality of the system slowly decays.
No exception.
No crash.
Just worse answers.
That is a very AI-native failure.
The Risk Review I Would Run Before Production 🔎
Before somebody tells me:
We're ready to launch.
I would ask these questions.
1. Where can this system make irreversible changes?
Payments?
Emails?
Database writes?
Deletes?
Those actions need stronger controls.
2. What happens when a tool times out?
Does the agent safely retry?
Or does it accidentally execute twice?
3. What happens when information is missing?
Does the model ask?
Or guess?
4. What happens after 20 tool calls?
Is there a limit?
5. What can the agent access?
Everything?
Or only what it actually needs?
6. How do we know the answer is correct?
Do we have an evaluation dataset?
Or are we eyeballing outputs?
7. What happens when retrieved content is malicious?
Can a PDF change agent behaviour?
8. How do we detect gradual quality decay?
Are we monitoring semantic behaviour?
9. Where must a human intervene?
Every production agent should have an answer to this.
My Pre-Launch Checklist ✅
Before moving an AI application from demo to production, I would want:
[ ] Regression Suite
At least a meaningful reference dataset representing real workflows.
Not just five favourite prompts.
[ ] Semantic Observability
Measure quality, not just uptime.
[ ] Idempotency
Any important write operation should survive retries safely.
[ ] Context Hygiene
Remove contradictory or redundant instructions.
[ ] Tool Permissions
Agents get the minimum permissions they require.
[ ] Human-in-the-Loop
Identify the actions where failure is too expensive to automate blindly.
[ ] Hard Limits
Maximum:
- Steps
- Tokens
- Cost
- Runtime
- Tool calls
[ ] Failure Tests
Deliberately test bad documents, missing fields, tool failures and strange workflows.
Basically...
Try to break your own application before your users do 😂.
The Demo Isn't Useless
I am not anti-demo.
Demos are incredibly useful.
They answer an important question:
Is this idea possible?
But people accidentally use that answer to conclude:
This product is ready.
Those are completely different questions.
A demo proves feasibility.
An evaluation pipeline proves repeatability.
Observability proves you can see degradation.
Guardrails prove failures have boundaries.
Production usage proves the business.
That is the last mile.
And unfortunately there is no magic foundation model that removes it.
The Most Expensive Sentence in AI 💸
So the next time somebody says:
“It worked in the demo.”
My next question would be:
How often does it work when nobody controls the demo?
Give it messy data.
Long conversations.
Unclear requests.
Tool failures.
Malicious documents.
Retries.
Thousands of users.
Then measure it.
Because the goal isn't to build an AI system that can succeed.
The goal is to build one that can fail safely, recover and keep producing useful results.
That is a much higher bar.
And that is where AI Engineering actually begins.
A demo proves the vision.
Production proves the engineering. 🚀
Top comments (0)