AI Proof of Concept to Production: What Demos Skip

A good AI demo takes an afternoon. A model reads a document, produces a summary, and everyone in the room is impressed. Then the project stalls for months, because the demo worked on five hand-picked examples and nobody knows what it does on the five-thousandth real one.

The gap between proof of concept and production is not mainly about choosing a better model. It is ordinary engineering that demos skip: knowing whether the output is right, making it fit your software, controlling what it can do, knowing what it costs, and rolling it out so a bad day does not reach every customer. This is the checklist we work through when taking AI features to production.

Demo vs production at a glance

Concern	In the demo	In production
Is it right?	Looks good on a few examples	Scored against an evaluation set of real cases on every change
Output format	Free text, read by a person	Schema-constrained output, validated before it touches your data
What can it do?	Whatever the API key allows	Acts as a specific user, with that user's permissions
Repeated actions	Not considered	Idempotent writes — retries cannot double-send or double-charge
Failures	Refresh and try again	Timeouts, retries, rate-limit handling and a non-AI fallback
Cost	Unknown	Tokens and cost tracked per feature and per customer
Visibility	Console logs	Traces of inputs, outputs, tool calls and latency
Rollout	Everyone at once	Shadow mode, feature flag, gradual exposure

1. Build an evaluation set before you tune anything

The single biggest difference between teams that ship AI features and teams that stall is an evaluation set: a collection of real inputs, with what a correct output looks like, that you can score automatically. Without one, every prompt change is a guess and every model upgrade is a gamble.

• Use real data, not invented examples. Pull cases from production logs, support tickets or documents — including the messy ones.
• Keep a held-out set. Tune prompts on one split and report results on another, so you are not grading your own homework.
• Score what matters. For classification, accuracy per class. For extraction, field-level correctness. For generated text, a rubric of specific yes/no questions ("does the reply invent facts not in the source?") rather than a vague 1–10 score.
• Add every production failure to the set. The evaluation set should grow with the feature.

2. If an LLM grades the output, grade the grader

For open-ended output, a second model scoring the first against a rubric ("LLM-as-judge") is often the only scalable option. It is also easy to get wrong: an uncalibrated judge happily passes bad answers or fails good ones, and you end up optimising toward its quirks.

The method we use, including in our own testing: have people label a sample of outputs against the same rubric, run the judge on that sample, and compare. Track false passes and false fails separately — a judge that passes everything looks great on average. Iterate on the rubric and judge prompt until agreement with human labels is high, and only then trust the judge on the full set. We have also found that a model's self-reported confidence is a poor stand-in for this; measure against labels instead.

Rubrics beat scores

Five specific yes/no questions written from how a human actually grades the output are far easier to calibrate than a single "quality out of 10".

3. Make the output fit your software

If code consumes the model's output, "usually valid JSON" is a production incident waiting to happen. Major model APIs support schema-constrained (structured) output: you supply a schema and the response is generated to match it. Validate it anyway before it touches your database.

In an AI assessment and reporting app we built for a leadership consultancy, templates are generated as typed JSON from a free-text topic, and executive summaries follow a fixed section contract. We also gave the summariser labelled findings instead of raw scores, which removed a whole class of confident, numerically wrong statements. Giving the model less — only what it needs — often improves accuracy and lowers cost at the same time.

4. Control what it can do, and make actions safe to repeat

Prototypes typically run with an admin API key. Production AI that takes actions must act as a specific user with that user's permissions, so it can never do more than the person on whose behalf it runs.

• Route actions through your existing validated API, not direct database writes, so business rules are enforced in one place.
• Make writes idempotent. Network retries, model retries and duplicate tool calls happen. An invoice must not be sent twice because a request timed out.
• Require confirmation for irreversible actions, especially when the AI reads untrusted content (emails, documents) that could contain injected instructions.
• Log every action with the user, inputs and result.

These are the defaults behind the remote MCP server we built so a consultancy's staff could run operations from Claude: each tool proxies an existing authenticated endpoint and inherits the signed-in user's permissions.

5. Plan for the model being slow, wrong or unavailable

• Timeouts and retries with backoff, distinguishing per-minute rate limits from daily quotas and from real errors — each needs a different response.
• Surface retry timing to users instead of failing silently when a provider returns a quota error.
• A non-AI fallback. If summarisation fails, show the source; if classification fails, route to a person. The feature degrades instead of breaking.
• Background jobs for slow work. Do not hold a web request open for a 30-second model call; queue it and stream or notify.

6. Know what each request costs

Track tokens and cost per feature, per customer and per model from day one. It turns pricing decisions — included, metered, add-on — into arithmetic, and it catches runaway loops early. It also lets you safely move easy steps to a smaller model: if the evaluation score holds, the saving is real. Our AI agent cost breakdown walks through the token arithmetic at current prices.

7. Trace everything

When a customer reports a strange answer, support needs to see exactly what the model received, which tools it called, what came back and how long each step took. LLM-observability tools such as Langfuse, or traces exported through OpenTelemetry, give you that per-request history. Store prompts and outputs only where your data agreements allow, and redact what you do not need.

8. Roll out gradually

1. Shadow mode. The AI runs on real traffic but only proposes; people still act. Compare its proposals with what they did.
2. Feature flag to a small group. Real users, real feedback, easy rollback.
3. Supervised autonomy. The AI acts on the cases the evaluation set shows it handles reliably; everything else goes to a person.
4. Wider rollout, with the evaluation suite running on every prompt or model change and production failures feeding back into it.

When the right answer is to stop

Sometimes the evaluation set shows the idea does not work well enough on real data, and no amount of prompt tuning closes the gap. That is a successful outcome of a proof of concept, not a failure — it cost weeks instead of a launch. The honest options are then to narrow the scope, fix the underlying data, keep a person in the loop permanently, or drop the feature.

If you have a prototype that needs to become a dependable feature, that is the core of our AI integration services. For agents that take actions across systems, see AI agent development.

FAQ

Why do so many AI proofs of concept never reach production?

Usually because nobody can show the feature is reliable on real data. Without an evaluation set, teams cannot tell whether changes help, cannot set an accuracy bar, and cannot justify the risk of launching. Missing permissions, cost tracking and failure handling add to the delay.

How big should an evaluation set be?

Large enough to cover the real variety of inputs, including the awkward cases. Many features start with 50 to 200 examples and grow as production failures are added. A small honest set beats a large set of easy, invented examples.

Can we trust an LLM to grade AI output?

Only after checking it against human labels. Compare the judge with people on a sample, track false passes and false fails separately, and refine the rubric until they agree. Then re-check periodically.

Do we need to rebuild our prototype for production?

Not usually. Most of the work is adding what the prototype skipped — evaluation, structured output, permissions, failure handling, tracing and a rollout plan — around the core idea, inside your existing codebase.

Sources

• Gemini API — Structured output (Google)
• Langfuse documentation — tracing and evaluation
• OpenTelemetry documentation

---

Originally published on the DablewDevs blog. DablewDevs builds production AI systems and software — AI agents, RAG, MCP servers and SaaS products. See what we build and what it costs.