A prototype agent that works in a notebook is a long way from one you can leave running against real users and real systems. The gap is rarely the model. It is everything around it: evaluation, tool permissions, failure handling, observability and the boring operational work.
This is the checklist we use before an LLM agent goes to production. It is model-agnostic and framework-agnostic. Copy it into your repo as PRODUCTION_READINESS.md and tick it off honestly.
1. Scope and contract
- [ ] One-sentence job description. "The agent classifies inbound support emails and drafts a reply for human approval." If you can't write it, you can't test it.
- [ ] Explicit non-goals. List what the agent must not attempt, and make sure the system prompt and the tool list enforce them.
- [ ] Input and output schemas. Structured outputs (JSON schema, typed tool arguments) for anything another system consumes. Validate them, and reject or retry on failure.
- [ ] Success metric agreed with the owner. Pick one, such as task success rate on the eval set, escalation rate or handling time, with a baseline.
2. Evaluation
- [ ] A versioned eval set in the repo: real (anonymised) inputs, expected outputs or grading rubrics, and edge cases. Start with 50–100 cases and grow it from production failures.
- [ ] Automated scoring in CI. Use exact match or schema checks where you can, and rubric-based LLM grading where you can't, spot-checked by a human.
- [ ] Regression gate. A prompt, model or tool change that drops the score below the threshold fails the build.
- [ ] Adversarial cases. Prompt injection in user input and in retrieved documents, off-topic requests, empty and huge inputs, and other languages.
- [ ] A pinned model version. Don't point production at a floating "latest" alias. Upgrade on purpose and run evals first.
# .github/workflows/agent-evals.yml (sketch)
on: [pull_request]
jobs:
evals:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: pip install -r requirements.txt
- run: python evals/run.py --set evals/cases.jsonl --min-score 0.90
3. Tools and permissions
- [ ] Least privilege per tool. Read-only credentials unless the tool must write. Give the agent its own scoped service account, never a developer's token.
- [ ] Human approval for side effects. Anything that sends a message, changes a record, moves money or deletes data waits for an approval step until the eval data justifies removing it.
- [ ] Idempotent tool calls. Retries happen. Use idempotency keys so a retried "create refund" doesn't run twice.
- [ ] Argument validation server-side. Treat tool arguments from the model as untrusted user input.
- [ ] Allow-lists over block-lists for URLs, file paths, SQL tables and email domains the agent can touch.
- [ ] Step and loop limits. Cap the number of tool calls per task and stop on repeated identical calls.
4. Failure handling
- [ ] Timeouts on every model and tool call, with sensible retries and backoff.
- [ ] A fallback path. When confidence is low, validation fails or limits are hit, hand off to a person with the context attached. Don't fail silently.
- [ ] Provider outage plan. Decide whether to queue, degrade to a simpler flow, or switch to a secondary model, and test it.
- [ ] Graceful user messaging. The user should know when a human is taking over.
5. Observability
- [ ] Trace every run. Inputs, the prompt version, model and version, every tool call with arguments and results, the final output, latency and token usage, all tied to one trace ID.
- [ ] PII handling in logs. Redact or hash sensitive fields before they reach your log store, and set retention.
- [ ] Dashboards for success rate, escalation rate, latency (p50/p95), error rate by tool, and cost per task.
- [ ] Alerts on spikes in errors, escalations, latency or spend.
- [ ] A feedback loop. Human corrections and approvals are captured and turned into new eval cases.
6. Cost and rate limits
- [ ] Per-task and per-day budgets enforced in code, not just watched on a billing page.
- [ ] Rate limiting per user and per tenant to stop runaway loops and abuse.
- [ ] Caching for repeated retrieval and deterministic sub-steps where it is safe.
7. Security review
- [ ] Prompt injection mitigations. Separate system instructions from retrieved content, don't let retrieved text grant permissions, and re-check authorisation on every tool call.
- [ ] Secrets in a secret manager, never in prompts, code or logs.
- [ ] Data residency and processor list documented for your security and legal teams.
- [ ] Threat model written down: what could a malicious user or poisoned document make the agent do, and what stops it? The OWASP Top 10 for LLM Applications is a good starting list.
8. Deployment and rollback
- [ ] Prompts and configs versioned alongside the code, deployed through the same pipeline.
- [ ] Staged rollout. Run in shadow mode first (the agent drafts, humans act), then send a small share of traffic, then more.
- [ ] Feature flag or kill switch that turns the agent off and routes everything to the human path in one step.
- [ ] Infrastructure as code for queues, workers, vector stores and secrets, so environments are reproducible.
If your platform basics such as CI/CD, environments and monitoring aren't in place yet, sort those first. An agent amplifies weak operations. Our notes on DevOps and SRE services cover the foundations we look for.
9. Handover
- [ ] Runbook: how to deploy, roll back, rotate keys, update the eval set and respond to each alert.
- [ ] Ownership: a named team on call for the agent, with access to traces and dashboards.
- [ ] Everything in the client's or team's repo: code, prompts, eval sets, IaC and docs. No knowledge stuck in one person's head.
We use a broader version of this in our software project handover checklist, which applies to any system, not just agents.
Using the checklist
You won't tick every box on day one, and that is fine for a pilot, as long as the gaps are written down and the risky ones (permissions, approvals, kill switch) are closed. The checklist works best as a pull-request template: each change to prompts, tools or models has to say which items it affects.
If you are scoping a first agent and want the evaluation and guardrails designed in from the start, our AI agent development team works this way by default.
What would you add? I'd like to hear which checks have saved you in production.
Kapil Singhal, Director, Axentis Technologies
About Axentis Technologies: https://axentis.tech | Axentis on LinkedIn: https://www.linkedin.com/company/axentis-technologies
Aranyix: https://aranyix.tech | Aranyix on LinkedIn: https://www.linkedin.com/showcase/aranyix/
Top comments (0)