Every team adding AI to its automations hits the same fork, usually without noticing: for this step, do we write code or call a model?
The default lately has been "call a model," because it's fast to prototype and the demo works on the first try. Then production arrives with its edge cases, its cost per call, its latency, and the incident where the model returned something plausible and wrong. Most of those incidents trace back to a step that never needed a model in the first place.
What the people building agents say
The two most cited guides on the subject agree on the starting point.
Anthropic's "Building effective agents" recommends finding the simplest solution possible and only increasing complexity when needed, which might mean not building agentic systems at all. When more is warranted, it separates workflows, where LLMs and tools follow predefined code paths, from agents, where the model directs its own process. Workflows give predictability for well-defined tasks; agents fit when flexibility and model-driven decisions are needed at scale.
OpenAI's "A practical guide to building agents" gives three signals that a task deserves an agent: complex decision-making (nuanced judgment, exceptions, context-sensitive decisions), difficult-to-maintain rules (rulesets so intricate that updates are costly or error-prone), and heavy reliance on unstructured data (interpreting natural language, extracting from documents, conversation). The guide's own conclusion for the other cases: a deterministic solution may suffice.
So the question isn't "AI or not." It's which steps need judgment over messy input, and which are just rules.
Five questions, step by step
Take one step of your automation (not the whole flow) and ask:
1. Can I write the rule down?
If you can express the decision as conditions on known fields ("if the invoice total is over the approved limit, route to finance"), write code. A model adds cost and variance to something that has a correct answer.
2. Is the input structured?
Fields from a form, a database row, a JSON payload: code. Free text, PDFs, emails, call transcripts: this is where a model earns its place, usually to extract structure, not to decide.
3. Does the same input have to give the same output, every time?
Billing, permissions, compliance rules, anything that will be audited: deterministic. If you can't explain a decision to an auditor by pointing at a line of code, it doesn't belong to a model.
4. What does a wrong answer cost, and who notices?
A slightly off summary in an internal digest is cheap and visible. A wrong refund, a wrong permission or a wrong email to a customer is expensive and may go unnoticed. The higher the cost, the more the model should propose and code (or a person) should decide.
5. How often do the rules change, and who changes them?
This is the honest case for a model: rules that change weekly, written by people who don't write code, with exceptions nobody fully remembers. OpenAI's "difficult-to-maintain rules" signal. Even then, keep the parts that don't change in code.
The pattern that works: model in the middle, code at the edges
In practice, the answer is rarely all code or all model. The reliable shape is a sandwich:
input ──▶ [code] validate & normalize
──▶ [model] interpret the unstructured part, return structured output
──▶ [code] validate the output against a schema and business rules
──▶ [code] decide and act (or route to a person)
A concrete example: incoming supplier emails with invoices attached.
| Step | Who does it | Why |
|---|---|---|
| Check sender domain against the supplier list | Code | It's a lookup |
| Extract supplier, number, date, total from the PDF | Model | Unstructured, varied layouts |
| Check the extraction: required fields, valid date, total is a number | Code (schema) | Catches malformed output |
| Compare total with the purchase order | Code | A rule with a correct answer |
| Approve if it matches within tolerance; otherwise route to a person | Code + person | Money moves; auditable |
| Draft a reply asking for a missing PO number | Model | Natural language |
The model does two of the six steps. Everything that moves money, grants access or must be explained later stays deterministic.
A sketch of the middle step, with the validation that keeps the model honest:
const InvoiceSchema = {
required: ['supplier', 'number', 'date', 'total'],
check: (x) =>
typeof x.total === 'number' && x.total > 0 &&
!Number.isNaN(Date.parse(x.date)),
};
async function extractInvoice(pdfText, callModel) {
const raw = await callModel({ task: 'extract_invoice', input: pdfText, format: 'json' });
let data;
try { data = JSON.parse(raw); } catch { return { status: 'needs_review', reason: 'invalid JSON' }; }
const missing = InvoiceSchema.required.filter((k) => data[k] == null);
if (missing.length) return { status: 'needs_review', reason: `missing ${missing.join(', ')}` };
if (!InvoiceSchema.check(data)) return { status: 'needs_review', reason: 'failed validation' };
return { status: 'ok', data }; // the decision happens later, in plain code
}
Notice what the function does not do: it doesn't approve anything. It turns a messy document into data or sends it to a person. The approval rule lives somewhere you can read, test and audit.
Signs you used a model where code would do
- The prompt contains a long list of if/else instructions in prose.
- You added "always return exactly…" three times and it still sometimes doesn't.
- The same input gives different results and nobody can say which is right.
- The step's cost per run is noticeable, and the input is a table.
- You're writing tests that accept "close enough" for a decision that has a right answer.
Signs you used code where a model would help
- A regex for parsing emails that grows every week.
- A rules table maintained by the business team in a spreadsheet, translated to code by hand, always out of date.
- A queue of "unrecognized format" items that a person processes by reading them.
Summary
| Use code when | Use a model when |
|---|---|
| The rule can be written down | Judgment over messy context is required |
| Input is structured | Input is free text, documents or conversation |
| Results must be reproducible and auditable | Some variance is acceptable and visible |
| A wrong answer is expensive or silent | A wrong answer is cheap and easy to spot |
| Rules are stable | Rules change often and live outside the code |
Start with code. Add a model to the specific step that needs judgment over unstructured input. Then surround that step with deterministic validation, and keep the decisions that matter where you can read them.
Which step in your current automation is using a model for a rule you could write in ten lines?
References
- Anthropic — Building effective agents: https://www.anthropic.com/engineering/building-effective-agents
- OpenAI — A practical guide to building agents (PDF): https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf
Top comments (1)
"Model in the middle, code at the edges" is a good rule. For a small task app, one split that makes sense: plain code for dates, reminders and storage, and the model only for the fuzzy part - reading a messy message and deciding whether it contains a request. Another thing that helps: the model drafts a result for a person to confirm and never acts on its own. How do you handle the cases where the model is unsure? A confidence threshold, or always ask a human?