AI is not a genie. Treat it like a function.
Most people use AI the way they use a search box: type a question, read the answer, move on. That works for one-off curiosity. It is a bad fit for the work you repeat every week, because you re-explain the context every time and never build anything you can trust.
A small assistant is different. It is one narrow task, wired up once, with a fixed input and a fixed output shape. You run it, check it, improve it. After a few iterations it stops being a demo and starts pulling real weight.
Here is how to build one without drowning in frameworks.
Start narrow: one task, one input, one output
Do not build "an assistant for my job." Build the thing that turns a messy meeting note into three bullet points. Pick a task that is:
- Repetitive (you do it weekly or daily)
- Boring (nobody will miss the manual version)
- Verifiable (you can look at the output and know if it is wrong)
That last one matters most. If you cannot tell good output from bad in ten seconds, you cannot trust the assistant and you cannot improve it.
Good starter tasks: drafting reply emails, summarizing documents, normalizing scrappy data, extracting fields from text.
Example: an email draft as a function
Think of your prompt as a function signature. Inputs go in, a structured draft comes out.
def draft_reply(incoming_email: str, tone: str = "friendly, brief") -> str:
prompt = f"""
You are drafting a reply on my behalf. Do not invent facts.
If information is missing, leave a [PLACEHOLDER].
Tone: {tone}
Incoming email:
---
{incoming_email}
---
Write only the reply body.
"""
return llm(prompt) # any model client you like
Two lines do the real work: "Do not invent facts" and the [PLACEHOLDER] rule. Together they turn a confident hallucination into a visible gap you can fill. The goal is to make errors loud instead of silent.
Example: summaries you can actually trust
The failure mode of summaries is a plausible sentence that never appeared in the source. Force evidence and it disappears.
Summarize the document below in 5 bullets.
For each bullet, quote the exact sentence it is based on.
If a claim has no supporting quote, drop it.
Now checking takes seconds: skim the quotes, confirm they exist in the text. You are not leaning on the model's judgment, only on its ability to copy, which is far more reliable.
Example: data cleanup, with a safety net
Cleaning inconsistent data (country names, job titles, date formats) is a great fit, and also where silent errors hide. Never let the model rewrite your data in place. Have it output a mapping you review first.
Input values: ["USA", "u.s.a", "United States", "Amrica"]
Return JSON: {original: normalized}.
Do not merge values you are unsure about; mark them "REVIEW".
You keep the original column, apply the mapping in code, and eyeball anything marked REVIEW. The model proposes, your code decides.
Verify outputs (the step everyone skips)
Before you rely on an assistant, build a tiny golden set: ten to twenty real inputs paired with the output you actually wanted. Re-run them every time you change the prompt. This is the difference between "it feels better" and "it got better."
Cheap checks that catch most problems:
- Assertions in code: valid JSON? required fields present? no placeholders left behind?
- A spot-check habit: read one in five outputs, even after it "works"
- A refusal path: when the model is unsure, it should say so, not guess
Add an agent only when there are real steps
An agent is just an assistant that takes more than one step and calls a tool or two. Reach for it when a task genuinely has stages: fetch, then decide, then act. A triage helper that reads an inbox, classifies each message, and drafts replies for the easy ones is a reasonable first agent.
Keep the non-AI parts deterministic. Let the model classify and draft; let ordinary code do the fetching, sending, and looping. Every step you hand to the model is a step that can drift.
Where this is weaker
Be honest about the limits.
- Tasks with no cheap verification (strategy, judgment calls, anything with legal or financial stakes) are a bad fit. If checking the output takes as long as doing the work yourself, you saved nothing.
- Outputs drift once real inputs get weirder than your test set. The golden set slows this down; it does not stop it.
- Volume amplifies mistakes. A small error rate is fine for ten drafts you read by hand and a disaster for a thousand you send unread.
None of this kills the approach. It just means verification is not optional garnish. It is the load-bearing wall.
Iterate: promote good runs into the spec
When a run comes out great, do not just enjoy it. Copy what worked back into the prompt as a rule or an example. Over a few weeks the prompt turns into a small spec of how the task should be done, and the assistant gets boring in the best way: predictable.
Pick one task this week. Ship the narrow version, verify it, and let it earn the next feature.
I write about turning AI from a chat toy into a working tool. I help build AGINE Academy, a game-based academy for learning Claude by real practice. It is an independent product and is not affiliated with Anthropic.
Top comments (0)