The hard part of building AI agents isn't the model. It's letting them act

George Zaletski — Mon, 20 Jul 2026 14:45:06 +0000

Getting an LLM to write a paragraph is easy — you can do it in an afternoon. Getting one to actually do things, like create a record, book a meeting, or send an email on someone's behalf, is where it gets hard. And the surprising part is how little of that difficulty is about the model.

I build Referent, practice-management software for law firms. The idea is that agents do the busywork and a person signs off on anything that leaves the office. Law is a brutal place to test this. A wrong action can be expensive, the people using it aren't developers, and "close enough" doesn't fly.

A few things bit us on the way from an AI that answers questions to one that takes actions. None of them are specific to law, so I figured they were worth writing down.

Context is the bottleneck, not the model

You'd think a smarter model gets you a better agent. Mostly it didn't. What held us back was getting a correct, current picture of the situation in front of the agent in the first place.

Take one client. Their information is spread across an inbox, a calendar, some documents, and a couple of lines someone typed into a notes field months ago. No single system has the full story, and the systems disagree with each other — the email says Tuesday, the calendar says Wednesday. Somebody has to pull all of that together and decide which version is true before the agent can safely act on any of it. That somebody is your code.

Our users said the same thing when we asked them. Their biggest complaint wasn't that the AI wasn't clever enough. It was two ordinary problems: their information is scattered across disconnected tools, and the AI they'd tried forgets everything from one session to the next. So we spend far more time fetching and reconciling data than we do on prompts. The prompt is almost the easy part.

Nothing writes without a human

Our one hard rule is that the agent prepares and a person approves. In code, that means the agent can't touch the database at all.

It reads whatever it needs, but it can't create a matter or send an email on its own. It produces a proposal instead — a typed record that says what it wants to do, why, and what it based that on — and drops it in a queue. Someone looks at it and confirms, edits, or bins it. Only then does anything happen for real.

The two best things about this we didn't really plan for. A wrong guess from the model costs one click to reject instead of turning into an incident. And you get an audit trail for free — every change points back to a proposal, its sources, and whoever approved it. In a regulated field, that trail is the thing that lets you ship at all.

Assume the agent is flaky

Model calls are nondeterministic, and everything retries eventually. Those two facts together will quietly wreck your data if you let them.

Let the same "create matter" step run twice and you've got duplicate matters. Make sending an email a direct side effect, and a retry sends it twice — to a client. So writes get idempotency keys, commits are transactional, and nothing the user can see happens outside that approval step from before. The agent is allowed to be unreliable. It just isn't allowed to have side effects.

What clicked for us was to stop treating the agent as a function we call, and start treating it like a flaky client hitting our API over a bad connection. You already know how to handle that one. Validate everything, make writes idempotent, and assume any given message shows up zero times, once, or five times.

The review screen is the whole product

Human-in-the-loop only works if checking the agent's work is faster than doing it yourself. Miss that and it breaks in one of two ways. If reviewing is a slog, people start approving things without reading them, and your safety story is fiction. If it's a real slog, they drop the tool and go back to doing it by hand, and you've automated nothing.

So a proposal has to carry its evidence with it. When the agent wants to open a matter, the reviewer sees the emails, the document, and the reasoning right there — enough to say yes or no in a couple of seconds without going digging. We spend a lot of design time on that one screen. It matters more than the chat box everyone puts in their demos.

A bonus we didn't see coming: every approve, edit, and reject is a person labeling the agent's output. Give it a few weeks and you've got a real evaluation set, which beats the synthetic benchmarks we started with.

So

If you're building agents that actually do things, the interesting work isn't picking a better model. It's the unglamorous stuff around it: getting the state right, putting a human in front of every write, making actions safe to retry, and building a review screen fast enough that people stay in the loop because they want to. That's most of the job.

We're doing this for law at Referent right now. I'd like to know how other people are drawing the propose–review–commit line, so if you've built something in this shape, tell me in the comments.

DEV Community: George Zaletski

The hard part of building AI agents isn't the model. It's letting them act

Context is the bottleneck, not the model

Nothing writes without a human

Assume the agent is flaky

The review screen is the whole product

So