Most people can get a Vapi voice agent talking in an afternoon. The quickstart is good, the docs are good, and a demo call will sound impressive enough to green-light a project.
Then it meets real calls.
I build production voice agents, and I spend a lot of my time fixing ones that were built by someone else and fell over. The pattern is consistent: the agent works exactly as designed, and the design never accounted for how it would fail. Nothing in the build was wrong. Things were just left undecided, and production decided them instead.
So if you are about to hire someone to build one, here are the five decisions that determine whether it survives. Ask about them before the build starts. The answers cost nothing at the design stage and a great deal afterwards.
1. What happens at the transfer boundary
The single most confusing production failure I get called about: the agent dials out, navigates an IVR menu correctly, gets transferred to a live person — and goes silent. You can hear the agent. The agent cannot hear the human. Pull the telephony provider's own recording and the audio is perfectly clear.
Almost everyone starts tuning the prompt. The prompt is fine.
What is actually happening is that the audio transport reconnects at the moment the call bridges to a live person, and inbound audio starts arriving in fragments the transcriber cannot assemble. It is a transport-level fault at the hand-off, not an intelligence failure. No amount of prompt engineering touches it.
The way to confirm it is to rebuild the same transfer flow on a rig you control and run it over two different transport paths. Whichever one breaks tells you where the fix lives.
Ask your builder: what happens to the audio session at hand-off, and how will we test that before launch rather than after?
2. Voicemail structurally resembles a conversation
An agent I was asked to look at was pushing a large minority of its "qualified leads" into the client's CRM as callbacks. They were answering machines.
The client's account of it was blunt: the agent was marking voicemail greetings as callbacks, and the sales team was ringing back people who had never actually spoken to it.
That is worse than a useless agent. Every false positive consumed a human's dial. The system built to relieve the sales team was manufacturing work for it.
The instinct is to tighten the classification prompt. That fights the symptom, because the mechanism is structural: an answering machine produces a transcript that looks like a conversation. Greeting, pause, speech. A classifier reading that transcript has no strong signal separating "a person said hello" from "a recording said hello." You are asking it to distinguish two things that genuinely resemble each other in the only data it has.
The fix is not a smarter first pass. It is a second pass: take the transcript plus the disposition the first agent assigned, and ask a separate model whether that disposition was correct. Post-hoc adjudication rather than better prediction.
Ask your builder: how does the system verify its own dispositions, and who reads transcripts?
That second half matters more than it sounds. On that project, nobody on the build side was reading transcripts at all. Every error was found by the client and reported upward. It was a process failure wearing a model failure's clothes, and the client had become the QA layer by default — knew it, and resented it.
3. Your verifier will be literal-minded, and it will bin good leads
Here is the trap in the fix I just described.
Add a verification pass and it will start throwing away real leads. Not because it is strict — because it is literal. An LLM asked "is this a qualified lead?" defaults to demanding an unambiguous yes. Any complicating clause reads as not-a-clear-yes.
The call that made this obvious: a prospect answered yes to every question, mentioned in passing that she was away that week, and got binned. A clear, workable lead, thrown out.
"Yes, but I'm on holiday" is a perfectly workable lead. To a literal-minded adjudicator it is a no.
And here is why this one is genuinely dangerous. False positives are loud — someone dials a dead number and complains within a day. False negatives are silent. A slightly lower lead count is indistinguishable from a slightly worse data batch. Nothing surfaces it. The only way that failure was ever found was a single named call, flagged by a human who noticed a recurring shape.
Ask your builder: how will we sample the leads the system discards?
If there is no answer to that, the system has a blind spot exactly where it hurts and no mechanism will ever report it.
4. Every rate limit in your stack is invisible until you scale
A client tripled their agent fleet and got roughly the output they had before. What caught their attention was the bill: three times the agents, and the cost had barely moved.
The tell was the flat cost. Spend barely moved despite the extra agents.
Cost is a proxy for work actually performed. Multiply capacity, watch the bill stay flat, and you know no additional work happened — which relocates the entire investigation from "quality" to "throughput" in a single step. Stop debugging the prompt.
The causes were stacked, and none of them were in the voice platform:
The model provider key was cold. Agents had been built against a warm, high-tier key. The client's own key was swapped in fresh, and a new key sits at the bottom rate-limit tier — it climbs only with elapsed time and accumulated spend. You cannot buy your way up. Worse, a key handed over at project start had never been wired in, so it accrued no warm-up during the entire period it could have been warming.
The record store had a per-minute write ceiling. At a few agents on a short trigger interval, nothing collided. Multiply the agents and triggers landed inside the same minute, writes errored, and call records silently failed to log. The mitigation — longer intervals with jitter — throttled the whole fleet to fix it.
At low agent counts every rate limit in your stack is invisible. Scaling does not reveal them gradually. It hits several at once, in layers nobody was watching, and the platform you are about to blame usually is not one of them.
Ask your builder: what is the per-minute ceiling on each layer — model provider, telephony, orchestrator, and the record store at the end?
That last one is the boring component nobody inventories, and it is the one I most often find holding the whole system down.
5. Whether anything can go out without a human seeing it first
The last decision is the one that sounds like overhead and is not.
The strongest builds I've seen put every outbound action — every call, every message — behind a dry-run gate by default. Nothing reaches a real person until a human has explicitly released it. That stays true after go-live, not just during testing.
Voice agents fail in a specific direction: they act. A broken dashboard shows you a wrong number. A broken voice agent phones somebody. If it is wrong, it is wrong at scale, out loud, to your customers, before you find out.
Ask your builder: what can this system send without a human approving it, and can that gate stay on permanently?
What I actually check before writing a prompt
Every item above is a design decision, not a bug. Each one gets made whether or not anybody makes it deliberately.
So before I write a line of prompt, I want to know: what the audio session does at hand-off, how dispositions get verified after the fact, how we sample what the system throws away, where every per-minute ceiling in the stack sits, and what can go out without a human in the loop.
None of that is Vapi-specific. It is what makes the difference between an agent that demos well and one that is still running in three months.
I am Ussama Assad. I build production voice agents, and I fix them when they break — latency, dropped calls, failing tool calls, transport errors at transfer, agents losing conversation state. I cannot promise you a booking rate; that depends on your offer, your list, and your market. What I will tell you is exactly what your system is doing and why, and I stand behind the build when it breaks.
If you are scoping a voice agent and want a second opinion on the design before it is built, that is a conversation worth having.
I'm Ussama Assad — I build and debug production outreach AI: voice agents, cold-email systems, lead-gen pipelines. Everything I write here is a real failure I traced to its cause and fixed. https://ussama.dev
Top comments (0)