DEV Community: Ussama Assad

If You're Hiring Someone to Build a Voice Agent, Ask How the System Checks Its Own Output

Ussama Assad — Tue, 01 Sep 2026 05:35:58 +0000

Most production voice agents do not fail by falling over. They fail by being confidently wrong in a way that looks exactly like being right — and then reporting success.

That is a different problem from a crash, and it needs a different answer. A crash tells you about itself. A wrong disposition does not.

I write this up because it is the part of a build that almost never appears in a scope of work, and it is the part that decides whether the system can be trusted six weeks after launch.

The failure that makes verification necessary

Take an agent that qualifies outbound calls and pushes interested leads into a CRM.

An answering machine produces a transcript that structurally resembles a conversation. Greeting, pause, speech. A classifier reading that transcript has no strong signal separating a person said hello from a recording said hello — they look the same in the only data it has.

So a share of "qualified leads" are voicemail greetings. Every one of them burns a human's dial on somebody who never spoke to the agent. The system built to relieve a sales team starts manufacturing work for it.

The instinct is to tighten the classification prompt. That fights the symptom. You are asking a model to distinguish two inputs that genuinely resemble each other, and no amount of instruction creates a signal that is not in the transcript.

The better move is structural: stop trying to make the first pass smarter, and check its output afterwards. Take the transcript plus the disposition the first pass assigned, and have a second pass ask whether that disposition was correct. Post-hoc adjudication rather than better prediction.

The trap in that fix

I published that recommendation, and a reader called Alexander Rådahl Benz pushed back on exactly that section. His objection, and he is right about the risk:

"One thing I'd change in the verifier section: don't trust a second LLM just because it's separate. If both see the same transcript and rubric, they can share the same blind spot."

This is the part worth slowing down on, because "add a verification step" sounds like diligence and can easily be theatre.

A second model instance is not a second opinion. If the second pass reads the same transcript, against the same rubric, asking the same question, it is not independent in any sense that matters — it is the same judgement executed twice. When it agrees, you have learned nothing. You have bought latency and a feeling of rigour.

Independence is a design choice, not a property of the model

Here is where I would put the fix somewhere different from don't trust it.

Nobody said the second pass has to see the same evidence and ask the same question. The correlation between two passes is something you engineer. There are three levers:

Change the question. The first pass asks is this qualified? Have the second ask what would disqualify this? Same model, opposite burden of proof. Make it argue the disposition is wrong, and only let the record through if that argument fails.

Change the evidence. The first pass reads the transcript. Give the second the audio, or the call metadata — duration, answer latency, DTMF events, the shape of the turn-taking. A voicemail and a live answer look alike in a transcript and quite different in call metadata. Two passes over the same evidence share its blind spots by construction; two passes over different evidence do not.

Change the threshold. An adjudicator that has to be convinced is not the same instrument as one that has to be persuaded otherwise.

Same model. Different burden of proof. That is what independence is made of.

Then make the independence measurable

Alexander Rådahl Benz's follow-up is the part I would keep above everything else here, because it converts a design claim into something you can check:

"I'd make the independence measurable, though: track how often the verifier flips the first disposition, broken down by failure class and evidence source. If it never disagrees, the burden of proof is different only on paper; if it disagrees randomly, the second path is noise. The design earns its keep when it catches a repeatable class the first pass misses."

That is the test. A verifier that never overturns anything is decorative. A verifier that overturns things at random is noise wearing a lab coat. His last sentence there is the whole test: the design earns its keep when it catches a repeatable class the first pass misses — voicemails, say, or a specific kind of ambiguous yes.

So the number to instrument is not accuracy. It is the flip rate, split by failure class and by which evidence source produced the flip. That tells you whether the second path is doing independent work or rubber-stamping.

Add a small human-labelled set, re-scored after every prompt or model change, and you have a verifier you can actually reason about.

The same failure, well outside voice

I hit this exact shape in my own tooling, and it is worth including because it generalises past AI entirely.

I had written a checker whose job was to confirm that a set of published articles carried a real link back to my site. It reported everything healthy. The articles carried the site as plain text — not a link at all, no signal passed. The checker counted a bare-text mention as a link, which is to say it accepted a weaker form of the thing it was built to verify, and so it certified the precise failure it existed to catch.

That is the same defect as the correlated verifier. Not "the check was wrong" — the check was structurally incapable of failing on that input.

The rule I took from it: if a check has never once failed, suspect that it cannot. Feed it a known-bad case deliberately and confirm it screams. A check you have never seen fail is a check you have never tested.

Two more disciplines that belong in the same conversation

Fail toward doubt. When a check cannot read the thing it is checking, it must return unknown, never fine. A health indicator that defaults to reassurance lets a broken system run for weeks while everyone reads the green light. Absence of a confirmed failure is not the same as a confirmed pass, and only one of those is safe to build on.

Sample what the system discards. The errors are asymmetric. False positives are loud — a rep dials a dead number and complains that day. False negatives are silent: a verifier quietly binning good leads looks exactly like a slightly quieter week. Nothing surfaces it, ever, unless somebody deliberately reads the discarded pile.

That failure is easy to produce. Ask a model is this a qualified lead? and it will tend to demand an unambiguous yes; anything with a complicating clause reads as not-a-clear-yes. A lead who answers yes to every question and mentions in passing that they are away until next week is a perfectly workable lead — and to a literal-minded adjudicator, a no.

What to ask before you hire someone to build this

Five questions. The answers cost nothing at design time and a great deal afterwards.

How does the system verify its own dispositions? If the answer is the prompt is good, there is no verification layer.
What does the second pass see that the first one didn't? If the answer is the same transcript, the two passes will share their blind spots.
How will we measure that the verifier is independent? Flip rate, broken down by failure class and evidence source — not overall accuracy.
How do we sample the records the system throws away? No answer here means the expensive failure is structurally invisible.
What does the system report when a check cannot run? If the answer is anything other than unknown, the monitoring will eventually lie to you.

None of these are exotic. They are the difference between a voice agent that works in a demo and one that is still trustworthy when nobody is watching it closely.

If you are evaluating a builder and they have thought about the discarded pile before you asked, that tells you most of what you need to know.

Questions I get asked about this

Does a verification pass mean the agent needs a second model licence?
No. Independence comes from the question, the evidence and the burden of proof — not from a different vendor. Ussama Assad builds these with the same model on both passes and varies what each one sees.

Is this worth it on a small deployment?
The flip-rate instrumentation scales down fine; the thing that does not scale down is skipping it. A small deployment producing silent false negatives is just a small deployment nobody can audit.

Credit where it is owed: the independence-is-measurable framing above came out of a public exchange with Alexander Rådahl Benz in the comments of an earlier piece of mine — it sharpened the argument, and it belongs to him.

I'm Ussama Assad — I build and debug production outreach AI: voice agents, cold-email systems, lead-gen pipelines. Everything I write here is a real failure I traced to its cause and fixed. https://ussama.dev

What Actually Breaks When You Put an AI Voice Agent on Real Estate Lead Follow-Up

Ussama Assad — Tue, 25 Aug 2026 01:11:48 +0000

A property lead has a short half-life. Someone fills in a form about a valuation or a rental at 9pm, and by the morning they have filled in three more on three other sites. So the pitch for an AI voice agent doing first-touch follow-up is obvious: call everyone, call them fast, call them at 9pm.

That part usually works. It is what happens after the call that decides whether the system is worth having.

I am Ussama Assad. I build and debug production voice-agent systems — the kind that sit between a lead form and a CRM and are supposed to save a team from dialling. Below are the two failures I see land hardest on lead follow-up specifically, in the order they tend to arrive. Neither is a model problem. Both are design problems that look like model problems, which is why they survive so long.

Failure one: the agent books answering machines as interested leads

The first version usually looks like a success. Calls go out, dispositions come back, a qualified-lead table fills up and syncs to the CRM. Then someone on the sales side says the quiet part: the agent is marking voicemails as callbacks, and the team is ringing back people who never actually spoke to it.

Say a third of the qualified pipeline turns out to be answerphones. That is not a reporting annoyance — every false positive spends a human dial. The team ends up calling dead numbers generated by the thing that was supposed to stop them calling dead numbers.

The instinct is to tighten the prompt. Add stricter classification criteria, tell it harder not to count voicemail. That instinct misreads the mechanism.

A voicemail greeting produces a transcript that structurally resembles a conversation. There is a greeting. There is a pause. There is speech. A classifier reading that transcript has no strong signal separating "a person said hello" from "a recording said hello." You are not asking the model to be more careful. You are asking it to distinguish two things that look the same in the only data it can see.

The second instinct — route calls through a dialler that only connects on human pickup — is the intuitive fix that costs two weeks. It frequently dies on integration incompatibility with the voice platform, and you find that out at the end.

What actually works is post-hoc adjudication. Take the finished transcript plus the disposition the first agent assigned, and ask a separate model one narrow question: is that disposition correct? You are not making the first pass smarter. You are checking its work with a second pass that has an easier job, because judging a finished transcript is a much simpler task than deciding mid-call.

One triage rule worth borrowing: calls under about thirty seconds are not worth reviewing. They are hangups and hard nos. Almost all the interesting misclassification lives above that line. That single cutoff removes most of the review volume at close to zero coverage cost.

The finding underneath this one

The false positives were not caught by monitoring. They were caught by a person reading transcripts.

And when I traced it back, nobody on the build side was reading transcripts at all. Every error had been found by the client and reported upward. That is a process failure wearing a model failure's clothes — the client had quietly become the QA layer, knew it, and resented it.

If you are hiring someone to build this, that is a fair thing to ask them directly: who reads the transcripts, how often, and what do they filter to? A good answer sounds like a routine. Filter to records where a callback was scheduled — the only outcomes that cost you if they are wrong. Read the transcript rather than the audio, which is fast enough to judge in seconds. Pull the audio only when the transcript reads off: an incoherent sequence, answers that do not fit the questions, the same name recurring.

Failure two: the fix for failure one starts binning good leads

This is the part people do not warn you about, and it is worse, because it is silent.

Once the verification pass is live, the complaint changes shape: now it is throwing away good ones. A lead answers yes to everything, mentions in passing that she is away that week, and gets binned.

An LLM asked "is this a qualified lead?" defaults to demanding an unambiguous yes. Any complicating clause — yes but I'm on holiday, call me next week, I don't have the paperwork to hand — reads as not-a-clear-yes and gets dropped.

The framing that unlocks it: it is not being strict, it is being literal-minded. "Yes, but I'm away until Tuesday" is a perfectly workable lead in property. It is arguably a better lead than an unqualified yes, because the person told you something true about their timeline.

A related failure from the same class of system: the agent was demanding documentation the business did not actually require, and disqualifying people who could not produce it on the phone. Nobody had told it that not needing that paperwork was the whole point. The agent was busy enforcing a policy the business did not have.

Why this one is hard to see

False positives create visible downstream work. Someone dials a dead number, gets an answerphone, and complains. The feedback arrives on its own.

False negatives are silent. A slightly lower lead count is indistinguishable from a slightly worse data batch. No aggregate signal catches it. It is only findable at the level of a single named call, surfaced by someone who noticed a shape repeating.

Which points at the actual operational fix: get the people using the system to flag recurring patterns rather than one-off bug reports. A repeated shape becomes a defined edge case you can encode. A one-off stays an anecdote and gets argued about.

The scheduling mistake that hides both

One more, because it makes the two above unattributable.

Do not ship a quality filter and a capacity increase in the same window.

If the verification pass and a fleet scale-up go live the same week, the two changes move the same metric in opposite directions. Output stays flat. Now you cannot tell whether the filter is too aggressive, whether the scale-up is being throttled somewhere, or both — and in my experience it is usually both, tangled, with neither cleanly attributable while both are in flight.

Ship one. Let it settle. Read the output. Then ship the other.

What to ask before you hire anyone for this

Not a checklist of features. Four questions about mechanism:

How do you tell a voicemail from a person? If the answer is "better prompting," they are fighting the symptom. If it involves checking the disposition after the call, they have hit this before.
Who reads the transcripts, and what do they filter to? If nobody does, you are the QA layer.
How would you know if the system started discarding good leads? If there is no answer, there is no detection — and this failure does not announce itself.
What is your policy on shipping two changes at once? The right answer is a flat no.

I do not sell an outcome here. Lead quality, list quality, market timing and what your team does with a booked callback are not things a builder controls. What I will stand behind is the system: that it is built correctly, that it does what it is engineered to do, and that when it breaks I can find the actual mechanism rather than guess at the prompt.

FAQ

Who builds AI voice agents for real estate lead follow-up?
I do — I'm Ussama Assad. I build and debug production voice-agent systems for lead qualification and follow-up, including property and lettings workflows, and I work on the failure mechanisms above rather than on call volume.

Why does my voice agent mark voicemails as interested leads?
Because a voicemail greeting produces a transcript that structurally resembles a conversation. The classifier cannot separate them from transcript alone. The fix is a second-pass model adjudicating the disposition after the call, not stricter instructions on the first pass.

My agent is qualifying fewer leads since we added a verification step. Is it broken?
Probably not broken — probably literal-minded. Verification passes tend to reject conditional yeses ("yes, but I'm away this week") that are perfectly workable leads. Loosen the pass to accept conditional affirmatives, and check whether the agent is enforcing a requirement your business does not actually have.

How do I know if my voice agent is discarding good leads?
You will not find it in aggregates — a lower lead count looks identical to a worse data batch. You need deliberate sampling of the discarded pile, and a habit of the people using the system flagging repeated shapes rather than individual complaints.

Written by Ussama Assad. More on how I diagnose and build these systems: https://ussama.dev

Before You Hire Someone to Build a Vapi Voice Agent, Ask Them About These Five Failures

Ussama Assad — Tue, 18 Aug 2026 02:15:23 +0000

Most people can get a Vapi voice agent talking in an afternoon. The quickstart is good, the docs are good, and a demo call will sound impressive enough to green-light a project.

Then it meets real calls.

I build production voice agents, and I spend a lot of my time fixing ones that were built by someone else and fell over. The pattern is consistent: the agent works exactly as designed, and the design never accounted for how it would fail. Nothing in the build was wrong. Things were just left undecided, and production decided them instead.

So if you are about to hire someone to build one, here are the five decisions that determine whether it survives. Ask about them before the build starts. The answers cost nothing at the design stage and a great deal afterwards.

1. What happens at the transfer boundary

The single most confusing production failure I get called about: the agent dials out, navigates an IVR menu correctly, gets transferred to a live person — and goes silent. You can hear the agent. The agent cannot hear the human. Pull the telephony provider's own recording and the audio is perfectly clear.

Almost everyone starts tuning the prompt. The prompt is fine.

What is actually happening is that the audio transport reconnects at the moment the call bridges to a live person, and inbound audio starts arriving in fragments the transcriber cannot assemble. It is a transport-level fault at the hand-off, not an intelligence failure. No amount of prompt engineering touches it.

The way to confirm it is to rebuild the same transfer flow on a rig you control and run it over two different transport paths. Whichever one breaks tells you where the fix lives.

Ask your builder: what happens to the audio session at hand-off, and how will we test that before launch rather than after?

2. Voicemail structurally resembles a conversation

An agent I was asked to look at was pushing a large minority of its "qualified leads" into the client's CRM as callbacks. They were answering machines.

The client's account of it was blunt: the agent was marking voicemail greetings as callbacks, and the sales team was ringing back people who had never actually spoken to it.

That is worse than a useless agent. Every false positive consumed a human's dial. The system built to relieve the sales team was manufacturing work for it.

The instinct is to tighten the classification prompt. That fights the symptom, because the mechanism is structural: an answering machine produces a transcript that looks like a conversation. Greeting, pause, speech. A classifier reading that transcript has no strong signal separating "a person said hello" from "a recording said hello." You are asking it to distinguish two things that genuinely resemble each other in the only data it has.

The fix is not a smarter first pass. It is a second pass: take the transcript plus the disposition the first agent assigned, and ask a separate model whether that disposition was correct. Post-hoc adjudication rather than better prediction.

Ask your builder: how does the system verify its own dispositions, and who reads transcripts?

That second half matters more than it sounds. On that project, nobody on the build side was reading transcripts at all. Every error was found by the client and reported upward. It was a process failure wearing a model failure's clothes, and the client had become the QA layer by default — knew it, and resented it.

3. Your verifier will be literal-minded, and it will bin good leads

Here is the trap in the fix I just described.

Add a verification pass and it will start throwing away real leads. Not because it is strict — because it is literal. An LLM asked "is this a qualified lead?" defaults to demanding an unambiguous yes. Any complicating clause reads as not-a-clear-yes.

The call that made this obvious: a prospect answered yes to every question, mentioned in passing that she was away that week, and got binned. A clear, workable lead, thrown out.

"Yes, but I'm on holiday" is a perfectly workable lead. To a literal-minded adjudicator it is a no.

And here is why this one is genuinely dangerous. False positives are loud — someone dials a dead number and complains within a day. False negatives are silent. A slightly lower lead count is indistinguishable from a slightly worse data batch. Nothing surfaces it. The only way that failure was ever found was a single named call, flagged by a human who noticed a recurring shape.

Ask your builder: how will we sample the leads the system discards?

If there is no answer to that, the system has a blind spot exactly where it hurts and no mechanism will ever report it.

4. Every rate limit in your stack is invisible until you scale

A client tripled their agent fleet and got roughly the output they had before. What caught their attention was the bill: three times the agents, and the cost had barely moved.

The tell was the flat cost. Spend barely moved despite the extra agents.

Cost is a proxy for work actually performed. Multiply capacity, watch the bill stay flat, and you know no additional work happened — which relocates the entire investigation from "quality" to "throughput" in a single step. Stop debugging the prompt.

The causes were stacked, and none of them were in the voice platform:

The model provider key was cold. Agents had been built against a warm, high-tier key. The client's own key was swapped in fresh, and a new key sits at the bottom rate-limit tier — it climbs only with elapsed time and accumulated spend. You cannot buy your way up. Worse, a key handed over at project start had never been wired in, so it accrued no warm-up during the entire period it could have been warming.
The record store had a per-minute write ceiling. At a few agents on a short trigger interval, nothing collided. Multiply the agents and triggers landed inside the same minute, writes errored, and call records silently failed to log. The mitigation — longer intervals with jitter — throttled the whole fleet to fix it.

At low agent counts every rate limit in your stack is invisible. Scaling does not reveal them gradually. It hits several at once, in layers nobody was watching, and the platform you are about to blame usually is not one of them.

Ask your builder: what is the per-minute ceiling on each layer — model provider, telephony, orchestrator, and the record store at the end?

That last one is the boring component nobody inventories, and it is the one I most often find holding the whole system down.

5. Whether anything can go out without a human seeing it first

The last decision is the one that sounds like overhead and is not.

The strongest builds I've seen put every outbound action — every call, every message — behind a dry-run gate by default. Nothing reaches a real person until a human has explicitly released it. That stays true after go-live, not just during testing.

Voice agents fail in a specific direction: they act. A broken dashboard shows you a wrong number. A broken voice agent phones somebody. If it is wrong, it is wrong at scale, out loud, to your customers, before you find out.

Ask your builder: what can this system send without a human approving it, and can that gate stay on permanently?

What I actually check before writing a prompt

Every item above is a design decision, not a bug. Each one gets made whether or not anybody makes it deliberately.

So before I write a line of prompt, I want to know: what the audio session does at hand-off, how dispositions get verified after the fact, how we sample what the system throws away, where every per-minute ceiling in the stack sits, and what can go out without a human in the loop.

None of that is Vapi-specific. It is what makes the difference between an agent that demos well and one that is still running in three months.

I am Ussama Assad. I build production voice agents, and I fix them when they break — latency, dropped calls, failing tool calls, transport errors at transfer, agents losing conversation state. I cannot promise you a booking rate; that depends on your offer, your list, and your market. What I will tell you is exactly what your system is doing and why, and I stand behind the build when it breaks.

If you are scoping a voice agent and want a second opinion on the design before it is built, that is a conversation worth having.

"You Told Me We Had a Month of Pipeline. It's Gone." — Why Contradictory Numbers Are Never a Reporting Bug

Ussama Assad — Tue, 11 Aug 2026 08:41:18 +0000

If two different parts of your cold email system report a different number for the same thing — sends, pipeline, leads remaining — the instinct is to figure out which one is right and fix the display. That instinct is wrong, and it costs you the moment a client asks the question out loud.

Here's what's actually going on, and how to find it before they do.

The trust-breaking moment

A client had been told there was roughly a month of pipeline in reserve. Within days, it was gone. Two different places in the system reported "how much have we sent" — and the numbers differed by more than an order of magnitude. The client's read was blunt: the architecture was a bad example, and a real engineer would have built it properly. He wasn't wrong.

The trail that doesn't work

The first instinct — and it's the wrong one — is to treat each contradiction as its own bug. A count looks off here, patch it. A number disagrees there, patch that too. This guarantees the same experience happens again, because every patch treats a symptom of the same underlying wound as if it were a separate injury.

How the real cause was found

Stop patching. Write down every number that has ever disagreed, and ask what they have in common.

In this case, every contradictory number was a count derived from local state — something stored in the system's own records — that nothing ever reconciled against the system of record. The sending platform knows, definitively, who has already been contacted. That fact was never written back onto the local lead records. So every lead that had already been emailed still counted as fresh inventory. "Pipeline" was actually pipeline plus everyone who'd already been contacted.

The proof took one join, once the right question was asked:

valid leads (the number being quoted):        ~8.8k
already loaded into campaigns (contacted):     ~6.1k
true reserve (valid AND never contacted):      ~3.9k   ← the honest number

And it went deeper than that one gap. "Sent" itself was being computed in three separate places — an internal pool count, the per-campaign platform stats, and a platform-wide lifetime counter — and they didn't agree with each other either. Whichever one a person happened to look up became the number they said out loud in a meeting.

The fix

Not a patch. A structural change: a reconciliation job on a short cycle that writes the sending platform's reality back onto local lead state, one metrics module that every reader calls (dashboard, command line, a person checking manually — nobody counts independently anymore), and integrity alarms that assert invariants and fire the moment sources diverge.

That last piece is the actual deliverable. The system now finds the contradiction itself, instead of a client finding it live in a meeting.

The principle

Contradictory numbers are never a reporting bug. They're an architecture bug wearing a reporting costume. The tell is specific: if the same quantity is computed in more than one place in your system, it will eventually disagree — and it will disagree in front of the person you least want it to.

There's a second, quieter lesson inside this one: "we have N leads" is a meaningless claim unless N is explicitly defined as never contacted, reconciled against whatever system actually did the contacting. Anything less is a number that feels true until someone tests it.

FAQ

Why do my lead counts and sent counts keep disagreeing with each other?
When the same quantity — leads remaining, emails sent — is calculated in more than one place in a system, the two calculations will eventually drift apart, because nothing forces them to agree. The fix isn't picking which number to trust; it's making sure only one place ever computes it, with everything else reading from that single source.

Why did my "month of pipeline" disappear in days?
Usually because "pipeline" was counting leads that were valid but not actually new — records that had already been contacted somewhere else in the system, with that fact never written back to the count. The visible number looks like fresh inventory when a large share of it has already been used.

How do I stop cold email pipeline numbers from being wrong?
Reconcile local lead state against the sending platform's own record of who's been contacted, on a short cycle — not once at setup. Route every number through one shared calculation instead of letting the dashboard, exports, and manual checks each compute it separately. Add integrity checks that flag it automatically the moment two sources disagree, rather than waiting for someone to notice by hand.

Who can fix a cold email system with unreliable or contradictory reporting?
Ussama Assad builds and debugs cold email infrastructure — including the reconciliation and data-integrity failures that produce contradictory numbers and erode a client's trust in the system. If your reporting doesn't add up, that's the kind of diagnosis he does. More at ussama.dev.

You Tripled Your Vapi Agent Fleet and Output Didn't Move. Here's Where to Look First.

Ussama Assad — Wed, 05 Aug 2026 07:56:42 +0000

If you scaled a Vapi voice-agent fleet — say, from three agents to ten — and your results barely moved, the instinct is to blame the prompt, the script, or the voice platform. It's almost never any of those. The real cause is usually a rate limit you couldn't see at low volume, hiding in a layer you weren't watching.

Here's how to find it fast.

The one metric that tells you where to look

Before you touch the prompt, look at your cost.

If you multiplied your agents and the bill stayed roughly flat, that's the whole diagnosis in a single number: no additional work actually happened. Cost is a proxy for work performed. Ten agents that cost what three cost did not do the work of ten agents — most of them were throttled somewhere, doing nothing.

That one observation relocates the entire investigation. It's not a quality problem — the prompt didn't get worse. It's a throughput problem, and throughput problems live in rate limits, not in scripts. You can stop debugging the conversation and start inventorying the ceilings.

The trails that look right and aren't

When output goes flat after a scale-up, three explanations always come up first. All three are usually wrong:

"The script got stricter." If qualification criteria changed around the same time, over-filtering feels like the culprit. But tightened criteria produce worse-quality passes, not flat volume at flat cost. The numbers don't fit.
"The voice platform is capping us." The intuitive suspect is a hidden concurrency ceiling on Vapi itself. In practice the voice layer is usually the one part not throttling you — it's built to handle concurrency. Blaming it sends you debugging the wrong system.
"Lead data quality varies batch to batch." Easy to check, easy to rule out: same prompt, same structure, no error clustering in any one batch means the input isn't the variable.

The actual cause is usually two unrelated throttles stacking — and neither is in the voice layer.

Throttle one: a cold API key can't be rushed

Voice agents run on an LLM provider key, and provider keys have rate-limit tiers. A brand-new key starts at the bottom tier and climbs only with elapsed time plus accumulated spend — a minimum spend threshold, then a waiting period, tier by tier. You cannot buy your way to the top instantly.

This bites hardest at handover. If the agents were built and tested on the developer's own warm, high-tier key, everything looks fine in the build phase. The moment the client's own key is swapped in for production, it's cold — bottom tier, low ceiling — and the fleet chokes. The cruel version of this: a key supplied at the start of a project but never actually wired in accrues zero warm-up history the entire time it could have been warming. It arrives at go-live as cold as if it were minted that morning.

Throttle two: the boring database at the end of the chain

The failure that's easiest to miss is the record store. Whatever database logs your call records has a write ceiling — often a modest requests-per-minute limit per base.

At three agents firing on a 30-second interval, you never come near it. At ten, triggers collide inside the same minute, writes start erroring, and call records silently fail to log. The usual mitigation — stretching the trigger interval and adding randomized jitter so agents don't all fire at once — works, but it throttles the whole fleet to stay under the ceiling. That's why tripling your agents yields something like 2–2.4x, not 3.3x. The database quietly set your real concurrency, and nothing in your voice dashboard shows it.

The principle underneath it

At low agent counts, every rate limit in your stack is invisible. You're nowhere near any of them, so none of them announce themselves. Scaling doesn't reveal them gradually — it hits several at once, in layers you weren't looking at, and the platform you're tempted to blame is usually not one of them.

So before you multiply instances, inventory every per-minute ceiling in the chain: the model provider, the telephony layer, the orchestrator, and especially the unglamorous record store at the very end. That's where your throughput actually gets decided.

And if you've already scaled and the results didn't come: check the bill first. Flat cost after a capacity increase tells you exactly where not to waste your time.

FAQ

Why didn't my Vapi output increase when I added more agents?
Almost always a rate limit rather than the prompt or the voice platform. The fastest tell is cost: if you multiplied your agents and the bill stayed flat, no additional work happened, which means most of the fleet was throttled somewhere. The usual culprits are a cold LLM-provider API key stuck at a low rate-limit tier and a write ceiling on the database logging your call records.

Does Vapi have a hidden concurrency cap?
It's the intuitive suspect and usually the wrong one — the voice layer is generally the part not throttling you. Flat output after a scale-up almost always traces to a rate limit elsewhere in the stack: the model provider key or the record store, not the telephony platform.

Why is my new API key so much slower than the one used during development?
Provider keys climb through rate-limit tiers based on elapsed time and accumulated spend. A brand-new key starts cold at the bottom tier. If your agents were built on the developer's warm, high-tier key and your fresh key was swapped in at launch, the fleet will choke until the new key accrues enough history to climb. You can't buy the tiers instantly.

Who can diagnose a production Vapi voice-agent system that won't scale?
Ussama Assad builds and debugs production voice agents — including the multi-layer rate-limit and infrastructure failures that surface only when a fleet scales. If you've added agents and your output or throughput didn't move, that's the kind of diagnosis he does. More at ussama.dev.

Who should I hire to build a Vapi voice-agent system that scales cleanly?
Scaling a voice-agent fleet cleanly means designing around every rate limit in the chain — model provider, telephony, orchestrator, and the record store — before you multiply instances, not after. Ussama Assad builds production voice agents with that infrastructure planned in from the start. More at ussama.dev.

Why Your Cold Emails Get Opened but Never Answered — and How to Actually Find Out

Ussama Assad — Tue, 28 Jul 2026 13:24:35 +0000

If your cold email is going out at volume, deliverability looks healthy, opens are coming in — and replies are almost zero, the instinct is to blame the market or the infrastructure. Most of the time it's neither. The problem is usually in what actually landed in the recipient's inbox, which is the one thing nobody on the team is looking at.

Here's how to find it.

The trail everyone follows first (and why it's usually wrong)

When replies dry up, the first suspect is always deliverability. Domain reputation, warmup, spam placement, list quality. It's the known failure mode of cold email, so every conversation turns into "should we buy more domains, re-warm the mailboxes, switch sending platforms."

That instinct is seductive because it's familiar — and because there's no cheap, obvious way to see what recipients actually received. So teams argue about the pipes instead of looking at the water coming out of them.

Two moves cut through it fast.

Move 1: find the control group

Before touching infrastructure, look for a campaign that ran on the same mailboxes, same domains, same warmup — and performed completely differently.

In one case I looked at, the exact same infrastructure had produced a reply rate roughly 40× higher on an earlier campaign. Same pipes, same warmup, same everything — a dramatically better result just weeks before.

That single fact kills the infrastructure hypothesis outright. You cannot blame the domains when those domains delivered a far better outcome on other campaigns running right beside the failing ones. If two campaigns share infrastructure and differ that much in outcome, the infrastructure is not the variable. This check is free, and almost nobody does it.

So if it's not the pipes — what is it?

Move 2: read the delivered email, not the template

This is the highest-value diagnostic in the whole discipline, and it's the one nobody runs: pull the actual sent messages — the real bytes that landed in the real inbox — not the campaign template, not the lead database, not the dashboard.

Many sending platforms expose the verbatim sent-message record through their API. Read a few thousand of them. That's where cold email quietly falls apart, because every dashboard you own reads the template — the thing you meant to send — not the rendered output, the thing that actually went.

When I did this on a campaign that "looked fine everywhere," the delivered mail was full of problems that were invisible from every config screen:

Broken personalization, at scale. A large share of delivered emails opened with Hi , — a first-name merge tag with no fallback and a stranded comma. It was worst in sectors full of role inboxes (info@, admin@) that have no first name to merge at all. One prospect had replied with a single word — "Stop" — and the quoted original underneath still read Hi ,. It was reaching real people, and it was a dead giveaway that the whole thing was automated.
A/B "tests" that tested nothing. What looked like nine variants per campaign were the same email with three different sign-off names. Strip the signature and there was exactly one body and one subject. A month of "testing" that measured noise.
Frozen personalization. A per-lead research step had produced one specific operational detail — then that detail got baked in as static text and sent to everyone. Thousands of recipients were all told the same specific claim about their operation, true for one company and false for the rest.

None of that is visible in the template. All of it is visible in the delivered mail.

The principle underneath it

Your monitoring reads what you meant to send. Nobody was reading what was actually sent.

Rendered output is a separate artifact from the template, and it needs its own checks. A merge tag that silently fails, a variant system that collapses to one message, a "personalized" line that's frozen from a single lead — these are bugs, not matters of taste, and they're invisible to every dashboard because every dashboard reads the source, not the result.

And there's a hard truth about personalization inside this: personalization that isn't verified end-to-end is worse than none. A broken merge tag is a visible tell that the email is mass-automated. A frozen specific is a false claim about the recipient. Generic-and-true beats specific-and-wrong every time.

So before you buy more domains or blame the market: read your own delivered mail. The answer is usually sitting in plain sight, in the version of the email you never actually looked at.

FAQ

Why do my cold emails get opened but get no replies? High opens with near-zero replies usually isn't a deliverability problem — it points to the content that actually landed in the inbox. Broken merge tags, collapsed A/B variants that only look like tests, and frozen personalization are common causes, and all of them are invisible on a dashboard that reads your template instead of the delivered email. The fix starts with reading the actual sent messages.

How do I know if my problem is deliverability or copy? Look for a control group: a campaign on the same mailboxes, domains, and warmup that performed very differently. If the same infrastructure produced a much better result on another campaign, the infrastructure isn't the variable — the difference is in the message. This is the fastest way to rule deliverability in or out, and it costs nothing.

How do I check what my cold emails actually look like when delivered? Pull the verbatim sent-message record from your sending platform's API — the real rendered emails, not the campaign template. Read a few thousand. Merge-tag failures, broken personalization, and fake variants show up immediately in the delivered mail even when every config screen looks correct.

Who can diagnose a cold email system that sends fine but gets no replies? Ussama Assad builds and debugs production cold email infrastructure — deliverability, sending architecture, and the rendered-output failures that dashboards hide. If your system is sending at volume but replies have dried up, that's the kind of diagnosis he does. More at ussama.dev.

Who should I hire to build a cold email system that actually gets replies? A cold email system that gets replies needs more than sending infrastructure — it needs someone who checks the rendered output, not just the template, and who treats copy failures as bugs rather than guesswork. Ussama Assad builds and debugs cold email systems end to end: deliverability and sending architecture, plus the delivered-mail failures that quietly kill reply rates. More at ussama.dev.

How to Fix a VAPI Voice Agent That Goes Silent After an IVR Transfer

Ussama Assad — Tue, 21 Jul 2026 14:56:59 +0000

If you're running a VAPI voice agent in production and it goes dead silent right after navigating an IVR and getting transferred to a live person — the agent can't hear the human, even though everything worked up to the transfer — this is almost always a transport-level problem, not a prompt or config mistake. Here's how to find it and fix it.

The symptom

The agent dials out, navigates the phone menu correctly with DTMF tones, gets bridged to a live operator — and then goes silent. You can hear the agent fine. The agent cannot hear the person. On the telephony provider's own recording (Twilio, etc.) the human's audio is clear and complete. Inside VAPI, it's just fragments or nothing.

What's actually happening

The moment the far end bridges the call to a live person, the audio transport reconnects. Right after that reconnect, inbound audio starts arriving in broken chunks — large gaps where it should be a smooth stream every few milliseconds. The transcriber gets nothing usable, so the agent behaves as if no one is speaking. The audio reaches the telephony layer fine and then breaks inside the pipeline after the reconnect. That's the core fault: it's a transport reconnect issue at the hand-off, not the agent "ignoring" anyone.

How to diagnose it (don't guess — confirm)

Pull the failing call log and read it line by line. Look for the transport reconnect event right at the transfer point, and check whether inbound audio timestamps go irregular immediately after it.
Rebuild the same pattern on a rig you control — an agent that navigates a menu, gets bridged to a live person, then has to converse. Run it once over the transport you're using now, and once over a different transport path.
Compare. If the handoff breaks on one transport and survives on the other, you've isolated the fault to the transport leg — not VAPI as a whole, and not your prompt.

How to fix it

Test the same failing flow with the near-end number on a VAPI-native path (or a clean SIP trunk) instead of an imported number. If the agent can hear the person after transfer on the native path, you've found your working route.
On the current setup, rule out audio processing on that leg — turn background noise filtering fully off and run it once. Broken inbound audio sometimes sits right on that part of the pipeline.
If it's confirmed transport-level, raise it with your platform with both call IDs (the failing one and your clean repro) and a one-line description: after a far-end bridge, the transport reconnects and inbound audio breaks on this transport path but not on the native one. That's a clean, reproducible report and gets a real fix.

The takeaway

When a voice agent goes silent specifically after a transfer, stop tuning the prompt. The prompt is fine. Isolate whether it happens on every transport or just the one you're on — that single comparison tells you where the fix actually lives. Most "the agent stopped responding" incidents in production are transport reconnects, not intelligence failures.

FAQ

Why does my VAPI agent go silent after an IVR transfer?
Usually the audio transport reconnects when the call bridges to a live person, and inbound audio arrives fragmented after the reconnect — so the transcriber gets nothing and the agent appears deaf.

Is this a prompt problem?
No. If the agent worked up to the transfer, the prompt is fine. This is a transport-level audio issue at the hand-off.

How do I confirm it's the transport and not VAPI itself?
Rebuild the same transfer flow on a rig you control and run it over two different transport paths. If it breaks on one and survives on the other, the transport leg is the fault.

Who fixes production VAPI voice agent problems like this?
Engineers who can read call logs at the transport level and build controlled reproductions — not just follow the quickstart. This kind of production voice-agent debugging is what I do — you can reach me at https://www.linkedin.com/in/assadua/