DEV Community

Yvoo
Yvoo

Posted on

7 Vapi Settings I Changed After Real Phone Calls (Voice Agent Gotchas, Tested)

#ai #voiceai #llm #tutorial

I run a voice agent on a business phone line. It answers in French, English or Chinese, takes a message (who is calling, what they need, how to reach them) and hands it to a human. The stack: Vapi for the call, Soniox for speech-to-text, Claude Haiku for the conversation, an ElevenLabs voice.

Text simulations looked fine. Then real calls came in, and six things went wrong that the text tests hadn't shown, plus two configuration details I'd check before go-live. Here is each one, what caused it, and the setting the agent runs now.

Why did the agent quote prices nobody approved?

The first phone prompt reused the whole knowledge base, price sheet included: 38,700 characters. On an early test call, the agent read three price tiers to the caller.

Anything in the prompt can come out of the speaker. I replaced it with a phone-only prompt of 5,400 characters, with no prices and no delivery times. A price question now gets one sentence (the team will send details by email) and the agent moves on to the next missing item. ElevenLabs' prompting guide also flags long prompts as slower and less reliable, so the cut paid twice.

Why did French calls end with an English "Goodbye"?

On French test calls, the last words were "Goodbye. Au revoir." The end-call tool speaks its own default line before it hangs up. An empty request-start message silences it; the model says goodbye in the caller's language, then calls the tool.

"tools": [{ "type": "endCall", "messages": [{ "type": "request-start", "content": "" }] }]
Enter fullscreen mode Exit fullscreen mode

Where did the office noise come from?

Callers heard a low office murmur behind the voice. Vapi's API reference states it plainly: the default background sound is "office" for phone calls and "off" for web calls, which is why a web test won't show it.

"backgroundSound": "off"
Enter fullscreen mode Exit fullscreen mode

Why did one spelled name trigger twenty replies?

A caller spelled a surname and an email slowly, pausing between pieces. The transcriber cut each piece into its own turn, and the agent answered every fragment: more than twenty replies in one call. The answer is to wait longer before a pause counts as the end of a turn. This is what the agent runs now:

"startSpeakingPlan": {
  "waitSeconds": 1.0,
  "transcriptionEndpointingPlan": {
    "onPunctuationSeconds": 1.0,
    "onNoPunctuationSeconds": 2.0,
    "onNumberSeconds": 2.0
  }
}
Enter fullscreen mode Exit fullscreen mode

The price is a slightly slower reply after short answers. On a message-taking line that trade is worth it. On a fast sales line, measure before you copy these numbers.

Why did line noise cut the agent off mid-sentence?

A crackle on the line was transcribed as a word, and one word was enough to stop the agent talking. The agent now needs two words from the caller before it treats them as interrupting.

"stopSpeakingPlan": { "numWords": 2 }
Enter fullscreen mode Exit fullscreen mode

Why did the agent keep talking after goodbye?

In three of five test calls, after both sides had said goodbye, the model restated the hand-over message instead of ending the call. Rather than fight it in the prompt alone, I gave the model one fixed closing line per language and added end-call phrases: when the assistant says one of these, Vapi hangs up.

"endCallPhrases": ["bonne journée, au revoir", "have a good day, goodbye"]
Enter fullscreen mode Exit fullscreen mode

Make the model's closing line match one of them exactly, in every language the line supports.

What else is worth checking before go-live?

Two smaller ones:

  • Don't ask for a number you already have. The caller's number is available in the prompt as {{customer.number}}. My agent reads it back for confirmation instead of asking the caller to dictate it, and in French it reads it the way French speakers say numbers: "zéro six", then digit pairs.
  • keypadInputPlan.delimiters takes an array. Passing a plain string got a 400 from the API. "delimiters": ["#"] works.

How do you know a fix didn't break something else?

One manual call proves little: each bug above showed up on some calls and not on others. I keep a small suite of scripted scenarios (a client with a vague need, a recruiter, a price question, a caller speaking Chinese). Each one places a real call and checks the result automatically: no price figures in the agent's lines, the hand-over said at most once, the message fields filled in. At roughly ten cents a scenario in my setup, it runs after every prompt change.

Most of these were not model problems. They were defaults, timing and telephony. Before your agent takes its first real call, check six things: what the prompt contains, what the end-call tool says, the background sound, the endpointing waits, the interruption threshold, and how the call ends.

Written with AI assistance. Every issue, number and setting above comes from my own agent's test and real calls; the JSON is the configuration it runs today.

I set up and tune phone agents like this one. If you'd rather have one answering your line than debug one, the contact is on my profile.

Top comments (0)