Building an AI voice roleplay coach for sales calls: what we learned

Franta Mráz — Thu, 12 Nov 2026 23:02:06 +0000

Practising a sales call on your own is awkward. Practising with a colleague is better, but it needs a second person, and colleagues tend to go easy on each other. We build DIRI AI, a voice roleplay coach where you talk to an AI "client" and get a report afterwards. Here are the engineering and product lessons that mattered most. They are opinions from building it, not benchmarks.

1. A voice loop is a latency budget, not a model choice

A text chatbot can take two seconds to answer and nobody minds. In a spoken call, a long silence reads as hesitation, disinterest or a dropped line, and the learner starts to fill it. That changes the training itself.

The pipeline is speech-to-text, a language model, then text-to-speech. Each stage adds delay, so we treat the total as one budget and look at the whole path instead of tuning the model alone. Things that helped in general:

Stream everything. Start speaking the first sentence of the reply before the model has finished the second.
Keep the AI's turns short. Short replies are cheaper, faster and closer to how real prospects talk.
Measure end-to-end from "user stopped speaking" to "first audio out", not per-component averages.

2. Turn-taking is where it feels fake

Most of the "this feels like a robot" moments came from turn-taking, not from the words. Two failures show up quickly:

The AI cuts in while the learner is still thinking mid-sentence.
The AI keeps talking when the learner interrupts.

Real prospects interrupt, trail off and say "hm". We had to treat barge-in (the learner talks over the AI) as a normal event: stop playback, drop the rest of the planned reply, and listen. We also learned that end-of-speech detection is a trade-off. Wait too long and the call drags. Cut too early and you break people off mid-thought. There is no perfect threshold, so we keep it a tunable setting instead of a constant buried in code.

3. The persona prompt is the product

For a coach, the quality of the practice depends on the quality of the simulated client. Generic "be a difficult customer" prompts produce clients who are either pushovers or random jerks. What works better is a small structured scenario:

who the client is and what they already know,
one concrete need and one concrete objection,
what would make them say yes, and what would make them end the call,
a rule to stay in character and not help the learner.

The last one matters. Language models want to be helpful, so an unconstrained AI client will quietly guide you to the right answer. A real client will not. We keep the "client" and the "coach" as separate roles: the client plays the scenario during the call, and the feedback only appears in the report afterwards.

4. Scoring: be specific, or it is just a number

A single score out of 100 feels objective and teaches nothing. What helped learners was feedback tied to something they actually said. A rubric with a few named criteria (for example opening, discovery questions, handling objections, next step) gives the model something concrete to check, and gives the learner something concrete to repeat.

Two cautions from our own testing:

Language-model scoring is not perfectly consistent. The same call can land a few points apart on two runs. We would rather show a short list of observations than imply false precision.
A report is only useful if the learner can act on it. One clear next thing to try beats ten generic tips, which is why a basic report on the free tier is a score plus one tip.

5. Languages are not a translation layer

We support 11 languages (English, Czech, Slovak, Polish, German, Spanish, Portuguese, Italian, French, Romanian and Hungarian). The lesson: do not translate English scenarios and call it done. Politeness conventions, how direct a salesperson may be, how formal the address is, and even how a phone call opens vary by market. We write scenarios and rubric wording per language, and we have to be honest that quality in some languages needs native review before we are fully comfortable with it.

6. Make the first call cost nothing and take a minute

For a practice tool, the biggest drop-off is before the first call. People will not read documentation, create a team, or enter a card to find out whether it is useful. We let a new account use the full individual plan for 7 days with no card (capped at 5 calls or 30 minutes of voice, whichever comes first). After that there is a Free plan: 3 calls per month, up to 5 minutes each, with a basic report. Paid individual plans are Start (€19.90), Plus (€49.90) and Pro (€99.90, 480 minutes per month), and there are team plans for companies.

Capping by calls and minutes (not just by days) also protects us: voice minutes are the real cost, and a hard cap lets us offer a no-card trial honestly.

7. Privacy and consent are design constraints

Recording and analysing someone's voice raises questions that a text app does not. Our rules of thumb: tell people plainly what is processed, keep each customer's data separate, and do not publish social proof (names, quotes, numbers) until it is real and consented. We have no testimonials on the site yet because we do not have enough consented ones, and we would rather show nothing than invent anything.

What we would tell someone starting this

Get a rough voice loop working end-to-end first, then measure the latency from the learner's point of view.
Spend your time on the scenario prompt and on turn-taking, not on a fancier model.
Separate the in-call "client" from the post-call "coach".
Give feedback anchored in what was said, and keep it short.
Treat every language as its own content project.
Cap free usage by the thing that actually costs money.

Where we are

DIRI AI is young, and there is plenty we have not solved: consistent scoring across languages, handling very noisy audio, and making scenarios feel less scripted. If you work on voice agents, I would like to hear how you handle barge-in and end-of-speech detection.

If you want to see how it plays out in practice, you can try a call at diriai.com. The 7-day trial needs no card.

DEV Community: Franta Mráz