DEV Community

Daniel Varela
Daniel Varela

Posted on

Voice agents on CPU vs the GPU incumbents: latency, cost, and a deliberate comparison

Two numbers frame this post. On Lokutor, a production voice agent starts speaking in 139 ms at the median (TTS streaming time to first audio, one CPU thread), and a full minute of a running agent costs about 2 cents with speech recognition, the language model, and the voice included. On the GPU incumbents' own pricing pages, a comparable minute lists at 6 to 8 cents and covers only part of the stack.

We picked this pairing on purpose, and it favors us. Below are the numbers, the sources, and the places where the comparison breaks.

Latency, measured in production

Versa 2.0, our TTS model, on live customer-facing infrastructure, single CPU thread (c8g.2xlarge, Graviton4):

  • 74 ms fastest streaming time to first audio
  • 139 ms median TTFA
  • 211 ms p90
  • 0.26 to 0.36 real-time factor

At the full agent level (turn detection, language model, voice, transport), caller-observed time to first audio is 385 to 442 ms in the best case and about 660 ms typical. In three recorded demo sessions, the server-side figure from end of turn to first audio came out between 67 and 207 ms.

The GPU reference point, from our earlier benchmark: Kokoro 82M on a spot RTX 4090 generated a short phrase in 47 ms warm. That is faster than Versa, as a good GPU model should be. Two caveats keep it honest: the 47 ms is full-utterance generation in a setup that did not stream, and the same setup needed 14.9 seconds to load the model. A CPU does not beat a 4090 on raw warm inference. The question is which infrastructure you want to operate when traffic is bursty and capacity goes cold.

Cost, from public pricing pages

Each company's own pay-as-you-go rate, read from their pricing pages on 23 September 2026 (details and billing rules in this breakdown):

Platform Published price per minute What that covers
Lokutor ~2 cents (1.4 cents on Business) Recognition, language model, and voice
Cartesia 6 cents Call duration
Deepgram Voice Agent 7.5 cents Standard tier, pay as you go
ElevenLabs Agents 8 cents Per minute beyond a plan's allocation

Bundles differ, so this is not a like-for-like price benchmark. It is still the comparison buyers make. A Lokutor minute includes the model and the voice; the other rates cover part of the stack, so the gap widens once you add recognition and the language model, and it narrows or flips if you already operate your own models.

Capacity: in a measured AMD Genoa test, one node handled about 19 concurrent calls, equal to roughly $22 of infrastructure per concurrent stream per month. That was a capacity test, not our current production deployment, so treat it as a capacity result rather than a guarantee.

Quality is gated, not assumed: English WER fell from 6.51% to 3.20% between the previous Versa version and 2.0, mean WER across ten benchmarks is 11.21%, and English UTMOS is 3.14. WER from a recognizer is a proxy for intelligibility, not proof of quality; we use it to catch pronunciation regressions before latency work ships.

Where this comparison breaks

  • Warm GPU TTS wins on raw speed. If you already operate GPU infrastructure well and need maximum throughput per node, use it.
  • List prices move. Check the current pages before deciding.
  • Bundles differ: support, languages, telephony, compliance, and what one minute buys.
  • Our median is a production median on our traffic. Yours will differ.

The claim is narrow: interactive latency and production economics for voice agents, on commodity CPUs, with no accelerator in the serving path.

Try it

The free plan includes 60 minutes a month. Get an API key at app.lokutor.com and read the docs at docs.lokutor.com. Benchmark cold and warm paths, then send us the numbers. We would rather compare real serving conditions than trade screenshots of ideal runs.

Top comments (0)