DEV Community: Daniel Varela

Ito: streaming speech synthesis in 4.89 MB, verified in ESP32-S3 emulation

Daniel Varela — Mon, 05 Oct 2026 09:07:59 +0000

We've released Ito's inference engine, ESP32-S3 firmware and two English voice models. The latest demo lists 4.05M parameters, about 4M. The chip weights occupy 4.89 MB per voice. The engine produces 24 kHz speech using integer arithmetic.

What runs where

Text becomes phonemes on the host using espeak-ng. Token IDs are sent to the firmware. A streaming acoustic front predicts duration, pitch, energy and a mel spectrogram. A Vocos-style vocoder turns that into audio using ConvNeXt blocks, a harmonic pitch source and an inverse STFT.

The C99 engine builds on a laptop and for ESP32-S3. The target has a 240 MHz dual-core CPU, no neural accelerator, and 8 MB PSRAM; the recommended board is N16R8 with 16 MB flash.

What we've verified

Espressif's QEMU runs the firmware and produces PCM that is bit-identical to the host engine. The public samples are that engine's output. This checks the implementation and arithmetic.

Nothing has run on a physical board yet. Time to first audio and real-time factor are estimates, so real-time playback on silicon is still an open question.

The latest firmware uses a 125 ms first audio chunk. The current firmware documentation estimates first audio at 137-143 ms optimistically, 200-207 ms centrally and 311-318 ms pessimistically. Estimated RTF is 0.53-0.54, 0.77-0.80 and 1.18-1.27 respectively. An RTF above 1 falls behind playback. The pessimistic case does that, so the range matters more than the central number.

These estimates use exact QEMU instruction counts with assumed CPI and PSRAM bandwidth. They are not measured board latency. First audio also differs from the buffering needed for gapless playback: the central estimate for that start delay is 215-240 ms.

Quality checks and limits

The repo contains automatic evaluations and a small listening exercise. The latter used one founder as listener and four sentences per system from the float model. It doesn't establish an independent MOS score, and the exact chip configuration hasn't been blind-rated.

Ito is English-only, with one voice per weights file and a fixed speaking style on the chip. We'd particularly like outside feedback on pronunciation, intonation and the flashing instructions.

Try the chip arithmetic on a computer

After accepting the weight terms at https://huggingface.co/lokutor-ai/ito and following the repo's dependency setup:

cd esp32/host && make && cd ../..
python esp32/tools/chip_wav.py "Good morning! The coffee is ready." hello_chip.wav

Full setup and flashing instructions: https://github.com/lokutor-ai/ito
Samples and comparisons: https://lokutor-ai.github.io/ito/

Licensing

Engine, firmware and Python code are GPLv3. The voice weights and Ito audio are CC BY-NC-SA 4.0 plus additional terms, with gated access on Hugging Face. Commercial use needs a written license from Lokutor. The GPL on the code doesn't make the weights commercially unrestricted. Training code and recipe are not public.

A separate Spanish ASR update

Oído now includes oido_es.tnm and oido_es.tlm, a Spanish Conformer model and language model for the same target. Together they occupy 14.0 + 1.3 MB. Spanish weights are CC BY 4.0; engine code is GPLv3.

Our own host evaluation with the language model reports WER 13.8% on Common Voice, 10.9% on MLS, 15.7% on VoxPopuli and 11.3% on FLEURS across the full test sets. Oído was fine-tuned on training data from these corpora, so this doesn't establish zero-shot generalization or performance on arbitrary regional accents. Spanish noise benchmarking is still missing, and board speed remains unmeasured.

Oído: https://github.com/lokutor-ai/oido

These are separate engines. We haven't validated a complete simultaneous STT/TTS loop on one ESP32-S3.

Oído: Open-Source Speech Recognition on a $5 Chip

Daniel Varela — Thu, 01 Oct 2026 10:24:51 +0000

Speech recognition on a microcontroller has, in practice, meant one of two things: a wake word, or a short list of commands the device was programmed to expect. Say anything else and it hears nothing.

Today we're releasing Oído: speech recognition for any English sentence, running entirely on an ESP32-S3. That's a $5 microcontroller with a 240 MHz dual-core CPU, 8 MB of PSRAM, 16 MB of flash and no neural accelerator. No cloud, no command list, and it's open source.

Watch the Oído demo on the original post

The transcripts in the video are Oído's chip-exact output, sped up. Footage from a physical board is coming.

¡Oído! is what cooks call out in a Spanish kitchen to confirm an order: heard, got it.

How accurate it is

Word error rate on LibriSpeech, the standard English benchmark (lower is better):

System	Runs on	test-clean	test-other
Oído, int8	ESP32-S3	3.7%	8.2%
Oído, int4 (8.3 MB)	ESP32-S3	4.6%	10.0%
Espressif MultiNet7	ESP32-S3	8.5%	21.3%
Moonshine tiny	laptop	5.0%	12.1%
Whisper tiny.en	laptop	6.3%	15.9%

On the same chip, Oído makes 2.3 to 2.6 times fewer errors than Espressif's own recognizer, which matches speech against a list of up to 200 predefined commands. It also makes fewer errors than Whisper tiny and Moonshine tiny running at full precision on a laptop. That holds in noise too: across 14 conditions built from real car, kitchen and cafeteria recordings, background chatter and room echo, Oído averages 8.4% against 12.1% for Whisper tiny.

Squeezing the model onto the chip costs almost nothing. The int8 version is within about a tenth of a point of the original full-precision model (3.70% vs 3.68% on test-clean).

The model is NVIDIA's. The work is the engine.

Oído runs NVIDIA's openly licensed Conformer-CTC Small, a 13-million-parameter speech model. We didn't retrain or distill it. The hard part was making it run on a chip whose fast internal memory is 512 KB, when the model alone is 14 MB.

The obstacle isn't arithmetic, it's memory bandwidth. The weights live in flash, which the chip reads through a small cache at a few tens of megabytes per second. So we wrote a new inference engine in C for the ESP32-S3's vector instructions, designed around that limit:

Integer matrix kernels that do 16 multiply-adds per instruction.
Integer attention, softmax included, so the whole model runs in int8 arithmetic.
A bandwidth-aware schedule that processes 64 frames of audio at a time, so each weight is read from flash once per block instead of once per frame. This cut weight traffic from 18 to 7.5 MB per second of audio.
Both CPU cores working in parallel.

There's also an int4 version that takes 8.3 MB of flash instead of 14 MB, leaving 6 MB for your own application, and an optional on-chip language model that brings the error rate down to 3.3% and 7.2%.

Where it stands

We want to be precise about this. Every transcript and accuracy number above comes from the exact arithmetic of the on-chip engine, and the real firmware produces the same transcripts, word for word, in Espressif's QEMU emulator. Speed is estimated from exact instruction counts at 0.7 to 0.95 times real time. Boards arrive this week, and we'll publish measured numbers here.

Two other limits: it's English only for now, and it transcribes after each utterance rather than word by word, so text appears about 3 seconds after you stop talking on a short command.

Why we built it

Voice is moving off the cloud. A device that streams audio to a GPU pays for inference for as long as it exists, stops understanding people when the connection drops, and sends their voice somewhere else. On-device recognition removes all three problems, but until now it also meant giving up open-ended speech. Oído shows that a commodity $5 chip is enough for the real thing.

It's the listening half of the on-device voice stack we're building at Lokutor for this class of hardware.

Try it

Everything is on GitHub at github.com/lokutor-ai/oido, and the models are on Hugging Face: int8 and int4.

You don't need a board to try it. The host build runs the chip's exact arithmetic on your laptop, including a live microphone demo (Python with numpy, soundfile, sentencepiece and sounddevice):

git clone https://github.com/lokutor-ai/oido
cd oido/esp32/host && make
python live_demo.py

To run it on hardware you need an ESP32-S3-DevKitC-1 N16R8 and an INMP441 microphone. The README has the wiring and a one-line flash script.

Open source, with a commercial option

The code is licensed under GPLv3. The int8 model is CC-BY-4.0 and the int4 model is CC-BY-SA-4.0. For products that can't meet GPLv3 terms, we offer commercial licenses, models for other languages and integration support. Write to us at contact@lokutor.com.

We're also writing up the full technical details as a paper. If you build something with Oído, we'd love to hear about it.

Try the voices in your browser on Hugging Face. Lokutor 2.0 launches on Product Hunt October 13.

Oído: open-source speech recognition for the ESP32-S3, without a command list

Daniel Varela — Wed, 30 Sep 2026 19:40:26 +0000

Oído is an open-source speech-to-text engine built by Lokutor for the ESP32-S3. It accepts open-vocabulary English speech rather than a fixed list of commands, with the recognition computation kept on the device. No cloud inference, GPU or NPU is required.

The release is available at github.com/lokutor-ai/oido.

There is an important boundary to this launch: as of September 30, 2026, the engine's arithmetic and firmware transcripts are verified on the host and in Espressif's QEMU emulator. Physical-board speed measurements are still pending. The demo uses chip-exact transcripts, but it is not footage of a physical board running in real time.

That distinction matters when evaluating an embedded speech system. Correct recognition, fitting in memory and keeping up with a microphone are separate claims.

Speech recognition without a command list

A fixed-command recognizer can map a few phrases to device actions. Open-vocabulary recognition has a different job: turn an English sentence into text without requiring the developer to enumerate every possible sentence first.

Oído targets the second case on an ESP32-S3 N16R8: a 240 MHz dual-core Xtensa LX7 microcontroller with 16 MB flash and 8 MB octal PSRAM. Those memory requirements are part of the target, not an optional upgrade. This release does not mean the model fits on every ESP32 board.

The public engine runs NVIDIA's Conformer-CTC Small model, with two supplied weight formats:

int8: a 14.0 MB model, with the stronger published accuracy.
int4: an 8.3 MB model, with a quantization-aware fine-tune. The supplied partition layout leaves a 6 MB app partition for your own code.

The choice is a concrete tradeoff between recognition accuracy and flash space for the rest of the device.

What the Whisper comparison actually says

The repository reports lower word error rates than Whisper tiny.en on its LibriSpeech evaluation and on its noise-and-reverberation evaluation. That is an accuracy comparison, not proof that Oído has beaten Whisper in a measured physical-board speed test.

The evaluation scope also matters. Oído's chip-arithmetic rows use the full LibriSpeech test sets; the laptop baselines use 500 evenly spaced utterances per set, with the same text normalization. These are project-reported results, not an independent benchmark on identical hardware or an identical full-set workload.

The useful conclusion is narrower than "microcontrollers are better than laptops": a model and engine designed around this memory and compute budget can provide useful open-vocabulary recognition without a neural accelerator. Check the README's accuracy table and evaluation notes before carrying the comparison into your own product claims.

The engine is the embedded work

The model architecture is only part of fitting recognition onto this chip. The public implementation includes:

A log-mel front end and convolutional subsampling to 25 Hz.
A 16-layer Conformer encoder, followed by CTC decoding over 1,024 BPE tokens.
C kernels that use the ESP32-S3's PIE vector unit for int8 matrix operations, alongside int4 kernels.
Quantized relative-position attention and a lookup-table softmax.
Scheduling across both cores and tiled access to weights stored in flash.
A VAD/AGC segmenter that determines when an utterance is ready to recognize.

The flash-access strategy is worth looking at if you work on embedded inference. The engine tiles computation so each weight is streamed from flash once per 64 frames. On this target, moving weights is part of the workload; counting model operations alone does not describe it.

You can inspect the implementation in esp32/components/tinyasr, the ESP-IDF application in esp32/firmware, and the host build in esp32/host.

Try the same arithmetic on a laptop first

The host build lets you test recognition before wiring a board. From a checkout of the repository, the documented path is:

cd esp32/host
make
./tasr_cli ../../models/nemo8.tnm recording.wav

The recording must be a 16 kHz mono PCM16 WAV. For microphone input, the repository documents python live_demo.py, with Python dependencies including NumPy, SoundFile, SentencePiece and sounddevice.

The point of this host path is to test the firmware engine's arithmetic. It is not a laptop throughput benchmark that can substitute for board timing. The live demo includes ESP32 time estimates; estimates remain estimates.

For the hardware path, the README specifies an ESP32-S3-DevKitC-1 N16R8, an INMP441 I2S microphone and ESP-IDF v5.5. An SSD1306 OLED is optional. The repository includes flashing scripts and an emulator path, so readers can inspect and reproduce the steps rather than rely on a video alone.

What to test before putting it in a device

This release is English-only and works in utterance mode. Text appears after a pause and recognition computation, not word by word as someone speaks. It is therefore not a drop-in promise of streaming captions or instant turn-taking.

The current real-time factor is estimated from emulator instruction counts and assumptions about execution and memory stalls. Physical hardware is the next check.

For a device trial, I would start with:

Recognition on the actual microphone, enclosure and acoustic environment.
End-of-speech behavior, including false segmentation and the delay before text appears.
Sustained operation, power draw and latency alongside the rest of the firmware.
Crowded speech and reverberant rooms, which the README explicitly lists as hard cases.

Keeping recognition local removes the need to send audio to a cloud recognizer. It does not, by itself, establish the privacy behavior of a whole product; the rest of its firmware still determines what is stored or transmitted.

Code and weights have different licenses

The engine and firmware code are GPLv3. The int8 weights and tokenizer are CC-BY-4.0; the int4 weights are CC-BY-SA-4.0. The supported NVIDIA transducer weights are not bundled and have separate NVIDIA terms.

Read the repository's NOTICE and commercial licensing notes before shipping a device. Lokutor offers commercial licenses for the engine and firmware where the GPL terms do not fit the product. That is separate from the weight licenses.

Try Oído, inspect the engine and share reproducible board results. The next useful evidence is recognition and timing on real hardware, under the conditions your device will actually face.

Try the voices in your browser on Hugging Face. Lokutor 2.0 launches on Product Hunt October 13.

Voice agent pricing at real volumes: the monthly bill from 500 to 100,000 minutes

Daniel Varela — Sun, 27 Sep 2026 09:45:38 +0000

Per-minute list prices are the easiest numbers in voice AI to compare and the least useful. Nobody pays list price: you pay a monthly plan plus whatever your minutes spill over it, and the plan decides the real bill. So here is the real bill, at five monthly volumes, for two platforms whose pricing pages publish enough to do the math: ElevenLabs Agents and Lokutor.

This comparison is deliberate. Lokutor is our product, and the math favors it at every volume below. The numbers are not ours: every ElevenLabs figure below is from elevenlabs.io/pricing, read 27 September 2026. Do your own math before you believe anyone's, including this post.

The two plan tables

ElevenLabs Agents, monthly billing, from their pricing page on 27 September 2026:

Plan	$/month	Agent minutes included	Effective per minute	Additional minute
Free	$0	15	-	$0.080
Starter	$6	75	8.0c	$0.080
Creator	$22	275	8.0c	$0.080
Pro	$99	1,238	8.0c	$0.080
Scale	$299	3,738	8.0c	$0.080
Business	$990	12,375	8.0c	$0.080

One thing worth crediting: 8 cents a minute, flat, on every tier. No volume discount, but no games either. Burst pricing when you exceed your concurrent call limit is $0.160 a minute, and the LLM (Gemini 2.5 Flash in their estimator) bills separately from credits. Telephony is excluded.

Lokutor, from the pricing post published 23 September 2026:

Plan	$/month	Minutes included	Effective per minute	Overage
Free	$0	60	-	-
Starter	$19	1,000	1.9c	2c
Growth	$99	6,000	1.65c	1.8c
Business	$349	25,000	1.4c	1.5c

The Lokutor minute includes speech recognition, the language model, and the voice. Phone minutes count double inbound and quadruple outbound, because carriers charge more to call a mobile.

The monthly bill at real volumes

Cheapest legal combination of plan plus overage on each side, voice agent minutes only:

Minutes per month	ElevenLabs Agents	Lokutor	Ratio
500	$40 (Creator + 225 over)	$19 (Starter)	2.1x
2,000	$160 (Pro + 762 over)	$99 (Growth)	1.6x
10,000	$800 (Scale + 6,262 over)	$349 (Business)	2.3x
50,000	$4,000 (Business + 37,625 over)	$724 (Business + 25,000 over)	5.5x

At 100,000 minutes the ElevenLabs bill is $8,000 (Business plus 87,625 overage minutes). Lokutor's published overage covers up to one extra month of plan minutes, so 100,000 is volume-contract territory; the first 50,000 of it costs $724.

The gap widens with volume because the overage rates do all the work: 8 cents against 1.5.

What this math leaves out, honestly

Concurrency. ElevenLabs Business allows 40 concurrent calls, Lokutor Business 20. If your peak is 30 simultaneous calls, the cheaper minute does not help you.

Languages. ElevenLabs lists 32. Lokutor lists 9: Spanish, Catalan, Galician, Basque, Portuguese, English, French, Italian, German. The Iberian set is first-class, which is the point of the product, but 9 is fewer than 32.

Catalog and features. ElevenLabs offers thousands of library voices, dubbing, sound effects, and a music product. Lokutor offers 10 voices and voice cloning on upper plans. If your product depends on a specific ElevenLabs voice, a per-minute table is not the decision.

The LLM line. On ElevenLabs the language model bills from credits on top of the minute. Their estimator puts Gemini 2.5 Flash at $0.34 for 270 minutes, so it is small, but it is not zero. On Lokutor it is inside the minute.

Free tiers. ElevenLabs gives 15 agent minutes a month. Lokutor gives 60, no card. For evaluation, both are enough to hear your own use case, which is the only benchmark that matters.

Where the difference comes from

Lokutor's voice model is a few tens of millions of parameters and runs on CPUs; the category norm is models tens of times larger on GPUs, priced to match. The 1.4 to 2 cent minute is what the architecture costs to run, plus a business. Whether that trade is worth it depends on the section above, not on the headline number.

The latency side of the same architecture is in Voice agents on CPU vs the GPU incumbents, and the migration mechanics are in Migrate from ElevenLabs to a CPU voice API in about 20 lines. Pricing sources: ElevenLabs pricing, Lokutor pricing, both as of late September 2026. Try the free tier at app.lokutor.com.

Try the voices in your browser on Hugging Face. Lokutor 2.0 launches on Product Hunt October 13.

Migrate from ElevenLabs to a CPU voice API in about 20 lines

Daniel Varela — Sat, 26 Sep 2026 16:34:31 +0000

Moving text-to-speech from ElevenLabs to Lokutor is a small diff: same install, same one call, same play-through-speakers result. What changes is what sits behind the call: a few-tens-of-millions-parameter voice model running on commodity CPUs, and a per-minute price that includes more of the stack.

Scope note up front: this post covers the real-time path (TTS for apps and voice agents). If you use ElevenLabs for dubbing, sound effects, or its voice library breadth, this migration is not aimed at you. The broader comparison lives in ElevenLabs alternative for real-time voice agents.

The swap, in Python

Before, from the ElevenLabs quickstart (checked 26 September 2026):

from elevenlabs.client import ElevenLabs
from elevenlabs.play import play

elevenlabs = ElevenLabs(api_key=ELEVENLABS_API_KEY)

audio = elevenlabs.text_to_speech.convert(
    text="The first move is what sets everything in motion.",
    voice_id="JBFqnCBsd6RMkjVDRZzb",  # George
    model_id="eleven_v3",
    output_format="mp3_44100_128",
)

play(audio)

After, with pip install lokutor and a key from app.lokutor.com:

from lokutor import TTSClient, VoiceStyle, Language

client = TTSClient(api_key=LOKUTOR_API_KEY)

client.synthesize(
    text="The first move is what sets everything in motion.",
    voice=VoiceStyle.F1,          # Claire
    language=Language.ENGLISH,
)  # plays through the speakers by default

That is the whole migration for the common case. There is also a JavaScript SDK (@lokutor/sdk) with the same shape.

What changes under you

	ElevenLabs	Lokutor
Package	`pip install elevenlabs`	`pip install lokutor`
Voice selection	`voice_id` string, 3,000+ library voices and clones	`VoiceStyle` enum, 10 voices (F1 to F5, M1 to M5)
Model choice	`model_id`: eleven_v3, multilingual_v2, flash_v2.5	One production model; `steps` 1 to 32 as a quality dial
Output	MP3 or your chosen format	PCM16 stream, 44.1 kHz mono
Languages	32 on multilingual_v2	9: en, es, ca, gl, eu, pt, fr, it, de
Free tier	10k characters per month	60 minutes per month

Read both sides honestly: ElevenLabs gives you a much larger voice library and more than three times the languages. Lokutor gives you one streaming PCM path, one model to reason about, and latency measured in production: 139 ms median time to first audio, 211 ms p90, on a single CPU thread (full benchmark). The platform also supports zero-shot voice cloning from an eight-second reference.

The deliberate pairing, admitted again: every comparison in this post was chosen by us and favors us where it can. Bundles differ. Check both pricing pages before deciding.

The agent case, also about 20 lines

If your ElevenLabs usage is Conversational AI rather than plain TTS, the equivalent object is VoiceAgentClient. One WebSocket carries speech recognition, the language model, and the voice, with barge-in:

from lokutor import VoiceAgentClient, VoiceStyle, Language

client = VoiceAgentClient(
    api_key=LOKUTOR_API_KEY,
    prompt="You are the receptionist for a dental clinic. Keep answers short.",
    voice=VoiceStyle.F4,          # Maria
    language=Language.SPANISH,
)

client.on('transcription', lambda t: print('user:', t))
client.on('response', lambda t: print('agent:', t))

client.start_conversation()

Tool calling, mid-session prompt updates, and transcripts are constructor parameters and methods on the same client. The Python SDK reference has the full surface.

Price, from the public pages

ElevenLabs Agents lists 8 cents per minute beyond a plan's allocation, and low-latency TTS as low as 5 cents per minute on the $990-per-month Business tier (their pricing page, 25 September 2026). Lokutor is about 2 cents per minute with recognition, the language model, and the voice included, 1.4 cents on Business, with 60 free minutes a month (details in this pricing breakdown).

When not to migrate

You need 32 languages or thousands of library voices.
You rely on dubbing, sound effects, or the voice changer.
You have a specific cloned voice in ElevenLabs that your product depends on.

If none of those apply and your cost per minute is becoming a problem, get a free key at app.lokutor.com, run both providers on the same script, and keep whichever wins your traffic. Docs at docs.lokutor.com.

Try the voices in your browser on Hugging Face. Lokutor 2.0 launches on Product Hunt October 13.

ElevenLabs alternative for real-time voice agents: CPU-native, 2 cents a minute

Daniel Varela — Fri, 25 Sep 2026 18:28:24 +0000

ElevenLabs is the default answer for synthetic voices, and for good reason: a large voice library, strong voice cloning, dubbing, and a broad creative suite. If you are producing narration, dubbing video, or exploring voices, it is the right tool.

If what you are building is a real-time voice agent, the requirements change. You need streaming time to first audio that survives a phone call, a per-minute price that holds up at call volume, and deployment that does not depend on GPU availability. That is the case for evaluating Lokutor as an ElevenLabs alternative.

What ElevenLabs costs, from its pricing page

Read on 25 September 2026:

Free: $0, 10k credits per month
Starter: $6 per month, 30k credits
Scale: $299 per month, 1.8M credits
Business: $990 per month, 6M credits, with low-latency TTS as low as 5 cents per minute at that tier

ElevenLabs Agents lists 8 cents per minute beyond a plan's allocation (read 23 September 2026). Credits, bundles, and what one minute buys differ between providers, so treat these as list rates rather than a like-for-like benchmark.

What Lokutor does differently

One price of about 2 cents per minute with speech recognition, the language model, and the voice included (1.4 cents on the Business plan). The free plan includes 60 minutes a month. The voice model, Versa, is a few tens of millions of parameters and runs on commodity CPUs, which is where the price comes from.

Latency from production traffic, single CPU thread: 139 ms median streaming time to first audio, 211 ms p90. At the full agent level, caller-observed time to first audio is 385 to 442 ms in the best case and about 660 ms typical.

Deployment runs on x86 and Arm CPUs: hosted API, private VPC, on-premises, or edge. In a measured AMD Genoa capacity test, one node handled about 19 concurrent calls, roughly $22 of infrastructure per concurrent stream per month.

Languages: Spanish, Catalan, Galician, Basque, English, Portuguese, French, Italian, and German. Zero-shot voice cloning from an eight-second reference.

Where ElevenLabs still wins

Voice variety and marketplace breadth
Dubbing and creative tooling
Ecosystem and brand recognition

If your product is content production, stay put. If your product answers phone calls and the GPU bill is becoming a problem, test us.

Evaluate in an hour

Get a free API key at app.lokutor.com, 60 minutes included. Docs at docs.lokutor.com. Run it against your current provider on the same script, cold and warm, and keep whichever wins your traffic.

Try the voices in your browser on Hugging Face. Lokutor 2.0 launches on Product Hunt October 13.

Voice agents on CPU vs the GPU incumbents: latency, cost, and a deliberate comparison

Daniel Varela — Fri, 25 Sep 2026 18:28:20 +0000

Two numbers frame this post. On Lokutor, a production voice agent starts speaking in 139 ms at the median (TTS streaming time to first audio, one CPU thread), and a full minute of a running agent costs about 2 cents with speech recognition, the language model, and the voice included. On the GPU incumbents' own pricing pages, a comparable minute lists at 6 to 8 cents and covers only part of the stack.

We picked this pairing on purpose, and it favors us. Below are the numbers, the sources, and the places where the comparison breaks.

Latency, measured in production

Versa 2.0, our TTS model, on live customer-facing infrastructure, single CPU thread (c8g.2xlarge, Graviton4):

74 ms fastest streaming time to first audio
139 ms median TTFA
211 ms p90
0.26 to 0.36 real-time factor

At the full agent level (turn detection, language model, voice, transport), caller-observed time to first audio is 385 to 442 ms in the best case and about 660 ms typical. In three recorded demo sessions, the server-side figure from end of turn to first audio came out between 67 and 207 ms.

The GPU reference point, from our earlier benchmark: Kokoro 82M on a spot RTX 4090 generated a short phrase in 47 ms warm. That is faster than Versa, as a good GPU model should be. Two caveats keep it honest: the 47 ms is full-utterance generation in a setup that did not stream, and the same setup needed 14.9 seconds to load the model. A CPU does not beat a 4090 on raw warm inference. The question is which infrastructure you want to operate when traffic is bursty and capacity goes cold.

Cost, from public pricing pages

Each company's own pay-as-you-go rate, read from their pricing pages on 23 September 2026 (details and billing rules in this breakdown):

Platform	Published price per minute	What that covers
Lokutor	~2 cents (1.4 cents on Business)	Recognition, language model, and voice
Cartesia	6 cents	Call duration
Deepgram Voice Agent	7.5 cents	Standard tier, pay as you go
ElevenLabs Agents	8 cents	Per minute beyond a plan's allocation

Bundles differ, so this is not a like-for-like price benchmark. It is still the comparison buyers make. A Lokutor minute includes the model and the voice; the other rates cover part of the stack, so the gap widens once you add recognition and the language model, and it narrows or flips if you already operate your own models.

Capacity: in a measured AMD Genoa test, one node handled about 19 concurrent calls, equal to roughly $22 of infrastructure per concurrent stream per month. That was a capacity test, not our current production deployment, so treat it as a capacity result rather than a guarantee.

Quality is gated, not assumed: English WER fell from 6.51% to 3.20% between the previous Versa version and 2.0, mean WER across ten benchmarks is 11.21%, and English UTMOS is 3.14. WER from a recognizer is a proxy for intelligibility, not proof of quality; we use it to catch pronunciation regressions before latency work ships.

Where this comparison breaks

Warm GPU TTS wins on raw speed. If you already operate GPU infrastructure well and need maximum throughput per node, use it.
List prices move. Check the current pages before deciding.
Bundles differ: support, languages, telephony, compliance, and what one minute buys.
Our median is a production median on our traffic. Yours will differ.

The claim is narrow: interactive latency and production economics for voice agents, on commodity CPUs, with no accelerator in the serving path.

Try it

The free plan includes 60 minutes a month. Get an API key at app.lokutor.com and read the docs at docs.lokutor.com. Benchmark cold and warm paths, then send us the numbers. We would rather compare real serving conditions than trade screenshots of ideal runs.

Try the voices in your browser on Hugging Face. Lokutor 2.0 launches on Product Hunt October 13.

Three real sessions on a CPU voice agent API

Daniel Varela — Thu, 24 Sep 2026 23:08:46 +0000

Three recorded sessions against the Lokutor voice agent API. Speech-to-text, LLM, and voice all run on CPU. The millisecond figure on screen is measured server-side: first audio after end of turn detected.

1. Booking a dentist, in Spanish

Two agents on the same API: a caller and a clinic receptionist. The caller asks for a cleaning this week, gets offered Thursday 5:30 PM or Friday 10 AM, picks Thursday, asks the price (45 euros), and books. Latency shown per turn: 207 ms, 206 ms, 144 ms. 40 seconds.

2. One voice, four languages

Same agent, same voice, switching live between Spanish, Catalan, Galician, and Basque. Latency on screen: 204 ms in Spanish, 144 ms in Catalan, 67 ms in Galician. The API also handles English, Portuguese, French, Italian, and German. 31 seconds.

3. Two agents debating

Two Lokutor agents debate in Spanish whether AI will end humanity. Unscripted: each one hears the other and answers on its own. Humanity survives, for now. 36 seconds.

Run it yourself

Free API key at app.lokutor.com, docs at docs.lokutor.com.

Try the voices in your browser on Hugging Face. Lokutor 2.0 launches on Product Hunt October 13.

Voice Agents for 2¢ a Minute, Everything Included

Daniel Varela — Wed, 23 Sep 2026 03:02:46 +0000

From today, a Lokutor voice agent costs about 2¢ a minute, with speech recognition, the language model and the voice all included. On the Business plan it is 1.4¢. There is no separate fee for the model, no separate fee for the voice, and a free plan with 60 minutes a month to try it.

What 2¢ buys, next to what the category charges

Most voice-agent platforms publish a per-minute price that covers only part of the call. Here is each company's own pay-as-you-go rate, read from their pricing pages on 23 September 2026:

Platform	Published price per minute	What that covers
Lokutor	1.4 to 2¢	Everything: recognition, language model, voice
Cartesia	6¢	Call duration
Deepgram Voice Agent	7.5¢	Standard tier, pay as you go
ElevenLabs Agents	8¢	Per minute beyond a plan's allocation
Retell AI	5.5¢ + LLM + voice	Voice engine; model and TTS billed on top
Vapi	5¢ + providers	Platform fee; recognition, model and TTS billed on top

The whole Lokutor minute costs less than most platforms' fee before you have chosen a model.

The plans

Plan	Price	Minutes included	Per minute	Concurrent calls	Beyond the allowance
Free	$0	60	n/a	1	n/a
Starter	$19/mo	1,000	1.9¢	3	2¢
Growth	$99/mo	6,000	1.65¢	8	1.8¢
Business	$349/mo	25,000	1.4¢	20	1.5¢

Every plan also includes text-to-speech and speech-to-text API allowances. 12 million characters of synthesis on Growth works out at about $8 per million, where the premium voice APIs charge $30 to $100. Growth and Business add voice cloning and phone numbers.

Why it can be this cheap

Because the models are small. Our voice model, Versa, is a few tens of millions of parameters, not billions. Our turn-taking and noise-suppression models are smaller still. Most of the category serves models tens of times larger on scarce GPUs, and prices the minute to match. Ours are built to run on commodity hardware, which is also why the same stack can run on your own servers or on a device. This is not an introductory price that doubles next quarter; it is what the architecture costs to run, plus a business.

How you are billed, all of it

The minute is measured per second of connected session, and includes recognition, the language model and the voice. Unused minutes don't roll over.
Phone calls count as two minutes per minute on the line, to cover the carrier: 3.3¢ on Growth, 2.8¢ on Business. Numbers are $5 a month.
Web search is off unless you turn it on for an agent; each search counts as four minutes.
At your limit, calls stop until next month, with no surprise bill. Or turn on overage and keep going at your plan's rate, up to one extra month's minutes, billed with your next invoice.
Changing plan: upgrade any time and a new billing month starts that day, less credit for what's left of the old one. A smaller plan applies straight away.

That list is the entire bill. It is on the pricing page too, with an estimator that shows the maths for your volume.

Built in Spain, native in the languages others skip

Lokutor speaks nine languages: Spanish, Catalan, Galician, Basque, Portuguese, English, French, Italian and German. Catalan, Galician and Basque are first-class, not an afterthought: if you serve customers across Iberia, you don't have to choose a platform that only does Spanish. And if your audio can't leave your infrastructure, the whole pipeline runs on-premise in the EU.

Start free with 60 minutes, no card, or talk to us about volume.

Try the voices in your browser on Hugging Face. Lokutor 2.0 launches on Product Hunt October 13.

Near-GPU TTS latency, zero GPU: what voice agents actually need in production

Daniel Varela — Sun, 20 Sep 2026 16:08:42 +0000

Versa 2.0 has produced first audio in 74 ms on one CPU thread.

In our RTX 4090 test, Kokoro's first cold synthesis took 2.44 seconds, with 14.9 seconds of model load time.

That is the attractive comparison. It is also incomplete.

Once warm, Kokoro generated a short phrase in 47 ms. It was faster than Versa, as a good GPU model should be. But that 47 ms was full-utterance generation, not streaming time to first audio. Our Kokoro setup did not stream. Versa's 74 ms figure is its fastest streaming time to first audio, while its production median is 139 ms.

These numbers are not an apples-to-apples model race. They expose the production decision teams actually face:

optimize for the fastest warm benchmark on a GPU;
or get interactive latency on ordinary CPU infrastructure, without making a GPU part of the serving path.

For many voice agents, the second option is more useful.

The benchmark, with the conditions left in

Metric	Versa 2.0	Kokoro 82M
Hardware	c8g.2xlarge, Graviton4 CPU	RTX 4090 spot instance
Execution	Single CPU thread	GPU
Model size	~58M parameters	82M parameters
Cold model load	Already resident in production	14.9 s
Cold first synthesis	Streaming service path	2.44 s
Fastest observed latency	74 ms to first audio	47 ms, warm short phrase
Production / warm reference	139 ms median TTFA, 211 ms p90	47 ms short, 124 ms medium, 132 ms long
Real-time factor	0.26-0.36	0.02 warm
Streaming in tested setup	Yes	No

Versa figures are from live customer-facing infrastructure, serving calls at six flow-matching steps. Kokoro was run through our test harness on a spot RTX 4090 supplied through amics.ai, created by Alvaro Fragoso.

The comparison needs two warnings:

Versa reports streaming time to first audio. Kokoro reports the time to finish generating the full utterance because the tested setup did not stream.
The 74 ms and 2.44 s figures are deliberately the best Versa observation and the cold Kokoro observation. For steady-state throughput, warm Kokoro wins comfortably.

The point is not that a CPU beats a 4090. It does not. The point is that a voice product is more than its best warm inference number.

Cold starts are a product metric

A benchmark often assumes the model is loaded, the accelerator is allocated, kernels are warm, and traffic is steady. Production traffic does not always behave that way.

Voice agents are bursty. A campaign begins. A queue goes from zero to hundreds of calls. A worker is replaced. A region fails over. An autoscaler adds capacity. A low-traffic language has been idle.

In those moments, model load and first-request latency become user experience.

A 47 ms warm result can coexist with a 14.9 second load. Both are true. Only one appears in most benchmark headlines.

This changes how teams should test TTS:

cold path = provision + model load + first synthesis
warm path = request arrival + first audio + generation
recovery path = replacement worker + readiness + first successful request

Report all three. If a system scales to zero, cold latency belongs in the product SLO. If it never scales to zero, the cost of keeping it warm belongs in the infrastructure plan.

CPU inference changes that tradeoff. CPU capacity is easier to find, easier to autoscale, and does not require a separate accelerator pool. A team can keep latency low without reserving a GPU for every serving unit or designing around GPU availability.

Time to first audio matters more than time to finish

For a voice agent, users do not wait for the whole sentence to be synthesized. They hear the first chunk while the rest is still being produced.

That makes streaming time to first audio, or TTFA, the useful number. Full-utterance latency measures something else: how quickly a system can generate a completed file.

Both metrics matter, but for different products:

Voice agents: TTFA, p90/p99 TTFA, chunk cadence, interruption behavior
Audiobooks and batch generation: full-utterance latency and total throughput
High-volume outbound systems: concurrency per node and recovery time
On-device or private deployments: hardware requirements and memory footprint

A system that generates a sentence in 47 ms but cannot emit audio until the sentence is complete may still feel slower than a system that starts speaking in 139 ms and continues streaming.

That is why we report Versa's production median and tail, not only its best run:

74 ms fastest time to first audio
139 ms median
211 ms p90
0.26-0.36 real-time factor

At the complete voice-agent level, our measured caller-observed time to first audio is 385-442 ms in the best case and about 660 ms typically. The model number matters, but the caller experiences the whole pipeline.

What latency does a voice agent actually need?

Past a certain point, shaving another 20 or 30 ms from isolated TTS inference has less effect than fixing the rest of the turn.

The practical target is not "the lowest TTS number possible." It is:

Start audio quickly enough that the exchange feels responsive.
Keep p90 and p99 under control when traffic changes.
Stream continuously without audible gaps.
Leave room in the latency budget for turn detection, language-model output, transport, and telephony.
Recover without multi-second stalls when capacity changes.

Teams should benchmark the system in the state users will hit, not only a warmed-up notebook.

A useful test matrix looks like this:

Test	What to record
First request on a new worker	Provisioning, load, TTFA
Warm short reply	TTFA, full generation time
Warm long reply	TTFA, chunk cadence, RTF
Burst from idle	p50, p90, p99, failed requests
Sustained concurrency	Calls per node, tail latency, CPU/GPU utilization
Worker replacement	Time until healthy traffic resumes

Latency is not enough if speech quality breaks

Fast speech still has to be intelligible.

We use held-out text and transcribe synthesized output with the same recognizer used on genuine human recordings. This does not prove that synthetic speech is "better than humans." WER is a proxy for intelligibility, and a recognizer can have its own biases. It does give us a repeatable way to catch pronunciation regressions.

From the previous Versa version to Versa 2.0:

English WER fell from 6.51% to 3.20%.
Mean WER across ten benchmarks fell from 14.03% to 11.21%.
On the held-out German, Italian, Catalan, and Basque sets, synthesized speech had lower WER than the genuine human comparator when both were scored by the same recognizer.

Each language set contained roughly 60 to 200 samples. We also measure an English UTMOS score of 3.14. Versa supports zero-shot voice cloning from an eight-second reference.

The important part is not one quality number. It is that latency work should be gated by repeatable quality checks. Otherwise a faster checkpoint may simply be speaking less clearly.

The infrastructure question behind the benchmark

GPU TTS can be extremely fast. Kokoro's 0.02 warm RTF on the 4090 makes that clear.

The tradeoff is operational dependency:

accelerator availability by region;
warm capacity during quiet periods;
GPU-aware scheduling and autoscaling;
recovery when a worker disappears;
separate deployment paths for cloud, private VPC, edge, or on-prem environments.

Versa runs on x86 and Arm CPUs. In a measured AMD Genoa test, one node handled about 19 concurrent calls, equal to roughly $22 of infrastructure per concurrent stream per month. That test is not our current production deployment, so we treat it as a capacity result, not a production guarantee.

The hosted API follows the same idea. Public voice-agent rates range from $0.035 to $0.116 per minute depending on plan. For directional context, current public list rates for ElevenLabs and Deepgram are around $0.08 per minute. Feature bundles and billing units differ, so this is not a like-for-like price benchmark.

When CPU-native TTS is the better choice

Use a GPU model when maximum warm throughput is the main constraint and you already operate GPU infrastructure well.

CPU-native TTS is worth testing when:

you want one deployment model across cloud and private infrastructure;
traffic is bursty and cold capacity matters;
GPU availability or regional coverage is a constraint;
you need low-latency streaming without maintaining an accelerator fleet;
voice is part of the product, but GPU operations should not become part of the company.

That is the problem we built Versa 2.0 to solve: near-GPU interactive latency, on a single CPU thread, in a production streaming API.

Try the live demo at lokutor.com, or get a free API key at app.lokutor.com.

If you benchmark it, test both cold and warm paths. Send us the numbers. We would rather compare real serving conditions than trade screenshots of ideal runs.

Try the voices in your browser on Hugging Face. Lokutor 2.0 launches on Product Hunt October 13.

Edge Voice AI: Why Voice Interfaces Are Moving Off the Cloud

Daniel Varela — Fri, 18 Sep 2026 16:23:09 +0000

Every voice interface shipped in the last decade has made the same architectural bet: send audio to the cloud, run inference on a GPU cluster, send audio back. That bet made sense when voice AI lived inside a phone app or a smart speaker with a permanent Wi-Fi connection and a company willing to subsidize the inference cost. It stops making sense the moment voice becomes the interface for a robot, a wearable, or an industrial device sold once and used for years.

The lifetime cost problem

A cloud-dependent voice stack doesn't just cost money to build. It costs money for as long as the product exists. Sell a voice-enabled device once, and GPU-based cloud inference means paying a recurring bill for every unit, indefinitely. That math works for a subscription software product. It works poorly for hardware, where the sale happens once but the voice interface needs to keep working for years.

This is the core reason robotics and wearable companies are increasingly asking whether voice AI can run on the device, or on infrastructure they control, instead of a third party's GPU cloud.

Latency and connectivity aren't optional

Real-time conversation has a biological rhythm to it. Human turn-taking gaps average around 200 milliseconds. A round-trip to a cloud GPU cluster, especially over an unreliable connection such as a warehouse robot, a car, or a wearable outdoors, fights against that rhythm directly. Voice interfaces that depend on connectivity to function at all are also voice interfaces that stop working exactly when connectivity is worst, which for a physical device is often the moment it matters most.

Edge and on-device inference removes the network hop by design. It doesn't guarantee low latency on its own because the model still has to be fast, but it removes the one latency source a company has zero control over.

Privacy is becoming a distribution advantage, not a compliance checkbox

For consumer devices, "does my voice data leave this device" is an increasingly direct purchasing question. For enterprise and regulated buyers in healthcare, finance, and government, it's a procurement requirement, not a preference. The GDPR and the EU AI Act both push toward systems where data processing is auditable and, ideally, contained within the customer's own environment.

A voice AI stack that can run entirely inside a customer's perimeter, rather than routing audio through an external GPU cloud, turns compliance from a limitation into a sales advantage. That's the case for regulated buyers specifically: an in-perimeter, auditable voice stack shortens procurement instead of complicating it.

What has to be true for this to work

Edge voice AI isn't just "take the cloud model and shrink it." Autoregressive transformer models, the dominant architecture behind most cloud TTS and a lot of cloud STT, are expensive to run well without GPU acceleration. That's precisely why the cloud GPU pattern became the default. Making voice AI genuinely edge-viable means designing models differently from the start:

Non-autoregressive generation where possible, so inference is parallelizable instead of sequential
Small parameter counts by design, not as an afterthought quantization pass
Architectures like ConvNeXt blocks instead of full transformer attention stacks, trading some of the attention mechanism's overhead for efficiency at a fraction of the compute

This is why "runs on CPU" and "runs on the edge" tend to be the same underlying engineering problem: both require models built to be small and efficient from the ground up, not GPU-scale models with the GPU removed after the fact.

Where this goes next

The near-term path for edge voice AI looks like three overlapping waves: a developer API and platform running on CPU infrastructure today, private enterprise deployments that run entirely inside a customer's own environment, and eventually on-device SDKs for robots, wearables, and appliances that need voice without a permanent cloud dependency. Each step removes another layer of GPU dependency and another reason voice AI has to live in someone else's data center.

The industry's default architecture assumed infinite, cheap GPU capacity would always be available to whoever needed it. For a wearable that has to work for years on a fixed hardware budget, that assumption was never going to hold. The interesting engineering problem isn't "how do we get a bigger GPU," it's "how do we need one less."

Try the voices in your browser on Hugging Face. Lokutor 2.0 launches on Product Hunt October 13.