Benchmarking Russian speech recognition on CPU: T-one, GigaAM-v3 and cloud models

Most speech-model benchmarks are in English. The Russian ones I found rarely say what hardware a number was taken on, which license the weights carry, or what broke during installation. Russian is also a harder target than the headline English models suggest: the Cyrillic alphabet, rich morphology, and everyday speech full of English tech terms that have to come out in the right script. If you ship anything for Russian speakers, you end up having to measure for yourself.

I needed a voice agent that understands Russian speech over the phone and answers out loud (the project is ru-voice-agent). I installed and measured everything I could get running: recognition, synthesis and turn detection. The results, scripts and license checks are in ru-speech-bench. This post covers recognition: what I got on a plain CPU with no GPU, and why I don't put WER in the table.

First, the constraints, because they determine how to read the numbers. The measurements were taken in September 2026 on two machines: an 8-vCPU VM with a Xeon E5-2695 v2 (CPU only, no AVX2) and a desktop with an RTX 3090 Ti on Windows 11. Numbers are comparable within a table row and its hardware, not across rows.

Why not WER

An honest WER needs a human-transcribed reference. I don't have one. The recordings are personal: voice messages from 7 seconds to two minutes of ordinary conversational speech with English terms, and they are not in the repo. So you won't find "accuracy" or "WER" for my measurements. Instead I compute disagreement: how many words you have to change to turn one engine's text into another's. It is word-level Levenshtein distance divided by the length of the reference, after lowercasing, replacing "ё" with "е" and stripping punctuation.

What that means in practice is covered in docs/methodology/stt-disagreement-and-streaming.md. In short, 20% disagreement does not mean every fifth word is wrong. It includes insertions, spelling variants and errors in the "reference" itself. The metric is good for comparing variants of one system with each other (greedy decoder vs beam search, 16 kHz vs 8 kHz) and bad as a ranking.

To compare many engines at once I use the mean distance to all the others, "distance to consensus". That has its own catch: the metric rewards resembling the majority, and as you'll see below, the majority in my set is Gemini models.

T-one: streaming on one core

T-one from T-Bank is a streaming CTC Conformer with roughly 71.7M parameters (per the model card), trained on 80,000+ hours, mostly telephone speech. Apache-2.0. It takes audio in 300 ms chunks and finds phrase boundaries itself.

Hardware results

On a 53-second clip, greedy decoder, best of four runs:

ONNX Runtime threads	Time	RTF	CPU time
1	16.39 s	0.309	16.4 s
2	11.39 s	0.215	22.2 s
4	12.78 s	0.241	41.0 s
8 (default)	21.80 s	0.411	105.1 s

By default the library takes all cores, and that is the worst option: slower, and more than six times as expensive in CPU time as a single thread. I kept one thread. The table is noisy: the VM shared its host with another container, and the host load average in some runs was 8.9-19.3. Source: stt/results_threads.json.

In streaming mode on 8 kHz clips, one thread spends 99-123 ms on average per 300 ms chunk, p95 104-175 ms. With the greedy decoder, none of the 1091 chunks exceeded 300 ms, so the stream doesn't fall behind.

Latency until a phrase appears is determined not by CPU speed but by how long the splitter waits for silence. The default is 20 frames of 30 ms, i.e. 0.6 s. Average lag came out at 1.05-1.09 s. Setting MIN_SILENCE_DURATION = 6 brings it down to 0.64-0.65 s, and on the two clips I checked no words were lost. At a value of 2 things break: words get chopped, and disagreement jumps from 20% to 46% and from 41.5% to 64.6%. Two clips is not much, and the repo says so.

You can run several lines at once with a shared model and shared language model in one process: 1, 2 and 4 lines stay real-time, p95 around 110 ms. I did not measure more than four lines.

What T-one can't do

The output alphabet is the 33 Russian letters and a space. No Latin letters, no digits, no punctuation. "Excel", "4417" or "12 500" come out as Cyrillic words, spelled as heard. The language model can't help, because those symbols are simply not in the vocabulary. Another unpleasant thing: the language model (kenlm) turns letter noise into real words, but sometimes the wrong ones. A smooth wrong phrase is harder to notice than visible garbage, and the gain from the model is only 1.4-2.3 points of disagreement.

Disagreement against GigaAM as the reference: 20-22% on a clean clip, 41.5-46.8% on a quiet recording of two people. Compression for a phone channel (G.711 mu-law) did not hurt quality, and 8 kHz was no worse than 16.

Installation, separately. Beam search is the default, and without kenlm from_hugging_face() fails, so you must explicitly ask for DecoderType.GREEDY. kenlm has no wheels on PyPI, and pypi-kenlm doesn't build on Python 3.12. I built it without root: the compiler was the ziglang wheel, headers came from a standalone Python from uv, I removed the cmake step from setup.py, and the compiler wrappers dropped the unsupported -Wl,--exclude-libs flag. If the greedy decoder is enough for you, none of this is needed. One more thing: the upstream demo server listens on 0.0.0.0:8080 with no authentication, so bind it to 127.0.0.1.

GigaAM-v3: more accurate, but not streaming

GigaAM-v3 from SberDevices is a family of models on a HuBERT-like encoder pretrained on roughly 700,000 hours of Russian speech. I used the sherpa-onnx export: RNN-T with punctuation and an int8 encoder, about 232 MB on disk. MIT license.

It is an offline model: text appears after the whole file or segment is processed. In return it adds punctuation and capital letters, and handles digits and Latin text.

On my seven recordings on the same VM, four threads, under parallel load: median 12.8 s per file for lengths of 7 to 130 s, range 3.8-26.6 s. On the desktop I saw roughly 40x faster than real time, but I didn't record whether that ran on CPU or GPU, so treat the figure as approximate.

The vendor-reported average WER over five domains is 8.4% for RNN-T, 9.2% for CTC and 25.1% for Whisper in the same table; T-one with a language model is at 19.4%. Those are model-card numbers and I did not reproduce them.

GigaAM's weak spot is Latin terms and brand names. It writes them phonetically in Cyrillic or mangles them. This is fixed with dictionary post-processing; an example is in stt/gigaam_words.example.json, hooked up in gigaam_serve.py.

Gotchas: input must be mono, 16-bit, 16 kHz (run ffmpeg first). A stereo recording of a conversation is better split by channel, recognized separately and merged by time; stt_one_shot.py --dialog does that. On the noisy recording of two people, GigaAM did noticeably better than T-one.

Cloud vs local: a small check

I ran the same seven recordings through Gemini models with audio input (via an API aggregator) and compared them with GigaAM. The full table and caveats are in stt/cloud-compare/SUMMARY.md. An excerpt, distance to consensus and median time per file:

Engine	Succeeded	Distance	Time
gemini-3.1-pro	5/7	7.0%	61 s
gemini-3.6-flash	7/7	7.5%	12 s
gemini-3.5-flash	7/7	8.2%	17 s
gemini-3.8-flash	7/7	8.4%	13 s
gemini-3-pro	7/7	8.5%	44 s
gemini-3.7-flash	6/7	8.6%	12.5 s
GigaAM-v3, local on CPU	7/7	9.4%	13 s

What you can and can't take from this. The spread is only 7.0-9.4%: on this material the engines mostly agree, and they differ in terms, verbal filler and small substitutions. GigaAM is the "farthest" from the group, but the group is mostly Gemini variants, so the metric is biased against it. The pro models were the most verbatim but 3-5 times slower and the least reliable: 3.1-pro had two timeouts at 180 seconds. On Latin terms 3.6-flash did best, but results are unstable from version to version. A strict prompt ("don't add or guess, mark anything unclear") gave no gain on 3.6-flash and slowed it from 12 to 58 seconds. All ~55 calls cost about $0.04 by the aggregator's balance, which I did not verify independently.

Whisper and dedicated recognition endpoints through the aggregator were unavailable or had no keys, so they are not in the comparison.

The rest of the repo

It didn't stop at recognition, since the agent also has to talk. Briefly; details are in the cards under docs/engines/:

• Kokoro-82M (Apache-2.0), Russian via espeak: RTF 0.76 on the Xeon, 0.2 on a 5950X, 0.05 on GPU; intelligible and lively, stress sometimes wrong.
• ESpeech-TTS-1 RL-V2 on the 3090 Ti: RTF 0.17-0.36, first audio chunk in 1.1-1.5 s, 6 of 6 phrases spoken verbatim, places stress itself.
• XTTS-v2: good streaming (first audio in 0.47-0.53 s), but the CPML license is non-commercial. F5-TTS with a Russian fine-tune: CC BY-NC 4.0.
• Silero VAD 6.2.1: 0.51-0.58 ms per 32 ms chunk, roughly 55-62x faster than real time.
• Smart Turn v3.2: about 171 ms per decision, but the default threshold over-triggered on my material.

I checked licenses against the official pages and put the check date in every card. The README's licenses section says commercial use is closed for XTTS-v2, F5-TTS, Fish open weights and Piper's ruslan voice, and that all four Russian Piper voices are fine-tuned from an English voice that is research-only, so commercial use there is unverified. This is not legal advice.

What to know before quoting these numbers

• Four clips for T-one and seven for the cloud comparison; the recordings are personal, mostly one speaker, and there is no reference.
• The splitter threshold was checked on two clips.
• Timings from the cloud run are noisy: runs of different engines overlapped in time on one machine and network.
• Before publication the scripts passed python -m compileall, but I didn't re-run most of them. The numbers come from the original runs, before cleanup. You will probably need to adjust paths and model names.
• No audio or weights are included; put your own recordings in ./refs/ and ./models/.

Reproducing it

pip install sherpa-onnx numpy
export GIGAAM_MODEL_DIR=./models/sherpa-onnx-nemo-transducer-punct-giga-am-v3-russian-2025-12-16
python stt/gigaam_serve.py clip.wav --timing        # mono, 16 kHz, 16-bit
python stt/stt_one_shot.py clip.ogg                 # any format, ffmpeg is called for you
python stt/cloud-compare/analyze.py raw stt/cloud-compare/timing.jsonl

If you have your own recordings and hardware, send a card for a new engine or numbers from another machine. The template and rules are in CONTRIBUTING.md: license with a link and date, hardware, command.

Repository: github.com/meshoklv/ru-speech-bench, MIT for code and docs; the engines keep their own licenses. Updates on Telegram at t.me/meshoklv. If this was useful, a star on GitHub helps others looking for the same thing find it.

Mike Meshok - github.com/meshoklv - meshoklv.github.io