Six months ago I launched VidClean with faster-whisper running on a Railway CPU worker. It worked. Accuracy was decent for English, the cost was around $25/month, and users could get a transcription without signing up for anything.
Then NVIDIA released Parakeet TDT 0.6B v3 in August 2025 and I started seeing it in every ASR benchmark thread. The numbers looked almost too good: 6.34% average word error rate on the Open ASR Leaderboard, 3,333x realtime throughput on GPU, 25 European languages with automatic detection. All in a 600-million-parameter model.
I run a free tool that processes real user files. I care about accuracy, speed on CPU (no GPU budget), and a model that does not hallucinate silence into phantom words. So I ran the bakeoff.
Here is what I actually found, including the parts that were harder than expected.
The Setup
VidClean runs on Railway. The transcription worker is a single CPU replica, 2 vCPU, running in a 2GB container. No GPU. The whole backend costs about $55/month across six compute services plus Redis.
On CPU, "3,333x realtime" does not happen. That number is from GPU benchmarks. Parakeet on a 2-vCPU Railway worker is meaningfully faster than Whisper small on the same hardware, but the gap is nothing like the benchmark suggests. If you are reading this expecting sub-minute transcription on cheap CPU hardware, calibrate accordingly.
My previous stack: faster-whisper with the small model for production and large-v2 for quality-sensitive jobs. The tradeoff was painful. Small dropped noticeably on accented English, Spanish, and Portuguese. Large-v2 on CPU was too slow for production on longer files.
For Parakeet I use onnx-asr with an int8-quantized ONNX export of the v3 weights, loaded via ONNX Runtime. Not the NeMo toolkit. The int8 ONNX is significantly smaller than the full-precision checkpoint and stays within the 2GB container budget with careful memory management.
What I Tested
I ran 40 files through both stacks:
12 English podcast clips (various accents, 2-15 minutes each)
10 Spanish YouTube videos (Mexican, Spanish, Colombian accents)
8 Portuguese files (Brazilian Portuguese, mixed quality)
6 heavily accented English files (Indian, Filipino, French accents)
4 noisy files (background music, cafe noise, wind)
I measured word error rate against manual transcription on a subset, relative processing time, and hallucination behavior on silence.
Accuracy
English: Parakeet and Whisper small are close on clean native-accent English. Parakeet edges it slightly. On accented English the gap widens: Parakeet handles Indian and Filipino accents noticeably better.
Spanish: Parakeet is clearly better. Whisper small was trained predominantly on English and its Spanish performance drops on non-standard accents. Parakeet v3 was trained on 670,000+ hours of multilingual audio, and the improvement on Mexican and Colombian Spanish is audible when you compare outputs side by side.
Portuguese: Same story as Spanish, maybe more pronounced. Whisper small produces noticeably more errors on Brazilian Portuguese. One caveat from NVIDIA's own model card: their training data uses European Portuguese, so Brazilian Portuguese gets a slight disadvantage. In practice on everyday creator content this has not been a meaningful issue.
Noisy audio: Neither model is great here. Parakeet degrades more gracefully. Whisper hallucinates more on noisy silence.
The Hallucination Problem
This was the primary reason I wanted to switch.
Whisper has a well-documented issue where it hallucinates text during silence or low-signal audio. "Thank you for watching" repeated at the end of a clip. Random phrases inserted during pauses. For a transcription tool used by real people on real content, this is not acceptable.
Parakeet does not do this. On every silent segment I tested, Parakeet returns nothing. The Token-and-Duration Transducer architecture handles silence differently at a fundamental level.
I had a post-processing step in my Whisper pipeline that tried to detect and strip common hallucinations. I removed it entirely after switching to Parakeet.
The Hybrid Architecture
Parakeet covers 25 European languages. A user uploading audio in Arabic, Chinese, Japanese, or Hindi gets routed elsewhere.
The routing works like this: a Whisper pre-pass on the first 30 seconds of audio detects the language. Parakeet does not expose its own detected language externally, so Whisper handles detection for both paths. The detected language is checked against Parakeet's 25-language set. If it matches, Parakeet handles the full transcription. If not, Whisper small takes the job.
The 25 languages include all major EU official languages: English, Spanish, Portuguese, French, German, Italian, Polish, and others including Maltese. Norwegian is notably absent. Russian and Ukrainian are included. Arabic, Chinese, Hindi, and most Asian languages fall to Whisper.
There is also a kill switch: setting TRANSCRIBE_ENGINE=whisper as a Railway service variable routes every job to Whisper instantly, no redeploy required. This was the safety net that made the migration low-risk to ship.
The Hard Part: Fitting Parakeet in 2GB
This is the part most posts skip. Getting Parakeet to run reliably in a 2GB CPU container required real engineering work.
ONNX Runtime memory arena must be disabled. With the default arena enabled, steady-state RSS grows by roughly 520MB over the course of a session and eventually busts the container. The fix is one line:
pythonimport onnxruntime as rt
import onnx_asr
so = rt.SessionOptions()
so.intra_op_num_threads = 2 # matches 2-vCPU allocation
so.inter_op_num_threads = 1
so.enable_cpu_mem_arena = False # arena-on costs ~520MB RSS
model = onnx_asr.load_model(
"nemo-parakeet-tdt-0.6b-v3",
quantization="int8",
sess_options=so,
providers=["CPUExecutionProvider"],
)
Windowed transcription, not full-file loading. Parakeet's ONNX export rejects inputs longer than roughly 200 seconds due to relative positional attention table limits. Inference buffer memory also scales with window length: a 60-second window peaks at about 1.65GB, a 30-second window at about 1.16GB. I settled on 20-second windows after measuring that 30-second windows hit 1,804MB, which is 4MB over the 1,800MB budget for a 2GB container.
ffmpeg splits the input into overlapping windows before inference. Only one window is ever resident at a time. This is what makes 90-minute files possible in a 2GB container, not RAM headroom.
Additional memory containment: MALLOC_ARENA_MAX=2 caps glibc malloc arenas and OMP_NUM_THREADS=2 caps the OpenMP thread pool. Both are set in the Dockerfile and Procfile.
Weights baked into the Docker image: The model weights are downloaded at build time, not at container startup. A restart loads from local disk in seconds.
The Language Detection Tax
Every auto-language job pays a Whisper pre-pass on the first 30 seconds of audio, even when the file ends up going through Parakeet. This adds 1-2 seconds per job.
If the user explicitly specifies a language at upload time, the pre-pass is skipped entirely and the job routes directly.
One Feature That Only Works on Parakeet
Filler word removal (removing "um," "uh," "you know" from transcripts) only works on the Parakeet path. It reconstructs whole words from Parakeet's token stream. The Whisper path returns empty word boundaries, so filler removal falls back silently. Users on non-European languages who go through Whisper do not get filler word removal.
Results in Production
VidClean has processed over 1,000 transcription jobs since the migration.
Accuracy complaints dropped to near zero. With Whisper small I received occasional complaints about wrong words, particularly from Spanish and Portuguese users. Since switching I have not received a single accuracy complaint.
The hallucination problem is gone. No more phantom phrases appended to transcripts. The post-processing step is deleted.
Processing time improved on longer files. Parakeet is meaningfully faster than Whisper small on CPU. The improvement is most noticeable on files over 10 minutes. Exact numbers vary by audio complexity but the improvement is real.
The 2GB container holds. Hundreds of jobs in production with the arena disabled, windowed inference, and memory-containment env vars. No OOMs since the tuning landed.
Should You Use Parakeet on CPU?
If your users are on European languages and accuracy on non-English matters, yes. The accuracy improvement over Whisper small is real and noticeable on Spanish and Portuguese. The hallucination elimination is practically valuable.
The caveats:
Use onnx-asr and ONNX Runtime, not the NeMo toolkit (easier for CPU deployment)
Memory management is not optional: arena-off and windowed inference are requirements, not optimizations
The language detection pre-pass adds overhead on every auto-language job
Non-European languages still go through Whisper, so you need both stacks
Speed on CPU is better than Whisper small but nothing like the GPU benchmarks
The migration took a weekend, most of it on the memory constraints rather than the model swap itself. If you are running a tight CPU container, the 4MB-over-budget window tuning story is probably the most useful thing in this post.
Melvin Bucio is building VidClean, a free browser-based video and audio tool suite for content creators. No account required, no watermark, no per-minute API bills. vidclean.net
Top comments (0)