Local Whisper transcription on a Windows PC: what hardware matters and how to check the output

We build ClapClip, a Windows app that runs Whisper models locally, so read this with that in mind. Most of the post is about checks, and they work the same if you run whisper.cpp from a terminal or use any other local Whisper setup.

Local speech-to-text sounds simple. Install something, drop in a file, read the text. In practice two questions decide whether it works for you. Is your hardware good enough for the model you need? Is the output good enough to use? You can answer both in about half an hour.

What actually runs

Most local transcription tools on Windows are built on OpenAI's Whisper model family. ClapClip runs it through whisper.cpp, a C/C++ port, and offers three quantized models:

Option in the app	Model	Use it for
Fast	Whisper small, 5-bit	Searchable drafts. Usable on a CPU. Most likely to miss names, numbers and quiet speech.
Balanced (default)	Whisper medium, 5-bit	Meetings, lectures and podcasts in a language you name.
Accurate	Whisper large-v3-turbo, 5-bit	Interviews you will quote, heavy accents, technical terms, non-English speech.

"5-bit quantized" means the weights are stored at lower precision. The files are smaller and inference is faster, with a small accuracy cost against full precision. Large-v3-turbo keeps the large-v3 encoder but has a much smaller decoder, so it runs far faster than the original large model.

What hardware matters

A GPU the runtime can use. whisper.cpp uses the GPU when its runtime supports your card and falls back to the CPU when it doesn't. For ClapClip we recommend a dedicated NVIDIA, AMD or Intel GPU for the fastest processing, on 64-bit Windows 10 or 11.

Memory for the bigger models. Bigger models need more RAM and video memory. Our app runs one large-model job at a time by default, so two jobs don't fight over memory. If you script whisper.cpp yourself, do the same: one big job per GPU, queue the rest.

Some disk. Our installer is about 2 GB, and the models live on your disk after setup.

Not the sample rate. Whisper works on 16 kHz mono audio internally, and everything gets resampled to that. A 96 kHz studio file has no accuracy advantage over a clean 16 kHz one. Mic placement and room noise matter far more.

How do you know which path you're on? Open Task Manager, go to Performance, pick your GPU, and start a transcription. If dedicated GPU memory climbs and one of the engine graphs moves, the GPU is doing the work. If the GPU sits idle and the CPU is pinned, you're on the CPU fallback. That is fine for the small model and slow for the others.

We won't give you minutes-per-hour figures. They depend on your card, drivers, model and audio. Measure on your own machine instead.

Step 1: Cut a hard five-minute excerpt

Don't test on your easiest file. Pick five minutes with the worst parts: names, numbers, crosstalk, the noisy stretch.

ffmpeg -ss 00:12:00 -t 300 -i meeting.mp4 -vn -ac 1 -ar 16000 excerpt.wav

-vn drops the video. -ac 1 -ar 16000 gives mono 16 kHz, which is what the model sees anyway.

Step 2: Type a reference transcript by hand

Write down what is really said in those five minutes. It's tedious. It's also the only way to get a number instead of a feeling.

Step 3: Run two models and score them

Transcribe the excerpt with the middle and the largest model. Note how long each run takes. Then compute word error rate (WER):

# pip install jiwer
import re
import jiwer

def words(path):
    text = open(path, encoding="utf-8").read().lower()
    return re.sub(r"[^\w\s']", " ", text)

ref = words("reference.txt")
for name in ["balanced.txt", "accurate.txt"]:
    print(name, round(jiwer.wer(ref, words(name)), 3))

The words() helper lower-cases and strips punctuation, so "Okay," and "okay" don't count as an error.

WER treats every word the same. A wrong "the" and a wrong client name each count as one. So do a second, targeted pass: list every name, number and negation in your reference and check each one in the output. A dropped "not" flips a sentence's meaning and barely moves WER.

If the larger model fixes the names and numbers you care about, the extra time is worth it. If both models get them right, take the faster one.

Step 4: Look for Whisper's known failure modes

• Text over silence. Whisper can invent sentences during long pauses or music. ClapClip runs a voice-activity detector before recognition to skip those parts. If your setup doesn't, read the silent stretches carefully.
• Repeated lines. The same phrase several times in a row is usually a decoding loop, not the speaker.
• Wrong language. Auto-detect decides from the first seconds of the file. A wrong guess ruins the whole transcript. Set the language when you know it.
• Timing drift. If you make subtitles, check a few cues near the end of a long file, not just the start.

Step 5: Check that "offline" really is offline

Installing an app or downloading models needs a connection. After that, a local tool should transcribe with no network at all. Test it: turn on airplane mode or disable the network adapter, run the excerpt again, and compare the output with your earlier run. For a closer look, keep Resource Monitor's Network tab open during a run and see which processes send traffic.

For confidential recordings this is the check that matters most, and it takes two minutes.

The short version

Start with the middle model. Move up when names, numbers or a non-English language go wrong in your excerpt test. Move down when you only need a searchable draft and the machine is slow. Check the GPU is actually in use, and check the network really is off.

If you'd rather use a desktop app than a terminal, ClapClip's offline transcription is free to download for Windows 10 and 11, with no ClapClip account needed: clapclip.net offline transcription. Run the same five-minute test on it before you trust it with a long file.

Written with AI help.