DEV Community

Cover image for Testing a speech app with no microphone, no network and no model
Mohsen Seyedkazemi Ardebili
Mohsen Seyedkazemi Ardebili

Posted on AI-assisted

Testing a speech app with no microphone, no network and no model

A speech app sounds like the worst possible thing to unit-test. It needs a microphone, a speech model that is hundreds of megabytes, sometimes a GPU, and for some features a webcam.

YazSes has none of those in its test suite. As of this week, pytest --collect-only finds 30,429 tests, and the whole suite runs offline on a normal CI runner with no audio device and no model download. A new contributor can clone, run two commands, and see it go green:

uv sync
uv run python -m pytest tests/
Enter fullscreen mode Exit fullscreen mode

Here's how, and the three tricks that keep it from lying.

The split: pure decisions, injected muscle

Almost every feature is built as two modules:

Pure (decides) Backend (does the heavy work)
meeting/segmenter.py meeting/silero_vad.py
gaze/zones.py gaze/mediapipe_backend.py
recimport/align.py recimport/diarizer.py

The pure side imports nothing heavy. It takes the heavy thing as a parameter. The meeting segmenter, which decides where one utterance ends and the next begins, doesn't know what a voice-activity detector is. It takes a function:

class UtteranceSegmenter:
    def __init__(self, ..., is_silent: Callable[[np.ndarray], bool], ...):
Enter fullscreen mode Exit fullscreen mode

In production, is_silent is a calibrated loudness check or Silero VAD. In a test, it's a lambda that returns whatever the test needs. The decision logic (drop leading silence, flush after enough trailing silence, never exceed the maximum length) gets tested exhaustively, in microseconds, with no audio at all.

Pure decision modules on the left take heavy backends on the right as parameters

The backend side is thin and does one thing: load the model and answer the question. It's tested separately, mostly for "what happens when the model isn't there".

Trick 1: heavy imports go inside functions

Optional backends are optional extras. Silero, for example, pulls in PyTorch. When I measured pip install yazses[silero] it resolved to 25 packages and about 3 GB, mostly the CUDA stack. Nobody should download that to get dictation.

So heavy imports live inside the function that needs them, and a missing extra degrades gracefully:

try:
    from silero_vad import load_silero_vad
except Exception as exc:
    log.warning("`silero` extra not installed (%s).", exc)
    return None
model = load_silero_vad(onnx=True)
Enter fullscreen mode Exit fullscreen mode

Rules like this rot unless something enforces them. There's now a static test (on main, in the next release) that parses every module a base install can import and fails if any of them imports an extra at module level.

Trick 2: make dependencies disappear before pytest starts

One CI leg runs on FreeBSD, where faster-whisper, ctranslate2 and friends have no wheels. Tests that touch the decoder should skip there, not fail.

Testing that locally used to mean a FreeBSD VM. Now there's a script:

uv run python scripts/simulate-missing-deps.py -- tests/ -q
Enter fullscreen mode Exit fullscreen mode

It makes those packages unimportable with a sys.meta_path finder installed through a generated sitecustomize, so the block is active before pytest collects a single file. It raises a real ModuleNotFoundError, which means pytest.importorskip and except ImportError branches actually run. Mocking an import inside a fixture is too late; by then something has usually imported it already.

Trick 3: fuzz the path to the keyboard

The most dangerous code in a dictation app is the text path between the speech model and the keyboard. Whatever comes out of there gets typed into someone's window. So that path, plus the config loader's promise that loading a config never raises, has coverage-guided fuzzing with Atheris:

python fuzz/fuzz_text_pipeline.py -atheris_runs=200000
Enter fullscreen mode Exit fullscreen mode

A crash there is treated as a real bug, and the crashing input becomes a regression test.

What this doesn't cover

Honestly: the real microphone, the real model and the real window. Mocked tests can prove the logic is right. They can't prove a particular Wayland compositor accepts the keystrokes, or that a particular Windows app decodes them. Two other posts in this series (2 and 5) are about exactly that gap. For those, nothing replaces a person on a real machine running yazses doctor.

What I took away

  • Put decisions where a fake can drive them. If testing a rule needs a model, the rule is in the wrong module.
  • "Optional" has to be enforced by a test, or the first convenient top-level import makes it mandatory.
  • Simulate a missing dependency before collection, not in a fixture.
  • Fuzz the code whose output is irreversible. For us, that's text going into a window.

If you're building anything on top of a big model (speech, vision, an LLM), the same split works: keep the model behind a function signature, and test everything around it without it.


YazSes is offline, open source (Apache-2.0) dictation: github.com/MSKazemi/yazses. The suite runs in a few minutes on a laptop, and the contributing guide explains the layout.

I wrote this with help from an AI assistant for drafting and editing; the architecture and the opinions are mine, and I checked every fact against the repository.

Top comments (0)