๐ Excited to announce the official release of driftcheck-audio (v0.1.0) on PyPI!
If you train Speech-to-Text (ASR) models (Whisper, Conformer, Kaldi) or curate speech datasets, you've hit this silent evaluation killer:
๐ Label drift and misaligned timestamps.
Speech datasets pair audio recordings with timed text (SRT, WebVTT, Lhotse manifests, Audacity labels). When these timestamps slipโdue to frame-rate mismatches (25 vs 23.976 fps), edits, or sync errorsโmodels silently train on pure noise or the wrong words. No exception is raised; Word Error Rate (WER) degrades silently.
driftcheck-audio solves this: an open-source Python library and CLI that audits timed labels against speech audio with mathematical precisionโwithout needing a heavy speech model.
๐ What Does driftcheck-audio Check?
โฑ๏ธ Constant Offsets: Finds whether labels are early or late by milliseconds or seconds and provides the exact time-shift value to correct them.
๐ Steady Drift: Detects offsets that grow over time (e.g. video frame-rate mix-ups), reports the drift rate (ms/min), and calculates the stretch multiplier.
โก Sudden Jumps: Identifies mid-stream timing discontinuities caused by inserted ads, cut audio, or spliced takes.
๐๏ธ Unlabelled Speech & Phantom Labels: Quantifies speech omitted from labels and flags labels mapped over dead silence.
๐ฒ Null Permutation Testing: Compares every measurement against shuffled null labels to eliminate false positives and coincidental alignments.
๐ Real-Data Benchmarks (LibriSpeech dev-clean)
Validated across multi-speaker recordings from LibriSpeech dev-clean:
โ Undamaged Labels: 0 failures, median measured offset of 0.001 s (1 ms). โ Planted Offsets: 6/6 (100%) recovered across ยฑ0.2s to +2.0s shifts (median error 10โ39 ms). โ Drift Detection: 100% caught on 1.004 stretch with slope accuracy of 0.014 s/min. โ False Positive Rejection: 0/6 false alignments when evaluated against unrelated text. โ Test Coverage: 110 / 110 unit tests passing.
โก Lightweight & Fast (Zero Heavy ML Bloat)
Requires no PyTorch, no Whisper weights, and zero GPU compute. Built purely on speech energy envelopes, cross-correlation, and NumPy/SciPy/SoundFile for instant, sub-second execution on any CPU or CI/CD runner.
๐ฆ Quick Start & Links
Install directly from PyPI: pip install driftcheck-audio
Run an automated audit on your dataset in 1 command: driftcheck check talks/ --out report/
Supports SRT, WebVTT, CSV, Audacity .tsv, and Lhotse .jsonl.gz supervisions!
๐ PyPI: https://pypi.org/project/driftcheck-audio/
๐ป GitHub: https://github.com/BoobalanArjunan/Driftcheck
If you curate ASR datasets or build speech pipelines, give it a spin! โญ
Top comments (0)