Every time I start a TTS fine-tuning project, I lose the first day to roughly the same set of tasks.
Not model training. Not hyperparameter tuning. Preprocessing audio. Figuring out which clips are actually clean enough to train on, trimming silence off the ends, converting everything to the right format, and writing a manifest file in whatever format the trainer wants this week.
None of it is hard work. It is just the same work, every single time, with a script I throw together that breaks in a slightly different way each time I write it.
Last time it died 3 hours into an overnight run. No way to resume. No log of what got processed and what didn't. Just a half-populated output folder and the knowledge that I had to start over from scratch.
So I built AudioTrove.
What it does
AudioTrove is a CLI tool and Python library that takes a folder of raw audio and produces a clean, training-ready dataset. The command looks like this:
pip install audiotrove
audiotrove curate ./recordings ./output --tts
Point it at your audio, point it at an output folder, add --tts to run the TTS curation pipeline. What comes out the other end:
- Clean 16kHz mono WAV files
-
filelist.txtin F5-TTS format (tab-separated: path, duration, text) -
metadata.csvin LJSpeech format (pipe-separated: filename, text, text) -
checkpoint.db, an SQLite database logging every processed file by content hash
The SQLite checkpoint is the part I care most about. If the run gets interrupted for any reason, you just re-run the same exact command. Files that were already processed get looked up by content hash and skipped right away. The run picks back up where it left off.
What the pipeline actually does, stage by stage
"Curation" is a vague word so here is what is actually happening.
Stage 1: Speech detection
Not all audio contains speech. A lot of it contains music, background noise, or silence that looks like audio at the file level. Silero VAD scores each clip for the proportion of frames that contain actual speech. Clips below the threshold get rejected here, before any of the other stages even run.
This is the most important filter in the whole pipeline. If a clip doesn't contain speech, nothing downstream will fix that.
Stage 2: Silence trimming
Leading and trailing silence gets stripped from each clip. The padding is configurable in frames rather than seconds, which matters when you're working with files at different sample rates and don't want duration estimates to drift.
Stage 3: SNR estimation
SNR, or signal-to-noise ratio, is a measure of how much louder the speech is compared to the background noise. A low SNR means the clip is too noisy to train on reliably, and you probably don't want it in your dataset.
AudioTrove estimates SNR in a VAD-aware way: the noise floor gets calculated from the non-speech frames, and the signal level from the speech frames. This gives a much cleaner separation than a naive global RMS ratio would, especially on clips with natural pauses in the speech where a global calculation would underestimate the noise floor. No scipy dependency anywhere in this, it's pure PyTorch.
Stage 4: Duration bounds
Duration gets checked after trimming, not before. So you're filtering on actual content duration and not raw file length. A 10-second file with 8 seconds of silence at the start will measure as roughly 2 seconds of content, not 10.
Stage 5: Export
Surviving clips get exported as 16kHz mono WAV. The sample rate is fixed because that's what Silero VAD expects internally and it's also the most common target rate for TTS trainers.
Stage 6: Manifest writing
Both LJSpeech and F5-TTS manifests get written atomically at the end of each batch. Every processed file gets written to checkpoint.db with its content hash so re-runs can skip it without needing to re-read the file at all.
Benchmarks
Tested on LibriSpeech dev-clean, which is 2,703 clips totalling about 5.4 hours of audio.
| Workers | Wall time | Real-time multiplier |
|---|---|---|
| 1 | ~52 min | 6.3x |
| 4 | ~33 min | 9.9x |
No GPU at any stage. The bottleneck is I/O and VAD inference, both of which are CPU-bound.
Worth noting: multi-worker scaling is more modest than you might expect. 4 workers gives about 1.57x wall-clock improvement rather than 4x, because manifest writes and checkpoint inserts are serialised in the main process to avoid corruption. This is a known limitation and SQLite WAL mode with write batching is the next thing I want to try.
Real use cases
Podcast to voice clone training data
Download a podcast with yt-dlp, split it into chapters, run AudioTrove on the output. You get clean per-sentence clips with a filelist already formatted for F5-TTS. The whole thing takes maybe 20 minutes on a laptop CPU for a typical hour-long episode.
Audiobook to custom TTS voice
Audiobooks are usually already clean and well-paced, which means VAD and SNR filters mostly pass everything through. AudioTrove's main value here is the manifest generation and the format conversion, which would otherwise take a couple of hours of scripting to get right.
Interview recordings to ASR fine-tuning data
Interview audio tends to be messy, with crosstalk, variable mic quality, and background noise. VAD filtering and SNR scoring automatically remove the worst clips. Add --tts-diarize to segment by speaker if you're dealing with a two-person conversation.
Lecture recordings to domain-specific ASR
Similar to interviews but usually one speaker and more consistent audio quality. The silence trimming is particularly useful here because lectures have long pauses that would otherwise inflate duration estimates.
Optional flags
# Add speaker diarization (requires a Hugging Face token for pyannote)
audiotrove curate ./audio ./output --tts --tts-diarize --tts-hf-token YOUR_TOKEN
# Add Whisper transcription for each clip
audiotrove curate ./audio ./output --tts --tts-transcribe
# Run with 4 parallel workers
audiotrove curate ./audio ./output --tts --workers 4
# Process FLAC files (default is WAV only)
audiotrove curate ./audio ./output --tts --extensions flac
# Segment long files at VAD boundaries before curation
audiotrove curate ./audio ./output --tts --segment
Using the Python API directly
AudioTrove exposes its internal components if you want to build custom pipelines rather than use the CLI.
from audiotrove import AudioDocument, AudioFilter, AudioTransformer
# Custom filter: reject clips where peak amplitude is below a threshold
class PeakFilter(AudioFilter):
name = "min_peak"
def __init__(self, min_peak=0.01):
self.min_peak = min_peak
def filter(self, doc: AudioDocument) -> bool:
return float(abs(doc.audio).max()) >= self.min_peak
# Custom transformer: normalize to a target peak amplitude
class NormalizeTransformer(AudioTransformer):
name = "normalize"
def transform(self, doc: AudioDocument) -> AudioDocument:
peak = abs(doc.audio).max()
if peak > 0:
doc.audio = doc.audio / peak * 0.95
return doc
Where it's still rough
I'd rather say this upfront than have you find out after running it on 10 hours of audio.
Multi-worker scaling is modest for the reasons described above. The per-filter rejection breakdown isn't surfaced in the CLI yet, so you see "Kept: N / Filtered: M" but not which specific filter rejected what. That breakdown is sitting in checkpoint.db and you can query it manually, but it should really just be in the summary output. Speaker consistency post-diarization is also unverified, diarization segments by speaker but doesn't check whether each clip actually contains only one speaker's audio. And there's no cloud storage support yet even though fsspec is already in the dependency tree.
All of these are open issues. If any of them sound interesting to work on, CONTRIBUTING.md has the setup guide.
Try it
pip install audiotrove
audiotrove curate ./your-audio ./output --tts --extensions flac
GitHub: https://github.com/onepizzateam/AudioTrove
If you've built something similar or have opinions on the SNR estimation approach or the checkpoint design (anything really), I'd really like to hear it.
Top comments (0)