How We Compressed a Studio-Grade Speech Restoration Model to 78MB (And Made It Run on CPU)
If you have ever tried cleaning up voice recordings—whether from an old interview, a phone call, or a podcast recorded in an echoing room—you quickly discover the limitation of standard noise removal tools.
Standard noise suppressors (like Zoom's background filter or Audacity's noise gate) only subtract noise. If the audio is muffled, clipped, or bouncing off the walls, subtractive filters can't fix it. They can mute the background hiss, but your voice still sounds like it was recorded inside an empty bathroom through a walkie-talkie.
To actually repair damaged audio, you need speech restoration—an AI architecture that can remove room echo and actively reconstruct the missing clarity and high frequencies.
The problem? Most neural audio restorers are massive (400MB to 1GB+), require heavy dedicated GPUs, and are painful to set up.
We built Phonos to solve this: an ultra-compact, standalone CLI that brings high-fidelity speech restoration down to a 78 MB package that runs on budget laptops and ordinary CPUs.
What Phonos Actually Does
Phonos addresses four common audio headaches in one automated step:
- Removes Cavernous Room Echo (Dereverberation): If someone recorded in a tiled room, church, or large hall, Phonos removes the bouncing room reflections and brings the voice upfront, as if spoken directly into a studio microphone.
- Unmuffles Low-Quality Audio (Bandwidth Extension): Phone calls and vintage recordings cut off all the crisp upper frequencies, making voices sound hollow and muddy. Phonos reconstructs those missing vocal harmonics so the voice sounds open and natural.
- Repairs Distorted & Clipped Audio: When someone speaks too loud and the microphone peaks, the waveform flattens out into harsh static. Phonos smooths the distortion and rebuilds the natural voice curve.
- Eliminates Heavy Background Noise: Suppresses loud crowd babble, engine rumble, and street traffic without making the voice sound robotic.
How We Squeezed It Down: 440MB $\rightarrow$ 78MB
Phonos is built on top of the open-source VoiceFixer research by Haohe Liu et al. While the original architecture produces remarkable audio quality, it was never designed for low-resource deployment.
To make it lightweight enough for everyday use, we applied a Selective Mixed Precision Architecture:
- 4-Bit NormalFloat (NF4) for Latent Stages: The center bottleneck layers of the neural network contain dense, normally distributed weights. Compressing these specific layers to 4-bit cuts the storage footprint by over 75% with zero audible loss in vocal clarity.
- 8-Bit Float (FP8) for Sensitive Audio Output: Quantizing the neural vocoder (the part that turns AI math back into sound waves) to 4-bit causes metallic ringing. Keeping the vocoder and outer feature layers in 8-bit prevents distortion while keeping the file size tiny.
- High-Frequency Presence Saturation: To ensure voices sound bright and natural after compression, we built a lightweight harmonic exciter that subtly enriches upper vocal overtones without adding processing lag.
The final result: A 78.7 MB model file (an 82% size reduction) that uses less than 1 GB of VRAM on consumer graphics cards.
Yes, It Runs on Regular CPUs (No GPU Required)
One of the biggest friction points with modern AI models is requiring expensive Nvidia GPUs.
Because the quantized weights are so small, Phonos runs on a standard laptop CPU with zero dedicated GPU:
- RAM footprint: Under 450 MB of system memory.
- Speed: 1.1x real-time on CPU (a 10-second recording restores in roughly 9 seconds).
- Speed on low-end GPU (RTX 3050 Laptop): 11.8x real-time (restores a 10-second recording in under 1 second).
How to Use It
Phonos is packaged into a single, self-contained Python CLI script. You don't need complicated frameworks or server setups:
# 1. Clone the repository
git clone https://github.com/jaadu611/phonos
cd phonos
# 2. Install minimal requirements
pip install -r requirements.txt
# 3. Restore any audio file
python cli.py my_recording.wav -o restored_studio.wav
(On first run, it automatically streams the 78MB checkpoint from Hugging Face or GitHub Releases).
Try It Out
The complete code, weights, and before/after audio samples are open-source on GitHub:
- GitHub Repository: https://github.com/jaadu611/phonos
- Model Weights (Hugging Face): https://huggingface.co/jaadu611/phonos
- License: MIT
Top comments (0)