Disclosure: This article is based on a benchmark conducted by the NeuralSound team. AI was used to help edit and structure the draft; the code, methodology, and technical claims were reviewed by the team before publication.
A music source-separation model can sound good and still receive a surprisingly poor score.
One common reason is not the model itself. It is timing. NeuralSound
A cloud-based AI vocal remover may add a short delay, trim a few samples, pad silence, or export a file that is slightly shorter than the original. To a listener, a 10–30 ms offset may sound almost irrelevant. To sample-by-sample metrics such as SI-SDR, that same offset can create a large penalty.
This tutorial explains how to detect and correct timing offsets before evaluating an AI stem splitter, vocal remover, acapella extractor, or background music remover.
The examples come from our public AI vocal-remover benchmark, where the same five songs were processed by NeuralSound, Moises, and Fadr.
Play the full benchmark with 35 audio samples
Why timing matters
Suppose the reference vocal is:
reference[n]
and the estimated vocal is identical except that it starts 1,000 samples later:
estimate[n] = reference[n - 1000]
At 44.1 kHz, 1,000 samples are only about 22.7 ms.
The audio may still sound almost identical. But a metric comparing reference[n] with estimate[n] sees different values at nearly every sample.
That creates a false conclusion:
“The separator is inaccurate.”
The more accurate conclusion is:
“The separator output and reference are not synchronized.”
What should be normalized before scoring?
A fair audio benchmark should make these conditions consistent:
- sample rate;
- channel layout;
- start time;
- duration;
- stem definition;
- output format;
- quality tier.
For our two-stem benchmark, every evaluated output was synchronized at 44.1 kHz before scoring. The reference vocal came from the dataset’s ground-truth vocal stem, and the instrumental reference was calculated from the mixture minus vocals.
Step 1: Resample and convert to mono
For a controlled experiment, every waveform should use the same sample rate.
from __future__ import annotations
from math import gcd
import numpy as np
from scipy import signal
def to_mono(audio: np.ndarray) -> np.ndarray:
"""Convert mono or channel-last audio to one float64 waveform."""
audio = np.asarray(audio, dtype=np.float64)
if audio.ndim == 1:
return audio
if audio.ndim == 2:
return audio.mean(axis=1)
raise ValueError("Expected mono audio or channel-last stereo audio")
def resample_audio(
audio: np.ndarray,
source_rate: int,
target_rate: int,
) -> np.ndarray:
"""Resample with a polyphase filter."""
if source_rate <= 0 or target_rate <= 0:
raise ValueError("Sample rates must be positive")
if source_rate == target_rate:
return np.asarray(audio, dtype=np.float64)
factor = gcd(source_rate, target_rate)
up = target_rate // factor
down = source_rate // factor
return signal.resample_poly(audio, up=up, down=down)
For a production benchmark, preserve the original stereo files separately. Mono conversion makes the timing example easier, but it does not evaluate spatial imaging or stereo artifacts.
Step 2: Estimate the lag with cross-correlation
Cross-correlation asks:
At what shift do these two signals match most strongly?
A practical implementation should limit the search window. Songs contain repeated beats and repeated sections, so searching across the entire duration can match the wrong chorus or drum pattern.
def standardize(audio: np.ndarray) -> np.ndarray:
"""Remove the mean and scale variance for correlation only."""
audio = np.asarray(audio, dtype=np.float64)
centered = audio - np.mean(audio)
scale = np.std(centered)
if scale < 1e-12:
raise ValueError("Cannot align a silent or near-silent waveform")
return centered / scale
def estimate_lag(
reference: np.ndarray,
estimate: np.ndarray,
sample_rate: int,
max_lag_seconds: float = 0.5,
) -> int:
"""
Return the sample shift to apply to the estimate.
Positive return value: shift estimate right.
Negative return value: shift estimate left.
"""
if sample_rate <= 0:
raise ValueError("sample_rate must be positive")
reference = standardize(reference)
estimate = standardize(estimate)
correlation = signal.correlate(
reference,
estimate,
mode="full",
method="fft",
)
lags = signal.correlation_lags(
reference.size,
estimate.size,
mode="full",
)
max_lag = int(round(max_lag_seconds * sample_rate))
allowed = np.abs(lags) <= max_lag
if not np.any(allowed):
raise ValueError("No candidate lags were available")
candidate_corr = correlation[allowed]
candidate_lags = lags[allowed]
best_index = int(np.argmax(candidate_corr))
return int(candidate_lags[best_index])
The function returns the shift that should be applied to the estimate.
A negative lag means the estimate is delayed and should be moved left.
A positive lag means it should be moved right.
Step 3: Apply the shift without changing content
Alignment should correct timing only.
It should not denoise, equalize, compress, or otherwise “improve” one product’s output.
def shift_to_reference(
estimate: np.ndarray,
lag_samples: int,
reference_length: int,
) -> np.ndarray:
"""Shift an estimate and return exactly reference_length samples."""
estimate = np.asarray(estimate, dtype=np.float64)
if reference_length <= 0:
raise ValueError("reference_length must be positive")
if lag_samples > 0:
shifted = np.pad(estimate, (lag_samples, 0))
elif lag_samples < 0:
advance = -lag_samples
shifted = estimate[advance:]
else:
shifted = estimate
if shifted.size < reference_length:
shifted = np.pad(
shifted,
(0, reference_length - shifted.size),
)
return shifted[:reference_length]
Now combine the steps:
def align_for_evaluation(
reference: np.ndarray,
estimate: np.ndarray,
sample_rate: int,
max_lag_seconds: float = 0.5,
) -> tuple[np.ndarray, int]:
"""Align estimate to reference and report the applied lag."""
reference = to_mono(reference)
estimate = to_mono(estimate)
lag = estimate_lag(
reference=reference,
estimate=estimate,
sample_rate=sample_rate,
max_lag_seconds=max_lag_seconds,
)
aligned = shift_to_reference(
estimate=estimate,
lag_samples=lag,
reference_length=reference.size,
)
return aligned, lag
Step 4: Verify the alignment
Never assume the largest correlation peak is correct.
Check:
- the lag in samples;
- the lag in milliseconds;
- the correlation before and after;
- a short waveform overlay;
- a listening test around transients.
def lag_in_milliseconds(
lag_samples: int,
sample_rate: int,
) -> float:
return 1000.0 * lag_samples / sample_rate
aligned, lag = align_for_evaluation(
reference=reference_vocal,
estimate=estimated_vocal,
sample_rate=44_100,
)
print("Applied lag:", lag, "samples")
print("Applied lag:", lag_in_milliseconds(lag, 44_100), "ms")
If the reported shift is unexpectedly large, do not continue automatically. Investigate whether:
- the wrong file was loaded;
- the stem starts at a different song section;
- the export was trimmed;
- there is a sample-rate mismatch;
- the signal contains long silence;
- repetitive music created a false correlation peak.
Why alignment can still fail
Repeated beats
A steady kick pattern can create several strong peaks. Restrict the lag window and use a section with distinctive transients.
Long silence
Silence contains little information. Skip silent intros or align on an active excerpt.
Different source content
If one tool removes part of the vocal or damages the transient structure, correlation may become less reliable.
Stereo phase differences
Averaging stereo channels can cancel information. For stereo evaluation, consider aligning channels separately or using a carefully chosen mono reference only for timing estimation.
Variable delay
A constant shift cannot correct clock drift, time stretching, or nonlinear resampling. If the offset changes through the song, inspect the sample rates and processing pipeline.
Where SI-SDR fits
After alignment, SI-SDR can answer the intended question:
How closely does the estimated stem match its reference after accounting for scale?
SI-SIR adds information about unwanted source leakage, while SI-SAR focuses on artifacts introduced by separation.
A strong music-separation benchmark should report several metrics and also provide playable audio. One overall number cannot explain missing cymbals, metallic textures, damaged reverb, or vocal residue.
Our five-song benchmark reported:
- NeuralSound: 15.80 dB overall SI-SDR;
- Moises: 14.47 dB;
- Fadr: 12.59 dB.
Those numbers are limited to the tested songs and configuration. They are not a universal ranking.
You can inspect the per-track audio and methodology in the NeuralSound vs Moises vs Fadr comparison.
A reproducibility checklist
Before comparing an AI vocal remover or stem separator, record:
- input file hash;
- sample rate and channel count;
- product and plan;
- separation mode;
- test date;
- output format;
- estimated lag;
- duration correction;
- metric implementation;
- failures and retries.
Also publish the raw results when possible.
NeuralSound AI Vocal Remover Benchmark 2026
NeuralSound vs Moises vs Fadr
A playable five-song comparison of AI vocal removal and two-stem music separation.
About NeuralSound
NeuralSound is an AI vocal remover, online music separator and multi-stem splitter available on the web, Android, iPhone and iPad. It can separate a mixed song or video into clean vocals, drums, bass, guitar, piano and other instrument stems, helping musicians, DJs, singers, producers and content creators work with individual parts of a recording.
Use NeuralSound to remove vocals from a song, create an instrumental or backing track, extract an acapella, split music into stems, reduce music behind a voice with the background music remover, or prepare practice tracks with the AI karaoke maker. NeuralSound also supports synchronized lyrics, pitch and tempo controls, stem mixing, audio/video input and downloadable separated tracks.
This repository publishes original benchmark evidence for people…
The live benchmark provides the playable inputs and outputs, while the GitHub repository contains the public project resources.
Product context without turning the article into an ad
This post focuses on evaluation rather than product claims.
For readers unfamiliar with the terms:
- an AI vocal remover separates vocals and instrumental;
- an AI music separator can additionally isolate drums, bass, guitar, piano, and other stems .
The engineering lesson applies to any provider:
Final takeaway
A few milliseconds can make a good model look bad.
Before trusting SI-SDR, SI-SIR, or SI-SAR:
- standardize the sample rate and channels;
- estimate a bounded timing offset;
- apply timing correction only;
- verify the shift visually and audibly;
- calculate multiple metrics;
- publish enough detail for others to reproduce the result.
That process creates a more honest music source-separation benchmark—and prevents timing errors from being mistaken for model errors.
References
- MUSDB18 dataset documentation
- SDR — Half-baked or Well Done?
- Interactive NeuralSound benchmark
Top comments (0)