DEV Community: orca_forge

The Problem of Robotic Voice When Changing Speech Speed - From Phase Vocoder to WSOLA

orca_forge — Fri, 07 Aug 2026 01:03:57 +0000

While building a "Voice Design" app, I added a slider to adjust speech rate (speaking speed). The goal was simple: speed up or slow down the tempo without changing the pitch. This is a very common requirement.

librosa has a function called librosa.effects.time_stretch designed exactly for this. It’s a one-liner. That’s what I used at first. However, as soon as I moved the slider even slightly, a faint metallic ringing would appear in the output. The voice sounded slightly "robotic" and "echoey," becoming muffled. The original voice was natural, but the moment the tempo changed, the quality dropped.

This article is a record of how I discovered that the cause was phase blurring in the phase vocoder and how I resolved it by implementing WSOLA (Waveform Similarity Overlap-Add) from scratch using numpy.

Premise: Changing tempo while preserving pitch

If you simply drop or duplicate audio samples, the pitch will shift along with the playback speed (the "chipmunk effect" you get when fast-forwarding). Time stretching is the process of changing only the tempo while avoiding this pitch shift.

There are two main approaches to this:

Phase Vocoder: Converts the signal into the frequency domain using STFT, then stretches/compresses it by adjusting the phase advancement of each frequency bin. It operates in the frequency domain.
WSOLA: Cuts the waveform (time domain) into short frames and finds the best positions to overlap and add them so they connect smoothly. It operates in the time domain.

librosa's time_stretch uses the former: the phase vocoder.

The Symptom: Metallic ringing (Robotic voice)

A phase vocoder treats each frequency bin independently and updates the phase to the "intended" advancement amount. While mathematically sound, in real speech, the phase relationships between harmonic components (overtones) gradually fall apart. This is known as a loss of phase coherence.

To the human ear, this manifests as:

A metallic or electronic sound (often called "phasiness").
A faint reverberation or echo effect.
Blurry vowel cores, making the voice sound "synthetic."

While this is often unnoticeable in music or percussive material, human speech relies heavily on formants and harmonic structures. Because of this, phase blurring is heard very clearly as a "robotic voice." Furthermore, the effect worsens as the stretch ratio increases. The conclusion was that it was a poor match for a speech-rate slider.

The Solution: WSOLA in the time domain

WSOLA does not enter the frequency domain; it simply cuts and pastes the waveform. Instead of "recalculating" the phase, it uses cross-correlation to find the position where adjacent frames connect most naturally. Therefore, phase blurring does not occur by design.

The logic works like this:

On the output side, frames are arranged at fixed intervals (syn_hop).
From the input side, we take frames at an interval corresponding to the stretch rate (ana_hop = syn_hop × rate) at the "ideal" positions.
However, instead of the exact ideal position, we search within a range of ±tol to find the frame that most closely matches the continuation of the previously placed frame (creating a natural waveform continuity).
The found frame is then combined using a Hann window (overlap-add).

The heart of the implementation (_apply_speed) is this "coarse search for the similar position."

ana_hop = int(round(syn_hop * float(rate)))
win = np.hanning(frame).astype(np.float32)
...
# The "natural continuation" of the previous frame = nat
nat = xp[off + prev_ana + syn_hop: off + prev_ana + syn_hop + frame]
best_d, best = 0, -1e18
for d in range(-tol, tol + 1, 8):     # Coarse search within ±tol to align phase
    cand = xp[off + ideal + d: off + ideal + frame + d]
    sc = float(np.dot(cand, nat)) / (float(np.linalg.norm(cand)) + 1e-6)
    if sc > best:
        best, best_d = sc, d
a = ideal + best_d
seg = xp[off + a: off + a + frame]
y[syn:syn + frame] += seg * win
norm[syn:syn + frame] += win

Here, nat represents the "natural continuation of the previously placed frame." By shifting the candidate frame cand within the range of ±tol, we find the position best_d where the normalized dot product (cross-correlation) with nat is maximized. Essentially, we are stitching the waveform where the periods align, which guarantees phase continuity. Finally, we normalize by dividing by the window weight norm (a standard practice in overlap-add).

The parameters used were frame=1024, syn_hop=512, tol=512. I used a coarse search with a step of 8 samples (range(-tol, tol+1, 8)) instead of a sample-by-sample exhaustive search. Since speech rate adjustment requires real-time performance, this provides a good balance between audio quality and processing speed.

Trade-offs: It is not a silver bullet

Switching to WSOLA does not solve everything. There are material-dependent trade-offs in method selection.

WSOLA is strong for speech (single speaker). Since periodicity is clear, it is easy to find similar positions, avoiding phase blurring. This was the best choice for this specific use case.
On the other hand, for complex polyphonic music or material with many transients, a phase vocoder may be less prone to artifacts. Since WSOLA commits to a single "stitching position," when multiple periodicities are mixed, the similarity search can get lost, leading to rhythmic fluctuations or "doubling" effects.
At extreme stretch ratios, artifacts increase regardless of the method. If you stretch too much with WSOLA, repetitions of the same frame become noticeable.

It’s not a matter of "phase vocoder is bad and WSOLA is justice," but rather that WSOLA was better suited for speech, which has very clear pitch structures.

Pitfalls and Lessons Learned

Question a single line of a library. librosa.time_stretch was working perfectly. It wasn't a bug; the characteristic of the phase vocoder method (phase blurring) simply didn't suit speech. The key was distinguishing between "the function is broken" and "the method doesn't fit the purpose."
Be aware of "Time Domain vs. Frequency Domain" from the start. If you can pinpoint that metallic sound as being phase-derived, you can logically conclude that you should move from frequency-domain processing (phase vocoder) to time-domain processing (WSOLA).
A custom implementation can be very small. The core of WSOLA is just "finding the similarity position via cross-correlation and performing overlap-add," which can be written with just numpy. A side benefit was reducing an external dependency.
Don't forget normalization for the window and overlap. In overlap-add, the amplitude only becomes correct when you divide by the sum of the window weights (norm). If you skip this, the volume will fluctuate.

Summary

The "robotic voice/metallic ringing" in speech rate adjustment is actually phase blurring (phasiness) caused by the phase vocoder.
For materials with clear pitch like speech, WSOLA—which stitches waveforms at similar positions—is more natural than re-calculating phases with a phase vocoder.
The core of WSOLA is searching within $\pm$tol for the position that has the maximum cross-correlation with the "natural continuation" of the previous frame, and then performing overlap-add with a Hann window. This can be done in a few dozen lines of numpy.
It is not a universal solution. For polyphonic music or transient-heavy material, a phase vocoder can be advantageous; the correct approach is to choose the method based on the source material.
Before suspecting that a library function is "broken," check if the underlying algorithm (methodology) is appropriate for your specific use case.

Visualizing Anchor Distributions to Create a 'Voice Map'

orca_forge — Fri, 07 Aug 2026 00:32:49 +0000

When You Collect 70 Voice Conversion Anchors, the Next Problem Emerges

Now that you’ve gathered 70 voice-conversion anchors, a new worry creeps in: Does this library truly cover everything? Are you overloaded with low male voices and missing high, bright female ones? Are you duplicating similar voices over and over?

You can guarantee the quality of each anchor with the selection filters and audits we wrote about in another post. But spotting systemic gaps—whether the collection is skewed as a whole—isn’t something you can see by looking at individual items. What you need is an overview, a single map of the entire voice library.

This post shows how we visualized an anchor set with two diagrams so you can check coverage and bias at a glance. No model required—just numpy and matplotlib.

Two Spaces You Want to Visualize

Voice anchors live in two different “spaces” with distinct properties.

Semantic-axis space
The app lets users adjust voices along eight axes (0–100 %): age feel, gender, pitch, body type, huskiness, clarity, warmth, roughness. Placing each anchor on these eight axes reveals the coverage of the design space.
Speaker-embedding space
A 192-dimensional vector from campplus. Similar voices cluster together; dissimilar ones drift apart. This map shows the spread of raw voice quality.

The first space answers “Do we have enough choices along the axes the user can control?” The second answers “Where are the dense clusters and empty gaps in pure voice quality?” We decided to draw both on one page.

(A) Semantic-axis coverage: strip plots

For each of the eight axes we lay out every anchor’s value in a horizontal strip. One strip per axis, with tiny jitter added so points don’t overlap vertically, and a crimson vertical line marking the mean.

for i, key in enumerate(axes_keys):
    vals = np.array([sv[n][i] for n in names])
    y = np.full_like(vals, i, dtype=float) + (rng.random(len(vals)) - 0.5) * 0.5
    axA.scatter(vals, y, s=18, alpha=0.55, color="#3a76b4")
    axA.scatter(vals.mean(), i, s=90, marker="|", color="crimson", zorder=3)  # mean

Reading the plot is straightforward: if points stretch from 0 % to 100 % on an axis, coverage is good; if they’re clustered on one side, there’s a gap. For example, a gender axis that reaches both ends tells you you can pick anything from masculine to feminine. A mean far off-center or a missing cluster immediately shows where anchors are missing.

Turning the vague feeling “I still need more ___ voices” into a concrete empty stretch on the axis is the real value of this diagram.

(B) Speaker-embedding scatter plot: PCA to 2-D

The second diagram compresses the 192-D speaker embeddings into two dimensions. We use a plain PCA—subtract the mean, run SVD, and take the first two principal components—no fancy libraries required.

X = np.asarray(bank.embeddings, float)
Xc = X - X.mean(axis=0)
U, S, Vt = np.linalg.svd(Xc, full_matrices=False)
Z = Xc @ Vt[:2].T                               # project to top 2 PCs
ev = (S[:2] ** 2) / (S ** 2).sum() * 100        # explained variance [%]

Each point (anchor) is labeled with the speaker name and colored by the gender-axis value (blue = masculine / red = feminine). We’re overlaying a human-interpretable axis on top of an abstract embedding space.

gi = list(axes_keys).index("gender")
gcol = np.array([sv[n][gi] for n in names])
axB.scatter(Z[:, 0], Z[:, 1], c=gcol, cmap="coolwarm", s=40, ...)

Here you see dense clumps (duplicate voices) and empty patches (gaps in voice quality). The coloring also shows how the embedding’s main axes relate to gender. Axis labels include the explained variance (PC1 (xx%)) so you know how much of the original variance this 2-D slice captures.

Quiet Implementation Details That Matter

Nothing flashy, but small touches that turn a visualization into a usable tool:

Explicit Japanese font selection.
Matplotlib defaults to tofu (□) for Japanese. We tried Hiragino Sans and similar candidates, then placed the working font at the top of the script. Without readable labels, the map is half useless.
Labels pulled from a lookup table.
If anchor_sources.json exists, we display the speaker’s real name (spk24); otherwise we fall back to the raw spk id. The labeling work from another post pays off here—you can talk about distributions in names, not numbers.
Model-agnostic.
As long as you have the embeddings and slider values, you can draw these plots. No need to load the heavy Seed-VC model. Just numpy and matplotlib. Lightweight enough for CI or a quick local run, which quietly nudges the team to make visualization a habit.

Turning the Maps Into Decisions

A visualization isn’t the end goal; it’s a tool for deciding what to do next. From the two maps you can derive concrete actions:

Axis with a one-sided gap (A) → prioritize collecting voices in that direction (e.g., more elderly voices, more husky voices).
Dense clump (B) → duplicate voices; stop adding more in that region.
Sparse region (B) → a hole in voice-quality coverage; hunt for material to fill it.

When “I feel like we’re missing something” becomes “this exact stretch on this axis is empty,” collecting anchors stops being guesswork. Treating the voice library as a design target is the real win of visualization.

Summary

We visualized an anchor set in two spaces:
(A) coverage along eight semantic axes,
(B) a PCA scatter plot of 192-D speaker embeddings.
(A) Strip plots reveal empty stretches and mean shifts on each axis—quickly spotting missing voice types.
(B) A simple PCA (mean-centering + SVD) projects embeddings to 2-D; coloring by gender shows how the abstract space aligns with a human axis. Dense clusters and empty patches become visible.
Explicit Japanese fonts, label lookups, and model-agnostic code make the visualization actionable.
The purpose of the maps isn’t just to look—they drive decisions. Fill the empty axis stretches and sparse regions to guide your next anchor hunt.

22.05kHz vs 44.1kHz — What's the Difference Between 'True Broadband' and Upsampling?

orca_forge — Fri, 07 Aug 2026 00:02:22 +0000

When comparing outputs from voice conversion, have you ever wondered about this?

The export sampling rate is indeed 44.1kHz. It says so in the file properties. Yet, it lacks the "air" and clarity of CD quality. The reason for this is that the sampling rate number and the actual frequency bandwidth contained in the audio are two different things. This article is a record of the definitive difference between "true wideband" and upsampling, which I verified when switching from a 22.05kHz model to a 44.1kHz F0-conditioned model in the "Voice Canva" app.

Premise: Sampling Rate Only Determines the "Ceiling"

First, let's cover the basics. According to the Nyquist-Shannon sampling theorem, the upper limit of the frequency that can be represented in audio with a sampling rate fs is fs / 2.

22.05kHz → Limit: approx. 11kHz
44.1kHz → Limit: approx. 22kHz

The crucial point here is that setting it to 44.1kHz means the audio can represent up to 22kHz; it does not mean the content is filled up to 22kHz. It determines the size of the container, but it doesn't automatically increase the contents.

Audio created at 22.05kHz simply has no components above 11kHz. Even if you resample (upsample) this to 44.1kHz, the empty high-frequency range remains empty. Even if you double the size of the container, what wasn't there cannot be created. This is the point that often defies intuition.

22.05kHz Models and 44.1kHz Models Are Different Entities

In the "Voice Canva" inference service, the model is switched via environment variables.

# Use the 44.1kHz F0-conditioned model (true wideband output). Default OFF (conventional 22.05kHz).
F0_COND = os.environ.get("VOICE_CANVA_F0", "0") == "1"

The startup script for 44k mode looks like this:

export VOICE_CANVA_F0="${VOICE_CANVA_F0:-1}"                    # Use 44k F0-conditioned model
export VOICE_CANVA_OUTPUT_SR="${VOICE_CANVA_OUTPUT_SR:-44100}"  # Output also at 44.1kHz (no downsampling)
export VOICE_CANVA_AUTO_F0="${VOICE_CANVA_AUTO_F0:-1}"          # Align carrier F0 to target pitch range

This is the decisive part. VOICE_CANVA_F0=1 is not simply a setting to "increase the output file rate." It replaces the generation model itself with a different model trained at 44.1kHz.

Internally, the 22k and 44k models use different checkpoints and vocoders. The 44k model utilizes a BigVGAN vocoder trained at a high sampling rate, meaning that during the stage where the mel-spectrogram is converted back into a waveform, the model can actually generate bandwidth above 11kHz. In other words, instead of mechanically filling the empty high frequencies via resampling, it creates the content based on its learned knowledge.

The sr (sampling rate) also switches to follow the model:

sr = mel_fn_args["sampling_rate"]   # 44100 in f0 mode

The output rate is maintained at 44100 without downsampling. By maintaining a consistent path of "Generate at 44k → Output at 44k," we ensure that the hard-won high frequencies aren't discarded midway.

Measurement: Is There Actually Energy in the High Frequencies?

To prove that the high-frequency content actually increased, we need numbers. I analyzed the spectrum of audio generated from the same input and measured the percentage of total energy occupying the bandwidth above 11kHz.

The results were as follows:

Output generated by 44.1kHz model: Approx. 7.4% of total energy was in the >11kHz band.
22.05kHz output upsampled to 44.1kHz: The >11kHz band was nearly zero.

Despite both being 44.1kHz files, the content is completely different. The upsampled version has a container that reaches 22kHz, but everything above 11kHz is empty. Meanwhile, the 44k model output has actual energy. This 7.4% is the component that the ear perceives as the clarity of consonants, the "air" of a breath, and that general "CD quality" feel.

While these numbers might seem obvious in hindsight, when comparing by ear, it's easy to conclude that "this one just sounds clearer." It was only by looking at the energy per band in the spectrum that I could objectively distinguish between "high frequencies being created" vs. "not being there."

Why Upsampling Doesn't Create High Frequencies

To explain the logic a bit further: upsampling is a process of interpolating between existing sample points. Whether using linear interpolation or higher-quality low-pass interpolation, what's happening is "connecting the existing waveform smoothly." It does not fabricate new frequency components (in fact, a proper resampler actively removes everything above the limit to prevent fake high frequencies from appearing as aliasing noise).

Therefore, "converting 22.05kHz audio to 44.1kHz" and "generating audio at 44.1kHz" result in fundamentally different content, even if the final "container" is the same. The former is interpolation; the latter is generation. Information like high-frequency detail can only be created by a model that "knows" what it should sound like.

Pitfalls and Lessons Learned

Don't use the sampling rate number as proof of quality. "44.1kHz output" often becomes mere window dressing on a spec sheet; it doesn't guarantee the content is there. You can only claim it's "real" after measuring the energy per band.
44k models are heavy. The startup script explicitly mentions that "the model is large and generation is slow on CPU (GPU recommended)." Wideband isn't free; it's a trade-off for model size and inference cost. It's realistic to design a system that switches between 22k and 44k depending on the use case.
Any downsampling in the pipeline ruins everything. Even if you generate at 44k, if you drop it to 22k at any later stage, the high frequencies vanish. It was critical to explicitly set VOICE_CANVA_OUTPUT_SR=44100 to ensure consistency from generation to output.
Question the root cause of "muffled" sound. When audio sounds muffled, boosting the high end with an EQ is only a temporary fix. You can't boost components that aren't there. The true solution was to switch to a model capable of generating high frequencies.

Summary

Sampling rate only sets the upper limit of representable frequencies; it doesn't automatically fill the content.
Upsampling 22.05kHz audio to 44.1kHz does not create high frequencies above 11kHz (it is interpolation, not generation).
VOICE_CANVA_F0=1 is not just an output rate setting, but a switch to a separate model trained at 44.1kHz (including the BigVGAN vocoder).
Measurements showed that 44k model output has approx. 7.4% of total energy in the >11kHz band, while 22k upsampled output has nearly zero.
Wideband comes at the cost of model size and inference overhead. The key is maintaining 44k consistency from generation to output without intermediate downsampling.

Recording Spec-Compliant WAV Files (16-bit/Mono/Uncompressed) Using Only Web Audio

orca_forge — Thu, 06 Aug 2026 01:20:45 +0000

Introduction to Clean Audio Recording

When it comes to recording audio in the browser, the first thing that comes to mind is likely the MediaRecorder API. It's easy to use and only requires a few lines of code to start recording. However, when it comes to using the recorded audio for machine learning preprocessing, such as voice conversion or feature extraction, MediaRecorder becomes inconvenient.

MediaRecorder typically outputs webm/opus, which is a non-reversible compression format. While it's sufficient for human ears, the fine details of the audio are already lost during the preprocessing stage. Moreover, many downstream pipelines require a strictly specified WAV format, which is 16-bit PCM, mono, and uncompressed.

In this article, we'll introduce an implementation that uses Web Audio to record raw PCM and encodes it into a WAV file manually. Although it's a tedious task to write binary data one byte at a time, once it's done, you'll have a high-quality recording with zero degradation.

Background: Why Raw PCM?

The Web Audio API allows us to redirect the microphone input to an AudioContext graph and extract the raw samples (Float32, -1 to 1) at any point. Since we're not passing the audio through any compression codec, we can capture the original signal from the microphone. By encoding these Float32 samples into a 16-bit WAV file, we can create a file that meets the specifications without any degradation.

Our approach consists of two stages:

Recording: Collecting Float32 raw samples from the microphone (using Web Audio)
Encoding: Converting the collected Float32 samples into a 16-bit PCM WAV byte array

Step 1: Opening the Microphone with Clean Settings

We use getUserMedia to access the microphone, but it's essential to disable all browser audio processing. Echo cancellation, noise suppression, and automatic gain control are useful for voice calls, but they're unnecessary for recording and can even degrade the audio quality.

this.stream = await navigator.mediaDevices.getUserMedia({
  audio: { channelCount: 1, echoCancellation: false,
           noiseSuppression: false, autoGainControl: false },
});

We set channelCount to 1 to require mono audio. To achieve "clean recording" as specified, it's crucial to explicitly set these flags to false.

Step 2: Collecting Raw Samples with ScriptProcessor

We create a MediaStreamSource from the microphone and capture the samples flowing from it. Here, we use a ScriptProcessorNode.

this.ctx = new AudioContext();
this.sampleRate = this.ctx.sampleRate;       // usually 48000 (>= 44.1kHz)
this.source = this.ctx.createMediaStreamSource(this.stream);
this.node = this.ctx.createScriptProcessor(4096, 1, 1);  // buffer, in=1, out=1
this.chunks = [];
this.node.onaudioprocess = (e) => {
  // getChannelData uses an internal buffer, so we must copy and escape it
  this.chunks.push(new Float32Array(e.inputBuffer.getChannelData(0)));
};
this.source.connect(this.node);
this.node.connect(this.ctx.destination);     // required for some browsers to trigger

There are a few key points to note:

Copy the samples: The Float32Array returned by getChannelData(0) is an internal buffer that will be overwritten on the next callback. If we don't copy it using new Float32Array(...), the entire recording will be overwritten with the last frame.
Connect to destination: Some browsers require the ScriptProcessorNode to be connected to the ctx.destination to trigger the onaudioprocess event. Even if we don't need to play the audio, we connect it to ensure the event is triggered.
Sample rate is device-dependent: ctx.sampleRate is dependent on the environment, and most devices use 48000 (which is greater than or equal to 44.1kHz). We record the actual sample rate here and use it to write the correct header later. Using the actual recorded sample rate in the header is crucial for creating a compliant file without any resampling.

When stopping the recording, we concatenate the collected chunks into a single Float32Array, release the microphone and AudioContext, and then proceed to the next encoding step.

Step 3: Manually Building the 16-bit PCM WAV

Now, let's create the WAV byte array. The WAV format consists of a 44-byte header followed by the PCM data. We use an ArrayBuffer and DataView to write the fields one by one.

export function encodeWav(samples: Float32Array, sampleRate: number): Blob {
  const buffer = new ArrayBuffer(44 + samples.length * 2);  // 16bit = 2byte/sample
  const view = new DataView(buffer);
  const writeStr = (o: number, s: string) => {
    for (let i = 0; i < s.length; i++) view.setUint8(o + i, s.charCodeAt(i));
  };
  writeStr(0, 'RIFF');
  view.setUint32(4, 36 + samples.length * 2, true);  // file size - 8
  writeStr(8, 'WAVE');
  writeStr(12, 'fmt ');
  view.setUint32(16, 16, true);            // fmt chunk size
  view.setUint16(20, 1, true);             // format = 1 (PCM uncompressed)
  view.setUint16(22, 1, true);             // channel count = mono
  view.setUint32(24, sampleRate, true);    // sample rate
  view.setUint32(28, sampleRate * 2, true);// bytes per second = sample rate * block align
  view.setUint16(32, 2, true);             // block align = mono * 16bit/8
  view.setUint16(34, 16, true);            // bit depth = 16
  writeStr(36, 'data');
  view.setUint32(40, samples.length * 2, true);       // data length
  // ... body ...
}

The key points in the header are:

Format number 1 means PCM (uncompressed).
Multi-byte values are little-endian, so we write them with true as the third argument to DataView.
Block align and byte rate are calculated from the channel count and bit depth (for mono 16-bit, block align = 2, byte rate = sample rate * 2). By writing the actual sample rate here, we ensure that the WAV file plays at the correct speed.

The body of the WAV file is where we quantize the Float32 samples into 16-bit integers. Note that the scale is asymmetric:

let off = 44;
for (let i = 0; i < samples.length; i++) {
  const s = Math.max(-1, Math.min(1, samples[i]));         // clamp to range
  view.setInt16(off, s < 0 ? s * 0x8000 : s * 0x7fff, true); // asymmetric scale
  off += 2;
}
return new Blob([view], { type: 'audio/wav' });

The 16-bit signed range is asymmetric: -32768 to +32767. For negative values, we multiply by 0x8000 (32768), and for positive values, we multiply by 0x7fff (32767) to utilize the full scale correctly. Some implementations may multiply both by 0x7fff, but this would lose some dynamic range on the negative side. Although it's a minor detail, it's essential for creating a compliant file. Clamping the input to the -1 to 1 range before quantization also prevents overflow.

Finally, we create a Blob (with type: 'audio/wav') that can be used as needed, such as downloading it using URL.createObjectURL or sending it to a server via FormData.

Conclusion

To achieve clean audio recording for preprocessing, we must:

Use Web Audio to record raw PCM instead of MediaRecorder.
Disable all browser audio processing using getUserMedia.
Collect Float32 samples using ScriptProcessorNode and copy them to avoid overwriting.
Connect the ScriptProcessorNode to the destination to ensure the onaudioprocess event is triggered.
Use the actual recorded sample rate in the WAV header.
Build the WAV file manually using ArrayBuffer and DataView, considering little-endian and asymmetric quantization.

By following these steps, you'll be able to create high-quality, degradation-free audio recordings that meet the required specifications for machine learning preprocessing.

Identifying Speakers by Voice Quality Alone — Labeling Unknown Audio Sources

orca_forge — Thu, 06 Aug 2026 00:50:25 +0000

When you are growing an audio dataset, you will inevitably run into the "Who's voice is this again?" problem. The anchors (target speakers for voice conversion) in this app were no exception. As I collected more material, I found anchors that had lost their labels—they were just numbered, like spk7 through spk18.

Files had missing names, some were renamed halfway through, and the mapping between the raw source directories and the extracted anchors became disconnected. For various reasons, the result is the same: you end up with "voice data that exists, but whose identity is unknown."

If left alone, this kind of material tends to become "dead data"—ignored because the source is unclear. But the clue actually lies within the sound itself. Since we know the vocal characteristics, we can just match them by voice quality. This article is a record of how I re-labeled unknown anchors by relying solely on speaker embedding similarity and rebuilt the anchor_sources.json file.

The Idea: Matching by Voice, Not by Label

Here is the breakdown of what I wanted to achieve:

I have a large amount of raw source material with labels (wav files where the speaker's name is in the filename).
On the other hand, I have unlabeled anchors (spk7, etc.).
I want to match the two using voice quality to restore the correct names to the anchors.

This is essentially classical speaker verification. Fortunately, this app already uses campplus speaker embeddings for its anchor definitions. By measuring the cosine similarity within the same embedding space, I can determine "which raw source file is most similar to this anchor."

The Source Side: Creating Robust Speaker Vectors

First, I create a "gallery" to act as the comparison target. I extract one speaker vector from each raw source file. However, taking just a single 4-second window is unstable; if the window hits silence, a breath, or a stutter, the resulting vector won't represent that person's voice accurately.

Instead, I extract embeddings from 12 distributed windows across the entire file, L2-normalize them, and then average them. I discard silent chunks based on their amplitude.

def _robust_embed(campplus, y16):
    win = int(WIN_S * SR16); guard = int(5 * SR16)
    hi = len(y16) - guard - win
    embs = []
    for p in np.linspace(guard, hi, N_WIN).astype(int):   # N_WIN=12 windows
        seg = y16[p:p + win]
        if float(np.max(np.abs(seg))) < 1e-3:             # Discard silence
            continue
        e = _embed(campplus, seg)
        embs.append(e / (np.linalg.norm(e) + 1e-9))       # Normalize each window
    v = np.mean(embs, axis=0)
    return v / (np.linalg.norm(v) + 1e-9)                 # Re-normalize the mean

The key is the sequence: "Normalize, then average, then re-normalize the average." If you average without normalizing first, the result is pulled too heavily toward windows that happen to have a large norm. By converting them to unit vectors before averaging, you create a robust speaker vector that treats each window equally.

The Anchor Side: Reusing Existing Embeddings

For the anchors being compared, there is no need to calculate new embeddings. Since this project already saves all anchor embeddings in anchor_embeddings.npz via precompute_anchors.py, I just need to load and normalize them.

d = np.load(os.path.join(DATA, "anchor_embeddings.npz"), allow_pickle=True)
names = [str(x) for x in d["names"]]
embs = d["embeddings"].astype(np.float64)
emap = {n: embs[i] / (np.linalg.norm(embs[i]) + 1e-9) for i, n in enumerate(names)}

By using the same vectors used for generation to perform the estimation, I maintain consistency: "Selection, definition, and estimation all using the same ruler."

Matching: Top 3 Candidates and Confidence Levels

Finally, I calculate the dot product (which equals cosine similarity since they are normalized) between the target anchor vector and the entire gallery, then look at the top results. If the gallery is stored as a matrix G, a single G @ v operation gives the similarity scores for all source files.

sims = G @ emap[t]
order = np.argsort(sims)[::-1][:3]
cand = "  ".join(f"{glabels[i]}={sims[i]:.3f}" for i in order)
conf = "○" if sims[order[0]] >= 0.55 else ("△" if sims[order[0]] >= 0.45 else "×")
print(f"{t}: {conf} {cand}")

The important thing is not to declare the result based on "only the top result." I list the top 3 candidates and use the best score to categorize the confidence into three levels:

○ (≥0.55): Same level as the single-speaker threshold. Can be considered certain.
△ (0.45〜0.55): A candidate, but a gray zone where a human should decide by looking at the top 3.
× (<0.45): No match in the gallery. Either a different person or the source material is missing.

This threshold of 0.55 is the same as homo-thresh used to identify single speakers during anchor selection. I standardized the criteria for "closeness that can be called the same speaker" across both selection and labeling.

Why Not Just Pick "Number 1"?

When people hear "automation," they often want to immediately adopt the top result, but in speaker verification, that leads to errors. There are real people with very similar voice qualities (e.g., deep-voiced men, high-pitched bright women), and source material recorded under similar conditions can look similar. If the scores for 1st and 2nd place are close, it is a signal that "the machine cannot decide."

Therefore, I output the top 3 candidates with their scores and decided that anything marked as △ or below must be reviewed by a human. This isn't a stopgap; it is a division of labor. The machine handles the vast number of obvious ○ cases, and the human only looks at the small number of truly ambiguous ones. This is orders of magnitude more efficient than listening to every single clip.

Once an estimation is confirmed, I write it back to anchor_sources.json (the mapping table of Anchor Name $\to$ Raw Source Filename). Having this mapping makes everything downstream much easier. Quality audits (audit_anchor_quality.py) and distribution visualizations (plot_anchor_distribution.py) can now use this table to display spk24 by its actual speaker name. Data that could only be referred to by numbers has been transformed into an asset that can be discussed by name.

Pitfalls and Lessons Learned

Don't trust a single-window embedding. A speaker vector should be a normalized average of multiple windows. If you are skewed by a single window containing silence or a stutter, the same person will look like someone else.
Explicitly show confidence and hand gray zones to humans. Automatically adopting the #1 result will definitely cause issues with speakers of similar voice qualities. I created a division of labor between machine and human using "Top 3 + 3 levels of confidence."
Use the same embeddings and the same thresholds for estimation and definition. By using the same campplus and 0.55 threshold for both, I prevent the discrepancy where "the machine says they are similar, but they don't sound similar in production."
Labels can be recovered from the voice. Even if metadata is lost, the voice itself is primary information. Before discarding data because the source is unknown, it is worth questioning if you can match them by voice quality.

Summary

I restored labels for unknown anchors using nothing but cosine similarity of speaker embeddings.
I created noise-resistant speaker vectors by calculating the normalized average of 12 windows per source file.
I reused the existing anchor_embeddings.npz for the anchors, ensuring estimation and definition use the same "ruler."
I implemented a division of labor: Obvious cases are handled by the machine (○ ≥0.55), while ambiguous cases are sent to humans (△ ≥0.45).
By writing the restored labels back to anchor_sources.json, audits and visualizations can use names instead of numbers, turning dead data into a valuable asset.

In-Depth Explanation of the Seed-VC Architecture — Decomposing Voice into 'Who, What, and How' in a 4-Stage Structure

orca_forge — Thu, 06 Aug 2026 00:16:14 +0000

Understanding "Voice" — Breaking Down the Components

When trying to convert your recorded voice into someone else's, the first question that comes to mind is:

Even if the same words are spoken, different speakers produce different sounds. Even the same speaker produces different sounds depending on what they're saying. Adding intonation changes it further. In other words, speech is a signal composed of multiple independent pieces of information mixed together — at least: who is speaking (speaker identity), what is being said (content), and how it's being said (prosody).

The core challenge of voice conversion (VC) lies here. If you naively process speech to replace only the speaker identity, the content and intonation often get altered as well. In this article, we'll explain how Seed-VC, which we adopted for the backend of our "voice design" app, solves this problem — by separating and handling information using four modules, while walking through the actual model loading and inference code.

Design Philosophy: Delegate Separable Information to Dedicated Modules

At the heart of Seed-VC is the idea: "Don't make a monolithic model do everything." Instead, decompose the components of speech by type and assign each to a dedicated module, then recombine them at the end. The model loading process at startup looks like this:

model, semantic_fn, f0_fn, vocoder_fn, campplus_model, mel_fn, mel_fn_args = load_models(a)

While seven components are returned, they can be grouped into four functional layers:

whisper (semantic_fn) — Extracts content (what is being said) as semantic features
campplus (campplus_model) — Encodes speaker identity (who is speaking) into a single vector embedding
CFM/DiT (model.cfm) — A diffusion model that generates a mel-spectrogram conditioned on the above two
BigVGAN (vocoder_fn) — Converts the mel-spectrogram back into an audible waveform

Each role — "extract meaning", "extract speaker", "draw the spectrogram", "convert to sound" — is cleanly separated. This division of labor is why we can replace only the speaker identity while keeping everything else intact. Let's go through each step.

Step 1: whisper — Extract Only "What Is Being Said"

For content extraction, we use the encoder of the speech recognition model Whisper. We don't use the decoder (transcription). Instead, we take the intermediate representation (a sequence of semantic features) directly as a feature vector.

s_alt = _semantic(torchaudio.functional.resample(src_t, _S["sr"], 16000))

A key point: the input is resampled to 16kHz before processing. Since Whisper is designed for 16kHz audio, even if the main conversion model operates at 44.1kHz, we always downsample to 16kHz before semantic extraction. The resulting sequence contains little to no information about pitch or timbre — it represents a representation aligned with the spoken content. This is the prerequisite that allows us to freely replace speaker identity later.

Note: Whisper has a limitation — it can only process up to 30 seconds of audio at a time. For longer audio, chunking is required. This constraint itself was a major pitfall (see our other article: "The Culprit Behind the 'Slow Speech' Bug in Voice Conversion Was Whisper's 30-Second Limit").

Step 2: campplus — Condense "Who Is Speaking" into a 192-Dimensional Vector

Speaker identity extraction is handled by CAMPPlus, a speaker embedding model. It takes audio, computes fbank features, and outputs a single 192-dimensional vector.

feat = torchaudio.compliance.kaldi.fbank(w16, num_mel_bins=80, dither=0, sample_frequency=16000)
feat = feat - feat.mean(dim=0, keepdim=True)
return _S["campplus_model"](feat.unsqueeze(0)).squeeze().detach().cpu().numpy()

The fact that "speaker identity = a single vector" is crucial to our "voice design" app. Because vectors can be added, blended, or interpolated, we precompute embeddings from clean recordings of 18 speakers as "anchors", then blend them using weighted averages based on slider inputs.

w = design_weights(slider_values, bank, **kw)
emb = (w[:, None] * bank.embeddings).sum(axis=0)

We define sliders for interpretable axes like "age", "pitch", and "huskiness", compute weights w, and take a weighted average of anchor embeddings. This "design by blending voices" operation is only possible because speaker identity is modularized as a standalone vector. If speaker identity were entangled with content, blending would corrupt the spoken words.

Step 3: CFM/DiT — Use Diffusion to "Draw" the Mel-Spectrogram

Once we have the semantic features (content) and speaker embedding (identity), we proceed to synthesis. This is where the diffusion model (CFM: Conditional Flow Matching; implemented as DiT) comes in. It starts from random noise and gradually generates a mel-spectrogram conditioned on the inputs.

vt = _S["model"].cfm.inference(cat, torch.LongTensor([cat.size(1)]).to(dev),
                               mel2, style, None, STEPS, inference_cfg_rate=CFG)
vt = vt[:, :, mel2.size(-1):]

Looking at the arguments reveals how this stage integrates outputs from previous modules:

cat — Semantic features (content). Concatenation of prompt audio (pc) and target condition (cond)
style — Speaker embedding from Step 2 (= the blended design embedding)
mel2 — Mel-spectrogram of the prompt audio. After generation, we trim the first mel2.size(-1) frames to keep only the new content
STEPS / inference_cfg_rate — Number of diffusion steps and strength of classifier-free guidance

The model receives content (cat) and speaker identity (style) as separate arguments. It interprets them as instructions: "Draw this content in this speaker's voice." If you fix one and change the other, only the intended attribute changes. The separation design is reflected directly in the inference interface.

Handling Prosody: `length_regulator` and F0

"How it's being said" — prosody (tempo and pitch movement) is a bit different. It cannot be fully disentangled from content, so instead of a dedicated module, it's handled as a connection point between content and speaker identity.

First, tempo (duration). The semantic sequence from Whisper needs to be stretched to match the desired number of mel frames. This is handled by length_regulator.

cond, *_ = _S["model"].length_regulator(s_alt, ylens=torch.LongTensor([mel.size(2)]).to(dev),
                                        n_quantizers=3, f0=None)

By passing the target frame length (ylens), the semantic sequence is adjusted to the desired duration. Next, pitch (F0). In a 44.1kHz F0-conditional model, length_regulator can accept an f0 argument, enabling prosody control such as shifting the carrier voice's pitch range toward the target speaker's range.

lf[F0_alt > 1] = lf[F0_alt > 1] - m_alt + m_ori   # Shift carrier F0 to target median
shifted = torch.exp(lf)
cond, *_ = _S["model"].length_regulator(S_alt, ylens=tgt_len, n_quantizers=3, f0=shifted)

This is a simple operation: shifting in the log domain by the difference in medians. Prosody isn't treated as a separate module but as an injected parameter. This reflects a pragmatic compromise — attempting full disentanglement can lead to unnatural results.

Step 4: BigVGAN — Convert Mel Back to Audible Sound

The diffusion model generates a mel-spectrogram — a kind of "blueprint" of sound. The final step, converting it into a waveform humans can hear, is handled by the vocoder: BigVGAN.

wav = _S["vocoder_fn"](vt.float()).squeeze().detach().cpu().float().numpy().reshape(-1)

Though it's just one line, this step greatly impacts audio quality. Even with the same mel-spectrogram, different vocoders produce different sounds. Our 44.1kHz model can output "true broadband" because BigVGAN is trained at high sampling rates (see our other article: "22.05kHz vs 44.1kHz").

Pitfalls and Lessons Learned

Here are some key lessons from implementing and operating this four-layer architecture:

Sampling rates differ per layer. Semantic extraction is fixed at 16kHz, the conversion model operates at 22.05kHz or 44.1kHz, and speaker embeddings use 16kHz fbank. Misunderstanding which layer expects which rate silently degrades quality. Explicitly resampling at each stage is safest.
Speaker identity as a single vector unlocks applications. If speaker identity were embedded inside the diffusion model, we couldn't implement "voice blending". Modular separation isn't just about quality — it directly enhances product expressiveness.
Prosody cannot be fully separated — design accordingly. Tempo and F0 are tied to content, so treating them as conditional parameters injected via length_regulator is the pragmatic solution. Forcing full disentanglement often leads to unnatural artifacts.
Each layer is replaceable. The vocoder can be swapped between BigVGAN, HiFiGAN, or Vocos. The semantic extractor can be swapped between Whisper and CNHuBERT. The modular design grants extensibility — each stage can be upgraded independently.

Summary

Seed-VC's voice conversion uses a four-layer architecture: whisper (semantic) + campplus (speaker) + CFM/DiT (diffusion mel generation) + BigVGAN (vocoder)
The design philosophy is to decompose speech into "who, what, and how", assigning each to a dedicated module
By representing speaker identity as a single 192-dimensional vector, we enable blending multiple speakers — the core of our "voice design" app
The diffusion model receives content (cat) and speaker identity (style) as separate arguments — the separation design is reflected in the inference interface
Prosody (tempo and F0) is not fully disentangled; instead, it's injected as a conditional parameter via length_regulator
Each layer operates at different sampling rates and roles, and is replaceable. This modularity improves quality, expressiveness, and extensibility.

Focus on Root Cause Resolution Rather Than Quick Fixes: A Collection of Bug Investigation Case Studies

orca_forge — Wed, 05 Aug 2026 01:14:15 +0000

When you encounter a bug, the quickest fix is to "eliminate the symptoms." If an error occurs, wrap it in a try-catch and swallow it. If it breaks only with a specific value, avoid that value via hardcoding. If the precision is off, boost it with a heuristic keyword to fake the result. All of these seem to work temporarily.

However, these quick fixes will inevitably come back to bite you. Because the root cause remains alive, the same problem will resurface through a different entry point. Swallowed errors leak downstream in much more cryptic forms. Hardcoded conditions become landmines for the next developer making a change.

When working with AI coding agents (like Claude Code), this temptation actually intensifies. Agents can suggest "fixes that work for now" at high speed. This is precisely why it is effective to explicitly impose a principle on the agent: "Ban quick fixes; always strive for the root cause resolution." In this article, I will introduce a pattern for investigation—reaching the root cause without hiding the symptoms—using three bugs I actually encountered while developing a voice conversion app.

The Grand Principle: Eliminate the Root Cause, Not the Symptom

First, let me establish the decision-making criteria that run through this article. When a proposed fix is presented, ask yourself the following:

Are you eliminating the symptom or the root cause?
Will this fix also eliminate other symptoms derived from the same root cause?
Can you explain why the fix works in a single sentence?

The third point is particularly crucial. A fix that you cannot explain is usually just hiding a symptom. Saying "If we avoid this value, it won't crash" is not an explanation. Saying "It crashes because the assumption of [X] breaks when this value is provided; therefore, I made it so the assumption is always met" is an explanation.

Let's look at three real-world examples.

Case Study 1: Converted Audio "Speaks Slowly" — Proportionality Points to the Root Cause

The first symptom was that only the voice-converted audio would play back with an unnaturally stretched cadence. The input recording was at a normal speed, but the output sounded like a slow, drunken speech.

One could think of endless quick fixes. For example, applying time-stretching to the output to force it back to normal speed. However, that explains nothing about why it was slow in the first place.

What worked here was the observation of proportionality. The issue didn't occur with short audio files, only with long recordings. Moreover, the longer the input, the slower the output became. A 131-second recording resulted in playback over 4 times slower than normal—this clue, that the "issue worsens in proportion to length," pointed me directly to the location of the root cause.

If it were a sampling rate mismatch, the audio would be consistently slow by a fixed ratio, regardless of length. The same applies to a time-stretch bug. A proportional relationship where "the issue scales with length" only exists when a fixed-length segment is being stretched to fit the total duration.

The root cause was the "30-second limit" of the Whisper encoder used for feature extraction. When I passed a 131-second recording, it only retrieved the content for the first 30 seconds. Since that 30-second chunk was being stretched to 131 seconds, it became 131 ÷ 30 ≒ 4.4x slower. This matched my "over 4x slower" perception perfectly.

The solution was to split the audio into overlapping 30-second chunks, run each through Whisper, and concatenate the results. This wasn't a symptomatic time-stretch; it was a fix at the source—ensuring correct information is obtained during the feature extraction stage. I documented the technical details of this investigation in a separate article: "The culprit behind the 'low speech' bug in voice conversion was Whisper's 30-second limit."

The lesson here is simple: Proportionality is an arrow to the root cause. If you measure what the symptom scales with, you can mechanically narrow down the suspects.

Case Study 2: 502 Errors During Long ML Inference — Don't "Extend" Timeouts, "Change the Mechanism"

Next was a bug where attempting to generate long audio with a 44.1kHz wideband model resulted in a 502 error at the frontend. Short audio worked fine, but if generation took too long, it inevitably resulted in a 502.

The easiest quick fix is to set the timeout value to a massive number. However, this is a classic symptomatic treatment: tinkering with numbers without understanding why the connection is dropping. Even if you increase the number, it will just crash again once an input exceeds that new limit. You've just postponed the landmine.

By chasing the root cause, I discovered that when the Next.js server relayed requests to the inference backend, the internal fetch implementation (undici) had a default timeout. It was closing the connection because it couldn't wait for the long-running response. The 502 was the result of the proxy layer giving up while the upstream server was still alive.

This is where the decision path diverges. "Disabling the undici timeout" would technically work, but the more robust solution was to replace the proxy relay with Node's standard http/https and allow unlimited waiting for a response. Given the nature of long-running inference, the very premise of "cutting off after a certain time" was incompatible with this endpoint. Therefore, the fix was to change the implementation so that this assumption was removed—treating the root cause.

The lesson here is: When you feel the urge to tinker with "numbers" like timeouts or retry counts, stop and ask if this is just symptomatic treatment. In many cases, you shouldn't be adjusting the number; you should be questioning "why is the architecture designed such that this limit exists?"

Case Study 3: Massive Model Download Stalls — Don't "Ignore and Proceed," "Ensure Placement"

The third issue involved the process of fetching multi-gigabyte model weights from HuggingFace stalling halfway through. In environments with unstable networks, the download would simply stop silently and hang.

The temptation for a quick fix here was to "swallow the download failure and attempt to continue starting the app." However, if you attempt inference with incomplete model weights, you'll just encounter much more confusing errors later in the pipeline. Swallowing the error merely hides the problem; it doesn't solve it.

The root cause was that the standard downloader could not detect "stalling" (silently stopping); once it got stuck, it couldn't recover on its own. It didn't crash, and it didn't return an error; it just sat there silently. This meant there was nothing to "swallow"—no exception was being thrown in the first place.

The solution was to use a downloader that supports stall detection and resumption (using specific curl options) and ensure the artifacts are reliably placed in the HuggingFace cache directory. If no data flows for a certain period, the process treats it as a failure, interrupts, and resumes from where it left off. This guarantees that a complete file eventually lands in the cache, even on unstable networks.

The lesson here is: "Silent stalling" is more troublesome than "crashing with an error." You cannot swallow an exception that is never thrown. The correct approach is to provide a reliable acquisition mechanism and define "completion" as the moment the artifact is successfully and accurately placed.

The Common Pattern Found in These 3 Cases

While these are three different bugs, the pattern used to reach the root cause is the same:

Measure the "Effectiveness" of the Symptom — As in Case 1 ("proportional to length"), observe what the symptom scales with. Proportionality, boundaries, and reproduction conditions are direct arrows to the root cause.
Eliminate the "Likely Suspects" First — Like testing sampling rates or time-stretching, eliminate suspicious candidates based on observed facts. What remains points to the root cause.
Use "Can you explain why it works in one sentence?" as a Gatekeeper — If a fix cannot be explained, suspect it is merely hiding a symptom. In Case 2, "increasing the timeout" was not an explanation, so it failed the test.
Be Wary of Tinkering with Numbers and Swallowing Errors — Increasing timeout values (Case 2) or swallowing errors (Case 3) are classic signals of symptomatic treatment. If you reach for them, stop and think.

Practicality When Working with AI Agents

This principle is worth institutionalizing specifically when working with AI coding agents. Because agents can rapidly mass-produce quick fixes, if left unchecked, you will end up with a mountain of code that "works, but only hides the symptoms."

What is effective is to define a permanent instruction for the agent: "Prohibit quick fixes. Follow this sequence: Identify root cause $\rightarrow$ Appropriate technology selection $\rightarrow$ Propose design-level solution $\rightarrow$ Implementation." Then, when a proposal is made, the human must put it through the gate: "Is this the symptom or the root cause?" and "Can you explain why it works in one sentence?" Agent productivity and the discipline of root-cause resolution can coexist.

Summary

Quick fixes (swallowing errors with try-catch / escaping via hardcoding / faking with heuristics) preserve the root cause and will inevitably recur.
Evaluate fix proposals using two gates: "Is this eliminating the symptom or the root cause?" and "Can you explain why it works in one sentence?"
Case 1: The proportionality of the symptom pointed to the root cause (Whisper's 30-second limit). Proportionality is an arrow to the root cause.
Case 2: The 502 during long inference was solved not by increasing timeouts, but by changing to a relay mechanism that doesn't cut off by time.
Case 3: The stall in massive downloads was solved not by swallowing the error, but by using stall detection + resumption to ensure the artifact is placed reliably.
AI agents are prone to mass-producing quick fixes. Make "strive for root-cause resolution" a permanent instruction, and use humans to act as the gatekeepers.

Creating Perceivable Vibrato/Pitch Fluctuations

orca_forge — Wed, 05 Aug 2026 00:40:56 +0000

Adding a Vibrato Slider to the Voice Design App

We added a "pitch variation" slider to the "voice design" app to reduce the robotic feeling of the voice and add a natural vibrato. The implementation was straightforward, and with a sine wave, we could verify that the pitch was indeed fluctuating periodically. However, when we applied it to an actual voice, even with the slider set to maximum, the difference was barely audible to the ear. This is a typical problem in DSP, where the parameters are correct, but the effect is not perceivable.

This article discusses the mechanism of creating vibrato using delay modulation and the design of modulation depth to achieve a perceivable effect.

Premise: Why Delay Modulation Makes Pitch Fluctuate

There are several ways to fluctuate pitch, but here, we used a time-varying delay (fractional delay). This is the same principle as chorus and flanger effects.

The key point is that it's not the delay time itself, but the rate of change of the delay time that fluctuates the pitch. As the delay quantity increases over time, the playback position slowly moves backward, causing the pitch to decrease due to the Doppler effect. Conversely, if the delay quantity decreases, the pitch increases. By making the delay quantity oscillate with a sine wave, the pitch fluctuates up and down, creating a vibrato.

Mathematically, the instantaneous pitch change is roughly proportional to 1 - d(delay)/dt. Therefore, both the speed of the fluctuation (modulation frequency) and the depth of the fluctuation (maximum delay amount) determine the strength of the vibrato.

Implementation: Oscillating Delay Quantity with a Sine Wave

The implementation (_apply_pitch_variation) is as follows:

n = len(wav)
t = np.arange(n) / sr
rate_hz = 4.5
max_delay = (0.0030 * amt) * sr           # maximum ~3.0ms (100% for ± approximately 65 cents of vibrato)
delay = max_delay * (0.5 - 0.5 * np.cos(2 * np.pi * rate_hz * t))
idx = np.clip(np.arange(n) - delay, 0, n - 1)
i0 = np.floor(idx).astype(int)
frac = (idx - i0).astype(np.float32)
i1 = np.minimum(i0 + 1, n - 1)
return wav[i0] * (1 - frac) + wav[i1] * frac

Let's break it down:

rate_hz = 4.5 is the vibrato speed. Human natural vibrato is around 4-7Hz, so we placed it near the lower end. If it's too fast, it sounds like a tremolo, and if it's too slow, it sounds out of tune.
delay oscillates between 0 and max_delay with a sine wave. The delay quantity oscillates, so its derivative, the pitch, fluctuates up and down.
idx is the "current sample position". It shifts backward by the delay amount. Since the position is not an integer, it becomes a fractional delay.
The last three lines are linear interpolation. We interpolate between i0 (integer part) and i1 (next sample) with frac (fractional part). This interpolation is necessary to achieve fractional delay; without it, the sound becomes jittery and noisy.

amt (0-1) linearly affects max_delay. Currently, the setting is a maximum delay of ~3.0ms at 100% slider value.

Main Topic: Solving the "Inaudible" Problem with Depth

In the initial implementation, max_delay was 1.0ms. Verification with a sine wave was normal, but the effect was not perceivable with an actual voice.

The reason was simple: the modulation was too shallow. From the above relationship, the modulation depth was approximately ±24 cents. Since one half-tone equals 100 cents, 24 cents is less than a quarter of a half-tone. While it's noticeable with a sine wave, it's not perceivable with an actual voice that has overtones and formants. The cause of the "existent but not perceivable" problem was this shallow modulation.

We increased the maximum delay from 1.0ms to 3.0ms. This resulted in a vibrato depth of ± approximately 65 cents at 100% slider value, which was clearly perceivable. This change is also reflected in the git history:

fix: Pitch fluctuation is not perceivable. Increased modulation depth from 1.0ms to 3.0ms (100% for ± approximately 65 cents)
Sine wave verification showed the effect was present, but ±24 cents was too subtle and not perceivable.

Interestingly, even with increased depth, it doesn't sound out of tune. As long as the speed (4.5Hz) remains within the natural vibrato range, increasing the depth doesn't make it sound out of tune; instead, it's perceived as a "fluctuating expression." The roles are divided: depth (max_delay) makes the effect stronger, and speed (rate_hz) ensures it sounds natural.

Pitfalls and Lessons

Sine wave verification only shows if it's working, but it doesn't guarantee the effect is perceivable with actual audio material. DSP requires separate evaluations for "operation correctness" and "perceivable effect".
Perception is non-linear. In terms of the perceived pitch scale (cents), ±24 cents and ±65 cents differ by a factor of roughly 2.7, but the perceived difference is qualitative, changing from "imperceptible" to "clearly audible". Parameters should be designed based on perceived quantities, not physical quantities.
Separate the roles of depth and speed. Effect strength is determined by depth (max_delay), while naturalness is ensured by speed (rate_hz). Adjusting both simultaneously makes it difficult to distinguish between their effects.
Fractional delay requires linear interpolation. Rounding to the nearest sample introduces jitter and aliasing, making the sound noisy and ruining the vibrato effect.

Summary

Vibrato, or pitch fluctuation, can be created using time-varying delay (fractional delay). The pitch is fluctuated not by the delay itself, but by the rate of change of the delay.
The speed (rate_hz) should be within the natural vibrato range (4-7Hz), and the delay quantity should oscillate with a sine wave. Reading is done using linear interpolation for fractional delay.
The main cause of the "existent but not perceivable" problem was insufficient modulation depth. ±24 cents was too subtle, while ±65 cents was clearly perceivable.
Increasing depth doesn't make it sound out of tune if the speed remains within the natural range. Depth makes the effect stronger, and speed ensures naturalness.
Verifying operation with a sine wave and verifying perception with actual audio material are distinct. Parameters should be designed based on perceived quantities (cents), not physical quantities.

Using Multiple AI Agents to Review UI Fidelity to Custom Designs

orca_forge — Wed, 05 Aug 2026 00:10:34 +0000

Have you ever been hit by a wave of anxiety after implementing a UI based on a mockup? You might think, "I tried to build it as faithfully as possible, but is it really matching the design?" As the person doing the implementation, it is easy to overlook small compromises made during coding or details missed in the mockup. This is the inherent limitation of self-reviewing your own code.

When humans work together, having multiple people involved increases the number of perspectives. My approach—which I introduce in this article—is to replicate this by running multiple AI agents. By having several independent review agents evaluate how faithfully your implementation matches the mockup (in terms of layout structure, positioning, and component mapping), you can reinforce the single review with multi-angled verification.

Why "One" is Not Enough

Self-reviewing by the original implementer has structural weaknesses:

Confirmation Bias — Since you already have the memory of "how you built it," you tend to view the mockup in a way that justifies your own implementation.
Reproduction of Oversights — If you missed a specific element in the mockup during implementation, you are likely to miss it again during review for the same reasons.
Fixed Perspectives — Being alone can lead to cognitive bias; for example, you might focus too much on "are the colors correct?" while overlooking the "hierarchical structure of the layout."

Even if you ask a single AI agent to "compare the mockup and the implementation," you will encounter similar issues, albeit to a lesser degree. A single response tends to lean toward one perspective; if you ask it to look at too many things at once, each individual check becomes shallow.

The solution is to split the perspectives and assign them to multiple agents. Instead of asking one agent to look at everything, assign each agent a specialized role and have them run independently.

Workflow: Parallel Review via Divided Perspectives

1. Define the Review "Axes"

First, break down the criteria for evaluating mockup fidelity. For UI design adherence, you might divide it like this:

Layout Structure — Does the overall grid, column configuration, and area division match? (e.g., the skeleton: a 2-column layout where the right column is further split vertically).
Element Placement and Hierarchy — Which elements are placed in which area, and in what order/nesting?
Component Mapping — Are all the parts from the mockup (sliders, charts, previews, lists, etc.) present in the implementation without excess or deficiency?
Visual Tone — The visual quality, such as color, spacing, and typography.

The key is to explicitly separate these axes. Because each agent focuses solely on its assigned axis, it can perform a deep dive.

2. Assign Independent Agents to Each Axis

Launch one specialized review agent for each axis. While you provide the same materials to everyone—the mockup (image or specs) and the implementation (the relevant component code or the actual rendered screen)—the instructions (prompts) must be specialized for the assigned axis.

For example, for the Layout Structure agent, you might instruct: "Ignore colors and text. Only compare the structural division of areas between the mockup and the implementation and list any discrepancies." This forces the agent to focus strictly on the skeleton without getting distracted by other details.

Crucially, you must run the agents independently. If they can see each other's conclusions, they will be influenced by one another, defeating the purpose of having independent perspectives. This is the same logic used in human code reviews: writing down your own comments before looking at what others have said.

3. Reconcile and Integrate Findings

A human (or an integration agent) then reconciles the feedback provided by each agent. You should look for three main types of findings:

Findings reported by multiple agents independently — High reliability. These should be prioritized for fixing.
Findings reported by only one agent — This is either a caught oversight or a deep insight unique to that specific axis. Evaluate these carefully.
Conflicting findings between agents — This indicates a discrepancy in how the mockup is being interpreted. This is where a human makes the final decision.

With a single reviewer, you can never get the powerful signal of "multiple independent sources reporting the same issue." This is the greatest value of running multiple agents.

4. Fix and Re-review

Once the issues are addressed, run the same group of agents again. This ensures that the fixes haven't introduced new discrepancies. In practice, for large changes—like restructuring a layout from a single column to a two-column setup—it rarely becomes perfect on the first try, making this iterative process essential.

Pitfalls and Lessons Learned

Here are some observations from running this workflow:

Dividing the axes is essential — If you give every agent the same prompt ("Check if this matches the mockup"), you'll just get multiple superficial, similar responses. Dividing the perspectives is the essence of the method.
Keep materials consistent — Ensure the mockup and implementation snapshots are identical for every agent. Otherwise, you won't be able to reconcile their feedback.
Protect agent independence — Do not let them share conclusions. The moment they do, you move from "independent perspectives" to "majority rule/conformity."
Humans hold final authority — Discrepancies between agents often visualize "ambiguity in the mockup interpretation." Instead of automating this away, let humans decide the intent.
Providing the actual screen is powerful — Simply reading static code might miss discrepancies in the actual rendering. Whenever possible, include the rendered screen as a material (I will cover the method of showing real screens via browser automation in another article).

Summary

Self-reviewing is prone to confirmation bias and narrow perspectives, making it easy to miss mockup discrepancies.
Divide the evaluation into axes (Layout Structure / Placement & Hierarchy / Component Mapping / Visual Tone) and run independent review agents for each.
By using the same materials but specialized prompts, ensure agents evaluate without seeing each other's conclusions.
Findings reported by multiple independent agents are highly reliable. Conflicts highlight mockup ambiguity for human judgment.
After fixing, re-review with the same agents to finalize the implementation.
The goal of multi-agent review is to reinforce a single review with multiple, independent perspectives.

Tips for Running Stable Background ML Inference on macOS

orca_forge — Tue, 04 Aug 2026 01:08:21 +0000

Running an inference service as a background process on macOS, with a Linux server mindset, can lead to subtle issues. Things like "a one-liner that works on Linux doesn't work on Mac" or "grepping logs results in garbled text errors and crashes" — these are minor but time-consuming problems.

This article compiles a collection of short tips gathered from running a Seed-VC based voice conversion service (FastAPI + uvicorn, local 127.0.0.1:8770) as a background process on macOS. It focuses on macOS-specific pitfalls not covered in Linux-centric articles.

TIP 1: `setsid` / `timeout` are not available on macOS

First, it's important to note that macOS (BSD-based) does not include GNU coreutils' setsid or timeout by default. If you use these commands, which are used for backgrounding and timed execution on Linux, directly in a Mac script, you'll get a command not found error.

There are two solutions:

Install coreutils via Homebrew and use gsetsid/gtimeout
Use standard tools as alternatives (next TIP)

For background scripts that avoid external dependencies, using standard tools as alternatives is a safer option.

TIP 2: Use `nohup` + `disown` for background processes

In environments without setsid, the combination of nohup and disown is reliable for keeping processes alive even after closing the shell.

# Run the inference service as a background process
nohup bash scripts/start-backend.sh > backend.log 2>&1 &
disown

nohup ... Ignores hangup signals (SIGHUP), keeping the process alive even after the terminal is closed
> backend.log 2>&1 ... Redirects standard output and standard error to a log file
& ... Runs the process in the background
disown ... Removes the job from the shell's job table, preventing it from being terminated when the shell is closed

While nohup alone usually keeps the process alive, adding disown ensures that closing the terminal won't accidentally terminate the process.

TIP 3: `tr` / `grep` fail with binary data in logs → use `LC_ALL=C`

This was the most problematic issue on Mac. Inference logs may contain progress bar control characters or, occasionally, garbled multibyte sequences. When processed by macOS's tr or grep, you'll see:

This happens because the locale is set to UTF-8, causing invalid byte sequences to be treated as "invalid characters" and throwing an exception.

The solution is to set the locale to C (pass-through as byte sequences) for those commands.

# Remove unwanted control characters from logs (avoids Illegal byte sequence)
LC_ALL=C tr -d '\r' < backend.log > backend.clean.log
LC_ALL=C grep "ERROR" backend.log

Setting LC_ALL=C treats text as "bytes" rather than "characters," preventing crashes due to invalid sequences. This is safer for pipelines that process or search logs programmatically.

TIP 4: Wait for startup completion using the health endpoint

Loading models takes time, so sending requests immediately after starting with nohup will fail because the service isn't ready. Using sleep 10 as a workaround is unreliable—too short for slow machines and too long for fast ones.

The proper approach is to poll the service's health endpoint until it returns a 200 status. In this setup, the health endpoint is http://127.0.0.1:8770/health, so we poll it.

# Wait for the health endpoint to be ready before proceeding
until curl -sf http://127.0.0.1:8770/health >/dev/null; do
  sleep 1
done
echo "backend ready"

curl -sf exits with a non-zero status on failure, so combining it with until allows you to wait until the service is ready. Waiting based on state, not a fixed delay, significantly improves the reliability of startup scripts.

TIP 5: Stop processes with `pkill` using pattern matching

For background processes without a saved PID, pkill with a command-line pattern is convenient for stopping them.

# Stop the inference service started with uvicorn
pkill -f "uvicorn server:app"

The -f option matches the entire command line, so including specific details like the port or app name in the pattern prevents unrelated processes from being affected. For more precision, save the PID during startup and target it directly.

Summary

macOS does not have setsid/timeout. Either install Homebrew's coreutils (gsetsid/gtimeout) or use standard tool alternatives
Use nohup ... & disown for background processes. Ignore SIGHUP and remove the job from the job table to prevent termination when the terminal is closed
Binary data in logs causes tr/grep to fail with Illegal byte sequence → Temporarily set LC_ALL=C to treat data as byte sequences
Instead of fixed sleep delays, poll the health endpoint with until curl -sf to wait based on state
Stop processes with pkill -f "specific pattern". Use a detailed pattern to avoid affecting unrelated processes

HuggingFace's Large File Downloads Keep Stopping — Resuming with curl for Reliable Retrieval

orca_forge — Tue, 04 Aug 2026 00:37:44 +0000

Development of a Voice Conversion App: Overcoming Seed-VC Model Download Issues

When developing a voice conversion app, I encountered an issue while trying to download the 44.1kHz Seed-VC model set (DiT weights, rmvpe, and BigVGAN vocoder) using huggingface_hub.

While small config.json files would download quickly, larger .pth or .pt files would get stuck at a certain point and freeze. This article describes how to use curl's stagnation detection and automatic resumption to ensure successful downloads, and create a custom Hugging Face cache structure to enable model reading even when HF_HUB_OFFLINE=1.

Symptoms: Only Large Files Freeze Silently

On my machine, large file transfers using hf_hub_download would stall halfway through. The suspected causes include IPv6 route unavailability and connection drops over extended periods. The issues are:

It doesn't fail with an exception (and retry doesn't work)
The progress bar freezes
Small files don't reproduce the issue

This isn't a matter of a slow network, but rather large transfers dying under specific conditions, so simply increasing the retry count won't solve the problem.

Approach: Use curl for Transfers and Create a Custom HF-Compatible Structure

The appeal of huggingface_hub lies in its cache management, not just as a downloader. load_custom_model_from_hf and from_pretrained assume a specific directory structure (blob entities + symbolic links + refs) and will re-download the model if this structure is broken.

To address this, I adopted a two-step approach:

Use curl to handle transfers (with stagnation detection and automatic resumption)
Manually configure the downloaded entities in an HF cache-compatible structure (enabling subsequent references by hf_hub)

Metadata (etag, commit hash, size, and entity URL) can be retrieved from the huggingface_hub API to construct the cache structure.

Using curl: Detecting Stagnation and Automatic Resumption

The key is combining curl options:

subprocess.run(
    ["curl", "-L", "--fail", "-C", "-",
     "--retry", "50", "--retry-delay", "3", "--retry-all-errors",
     "--speed-time", "20", "--speed-limit", "2000",   # 20 seconds at 2KB/s or lower, consider transfer failed and retry
     "-o", blob, loc],
    check=True,
)

The roles of each option are as follows:

--speed-time 20 --speed-limit 2000 — Consider transfer failed and retry if 20 seconds pass at a speed of 2KB/s or lower. This proactively cuts off silent stagnation.
-C - — Resume download from where it left off (Range request). This avoids restarting from scratch each time.
--retry 50 --retry-delay 3 --retry-all-errors — Retry after 3 seconds, up to 50 times. Transfers cut off by --speed-time are also retried.
-L (follow redirects) and --fail (non-zero exit on HTTP errors).

This combination automatically loops through "stagnation → cut → resume" and eventually completes the transfer. This was the most reliable method in environments where large files would freeze.

Creating a Custom HF Cache Structure

The core of the Hugging Face cache (e.g., ~/.cache/huggingface/hub) has the following structure per repository:

models--{org}--{repo}/
├── blobs/
│   └── {etag}                      # File entity, named by etag (content hash)
├── snapshots/
│   └── {commit_hash}/
│       └── {filename}              # Symbolic link to blob
└── refs/
    └── main                        # Branch name → commit hash text

The key points are:

Entities are stored in blobs/ with etag names. This design allows a single entity to serve multiple revisions.
snapshots/{commit}/ contains symbolic links to blobs. User code and libraries access these human-readable file names.
refs/main contains the commit hash. This enables resolution for revision="main" and allows hf_hub to find the correct snapshot even when HF_HUB_OFFLINE=1.

Metadata can be obtained from the huggingface_hub API:

from huggingface_hub import hf_hub_url, get_hf_file_metadata

url = hf_hub_url(repo_id, filename, revision=revision)
m = get_hf_file_metadata(url)
etag = (m.etag or "").strip('"')   # Becomes the blob file name
commit = m.commit_hash or revision # Becomes the snapshots/ directory name
size = m.size                       # Used for download completion detection
loc = m.location or url             # Entity URL (redirected)

The rest involves using curl to download the entity to blobs/{etag}, creating a relative symbolic link from snapshots/{commit}/{filename}, and writing the commit to refs/main. The relative link ensures the cache remains intact even when moved to another machine.

os.symlink(os.path.relpath(blob, os.path.dirname(snap)), snap)
with open(os.path.join(refsdir, "main"), "w") as f:
    f.write(commit)

If the size is known, checking it against the existing blob size and skipping if complete can make re-runs idempotent.

Pitfalls and Lessons Learned

Download Only Necessary Files

Some repositories, like BigVGAN, contain large unnecessary files for inference (e.g., discriminator or optimizer states). Downloading the entire repository can result in getting stuck with the largest file. Specify the necessary files explicitly to avoid this.

Create refs for Offline Use

Failing to create refs/main alongside snapshots and blobs will prevent hf_hub from resolving the correct commit when HF_HUB_OFFLINE=1, even if the cache exists. Always create the three-part set (blob, snapshot link, and refs).

HEAD Verification May Require Online Access

Even with a cache, hf_hub performs a HEAD request at startup to check for updates (without re-downloading large files). To ensure complete offline functionality, set HF_HUB_OFFLINE=1 or TRANSFORMERS_OFFLINE=1. Conversely, if you want to see updates but prevent downloads, you can leave it online and just allow HEAD requests.

Summary

In environments where large hf_hub transfers freeze silently, use curl's --speed-time and --speed-limit to proactively cut off stagnation, and -C - with --retry for resumption.
Configure downloaded entities in an HF cache-compatible structure (blobs/{etag} + snapshots/{commit}/ with relative symbolic links + refs/main).
Metadata (etag, commit, size, location) can be retrieved using get_hf_file_metadata.
Create relative symbolic links to ensure the cache remains valid even when moved.
Explicitly exclude unnecessary large files (like discriminators or optimizers) from downloads.
For complete offline functionality, ensure refs/main is created to enable HF_HUB_OFFLINE=1.

Fixing Visual Discrepancies with Claude Code + Chrome Extension

orca_forge — Tue, 04 Aug 2026 00:07:22 +0000

You've got a code that looks correct when read, but when you open it in the browser, it's slightly different from the mockup - this "visual discrepancy" is the most troublesome part of UI development. A slight CSS specification, nesting of elements, and flex wrapping. Discrepancies that cannot be noticed by statically reading the code together will only appear when actually rendered.

Until now, it was necessary for a human to open the screen in a browser, compare it with the mockup image, and verbally communicate the differences to the AI. This workflow replaces the process of "humans visually seeing and verbalizing" by showing the screen to the AI agent itself via the browser. By combining Claude Code and browser automation extensions (Chrome extensions), we will "see" the screen actually rendered on localhost, compare it with the mockup, identify layout discrepancies, and fix them.

Why is it necessary to "show the actual screen"?

There are limitations to just handing over the code for UI review. It's difficult for both humans and AI to completely reproduce the final rendering result in their minds from the code. In particular, these discrepancies are difficult to detect just by looking at the code.

Layout skeleton discrepancies - One area is crushed when it's supposed to be a 2-column layout, or the vertical split ratio is different from the mockup, resulting in structural-level discrepancies
Element placement errors - A preview that should be in the upper right column is wrapped around to the bottom
Unexpected wrapping and overflow - The component wraps due to insufficient width, changing the impression from the mockup

These discrepancies cannot be determined without seeing the "rendering result" as a fact. That's why we show the actual screen to the AI.

Workflow: Show, Compare, and Fix

1. Provide the mockup as a baseline

First, provide the target mockup image to the AI and share the baseline that "this is the correct appearance". It's particularly helpful to verbalize the layout skeleton to make the comparison more accurate later. For example, describe the structure as "The whole is a 2-column layout. The left column is the adjustment UI. The right column is divided into upper and lower parts, with a radar chart and preview at the top, and voice samples and advanced adjustments at the bottom".

2. Launch the actual screen locally and "show" it to the AI

Start the development server on localhost and have Claude Code open that screen through the browser automation extension. The role of the extension here is to deliver the actually rendered DOM and screenshot to the AI's eyes. The AI no longer imagines the code but judges based on the rendered actual object.

If necessary, operate the screen to a specific state (after recording audio, after moving a slider, etc.) before taking a screenshot. This is because dynamically changing UIs cannot capture all discrepancies just by looking at the static initial screen.

3. Compare the mockup with the actual screen to identify discrepancies

With the baseline (mockup) and the actual object (screenshot) in hand, have the AI list the differences. What works here is that the skeleton was verbalized in step 1. Have the AI specifically point out structural-level discrepancies, such as "The preview that should be in the upper right column is at the bottom" or "The right column is not divided into upper and lower parts and is in one column".

The key is to break down the abstract "something is different" into specifics, such as "which area, which element, where it is in the mockup, and where it is in the actual screen". This becomes the correction instruction as is.

4. Fix and show again

Based on the feedback, have Claude Code fix the code and open it in the browser again for comparison. Check if the fix is actually reflected in the rendering result and if it hasn't created new discrepancies. In reality, layout reconstructions such as 2-column + right column vertical splits often don't work on the first try, and this "show, fix, and show again" loop is essential. If you skip the confirmation and just fix the code, discrepancies that you thought were fixed may remain.

Pitfalls and Learnings

Here are some cautionary notes that have emerged from running this loop.

Take a new screenshot with each fix - Reusing a previously taken screen won't allow you to evaluate the fixed state. Take a new screenshot of the actual object every time.
Reproduce the state before showing - Not just the initial display, but also operate to the state assumed by the mockup, such as after recording or slider operation, before taking a screenshot.
Match the viewport width - Wrapping changes with different display widths. Compare after matching the browser to the width assumed by the mockup.
Verbalize the skeleton first - Having a structural description like "2-column / right column vertical split" keeps the AI's feedback from becoming abstract and makes it a fixable granularity.
Code review and screen review are different - Consistency in the code and consistency in the rendering result do not match. Always make the final judgment on the actual screen.

Summary

Discrepancies in appearance, such as layout skeleton discrepancies and wrapping/overflow, cannot be detected by statically reading the code together
Use browser automation extensions to "show" the actual screen on localhost to Claude Code, compare it with the mockup image, and identify discrepancies
Specify discrepancies as specifically as "which area, which element, and how it differs between the mockup and actual screen", and use this as the correction instruction
After fixing, open it in the browser again for comparison. The loop of showing, fixing, and showing again is essential
Take a new screenshot every time, match the state and viewport width, and then compare
Make the final judgment on the actual screen, not the code - the new practice is to show the screen to the AI and have it fix it

DEV Community: orca_forge

The Problem of Robotic Voice When Changing Speech Speed - From Phase Vocoder to WSOLA

Premise: Changing tempo while preserving pitch

The Symptom: Metallic ringing (Robotic voice)

The Solution: WSOLA in the time domain

Trade-offs: It is not a silver bullet

Pitfalls and Lessons Learned

Summary

Visualizing Anchor Distributions to Create a 'Voice Map'

When You Collect 70 Voice Conversion Anchors, the Next Problem Emerges

Two Spaces You Want to Visualize

(A) Semantic-axis coverage: strip plots

(B) Speaker-embedding scatter plot: PCA to 2-D

Quiet Implementation Details That Matter

Turning the Maps Into Decisions

Summary

22.05kHz vs 44.1kHz — What's the Difference Between 'True Broadband' and Upsampling?

Premise: Sampling Rate Only Determines the "Ceiling"

22.05kHz Models and 44.1kHz Models Are Different Entities

Measurement: Is There Actually Energy in the High Frequencies?

Why Upsampling Doesn't Create High Frequencies

Pitfalls and Lessons Learned

Summary

Recording Spec-Compliant WAV Files (16-bit/Mono/Uncompressed) Using Only Web Audio

Introduction to Clean Audio Recording

Background: Why Raw PCM?

Step 1: Opening the Microphone with Clean Settings

Step 2: Collecting Raw Samples with ScriptProcessor

Step 3: Manually Building the 16-bit PCM WAV

Conclusion

Identifying Speakers by Voice Quality Alone — Labeling Unknown Audio Sources

The Idea: Matching by Voice, Not by Label

The Source Side: Creating Robust Speaker Vectors

The Anchor Side: Reusing Existing Embeddings

Matching: Top 3 Candidates and Confidence Levels

Why Not Just Pick "Number 1"?

Pitfalls and Lessons Learned

Summary

In-Depth Explanation of the Seed-VC Architecture — Decomposing Voice into 'Who, What, and How' in a 4-Stage Structure

Understanding "Voice" — Breaking Down the Components

Design Philosophy: Delegate Separable Information to Dedicated Modules

Step 1: whisper — Extract Only "What Is Being Said"

Step 2: campplus — Condense "Who Is Speaking" into a 192-Dimensional Vector

Step 3: CFM/DiT — Use Diffusion to "Draw" the Mel-Spectrogram

Handling Prosody: length_regulator and F0

Step 4: BigVGAN — Convert Mel Back to Audible Sound

Pitfalls and Lessons Learned

Summary

Focus on Root Cause Resolution Rather Than Quick Fixes: A Collection of Bug Investigation Case Studies

The Grand Principle: Eliminate the Root Cause, Not the Symptom

Case Study 1: Converted Audio "Speaks Slowly" — Proportionality Points to the Root Cause

Case Study 2: 502 Errors During Long ML Inference — Don't "Extend" Timeouts, "Change the Mechanism"

Case Study 3: Massive Model Download Stalls — Don't "Ignore and Proceed," "Ensure Placement"

The Common Pattern Found in These 3 Cases

Practicality When Working with AI Agents

Summary

Creating Perceivable Vibrato/Pitch Fluctuations

Adding a Vibrato Slider to the Voice Design App

Premise: Why Delay Modulation Makes Pitch Fluctuate

Implementation: Oscillating Delay Quantity with a Sine Wave

Main Topic: Solving the "Inaudible" Problem with Depth

Pitfalls and Lessons

Summary

Using Multiple AI Agents to Review UI Fidelity to Custom Designs

Why "One" is Not Enough

Workflow: Parallel Review via Divided Perspectives

1. Define the Review "Axes"

2. Assign Independent Agents to Each Axis

3. Reconcile and Integrate Findings

4. Fix and Re-review

Pitfalls and Lessons Learned

Summary

Tips for Running Stable Background ML Inference on macOS

TIP 1: setsid / timeout are not available on macOS

TIP 2: Use nohup + disown for background processes

TIP 3: tr / grep fail with binary data in logs → use LC_ALL=C

TIP 4: Wait for startup completion using the health endpoint

TIP 5: Stop processes with pkill using pattern matching

Summary

HuggingFace's Large File Downloads Keep Stopping — Resuming with curl for Reliable Retrieval

Handling Prosody: `length_regulator` and F0

TIP 1: `setsid` / `timeout` are not available on macOS

TIP 2: Use `nohup` + `disown` for background processes

TIP 3: `tr` / `grep` fail with binary data in logs → use `LC_ALL=C`

TIP 5: Stop processes with `pkill` using pattern matching