DEV Community: MadEnvel

Gapless playback with one ALSA device

MadEnvel — Sat, 25 Jul 2026 21:56:10 +0000

At some point Kalinka had become quite good at the basic job of a music player. It could open a stream, decode it, send PCM to ALSA, recover from errors and move through a queue reliably.

But it still paused between tracks.

For many albums that is only a minor annoyance. For others it changes the music. Pink Floyd’s The Dark Side of the Moon was developed and performed as one complete piece, and many of its tracks run directly into the next. Parts of The Division Bell use the same kind of transitions. Jean-Michel Jarre’s Oxygène is described on his official site as “one long flowing instrumental journey”. A player that inserts even a short pause makes those albums sound wrong.

That was when gapless playback stopped being a nice extra for me and became something I wanted Kalinka to do properly.

By gapless I mean a narrow, testable property: when two files are meant to be adjacent, the engine must not insert, remove or duplicate PCM samples at their boundary. It should behave as though the two decoded streams were one continuous stream.

This article is about how I implemented that for tracks with the same PCM format while keeping one ALSA device open. It also explains why a transition between different sample rates or channel formats is a different problem, especially if the player is trying to preserve the source format rather than resample everything.

The result, in one paragraph, is this: Kalinka prepares the next decoder before the current track ends, keeps the existing ALSA buffer and PCM handle alive, and asks the output node whether the new stream is compatible. If the complete PCM format matches, playback simply continues. If it does not, the player must either convert the audio or reconfigure the device; Kalinka chooses reconfiguration, so that case is not guaranteed gapless.

Kalinka is an open-source music player for Raspberry Pi and other small Linux systems. Its playback engine is a C++ audio graph using libFLAC++ or minimp3 for decoding and ALSA for output. It is not built around MPD, so track switching, buffering and output-format policy all live inside Kalinka.

The key decision was simple:

Keep the PCM device open for the whole playback session, not for one track.

Why separate ALSA sessions cannot be made gapless

The first implementation anyone is likely to write looks roughly like this:

Track A → open and configure ALSA → write PCM → drain or stop → close
                                                                    ┆
Track B → open and configure ALSA → write PCM → drain or stop → close

The important part of this diagram is not drain() by itself. It is the boundary between two complete PCM sessions.

For ALSA, a PCM handle is a stream with a particular hardware configuration and a state of its own. ALSA knows which frames have been queued and whether the stream is running, prepared or stopped. It does not know that the file which just ended and the file about to begin are two parts of one album.

If the player finishes the whole ALSA cycle at every EOF, it tells the output layer that the stream itself has ended. snd_pcm_drain() is only one possible step in that cycle: it preserves the queued tail, waits for it to play, and then stops the PCM. snd_pcm_drop() stops sooner but discards the tail. Closing the handle, reopening it, applying hardware parameters, filling its buffer and starting it again add further work, but removing any one of those calls does not restore continuity.

The crucial loss is that the queue reaches zero. The first frames of track B are not sitting immediately behind the last frames of track A in the same running PCM buffer. Once A has been treated as a completed ALSA stream, B has to begin as a new run: the handle must be opened or prepared, its format must be valid, some data must be queued, and playback must be started explicitly or through start_threshold.

At 48 kHz neighbouring PCM frames are only about 20.8 microseconds apart. That number is useful for understanding the requirement, but it is not a realistic restart budget. It describes a stream that is already running. ALSA provides no portable operation that means: “start this separately prepared PCM session exactly one frame after the previous session ends”. mmap access can reduce copying, and a low start threshold can reduce initial buffering, but neither creates a shared sample timeline between two stopped-and-started sessions.

This is why the problem cannot be repaired by finding a cleverer sequence of ALSA calls. Once the application has divided playback into one PCM session per track, ALSA has no information or buffered data with which to join those sessions sample for sample. The application has already discarded the continuity it is asking ALSA to reconstruct.

ALSA does have mechanisms for synchronising several PCM streams. snd_pcm_link() can link their state transitions, and hardware can advertise sample-resolution synchronised start. But that depends on having multiple compatible hardware streams with a shared synchronisation domain. It is not a general sequential-handover mechanism for a simple DAC exposing one playback substream, and it does not turn two independently configured track sessions into one sample-continuous stream.

ALSA plugins or an audio server can solve a different version of the problem by keeping one fixed-format slave stream alive and resampling or mixing the tracks into it. That can preserve audible continuity, but it changes the output policy and is no longer a direct source-native path.

For a direct PCM path, the solution has to sit above ALSA: do not end the output stream when a file ends. Prepare the next source in advance and append its first frames to the same running buffer immediately after the previous source’s final frames.

One output stream, replaceable decoders

A track and an output device have very different lifetimes. A decoder may live for four minutes. The ALSA device may stay open for an evening.

So I separated them:

Only one decoder supplies samples at a time. The next one can still be opened, initialised and buffered while the current track is playing.

This gives two advantages:

the next source is ready before the boundary arrives;
the ALSA ring buffer survives the boundary and gives the software time to switch producers.

The second point turned out to be just as important as the first. Kalinka normally keeps about 160 ms of audio queued in ALSA. During that time the DAC continues playing without caring which decoder will provide the following frames. The software does not have to switch at an exact CPU instruction; it only has to finish before the already queued audio runs out.

Why the output node must know about the boundary

My first mental model was that the stream switcher should hide the track change. The output would see one uninterrupted sequence of frames and would not know that a new decoder had taken over.

That only works while the complete PCM format stays the same: sample rate, sample representation, channel count and layout.

Suppose track A is 16-bit stereo at 44.1 kHz and track B is 24-bit stereo at 96 kHz. If the switcher silently starts passing B’s frames into a device still configured for A, ALSA has no way to infer the change. The bytes are interpreted using the old configuration. At best the result is played at the wrong speed or with the wrong sample layout; at worst the write fails.

The hw: ALSA plugin communicates directly with the kernel driver and performs no conversion. A PCM handle has one active hardware configuration installed through snd_pcm_hw_params(). Changing that configuration means stopping or draining the current stream, applying new parameters and preparing it again.

There are several valid ways to avoid that reconfiguration, but each chooses a different output policy:

Resample and convert everything to one internal format. The hardware might stay at 48 kHz, 32-bit stereo for the whole session. Tracks at 44.1, 88.2 or 96 kHz are converted before reaching it. This can remain gapless to the listener, but it is no longer source-native or bit-perfect.
Use ALSA conversion and mixing plugins, PipeWire or another audio server. ALSA’s plug plugin can convert channels, rate and format; dmix can combine multiple streams into one fixed slave configuration. This is a perfectly reasonable desktop design, but the conversion and mixing policy is now delegated to another layer.
Use multiple hardware playback subdevices and a hardware mixer. Some devices expose that capability. Many simple DACs and Raspberry Pi audio boards do not. Even when it exists, independently configured hardware streams are not automatically sample-aligned at the handover, so it is not a portable route to sample-perfect switching.
Keep a direct, source-native path and reconfigure when the format changes. This is Kalinka’s choice. It preserves the input format where the device supports it, but it means that only equal-format transitions use the guaranteed gapless path.

This distinction matters because “gapless” and “bit-perfect” are separate requirements. Resampling can solve continuity by sacrificing source-native output. Direct output can preserve the source format, but a format change becomes an explicit event and may introduce a pause.

For that reason the track boundary cannot be hidden from the output node. The output is the component that knows the current device configuration and can decide whether the next stream is compatible.

The source-change handshake

AudioStreamSwitcher keeps a queue of input nodes and a pointer to the active one. When the current input finishes and another one is waiting, it does not immediately start reading from the next node:

if (state.state == AudioGraphNodeState::FINISHED && !inputNodes.empty()) {
  currentInputNode = nullptr;
  setState(StreamState(AudioGraphNodeState::SOURCE_CHANGED));
  return false;
}

It removes the finished input and reports SOURCE_CHANGED. While no source is active, reads return no data:

size_t AudioStreamSwitcher::read(void *data, size_t size) {
  // ...
  if (currentInput == nullptr) {
    return 0;
  }
  return currentInput->read(data, size);
}

The transfer remains paused until the consumer calls acceptSourceChange():

// Accept the source change. This is called when the source has changed and
// the reader should accept the new source.
// The data transfer stops until the source is accepted.

That pause is deliberate. It gives the ALSA output node a chance to inspect the next stream before any of its frames are written using the old configuration.

It sounds dangerous to stall a live audio graph, but the hardware is still playing the frames already in its buffer. The handshake takes a tiny fraction of the available 100–160 ms, so it never becomes an audible underrun under normal conditions.

The same-format path does almost nothing

The ALSA emitter handles the source-change state:

if (inputNodeState.state != AudioGraphNodeState::STREAMING ||
    newStreamInfo.value().format != currentStreamAudioFormat || paused) {
  drainPcm();
  setState(StreamState(AudioGraphNodeState::SOURCE_CHANGED));
  return false;
}

If the next source is already streaming, its PCM format matches the current device configuration and playback is not paused, the condition is false.

Nothing is drained. Nothing is dropped. No parameters are changed. The PCM handle stays open and the emitter continues writing to the same mmap-backed buffer. The only difference is that the next frames came from another decoder.

That is the whole fast path.

For a different format, Kalinka deliberately takes the slower path: drain pending audio, apply the new hardware parameters and continue. That transition is not guaranteed gapless. It could be made continuous by converting every track to a fixed output format, but that is not the policy I chose for Kalinka.

This is also why “gapless across any files” is too broad a claim. The precise claim is:

Kalinka adds no samples, removes no samples and does not stop the ALSA stream when two adjacent tracks have the same output PCM format.

Why avoiding reconfiguration matters in practice

Avoiding device setup is not only an optimisation. Hardware and compatibility layers can behave differently around format changes.

On my Raspberry Pi 4 with a HiFiBerry Digi2 Pro, an older kernel/firmware combination sometimes reported the device ready before the I2S path was actually producing stable output. The first part of the next track disappeared. The only workaround that helped was a configurable delay after applying the new format:

// Hack for HiFiBerry boards on Raspberry Pi
// Sleep to make sure RPi is ready to play.
// I2S sync mechanism doesn't work properly
// wich results in the first ~500 ms of the track being cut off.

I have not reproduced that issue on current kernels, so the delay defaults to zero.

On a development laptop using pipewire-alsa, I encountered a different problem: at the time, changing the format required closing and reopening the compatibility device, although direct ALSA output did not. Kalinka has a second optional fixup for that case, the issue has since been fixed.

Neither workaround is elegant, but both reinforce the same practical point: a format-change path touches more hardware- and stack-specific behaviour than a normal write. The best boundary is the one that does not enter that path at all.

One caveat is important here. Kalinka’s default device is default, not necessarily a raw hw: endpoint, and ALSA conversion may be present depending on the user’s configuration. Kalinka’s guarantee is therefore narrow: it does not resample merely to keep the device configuration fixed. Whether the final path is bit-perfect still depends on the selected ALSA device and its configuration.

Two details that make the design actually work

Prepare the next track before EOF

Opening the next track after the current decoder reaches EOF is too late. The source may be remote, headers must be read and the decoder must fill its first buffer.

Kalinka begins this work five seconds before the current track ends:

PREFETCH_TIME_MS = 5000

It builds the next source and decoder chain and connects it to the switcher while the current track is still playing. The buffers are bounded, so the producer blocks once they are full rather than consuming unbounded memory.

By the time the boundary arrives, the next source is usually already in the streaming state. The handshake then only selects it.

At the end of a track, the final decoder read is usually shorter than the normal block size. The graph must preserve that exact frame count so that nothing is inserted, dropped or duplicated at the boundary.

Testing a track boundary without audio hardware

Listening is useful, but it is not a strong test. A click may be hard to notice, and a few missing samples may be masked by the music. I wanted a test that could answer a simpler question: are the output bytes exactly right?

1. Capture the ALSA output

ALSA includes a file plugin. Kalinka can use it as the output device and write every PCM frame to a raw file while the slave device is null:

pcm.kalinka_capture {
    type file
    slave.pcm "null"
    file "/tmp/capture.raw"
    format raw
}

The real graph, decoders, switcher and emitter still run. Only the physical sound card is removed.

The test uses latency_ms = 100 and period_ms = 25. At 44.1 kHz, the ALSA buffer contains 4410 frames split into periods of 1102 frames. The boundary therefore has to be completed inside a realistic playback window.

2. Create a signal with a known continuation

The source is a three-second, 441.5 Hz sine wave at 44.1 kHz, 16-bit stereo and −6 dBFS. It is cut exactly in half at frame 66,150 and encoded as two FLAC files.

Using one tone cut into two files is important. Track B must continue the same waveform, in phase, at the exact next sample. If I used two unrelated tones, a legitimate discontinuity between them could hide an implementation error.

The cut is also placed near a positive crest rather than a zero crossing, making inserted silence or duplicated samples easy to detect.

3. Compare the capture with the original PCM

The test checks:

total frame count;
byte-for-byte equality;
the longest run of digital silence;
the largest change between neighbouring samples;
that no drain() occurred for the equal-format transition;
that a 44.1-to-48 kHz transition does perform a drain and reconfiguration.

The result for the equal-format case is sample-identical: 132,300 frames, with the join at frame 66,150. No frame is inserted, removed or duplicated.

The first file is shown in blue and the second in orange below. The colour is the only visible trace of the boundary:

The same join in Audacity:

For comparison, this is the same signal with 3 ms of silence inserted at the boundary:

The audio examples are short enough to compare directly:

captured-joined.wav — the two tracks played back to back;
captured-with-gap.wav — the same output with 3 ms of silence inserted;
reference-joined.wav — the original continuous tone.

The maximum step between neighbouring samples is 1031 in the clean capture and 16,384 after inserting the gap. The latter is a full-amplitude jump and corresponds to an obvious click.

Two complementary assertions verify the decision at the boundary. When the formats match, the emitter must continue without draining the PCM device. When the next track uses a different format, the test requires the reconfiguration path instead.

The test is in test_gapless_playback.py. It runs without audio hardware and can write the WAV files and plots when KALINKA_GAPLESS_ARTIFACTS points to a directory.

Reproducing the byte comparison

The result does not depend on Python or on the test implementation. Decode the two source files with the reference flac tool, concatenate them and compare that PCM with Kalinka’s capture:

$ flac --decode --force-raw-format --endian=little --sign=signed a.flac -o a.raw
$ flac --decode --force-raw-format --endian=little --sign=signed b.flac -o b.raw
$ cat a.raw b.raw > reference.raw
$ cmp reference.raw captured-joined.raw && echo identical
identical
$ sha256sum reference.raw captured-joined.raw
482bd54a14ee9ce215008b4ad57e14bcdbcc36167ec2fe6aabb73a0b6828d5d0  reference.raw
482bd54a14ee9ce215008b4ad57e14bcdbcc36167ec2fe6aabb73a0b6828d5d0  captured-joined.raw

The inputs, raw capture and verify.sh are published with the test assets.

A null test in an audio editor says the same thing. Invert the reference, mix it with the capture, and the result is digital silence:

This verifies the digital path from the decoders to the ALSA capture device. It is not a measurement of the analogue output. A physical check would require an S/PDIF loopback for bit-exact capture or an ADC for the analogue path. I have not run that experiment yet.

What the engine cannot repair

A player can avoid adding its own gap, but it cannot remove every gap already present in the files.

FLAC is convenient here because it is lossless: the decoded PCM is bit-for-bit identical to the encoded input. Two FLAC files cut from one PCM stream can therefore be joined exactly.

Lossy formats have additional complications. MP3 encoders normally introduce encoder delay and padding. Gapless-capable decoders need metadata describing how many samples to trim. minimp3 handles the Xing/Info header with the LAME extension, but files relying only on other metadata conventions may still produce a small discontinuity.

The engine also cannot know whether a second of silence at the end of a track is deliberate. Gapless playback means not adding an artificial boundary; it does not mean rewriting the album.

The useful model

The implementation became much easier once I stopped thinking of playback as “open a file, play it, close everything, repeat”.

The output stream belongs to the session. Files and decoders are temporary producers behind it. At a track boundary the producer changes, but the sink continues.

There is one important qualification: the output node must approve the new producer. If the complete PCM format matches, the correct action is almost nothing — keep writing to the same device and buffer. If the format changes, the player must choose between conversion and reconfiguration. Kalinka chooses source-native output and accepts that those transitions are not guaranteed gapless.

The resulting model is simple: the player keeps one output stream running while decoders are connected and replaced behind it. A track boundary changes the source of the next PCM frames, not the lifetime of the ALSA stream.

References

The circular_buffer trap: how a 'faster' data structure spiked my CPU from 2% to 25%

MadEnvel — Tue, 21 Jul 2026 22:35:35 +0000

There's a special kind of bug that only shows up after you've made your code "better." You swap in the data structure everyone agrees is more efficient, the microbenchmark cheers, you commit — and then the real system quietly starts burning CPU. This is the story of how I replaced std::deque with boost::circular_buffer in my audio player, watched playback CPU jump from 1–3% to 20–30%, and ended up reverting the whole thing a week later.

The punchline up front: for a byte-streaming FIFO, std::deque beat boost::circular_buffer by at least 2×, and the "obvious" reasons circular buffers are supposed to be faster never applied.

The setup

Kalinka Player is a music player that runs on a Raspberry Pi. At its heart is a bounded producer/consumer buffer: a network/decoder thread writes audio bytes to the back, and the audio output thread reads and removes them from the front. The element type is uint8_t, capacities are big (the HTTP buffer is ~768 KB, FLAC ~1.5 MB), and data moves in large chunks (~16 KB, matching CURL_MAX_WRITE_SIZE).

The original buffer was backed by std::deque<T>. The two hot paths look like this:

size_t write(const T *source, size_t size) {
  std::lock_guard<std::mutex> lock(m);
  size_t n = std::min(size, capacity - data.size());
  data.insert(data.end(), source, source + n);   // append to back
  return n;
}

size_t read(T *dest, size_t size) {
  std::lock_guard<std::mutex> lock(m);
  size_t n = std::min(data.size(), size);
  std::copy_n(data.begin(), n, dest);             // copy from front
  data.erase(data.begin(), data.begin() + n);     // remove from front
  return n;
}

Look at that read(). A ring buffer is literally the textbook data structure for this access pattern. deque has to manage a map of fixed-size blocks and, so my reasoning went, all that front-erasing must be doing real work. A circular_buffer is a single fixed allocation with two indices — no allocation churn, perfect for a bounded FIFO. It felt like free performance.

I even wrote a synthetic benchmark to confirm it.

The synthetic benchmark that lied to me

The microbenchmark pushed fixed-size records onto the back of a bounded queue and drained them from the front — exactly the producer/consumer shape, no overwrite. circular_buffer won comfortably:

Bounded FIFO, 32-byte records, 20M ops (higher is better):

capacity	boost::circular_buffer	std::deque	winner
64	845 Mops/s	304 Mops/s	circular 2.78×
1024	813 Mops/s	295 Mops/s	circular 2.75×
65536	614 Mops/s	280 Mops/s	circular 2.19×

Case closed, right? circular_buffer is 2–3× faster, it's the correct data structure for a ring, and the benchmark agrees. On 2024-09-11 I committed:

bad7d04 — Switch to boost::circular_buffer, give DequeBuffer a generic name

Reality disagrees

The unit tests passed. Playback worked. And then I looked at top.

The KalinkaPlayer process, which used to sit at a comfortable 1–3% CPU during playback, was now churning at 20–30% — on a Raspberry Pi, where that actually matters for heat and battery. Nothing else had changed. The only difference was the buffer.

My first instinct was that I'd used the container naively. And I had: I'd left the read() path calling the generic erase(begin, begin + n):

data.erase(data.begin(), data.begin() + sizeToCopy);

boost::circular_buffer has a dedicated front-removal method, erase_begin(n), that just advances the internal start index instead of going through the general erase machinery. So on 2024-09-18 I fixed it:

- data.erase(data.begin(), data.begin() + sizeToCopy);
+ data.erase_begin(sizeToCopy);

402cdd1 — Improve CPU usage by changing erase with more efficient erase_begin call on circular buffer — time down from 2ms to 0.02ms

That's a 100× improvement on that single call — 2 ms down to 0.02 ms. Huge! Surely that was the CPU regression solved.

It helped. It was not enough. CPU was better but still nowhere near the old deque baseline. So I added a performance monitor (60976db) to actually measure the hot paths instead of guessing — and the numbers were damning.

Measuring the real workload

Synthetic record-pushing is not what the buffer actually does. The real buffer moves bulk byte ranges: insert(end, src, src+n) and copy_n(begin, n, dst) over ~16 KB chunks of uint8_t. So I rebuilt the benchmark around the actual Buffer<uint8_t> read/write hot paths, with the correct erase_begin for the ring, and verified both containers produced byte-identical output before trusting any number.

Buffer<uint8_t>, capacity 768 KB, 16 KB chunks, 4 GB streamed (higher is better):

Scenario A — fill to capacity, then drain to empty:

container	write MB/s	read MB/s
std::deque	6387	2798
boost::circular_buffer	1655	1785
circular vs deque	0.26×	0.64×

Scenario B — steady interleave, buffer kept near full:

container	write MB/s	read MB/s
std::deque	33420	4046
boost::circular_buffer	3671	2268
circular vs deque	0.11×	0.56×

The "faster" data structure was 1.6× to 9× slower on the workload that mattered. Even with the erase_begin fix in place. On 2024-09-18, the same day, I gave up:

5b281fc — Switch back to std::deque as buffer implementation as it is at least 2x faster than circular buffer.

Total lifespan of the circular_buffer experiment: one week.

Why the ring loses: the memmove fast path

Here's the mechanism, and it's the whole story.

For a trivially-copyable type like uint8_t, the standard library reduces bulk copies (std::copy, std::copy_n, and deque's range insert) to a single vectorized memmove — but only when it can prove the iterators are contiguous. Raw pointers qualify. std::deque's block iterators qualify (the library special-cases them, copying block-by-block via memmove).

boost::circular_buffer's iterator is not contiguous — its storage wraps, so every ++ does modular index arithmetic. The library can't take the memmove path, so both hot ops collapse to a scalar, per-element, unvectorizable loop:

write: deque::insert(end, src, src+n) → memmove into the tail block. That's the 33 GB/s in Scenario B — essentially raw memcpy. The ring can only copy one byte at a time through its wrapping iterator.
read: copy_n(begin, n, dst) → memmove out of the front blocks for deque; byte-at-a-time for the ring.

That single fact — no memmove for a wrapping buffer — accounts for the entire gap, on both reads and writes.

And every intuition that sent me down this path turned out to be irrelevant here:

"No allocation churn." True, but at 16 KB chunks deque mostly reuses its already-allocated blocks — writes hit an existing tail block and memmove into it. There's almost no allocation to avoid.
"Better locality — one contiguous buffer." Also true, and it genuinely favors the ring... but the win is buried under per-element iterator overhead that dwarfs any cache benefit.
"erase_begin is the fix." It was a real 100× win on that one call, and still the ring lost overall — because the copy and insert around it were the actual bottleneck, not the erase.

The synthetic benchmark wasn't wrong — it faithfully measured element-by-element pushes of 32-byte records, and for that workload the ring really is faster. It just measured a workload my program never runs. My program moves big contiguous runs of bytes, and that's precisely where memmove makes deque untouchable.

Lessons

Benchmark the real operation, not the shape of the problem. "Bounded FIFO" made me benchmark pushes and pops of single records. The real workload was bulk memcpy of byte ranges. Same data structure, completely different performance regime.
Trivially-copyable + contiguous = memmove, and it's enormous. If your hot path is bulk-copying POD, the single most important question is whether the library can lower it to memmove. A container that breaks contiguity (a wrapping ring, a linked structure) forfeits that, and no amount of micro-optimizing the other calls buys it back.
"The textbook data structure for X" is an argument, not a measurement. A ring buffer is the canonical structure for a bounded FIFO. It was still the wrong choice here. The canonical answer optimizes for allocation and wraparound; my bottleneck was neither.
Watch the system, not just the unit test. The tests were green the whole time. The regression only showed up as CPU on a running Pi. A cheap always-on performance counter (60976db) turned "it feels sluggish" into "here are the microseconds," and that's what ended the debate.

std::deque has quietly backed KalinkaPlayer's audio buffer ever since. The best data structure was the one I already had.

The benchmarks in this article are reproducible. The full single-file C++ harnesses — one reproducing the Buffer<uint8_t> read/write hot paths across both containers (verifying byte-identical output before timing), and one for the synthetic record-push benchmark — are available as a GitHub Gist. Build with g++ -O2 -std=c++17 (Boost's circular_buffer is header-only) and run; they print the tables above.

"Something melancholic for tonight": Building semantic music search that runs on a Raspberry Pi

MadEnvel — Mon, 20 Jul 2026 16:22:30 +0000

I wanted to type "something melancholic for tonight" and get a playlist from my own FLAC collection — matched by how the music sounds, not by tags. And I wanted it to run entirely locally, on the same Raspberry Pi 4 with 4 GB of RAM that plays the music. No cloud, no external APIs.

This post is about getting there: two failed approaches, a benchmark that told me exactly what my model couldn't do, a tiny 66k-parameter network that fixed it, and the memory tricks needed to make everything fit next to a running audio server.

Some context first. For the last three years I've been building Kalinka (GitHub), an open-source Hi-Fi music streamer: a server that runs on a Raspberry Pi connected to a DAC and plays audio bit-perfect and gapless through ALSA, with desktop/mobile apps acting as remote controls. It grew out of frustration with my receiver's built-in streaming (every 192 kHz track started with an audible stutter — I only noticed after two years, when I heard the same song open cleanly on YouTube), and with the alternatives: Roon wanted a subscription, Volumio put Qobuz behind an extra paywall on top of the Qobuz subscription I already paid for. So I did the reasonable thing and wrote my own.

The C++ audio graph, gapless playback against CDN links that expire mid-queue, and the real-time client sync each deserve their own write-up. Today: search.

The problem with tags

By the time Kalinka's local library became central, I had a collection big enough to hit a familiar wall: lots of music, no way to pick something for right now. Genre, year, album — none of it describes how a track sounds. Text search assumes you already know what you're looking for. I wanted the opposite: describe a mood, get tracks.

Attempt 1: pre-trained taggers (it went badly)

The obvious approach: auto-tag the whole library, then search the tags. The Essentia project ships pre-trained TensorFlow models for exactly this, so I wired up three:

Predicts	Model (backbone → head)	Output
Genre	EffNet-Discogs → Discogs-400	top-N genre labels + scores
Mood	VGGish → MIREX mood	one of 5 mood clusters
Danceability	VGGish → danceability	a single float

It worked… disappointingly. The genre classifier was noisy — a suspicious number of tracks came back as electronic---ambient regardless of what they actually were — and produced usable genres for only about a third of my library. The mood model collapsed the entire collection into three or four clusters. The published numbers explain why: PR-AUC below 0.2. Multi-label music tagging is just a hard problem.

And even ignoring quality, the stack was too heavy. TensorFlow plus its dependency tree is not something you want to install on a Raspberry Pi.

Attempt 2: betting on CLAP

In parallel I was already computing an embedding for every track with CLAP (Contrastive Language-Audio Pretraining). CLAP's core idea is what I actually needed: audio and text map into the same vector space, so a text query becomes an embedding and search becomes nearest-neighbour lookup — no intermediate tags at all.

It quickly became clear this was the real solution and the tagging pipeline had been a temporary crutch.

Making CLAP behave on a Pi took a few iterations:

Dropped PyTorch + laion-clap for ONNX Runtime — far easier to distribute, much lighter at startup.
Swapped the general audio model for an HTSAT-based CLAP trained specifically on music.
Since the model only "hears" ~10 seconds at a time, I sample several fragments per track and average the embeddings.

Measure before you optimize

"Vibe" is too subjective to develop against, so before tuning anything I built a proper benchmark on MTG-Jamendo, where tracks are labelled by humans. The results were clarifying - and a little sobering:

Query category	P@10	Lift over random	MRR	Hit-rate@10
Genre	0.296	+0.212	0.670	0.880
Instrument	0.273	+0.157	0.476	0.933
Mood / theme	0.140	+0.050	0.315	0.733
Overall	0.247	+0.153	0.520	0.855

CLAP is genuinely good at concrete sonic properties. Instrument queries shine — piano hits P@10 = 0.80, meaning eight of the top ten results are relevant. Guitar, strings, violin all do well; among genres, hip-hop and jazz stand out.

Abstract emotional concepts are another story. uplifting, inspiring, love — effectively zero. Worse than picking tracks at random:

	Strong (P@10)	Weak (P@10)
Instruments	piano 0.80, guitar 0.50, acoustic guitar / strings / cello 0.40	synthesizer 0.30, voice 0.10, "electronic production" 0.00
Genres	hip-hop 0.80, jazz / rock / electronic 0.60	pop 0.20, techno / world 0.00
Mood / theme	epic 0.50, film / meditative 0.30	uplifting / inspiring / love 0.00

This turns out to be a well-known property of CLAP. But reproducing it on my own benchmark was oddly satisfying: instead of a vague feeling that mood search was weak, I had a number — and a direction.

Fixing mood: a small model on top of CLAP

My hypothesis: if CLAP embeddings describe the sound well, maybe a small extra model can learn to extract the emotional component from them.

The DEAM dataset fit perfectly — ~1,800 tracks hand-annotated on two axes: valence (how positive the music feels) and arousal (how energetic it is). I computed CLAP embeddings for DEAM and trained a two-layer MLP with just 66k parameters to map an embedding to a (valence, arousal) point.

(Training ran on my Lenovo ThinkBook 13x, which got hot enough to burn my fingers twice. The eventual fix was parking it under the air conditioner. A cooling pad is probably in my future.)

The part where I got a negative R²

My first attempt was a textbook lesson in train/serve skew. I had computed DEAM's embeddings with the general-audio CLAP model — while production had already moved to the music-trained one. On real production embeddings the results were catastrophic: R² = −0.42. The model performed worse than a baseline that assigns every track the same average value.

After finding the mismatch and retraining on the exact embeddings used in production:

Axis	Model	R²	CCC	RMSE
Valence	MLP head	0.594	0.761	0.755
Arousal	MLP head	0.711	0.827	0.683
Valence	ridge baseline	0.480	0.626	0.855
Arousal	ridge baseline	0.631	0.742	0.773

I plug this mood signal into ranking through a confidence gate, so it only influences queries that are actually about emotion — it stays out of the way when you search for piano or hip-hop.

Re-running the benchmark: mood search improved by ~80% (lift over random went from +0.05 to +0.09), and P@10 for uplifting went from 0.00 to 0.30. Genre and instrument search degraded by an amount small enough to ignore. A trade I'll take:

Query category	P@10	Random baseline	vs random
Genre	0.296	0.084	3.5×
Instrument	0.273	0.116	2.4×
Mood / theme	0.140	0.090	1.6×
Overall	0.247	0.094	2.6×

Tag	P@10	Random baseline	vs random
hip-hop	0.80	0.063	12.7×
trance	0.40	0.032	12.5×
jazz	0.60	0.067	9.0×
rock	0.60	0.083	7.2×
piano	0.80	0.277	2.9×

Making it fit in 4 GB

On the Pi, the bottleneck was memory, not compute. Indexing several tracks in a row kept growing RSS until the OOM killer took the process down; forcing Python's garbage collector after every iteration stopped the bleed. Beyond that, two optimizations did the heavy lifting:

Quantized stored embeddings to int8. Vectors shrank 4×, and ~99% of top-10 results stayed identical. Practically free.
Split CLAP in two. The text encoder (needed for every user query) stays resident. The much heavier audio encoder is only needed for indexing — so it unloads automatically once indexing finishes and comes back when new tracks arrive.

Deleting the scaffolding

At this point CLAP embeddings fully covered semantic search, and the little valence/arousal head covered mood — both jobs the Essentia taggers were originally there for, done better and cheaper. Meanwhile the taggers had turned into pure liability: essentia-tensorflow ships ARM wheels only for Python 3.11, which had silently pinned my entire environment to 3.11 right when I needed 3.13.

So the whole auto-tagging pipeline went in the bin — TensorFlow (a multi-hundred-megabyte dependency), the Python version pin, the tags_predicted column, and all the tag-based ranking logic. What replaced genre and danceability detection? Nothing separate — CLAP already does it. What replaced the mood model? The one I trained myself.

What shipped

Kalinka's smart search today is exactly two components: CLAP audio embeddings and a 66k-parameter valence/arousal model, both running locally through ONNX Runtime. No TensorFlow, no external taggers, no cloud.

A nice side effect: a Russian song and a Pink Floyd track can land next to each other in the results if they share a mood. The system ranks by how music sounds, not what it's called. And all of it fits on a Raspberry Pi 4 with 4 GB of RAM.

(For completeness: semantic search is one layer of a larger query system — SQLite FTS for full-text, streaming-service search, and semantic catalog matching, with RapidFuzz-based ranking and smart-search results surfacing only when there's no exact match. One search box, several engines behind it.)

What's next

Recent work went into first-run setup and easier installation, plus a second free streaming source (Jamendo) with semantic search built the same fully-local way. The model still can't search by origin or era — italian 80s won't find my Toto Cutugno records — so there's plenty left to explore.

Kalinka is open source, and I'd genuinely welcome testers and contributors — not just code: trying it on different hardware, bug reports, UI feedback, docs, new plugins. AI-assisted PRs are fine by me; what matters is that the author understands the solution, can explain and maintain it, and the code is tested and fits the architecture.

Kalinka Player GitHub: Kalinka Player
Kalinka Player Client: Kalinka AI App
Site: https://kalinkaplayer.com