DEV Community: zhiyi guo

Inside murmur: prefetching, interruptions, and audio scheduling

zhiyi guo — Wed, 16 Sep 2026 10:09:04 +0000

murmur is a companion radio that runs in a terminal. It talks and picks songs without waiting for a question. The listener can type a line, get a spoken reply from the host, and then let the show continue.

Writing a segment, synthesizing speech, and finding music all take time. If preparation starts only after the previous segment ends, the listener hears a gap. An interruption can make already prepared content obsolete. Music adds another uncertainty: finding a song does not mean its audio source will play.

A local scheduler, the Director, handles these cases. It prepares content during playback, updates the queue when the listener interrupts, and sends playable audio to the mixer. The model writes segments and selects content; it does not directly control the speakers.

Where the waiting happens

Generate text → synthesize speech → play it → generate the next segment.

Running these steps serially means waiting for the model and the speech service between every pair of segments.

In two historical full runs, preparing the first batch of two segments took 24.5 and 33.9 seconds, from text generation through completed speech synthesis. In another log, generating one refill segment took 9 to 14 seconds for the model call alone, before speech synthesis.[1]

What the listener notices depends on when that work happens:

Moment	What the listener notices	Metric
Opening the radio	Why hasn't anyone started talking?	Startup to first audible voice
Moving between segments	Why did it stop?	Extra wait beyond the configured pause
Typing a message	When will it answer me?	Input to the start of reply playback

Preparing ahead can reduce the wait between segments. The first batch still has to be generated after startup. An interruption also needs a fresh generation request: the old program can keep playing while that request runs, but the reply still takes time. These are three separate latency measurements.

The local program owns the schedule

The main application is TypeScript running on Node.js. Its terminal UI runs in a separate process using Bun and OpenTUI. Model inference and production speech synthesis depend on external services.

Figure 1 groups the components by responsibility: blue for local orchestration, yellow for content preparation, purple for context, and green for audio. The arrows show interactions; preparation tasks can run concurrently.

Figure 1. The Director controls when content airs; AudioEngine controls audio state. Music preparation also calls the model to select songs. That work is grouped inside Music here.

The model submits results through tools

Brain uses the Claude Agent SDK. Each task gets its own tool set. The harness disables the CLAUDE.md files, skills, and arbitrary MCP tools from the listener's everyday coding environment so those settings do not affect the radio.[2]

For example, the model submits a batch of segments through emit_talk_beats. The program validates the tool arguments against a schema and reads the text, without having to extract a result from free-form prose. When the listener asks for another song, the model calls a music-switching tool. A callback owned by the Director performs the action.

Song introductions follow the same separation of responsibilities. The Director starts the source and waits for the engine to confirm that actual audio has been queued. Only then does it update "now playing," record the song, and play the introduction. This avoids introducing a song whose source never started. The check reaches as far as audio scheduling; it cannot prove that sound came out of the speakers.[3]

What goes into each generation request

Before each request, the program assembles a bounded context package:

The persona describes the host's identity. Once created, everyday conversations do not automatically rewrite it.
The listener profile holds selected, consolidated long-term information. Executed one-off control instructions are not directly learned as preferences.
Recent conversation keeps the current thread going. Earlier material can be retrieved when needed.
Broadcast history records songs, topics, and time-based anchors to help avoid repetition.

These live in local Markdown and JSONL files. Profile consolidation runs in the background without blocking playback. If it fails, the previous profile and processing checkpoint remain intact.[4]

Selected context is sent to the inference service, and text to be spoken is sent to the speech service. Data still leaves the machine.

The terminal UI only displays state and receives input. It exchanges newline-delimited JSON with the engine over a Unix socket, with schema validation at both ends. Without Bun, the application can fall back to a plain-text interface using the same scheduler.

Preparing the next segment during playback

Suppose preparation begins with R seconds left until the planned handoff, and the work takes P seconds. Ignoring playback startup overhead and failure retries, the extra wait at the handoff is:

Extra wait at the handoff: W = max(0, P - R)

Reducing P helps. Starting earlier also helps, because preparation can overlap the current segment's playback.

Figure 2 assumes a 30-second segment and 12 seconds of preparation for the next one. Both numbers are illustrative.

Figure 2. Prepare B while A plays, then play B after A ends. Diagram lengths do not represent elapsed time.

The Director maintains two kinds of prefetch.

The talk buffer has a target depth of two segments. Each entry holds text and a speech-synthesis Promise that has already started. After consuming an entry, the Director refills the buffer in the background, with at most one refill task in flight. Being in the queue does not mean synthesis is complete; playback may still have to wait for the Promise.

Music prefetch has one slot. Search, selection, and source resolution run in the background. If the pick is not ready at a planned music boundary, the Director plays another talk segment and checks again at the next boundary.

A deeper queue could absorb larger variations in preparation time. It would also spend more on generation, and content farther back in the queue could become stale before airing. The current depth has not been established as a global optimum.

Prefetched content can become stale

When refilling the talk buffer, the model needs to know what is already queued. Otherwise, it may write about the same topic again. We include queued text in the generation context so new segments follow it. Broadcast history is still updated only when the content airs.

The transition out of a song needs separate handling. If the next talk segment was written before the song started, its context does not include that song. Playing it several minutes later can abruptly return the listener to an earlier topic.

After a song starts, the system generates a short closing link, or coda, with the current song in context. It can play over the song's ending or go to the front of the queue after the song finishes.

What happens to the queue when the listener interrupts

Suppose the radio is playing A, with B and C buffered. The listener types, "Let's stop talking about this."

B and C no longer fit. Appending the reply to the queue would make the listener sit through two more segments on the subject they just asked to leave.

murmur handles it in this order:

Clear the old talk queue and invalidate its in-flight refill work.
Keep the current audio playing while generating and synthesizing the reply.
Once the reply audio is ready, stop any remaining old voice playback and start the reply.
Refill the queue using the updated conversation context.

If another line arrives while the reply is being prepared, it is merged into the reply, and the superseded preparation task is invalidated. An ordinary interruption leaves the song playing. The mixer lowers its volume when the voice comes in.

Clearing the queue alone is not enough. An old request could return a few seconds later and put obsolete text back into it.

The code uses an incrementing version number, epoch, to decide whether a result is still valid. The talk-refill path can be simplified to:

const startedIn = talkEpoch
const beats = await generateTalks()

if (startedIn !== talkEpoch) return // Context changed; discard this batch.
enqueue(beats)

An interruption increments talkEpoch. An old task may keep running, but its result is discarded when the versions no longer match. Interactive tasks use similar checks to prevent later tool calls from a superseded task from changing state.

This only blocks results that have not yet taken effect. Completed actions are not rolled back, and model requests already sent may continue consuming resources.

murmur tries to wait until the reply audio is ready before switching, so the old voice may continue for a while after the listener presses Enter. That reduces silence, but the time until the reply is heard still depends on generation and synthesis. Tests need to measure silence and reply latency separately.

Mixing voice and music

murmur uses node-web-audio-api to build one audio graph for voice, the main song, and the background bed. The engine schedules gain changes ahead of time on the audio clock, without relying on JavaScript timers to change volume during playback.[5]

Figure 3. Solid arrows carry audio; the dotted arrow shows control. Voice changes the song channel's gain without pausing the song.

The current settings lower the main song to a linear gain of 0.3 over about 0.3 seconds, then restore it over 2.5 seconds after the voice ends. A gain of 0.3 is an amplitude ratio. It does not mean "30% as loud."

The drop needs to be quick enough to keep the song from covering the voice. A slower recovery avoids a sudden jump in volume immediately after a sentence ends. These values were adjusted by listening; they are not universal.

The background bed stays steady during speech. It only crossfades when the main song enters or leaves. Making it rise and fall with every sentence would produce audible pumping.

Speech synthesis currently waits for the service to return a complete audio clip before playback starts. Knowing the clip's duration makes it easier to schedule music recovery, handle interruptions, and join clips. It also means the first line and each reply must wait for the whole clip to be ready.

Long sources such as songs are decoded and queued onto the playback timeline in chunks. The whole song does not have to be loaded first.

Measurements

Across two historical runs using the real model, real music-selection tools, and the production speech service, all 13 boundaries that consumed prefetched talk entered playback in the same logged second as talk.buffer warm. These included two transitions from music back to talk. The program still kept its default two-second pause between segments.[1]

Prefetch covered the generation wait at those boundaries. The logs only have one-second resolution, so they do not establish zero latency. There are also too few samples to calculate a meaningful long-run P95.

There is a separate before-and-after record for the first song:

Metric	Cold-start run	Subsequent run with memory from the previous session
Before optimization: startup to first song	136 s	195 s
After optimization: startup to first song	71 s	78 s
After optimization: music preparation itself	40.2 s	54.7 s

The experiments used separate data directories, preset personas, cached background beds, and a fixed "listener present" signal. The changes included starting music selection earlier, simplifying search, and limiting the selection context. The improvement in the table reflects those changes together. These numbers come from small-sample engineering records in the repository; I did not rerun the measurements for this article.

After optimization, the song was ready roughly 30 seconds before it played in both runs. It still had to wait for the third segment boundary. The scheduling rule at the time required two talk segments first, so making search faster would not have brought the song forward in those runs.

Cold-start waiting remains. Historical measurements put the first audible voice at roughly 29 to 39 seconds. Prefetch did not cover that first batch.

Music discovery also varies considerably. In another real log, five selections took roughly 82 to 192 seconds each. Talk can continue during selection, but the listener may wait a long time for the next song.

Natural transitions still need listening tests. Even with a buffer hit, ready audio, and correctly scheduled gain changes, a sentence can fail to follow the previous content. An interruption can land at an uncomfortable moment.

We use model-free tests for scheduling and invalidation rules, and offline audio rendering for gain changes and handoffs. Latency, source failures, and continuity need runs against real services, followed by actually listening to the result.

Implementation and measurement sources

The implementation described here was checked against d6c3619. Performance figures come from existing engineering records and were not remeasured for this article.

Project repository: wine-fall/murmur.

I built a radio station with one listener, and it runs in a terminal

zhiyi guo — Sun, 13 Sep 2026 03:23:31 +0000

The film above is 86 seconds, and the middle of it is a real on-air moment. There is a longer unedited recording at the top of the repo README. Watch either first, with the sound on. The rest of this post is why it exists and how it is put together.

What it is

murmur runs in a terminal window and behaves like a radio station whose only listener is you. It picks a topic on its own and talks about it in a voice that sounds like a person. Then it plays a song, comes back, and keeps going. At the right hours it says good morning and good night. You do not have to answer any of it.

When you do type something, the host replies in character. You chat for a bit. Then it eases back into the program.

There is one host. A few questions on the first run write its character to a file. After that, nothing rewrites that file except you. What grows over time is a separate thing: what it knows about you.

Why I built it

Every AI tool on my machine wants me to go faster. That is fine for work, and it is exhausting as company. I wanted something that sits closer to the person than to the task. A late-night radio host does that. They talk whether or not you are listening, they never ask you to respond, and when you do call in, they are glad and then they get back to the show.

Nothing I found did the proactive half. Chat assistants wait for a prompt. Voice tools want to drive my editor. So the bet was simple: what if the AI speaks first and needs nothing back?

How it is built

It is TypeScript on Node 24, no build step, run straight from source.

The brain is Claude, opened through the Claude Agent SDK. It reuses your local Claude Code login, so there is no API key and no separate bill. It is a harnessed agent rather than a one-shot call. murmur gives it a small set of its own tools: search for a song, judge the candidates, commit a pick, update memory, run the setup guide. It is sealed off from your own Claude Code setup. None of your CLAUDE.md files, skills, MCP servers, or hooks reach it.

The voice is fish-speech, reached over a hosted HTTP endpoint you point it at. That is the reason it sounds like a person and not a screen reader. It is also the reason murmur is not a local tool, and I want to say that plainly: the brain and the voice are network services. What stays on your machine is the program logic, the keyboard, the mixer, the persona, and the memory.

Music comes through yt-dlp. The rules it picks by live in a markdown file under ~/.murmur. The pick task re-reads that file before every song, so if you write "more Cantonese, no covers" while a track is playing, the change reaches the next pick or the one after.

The mixer is a Web Audio graph on node-web-audio-api. Voice and music share one output stream. A gain envelope ducks the song under the host, so the lead-in is spoken over the head of the track and the back-announce over its tail. The song never stops for the voice.

Two processes. The engine owns the program and the audio. The TUI is a separate process on OpenTUI under Bun, attached over a unix socket carrying newline-delimited JSON. Without Bun it falls back to a plain text host in the same process, and every command still works typed in full.

The decision people ask about

The host's character does not evolve. I built the evolving version first. A background loop watched the conversation and rewrote the persona to fit the listener better. Within days it had drifted into the same generic assistant voice every chatbot has. Language-model rewrite loops mean-revert, and there was no checkpoint where a person could catch the drift.

So the persona is frozen. It is a text file. You can open it and rewrite it whenever you like, and nothing changes it behind your back. The memory of you is a separate tier that does grow, with dates the code owns and citations back to the conversation that produced each fact.

What is rough

It needs a Claude Code subscription for the real brain. --brain stub gives you a canned stand-in brain for a look around, and that is all it is.
The voice is hosted. If you have no endpoint configured, it prints its lines instead of speaking. A local TTS is on the list, not in the code.
Music picks sometimes stall or run long. There is an open issue with measurements.
It is developed on macOS. Linux should be fine. Windows is untested.
Onboarding is still being tuned by ear, and one path after /quit misbehaves.

Try it

Node 24 or newer:

npm install -g murmur-radio
murmur

It launches even with pieces missing and offers to walk you through installing ffmpeg, yt-dlp, and a voice endpoint by talking to you. Say no once and it stops asking.

Repo: https://github.com/wine-fall/murmur (MIT). Landing page: https://wine-fall.github.io/murmur/.

If you leave it on for an afternoon, I would like to know one thing: did it feel like radio, or like a chatbot with a timer?