Piper TTS over Wyoming: the frame format and empty model that broke my podcast

I wanted my own articles as audio: something I could listen to on a walk, as a conversation between two voices rather than one voice reading. Not a cloud service. A thing I run, on the hardware I already have, with no GPU.

It works now. The first episode came out at 6 minutes 46 seconds, 58 turns, normalised to -16 LUFS. Getting there took two bugs that both looked like something else, and a licence check that ruled out the voices I wanted most. Here is what actually happened.

The shape of it

Four stages: fetch → script → voice → mix. Each one writes its output to disk (the article JSON, the dialogue turns, one WAV per turn, the final MP3), so any stage can be re-run on its own. That turned out to matter more than anything else in the design, because every bug below was found by re-running one stage, not the whole thing.

• Script: a local 7B model writes the dialogue, one article section at a time.
• Voice: Piper, spoken to over the Wyoming protocol, with two stock British voices, one per speaker.
• Mix: ffmpeg concat plus short pauses, then loudness normalisation.

Voice: Piper over Wyoming, no new service

I already had a wyoming-piper service running for my home voice assistant. It turns out that if you send it a synthesize request naming a voice it has never seen, it downloads that voice on first use. So two distinct speakers cost nothing: no new container, no GPU, no API key.

Measured on a 2-core container: 8.4 seconds of audio in 2.9 seconds of wall time, a real-time factor of about 0.35. A 15-minute episode is roughly five minutes of synthesis. Clips are cached by a hash of (voice, text), so fixing one line re-renders one line.

Bug one: the frame has three parts, not two

The first version of my Wyoming client read a JSON header line, then read the audio payload after it. It failed straight away with:

Extra data: line 1 column 62

That reads like a broken server sending malformed JSON. It wasn't. A Wyoming event can carry three parts: the header line, then an optional block of JSON data, then an optional binary payload. The header announces the sizes of both. If it declares a data_length, that data is its own block of bytes after the newline, not a field inline in the header. My reader skipped it, landed in the middle of a JSON object, and choked.

The fix is to read what the header tells you to read, in order:

def _read_event(f):
    line = f.readline()
    if not line:
        return None, b""
    head = json.loads(line)
    dn = head.get("data_length") or 0
    if dn:
        head["data"] = json.loads(f.read(dn))
    pn = head.get("payload_length") or 0
    return head, (f.read(pn) if pn else b"")

The audio-start event puts the sample rate, width and channel count in that data block, which is why skipping it was fatal rather than cosmetic.

Bug two: the model that answered in a thinking key

The first full run produced a cheerful log line: intro: 0 turns. No error. The natural assumption is a bad prompt, and I spent time there first.

The real cause was the model. My default local model is a thinking model, and I was using a variant whose Modelfile is meant to switch thinking off. Measured through Ollama's API: message.content was an empty string, and the entire 200-token budget had gone into a separate thinking key. The "no thinking" Modelfile parameter was accepted and ignored.

Two fixes. Send "think": false in the request body (the API-level switch does work where the Modelfile one didn't), and refuse to return an empty answer silently:

content = msg.get("content") or ""
if not content.strip() and (msg.get("thinking") or msg.get("reasoning")):
    raise RuntimeError(f"{model!r} spent its whole budget in a `thinking` "
                       "block and returned no content.")

An empty content from a thinking model looks identical to a model with nothing to say, and everything downstream just counts zero turns. That sends you to the prompt when the problem is the model. Now it fails loudly and says why.

I also switched models on measurement rather than habit. On my 6 GB card, which is already mostly holding an image model, the default ran at 5.3 tokens/s (about 90% on CPU). A plain 7B non-thinking model ran at 12.6 tokens/s. Faster, and it actually returns content.

The small 7B problems

A 7B model writing structured dialogue needs guard rails:

• Ollama's JSON mode returns an object, and I wanted an array, so the model wrapped the array in a key it invented, differently each time. Asking plainly and parsing defensively was more predictable.
• Sometimes it wraps each object in quotes, which is invalid JSON. A last-resort regex pulls out the speaker/text pairs.
• A check() step runs before synthesis and rejects monologue turns, empty turns, unknown speakers, URLs and bullet points. That's cheaper than finding them after five minutes of CPU spent speaking them.

The rule that keeps it honest: the source sets the ceiling

The model writes dialogue; it never supplies facts. Each prompt carries only the section it's allowed to talk about. And the target length is capped at 1.15 × the article's word count. An 828-word article becomes about six minutes, not fifteen. The only way to stretch a short article into a long episode is to make things up, so the pipeline refuses to ask for it, and says so.

The mix

ffmpeg -f concat copies the PCM straight through, so stitching takes seconds, not minutes. I insert a 320 ms pause between lines from the same speaker and 520 ms on a handover, which is enough to hear the conversation change hands. Then loudnorm=I=-16, because Piper's two voices aren't equally loud out of the box.

The licence check

I wanted to go further and have one of the voices be mine, cloned from an old recording. The two most-recommended cloning models were out as soon as I read the licences: XTTS-v2 is under a non-commercial licence (CPML), and the F5-TTS weights are CC-BY-NC. This podcast links to things I sell, so neither was an option.

I used OpenVoice v2 with MeloTTS instead (both MIT). The lesson there was a different one: OpenVoice changes timbre, not rhythm. My vocal colour on top of Piper's cadence still sounded like a machine. No better sample fixes that; the cadence comes from the underlying TTS. Kokoro (Apache-2.0) is the next candidate. Measured on CPU, it speaks the same line about 24% slower because it takes the pauses a person would.

What I'd tell you if you're building one

1. Read the protocol spec for framing before writing a client. A three-part frame read as two gives you an error that blames the server.
2. Treat empty model output as an error, not an answer. Check for a thinking or reasoning field when content is empty.
3. Write every stage to disk. Being able to re-run one stage is what turned each of these bugs into a short investigation.
4. Read the model licence before you fall in love with the demo.

---

🤖 Drafted with AI assistance from my own homelab notes, logs and code, then reviewed and edited before publishing.