Drawing Voices via Gacha — Reproducing the Same Voice with Just a Caption and a Random Seed

📝 Originally published (in Japanese) at forge.workstyle.tech.

I'm using a TTS system that lets you design a voice from a caption. You hand it a text description of the voice plus a random seed, and it speaks in exactly that voice.

{
  "input": "こんにちは。本日はお集まりいただき、ありがとうございます。",
  "irodori": {
    "caption": "落ち着いた知的な大人の女性の声。滑らかで聞き取りやすく、上品で信頼感のある話し方。",
    "seed": 1042
  }
}

Fix the caption and vary the seed, and you get a stream of voices in the same family but subtly different from one another. It's a gacha.

And the same caption with the same seed produces the same voice, every time. That determinism turned out to be the single most useful property in day-to-day operation.

Treating a voice as a set of "design values"

If generation is deterministic, then the (caption, seed) pair is the identifier for a voice. You don't need to store the audio file — just those two values.

When I built a set of seven characters, I kept a ledger like this.

| key    | 名前   | seed | caption |
|--------|--------|------|---------|
| luna   | ルナ   | 1042 | キラキラした明るいアイドル風の若い女性の声。高めで華やかで、ファンに語りかけるように楽しそうに話している。 |
| haruto | ハルト | 1042 | 爽やかで明るい若い男性の声。人懐っこく、ハキハキとした聞き取りやすい話し方。 |
| mio    | ミオ   |  777 | 元気いっぱいのアイドル風の少女の声。少し高めでハツラツとして、笑顔が伝わる話し方。 |
| shiori | シオリ | 1042 | 落ち着いた知的な大人の女性の声。滑らかで聞き取りやすく、上品で信頼感のある話し方。 |
| sora   | ソラ   | 1042 | 少年のような元気な声。中性的でボーイッシュ、はきはきとした明るい話し方。 |
| gen    | ゲン   | 1042 | 深く渋い大人の男性の声。低音で落ち着いたトーン、ナレーターのように力強く語る話し方。 |

The seeds skew heavily toward 1042 because once one seed gave a good result, I reused it across other captions. That's fine — a different caption yields a different voice even with the same seed.

The day the ledger paid off

The system is two-stage: it uses the generated audio as training material to bake a lightweight model. At some point, the driver script that ran generation disappeared.

The trained models survived, but I had no way to rebuild them without the design values. Change the caption by a single character and you get a different voice.

Without the ledger, I'd have had to redesign all seven voices from scratch. As it happened, I recovered the design values from my work logs, transcribed them into the ledger, and re-baked the exact same voices.

2026-08-06承認・2026-08-27にセッション履歴から復元(生成スクリプト消失のため)。
同caption+seedで完全決定的。

A trained model file is hundreds of megabytes — not a great backup candidate. The design values are a few hundred bytes: a text file you can commit to git. The recovery cost differs by orders of magnitude.

What goes in the ledger

It eventually settled into this shape.

| key  | 名前             | seed   | 話体         | caption |
|------|------------------|--------|--------------|---------|
| narF | 女性ナレーター   |  55555 | narration    | 落ち着いた大人の女性ナレーターの声。ゆったりとした語り口で、温かみと信頼感があり、長い文章を丁寧に読み上げる。 |
| narM | 男性ナレーター   |   3407 | narration    | 深く渋い男性ナレーターの声。低音で落ち着いた語り口、ゆったりと重厚に、長い文章を丁寧に読み上げる。 |
| cnsF | 女性カウンセラー |   1042 | counseling   | とても優しく穏やかな女性の声。ゆっくりと柔らかく、相手を安心させるように語りかける。息づかいのやわらかい話し方。 |
| salM | 男性営業         |      7 | sales        | 明るく信頼感のある男性の声。前向きでハキハキとして、押し付けがましくない爽やかな提案の話し方。 |

Adding the speaking style (conv_style) came later ([[speaking-style-is-baked-into-the-corpus|You can't change speaking rate after training]]). With the same caption and seed, changing the scripts in the training corpus changes the sentence-ending habits and the speaking rate. In other words, (caption, seed) alone doesn't uniquely determine a voice — you need the triple (caption, seed, speaking style).

For models baked before I added that column, there's now no way to tell which speaking style they were trained on. You can't retroactively backfill a field that turns out to be part of the identifier.

Why I deliberately didn't make the ledger machine-readable

The ledger is a Markdown table, and I never built anything to parse it. Three reasons.

Humans read it more often than machines do. Answering "which one was the narrator-ish voice?" is a human task, and for that a readable table is plenty.

Captions are long. They run 50–60 Japanese characters, which makes CSV or JSON unpleasant to read — plus you inherit escaping problems.

The system of record already lives in the DB. Every job actually submitted persists in the voice_design_jobs table as name / caption / seed / progress.params.conv_style. When I need to query programmatically, I go there.

SELECT name, seed, progress->'params'->>'conv_style' AS conv_style, caption
FROM voice_design_jobs WHERE status = 'ready' ORDER BY created_at;

So the split is: the ledger is the human-facing index, the DB is the machine-facing system of record. Yes, that's duplicated state — but the ledger only carries approved design values, so failed experiments never contaminate it. Different purposes.

Verifying determinism in practice

If you're going to build a ledger on the assumption of determinism, you should confirm the assumption holds. I generated the same input twice and checked whether the byte streams matched.

a = gen("テスト文です。", caption, seed=1042)
b = gen("テスト文です。", caption, seed=1042)
assert a == b      # 完全一致した

It even matched in cases where the model hallucinated.

1回目: 「なぜだと思いますか?そうだ!これに似てる円形と思いました」
2回目: 「なぜだと思いますか?そうだ!これに似てる円形と思いました」

Even the hallucinations reproduce. That turned out to be useful for testing: when chasing a bug, you can keep an input that reliably fails as a fixed asset.

⚠️ That said, results change when the model version changes. Update the TTS server image and the same design values may produce a different voice. I date-stamp entries in the ledger, and on any major update I regenerate the key voices and listen for differences.

A side benefit: you can expand a voice

With the design values on hand, you can bake additional variants of the same voice for different use cases.

The character "Shiori" was baked for narration. If I later want a call-center version, I keep the caption and seed as-is, switch only the speaking style to support, and bake one more. Each takes one to two hours.

In practice, one character has two variants: "streaming (casual)" and "business (polite)". Same voice, different sentence-ending habits and a different set of available emotional styles.

This works only because voice identity (caption + seed) is decoupled from delivery (speaking style). Without that separation, every new use case would mean redesigning the voice from scratch.

Takeaways

• If generation is deterministic, the design values become the voice's identifier. Orders of magnitude smaller than a model file, and they fit in git
• Fields that belong to the identifier can't be backfilled later. Models baked before I added speaking style now have an unknown speaking style
• Separate the human index from the machine system of record. Different purposes, so duplication is acceptable
• Measure determinism before you rely on it. Generate the same input twice and check for a byte-for-byte match — that's it
• Keep the design values and you can grow the voice. Baking extra use-case variants of the same voice becomes a viable workflow

---

Series: Mass-producing production voices from a diffusion TTS

A record of designing a voice from a single line of caption text, manufacturing a training corpus, and mass-producing role-specific production voices. This article is Part 1: Design.

← Previous: [[diffusion-tts-too-slow-for-conversation|The TTS I picked for audio quality was too slow for conversation]]
→ Next: [[screening-voices-by-metrics-not-ears|Letting a machine pick the "narrator-ish voice" out of 24 candidates]]

All 18 articles in the series

1. [[diffusion-tts-too-slow-for-conversation|The TTS I picked for audio quality was too slow for conversation]] 2. [[deterministic-voice-gacha-and-design-ledger|Pulling voices from a gacha]] ← you are here
2. [[screening-voices-by-metrics-not-ears|Letting a machine pick the "narrator-ish voice" out of 24 candidates]]
3. [[quality-gate-selection-bias-flat-takes|The stricter the quality gate, the more monotone reads survive]]
4. [[speaking-style-is-baked-into-the-corpus|You can't change speaking rate after training]]
5. [[tts-changes-recording-room-every-time|The TTS that changes "recording rooms" on every generation]]
6. [[one-rough-clip-ruins-the-whole-style|One rough clip makes the entire style sound hoarse]]
7. [[where-did-the-elongated-ending-come-from|Where did the AI pick up its habit of drawling "konnichiwaa"?]]
8. [[the-character-that-broke-the-tts-input|When "少々" became "しょも" — an allowed-character list was eating Japanese]]
9. [[hallucination-guard-that-never-fired|The hallucination guard that only failed to run during hallucinations]]
10. [[three-chars-became-a-verbal-tic|The "3 characters" the quality gate allowed became the model's verbal tic]]
11. [[measuring-factory-defects-as-product-traits|I was rejecting candidates over defects I could have fixed]]
12. [[defects-invisible-to-transcription|Some defects transcription will never find]]
13. [[70-minutes-lost-to-a-network-blink|70 minutes of training data, gone in a single network blink]]
14. [[ja-vs-JP-babbling-model|How writing "JP" instead of "ja" produced a babbling model]]
15. [[four-registration-paths-one-exit|Four registration paths, zero admin screens]]
16. [[who-is-rolling-back-whom|We were erasing each other's work on every deploy]]
17. [[chasing-unmeasured-targets-with-thresholds|Chasing unmeasured targets with thresholds always fails]]

The notes this article draws on are collected in [[拡散TTSから実用ボイスを量産する製造パイプライン]].