Most text-to-speech integrations treat delivery as a settings problem. You pick a voice, then you set a speed value, a pitch value, and a style label from a fixed list. That works for a paragraph. It breaks the moment one paragraph needs two emotions.
I build a voice tool, so I have spent a lot of time on this problem. Here is the design I ended up with, and what it costs to use it.
The tag lives in the text
Instead of a global style, the direction sits inside the sentence:
[excited] We shipped it.
[whispering] Nobody knows yet.
Three things follow from that choice.
1. Direction is local. A tag changes the delivery of the next clause, not the whole take. One generation covers a paragraph that starts calm and ends loud, so you stop splicing five renderings together in an audio editor.
2. The script stays the source of truth. A style label lives in a dropdown and disappears from the file. A tag lives in the text, so it survives a copy-paste, a review, and a diff. When a co-writer edits the sentence, the direction moves with it.
3. The tag list becomes a small language. You need a fixed vocabulary: excited, whispering, sad, laughing, sighing. A free-text instruction such as "make it warmer" is not testable. A vocabulary of twelve tags is.
What it costs
Model support is uneven. A tag the model does not know is silently dropped, and the line reads flat. That failure is invisible in a waveform, so the tool has to show which tags the current model honours, and the writer has to listen to the result.
Counting characters gets harder. Tags are not speech, so the text splitter must strip them before it counts, and the billing counter must ignore them. If you charge by character, a user who writes [whispering] ten times must not pay for the extra characters.
Two systems edit the same string. The editor highlights tags, the synthesiser consumes them, the export writes them back. Every one of those paths needs the same parser, or the tags drift.
Multilingual is close, not exact
One voice can read the same script in Spanish, English and Portuguese and keep its timbre. It is close, not identical: vowels carry accent, and a voice cloned in Spanish will read German with a trace of it. Say that in the product, not in the small print. People forgive an accent. They do not forgive a promise you did not keep.
Consent belongs in the same flow
In a cloning tool the risk is not the model, it is the use. Two rules are cheap to build and they change the product:
- A new voice starts private, and publishing it needs an explicit consent step.
- Generated audio can carry a forensic mark before delivery.
Neither rule makes the product safe by itself. Both make the careless case harder to reach.
When not to use tags
If you need a stable, neutral read for a long narration, tags add nothing. Set the voice, write clean sentences, and leave the text alone. The feature earns its place only when the delivery changes inside the paragraph.
I built this into Vocloner, a browser voice cloning and text-to-speech studio. There is a free plan, and the tags work the same way in the API and in the MCP server. If you have shipped a similar editor, I would like to know how you handled the splitter problem above.
Top comments (0)