Wakaru is a menu bar app that turns any audio playing on your Mac into translated subtitles, fully on-device.
In version 1.1.2, I fixed two issues: "right?" getting translated as the direction "right", and the last word of a sentence popping up again as its own caption.
- Short questions and tag questions such as "right?" were translated with the wrong meaning
- Right after a sentence was shown, its last word sometimes came back as a caption of its own
1. "right?" became the direction
What was happening
In English meetings people end sentences with "right?" or "you know what?" all the time. In my captions, "right?" came out as 右, the Japanese word for the right-hand side. If you are following a conversation through subtitles, that is the kind of thing that makes you lose the thread for a second.
Wakaru translates whatever the speech recognizer marks as a finished sentence. In 1.1, when I cut the delay between a sentence ending and its translation appearing, I added a step that strips all trailing punctuation from the sentence before it goes to the translator. The reason was caching: the app translates the unfinished sentence in the background while it is still being spoken, and that unfinished text has no punctuation yet. Stripping it from the finished sentence too made both strings identical, so the cached translation was reused the moment the sentence ended.
The side effect is that the translator only ever saw "right". With the question mark it is a tag question. Without it, it is a direction.
I ran a few short phrases through Apple's on-device translator to see how much the punctuation mattered:
| Input | Japanese output | Meaning |
|---|---|---|
| right | ライト | "light" |
| right? | そうでしょう? | "isn't it?" |
| Nice | ニース | the city in France |
| Nice. | いいね。 | "nice" |
| Sure | 確かな | "certain" (adjective) |
| Sure. | もちろん。 | "of course" |
It is not only the question mark. A bare "Nice" becomes a French city. The shorter the phrase, the more the punctuation decides what it means.
What changed
The app now keeps a sentence-final "?" or "!" when it sends text to the translator. If a sentence has neither, it appends a period before translating.
The period is there to keep the speed. The background translation of the unfinished sentence gets the same period, so the finished sentence still hits the cache. Questions are the one case that misses: the "?" arrives and the sentence ends in the same instant, so the background translation never had a chance. Those sentences pay for one translation call, about 0.3 seconds for a short phrase. I decided a correct caption is worth that.
On screen nothing else changes. A trailing period is dropped from the translation like before, and question and exclamation marks stay.
What you will notice
Before shipping, I ran the new build through about 20 minutes of a real meeting and logged every finished caption. There were 19 questions in that stretch, and all 19 were translated as questions. The original "right?" showed up too, and became そうでしょう?, the tag-question reading ("isn't it?") instead of the direction.
Short sentences in general read better now. "Thank you." comes out as ありがとうございます, the ordinary polite thank-you, and "I was over the moon." as とても嬉しかった, which reads as "I was really happy" rather than anything about the moon. These used to be a bit of a lottery.
2. A stray one-word caption
What was happening
While reading that same 20-minute log, I noticed a second problem. Right after "Maybe I'm the crazy one." the app showed a separate caption: "one." translated as 一つ, the word you would use when counting one item. The same thing happened with "for." and "mouth.", three times in 20 minutes.
The cause is the recognizer revising its own output. It first produces "Maybe I'm the c one.", and a few hundred milliseconds later rewrites it as "Maybe I'm the crazy one.". Wakaru had already shown the first version. To know where the unshown text starts, it remembers the last four words of what it showed and looks for them in the revised sentence. When the rewrite changes one of those words, the four-word anchor is gone, and the app falls back to a character count from the old text. The revised sentence was four characters longer, so those four characters, "one.", were treated as new speech and captioned.
What changed
When the four-word anchor is missing, the app now tries three words, then two, then one. Short anchors are common words that appear everywhere, so a shorter match only counts if it sits within 24 characters of where the count said the boundary was, and directly before the punctuation that ended the sentence. Without those two conditions, a sentence like "I know you know." rewritten as "I know you knew." would snap to the earlier "know" and show the sentence twice. That case came up in review and is covered.
What you will notice
I replayed the three rewrites from that log ("the c one" to "the crazy one" and the two others) against the new code, and none of them leaves a word behind any more. Captions should flow from one sentence to the next without the odd single-word interruption.
Closing
Neither bug was about translation quality. Both came from the small amount of tidying the app does before handing text to the translator, and one of them was a direct consequence of the speed work in 1.1. The new version keeps that speed and stops throwing meaning away.
Wakaru is on the App Store. If you use it for meetings or videos and a caption looks off, I would like to hear about it.
https://apps.apple.com/us/app/wakaru-live-caption-translate/id6796070776
Top comments (0)