DEV Community

Andrew Gnasevych
Andrew Gnasevych

Posted on Edited on

Building a voice-follow teleprompter on iPhone without sending audio anywhere

A teleprompter that scrolls as you speak sounds simple: recognize speech, find the words in the script, move the text. I built that for iOS and the recognizing part was the easy part. The hard parts were making the scroll feel calm and not trusting the recognizer too much. Here is what I learned. Disclosure: this is my app, Teleprompter Offline & Private.

1. Keep recognition on the device

I use Apple's speech recognition and require on-device mode. If the language has no on-device model, the app says so instead of silently falling back to a server. That choice makes the app work in Airplane Mode and means your voice never leaves the phone. The cost: some languages need the offline speech pack, downloaded once in iOS settings.

2. Match words, not text

The recognizer returns a stream of partial transcripts. I align them against the script with a fuzzy matcher that searches a lookahead window of 40 to 70 words past the current position. Fuzzy, because people skip words, say "gonna" for "going to", and the recognizer mishears.

3. The bug that looked like a smoothing problem

The scroll jerked. I spent days tuning easing curves. Then I found the cause was the matcher, not the animation. One stray match on a breath or a noise could jump the cursor across the whole lookahead window, 40 words at once, and the text lurched.

The fix was boring: cap how far the cursor can advance per update (12 words). If the speaker really did jump ahead, the next updates catch up. A jump that fast is almost always noise.

4. Start every animation from zero velocity

My first smoothing carried velocity from one animation segment to the next. It looked elegant and sometimes overshot the target wildly after a noisy update. Now every segment starts at zero velocity and eases toward the target with a critically damped curve. From zero velocity a critically damped ease approaches the target monotonically and never crosses it, so the text cannot overshoot.

5. Plan for the recognizer going quiet

During video recording the recognizer can lose the audio stream. If the stream stops for more than two seconds, the app falls back to a steady words-per-minute scroll, keeps recording, and tells the user. A teleprompter that stops dead on camera is worse than one that scrolls at the wrong speed.

6. Rewind is rarer than you think

Going back needs a stricter rule: the cursor must have been standing still for a couple of updates before I allow a relocation, and a forward jump must match at least four words. Otherwise repeated phrases in a script pull the cursor to the wrong place.

What I would tell myself at the start

  • Put the limits in the matcher, not in the animation.
  • Prefer a rule you can prove (monotonic approach to target) over a clever one.
  • Test with a script that repeats phrases. Real scripts do.

The app is free to download, works offline and needs no account: https://apps.apple.com/ua/app/teleprompter-offline-private/id6808964093?pt=128162591&ct=devto_voicefollow&mt=8

Comparison with other teleprompter apps: https://www.a-devgroup.com/teleprompter/best-teleprompter-apps/

Top comments (0)