When building a voice-controlled teleprompter, accurately tracking spoken words against a script is critical. The challenge lies in handling imperfect speech recognition—common with real-world audio—while maintaining precise synchronization. This article explains a lightweight, effective approach using fuzzy matching with a lookahead window, designed to tolerate minor recognition errors without derailing the prompter.
The Core Idea: Pointer and Lookahead Window
The system maintains a pointer that tracks the next expected word in the script. As speech recognition outputs words (via the Web Speech API), the algorithm searches forward from the pointer within a configurable lookahead window—typically 12 words ahead. This window allows the system to recover from misheard or skipped words by checking if the recognized word matches any word in the upcoming segment.
If a match is found, the pointer advances to the next word after the matched one. If no match is found, the pointer remains unchanged—treated as noise or an irrelevant utterance. This design ensures robustness against transient errors while avoiding false advances.
Fuzzy Matching with One-Edit Tolerance
To handle common recognition errors (e.g., "telepromter" instead of "teleprompter"), the system uses a one-edit-distance algorithm. The withinOneEdit function checks if two strings differ by at most one substitution, insertion, or deletion.
However, to prevent false positives on short, common words (e.g., confusing "the" with "then"), fuzzy matching is applied only to words of four or more characters. The wordsMatch function enforces this rule:
const wordsMatch = (a, b) => !!a && !!b && (a === b || (a.length >= 4 && b.length >= 4 && withinOneEdit(a, b)));
This ensures that only meaningful, longer words are subject to fuzzy matching, preserving accuracy for short function words.
Code Implementation
The ScriptFollower class manages the logic:
class ScriptFollower {
constructor(script, lookahead = 12) {
this.tokens = script.split(/\s+/).filter(Boolean).map(normalize);
this.lookahead = lookahead;
this.pointer = 0;
}
hear(text) {
let p = this.pointer;
for (const w of text.split(/\s+/).map(normalize).filter(Boolean)) {
const end = Math.min(this.tokens.length, p + this.lookahead);
for (let q = p; q < end; q++) {
if (wordsMatch(this.tokens[q], w)) { p = q + 1; break; }
}
}
this.pointer = p;
return p;
}
get done() { return this.pointer >= this.tokens.length; }
}
The hear method processes each recognized word, scans the lookahead window, and updates the pointer only on a match. Unmatched words are ignored, and words beyond the window are not considered.
Web Speech API and Deployment Considerations
This system uses the browser’s Web Speech API for real-time speech recognition. In Chrome, audio is typically sent to Google’s servers—not processed fully on-device—raising privacy considerations. Support is best in Chrome and Edge, partial in Safari, and absent in Firefox. As a fallback, a manual scroll mode is available.
What the tests check
The repo's npm test covers these cases. Given the script "Welcome back to the channel...", hearing "welcome back to the channel" moves the pointer to index 5. A one-typo word like "telepromter" still matches "teleprompter". An unrelated word (the test uses "banana") leaves the pointer unchanged. A word outside the window (a window of 3 in the test) is not matched.
For a working implementation, see the open-source project: follow-the-script. The same idea is used in a voice-controlled teleprompter that runs in the browser.
Top comments (0)