Updating text at the cursor as a transcript changes, without retyping the whole thing

#node #windows #javascript #performance

Disclosure: I'm the developer of Bolo, the app this code comes from, and this post is published from my account. It was drafted and published by an AI agent working from the project's source code.

Bolo turns an Android phone into a wireless microphone for a Windows PC: you speak, and the words appear at the PC cursor in whatever app is focused. The hard part isn't the final text. It's the live preview — the partial transcript that updates several times a second while you're still talking, each version a little different from the last.

Naively, every partial means: delete what you typed, type the new version. That is slow, it flickers, and over Remote Desktop the deletes and types arrive out of order and scramble. Here is how the Windows companion (Node.js) avoids all three.

Don't retype what didn't change

Two consecutive partials usually share a long prefix. "the quick brown" becomes "the quick brown fox" — the first fifteen characters are identical. So the update is: find the common prefix, backspace only the characters after it, and type only the new tail.

async function replaceTypedText(previousText, newText) {
  if (previousText === newText) return true;

  let commonPrefixLength = 0;
  const maxPrefix = Math.min(previousText.length, newText.length);
  while (commonPrefixLength < maxPrefix &&
         previousText[commonPrefixLength] === newText[commonPrefixLength]) {
    commonPrefixLength += 1;
  }

  const backspaceCount = previousText.length - commonPrefixLength;   // delete only the changed suffix
  const suffixToType   = newText.substring(commonPrefixLength);      // type only the new suffix
  return sendCommand(suffixToType, backspaceCount);
}

For a correction near the end of a sentence this is a few keystrokes instead of a hundred. The cursor never jumps; it only ever backspaces and appends, so it works in any text field without caret APIs.

The real bottleneck wasn't typing, it was spawning

The first version spent about 925 ms per update — and that fires on every partial, so a long sentence paid it dozens of times. That was the "types one letter at a time, very laggy" bug.

The cost wasn't SendInput. It was that every batch spawned a fresh powershell.exe and recompiled the C# SendInput helper with Add-Type. The fix: start one long-lived PowerShell process, compile the helper once, and feed each command over stdin. Commands are serialised and matched to a DONE:<id> marker on stdout:

ps.stdin.write(INIT_SCRIPT + '\n');     // compiles the helper once, prints "READY"

// per update, over the same stdin:
const line =
  `$t=[Text.Encoding]::UTF8.GetString([Convert]::FromBase64String('${b64}'));` +
  `if(${bs} -gt 0){[SendInputHelper]::PressBackspace(${bs},$r)|Out-Null};` +
  `[SendInputHelper]::TypeText($t,$r)|Out-Null;` +
  `Write-Output "DONE:${id}"`;
ps.stdin.write(line + '\n');

Passing the text as base64 sidesteps every PowerShell quoting and code-page problem, so emoji and non-Latin scripts survive the trip into the shell intact. A DONE:<id> reply per command keeps updates ordered; a 20-second watchdog kills and respawns the helper if one ever hangs.

Remote Desktop needs the slow path, on purpose

In a normal foreground window you can push the whole batch in a single SendInput call. Inside a Remote Desktop client (mstsc), a big synthetic batch arrives reordered — letters land scrambled. So the helper checks the foreground window class and, when it's RDP, sends one character at a time with a 2 ms sleep so ordering is preserved:

public static bool IsRdpForeground() {
    IntPtr h = GetForegroundWindow();
    StringBuilder sb = new StringBuilder(256);
    GetClassName(h, sb, sb.Capacity);
    string c = sb.ToString();
    return c == "TscShellContainerClass" || c.StartsWith("IHWindowClass") || c == "RAIL_WINDOW";
}

Local window: fast batch. RDP: throttled. Same code, decided per keystroke from the actual window in front.

Never press a bare Enter

One subtle rule for dictation into a chat app: the end of a message must never submit it. There is no "send" in the protocol, so the typer strips trailing newlines, and an interior newline is typed as Shift+Enter using scan codes (not virtual-key events). With vk-only events, Chromium/Electron apps sometimes dropped the Shift and treated the synthetic Enter as a bare Enter — which submits. Scan codes make the modifier register reliably, so a newline stays a newline.

// Left Shift down (scan 0x2A) → Enter down/up (scan 0x1C) → Left Shift up, all via KEYEVENTF_SCANCODE

Takeaways beyond dictation

When you emit a stream of near-identical outputs, diff against the last one. A common-prefix walk turns an O(n) retype into an O(changed) update.
If a per-event operation shells out, the spawn is usually the cost, not the work. One persistent process with a request/response marker fixes it.
Encode payloads you hand to a shell (base64) so quoting and code pages can't corrupt Unicode.
Synthetic input is not always delivered the way you queued it. Detect the hostile case (RDP) and slow down only there.

Bolo is on Google Play (search "Bolo AI"), and the open Windows companion is on GitHub releases. It types at the cursor over Wi-Fi; the clipboard is never touched by this path.

DEV Community