DEV Community

Cover image for How I Built an Ultra-Low Latency, Local-First Interview Copilot with Electron, React 19, and Groq Whisper Turbo
Vishwjeet Singh Vilkhu
Vishwjeet Singh Vilkhu

Posted on Originally published at wishpilot.vercel.app AI-assisted

How I Built an Ultra-Low Latency, Local-First Interview Copilot with Electron, React 19, and Groq Whisper Turbo

#webdev #javascript #ai #opensource

Most conversational AI speech assistants have a fatal flaw: crippling latency.

You ask a question, an audio buffer accumulates, an upload takes 2 seconds, speech-to-text takes another 3 seconds, and the LLM takes 4 seconds to respond. By the time an answer arrives, 8–10 seconds of awkward silence have passed.

In a high-stakes interview or live presentation, that delay is unacceptable.

Furthermore, most commercial interview tools force candidates to stream their raw voice audio, private resumes, and confidential credentials to opaque third-party cloud servers.

Over the past few weeks, I set out to solve this from scratch.

Today, I’m open-sourcing WishPilot — a universal, local-first stealth interview copilot and real-time speech intelligence engine built under the GNU General Public License (GPL v3).

In this article, I want to share the architectural decisions, low-level audio engineering, and streaming pipelines that make sub-second speech intelligence possible on the desktop.

The Architecture at a Glance

WishPilot is built on a high-speed desktop stack:

  • Desktop Runtime: Electron v44 with isolated context bridges
  • Frontend Architecture: React 19 + Vite 8
  • Speech Intelligence: Groq Whisper Large v3 Turbo (~180ms - 350ms STT)
  • Inference Gateway: Unified BYOK streaming supporting 9 providers (Cerebras, Groq, Together AI, Fireworks, NVIDIA NIM, OpenAI, Gemini)
  • Display Layer: Native Win32 WDA_EXCLUDEFROMCAPTURE

Here is how the data flows from raw sound waves to a floating heads-up display:

[ Microphone Signal ] 
       │ 
       ▼
[ Web Audio API / AudioWorkletNode (16kHz PCM) ]
       │  (< 20ms processing)
       ▼
[ Groq Whisper Large v3 Turbo ]  <-- ~250ms STT
       │
       ▼
[ Context Synthesizer (Resume + Category Domain + Vision Context) ]
       │
       ▼
[ Unified AI Streaming Engine (BYOK Direct HTTPS) ]  <-- Up to 1,800 tps
       │
       ▼
[ Candidate Floating HUD / Stealth Notch ]
       │  (Excluded from Zoom/Teams via WDA_EXCLUDEFROMCAPTURE)
Enter fullscreen mode Exit fullscreen mode

1. Sub-Second Audio Pipeline (Web Audio DSP)

To achieve true real-time performance, audio processing cannot happen on the main JavaScript UI thread.

WishPilot implements a dedicated AudioWorkletNode that taps into the browser's AudioContext. The worker:

  1. Captures native microphone signals (typically 44.1kHz or 48kHz).
  2. Performs low-pass filtering and downsamples the stream directly to single-channel 16,000 Hz 16-bit linear PCM.
  3. Runs client-side Voice Activity Detection (VAD) by monitoring root-mean-square (RMS) speech energy levels.

When a natural conversational pause is detected, the audio chunk is dispatched immediately to Groq's Whisper Large v3 Turbo endpoint via direct HTTPS.

The result? The spoken question is transcribed with technical vocabulary accuracy in under 250 milliseconds.

2. Zero Telemetry & The BYOK (Bring Your Own Key) Model

Privacy was my non-negotiable requirement.

WishPilot has zero central backend servers. No telemetry pings, no user tracking, and no database holding interview transcripts.

  • API keys are stored locally using encrypted desktop storage.
  • All requests flow directly from the user's desktop to the provider's official HTTPS endpoints (Groq, Cerebras, OpenAI, etc.).
  • There is zero middleman markup. Users can leverage generous free tiers from providers like Groq and Cerebras without spending a single dollar.

3. Native Windows Display Protection (WDA_EXCLUDEFROMCAPTURE)

A major engineering challenge with desktop copilots is preventing overlay windows from interfering with screen shares during mock technical drills or system design presentations.

WishPilot interfaces with the native Windows DWM compositor through an Electron native bridge:

// Native window display affinity hook
mainWindow.setContentProtection(true);
Enter fullscreen mode Exit fullscreen mode

Under the hood, Windows sets the window display affinity:

SetWindowDisplayAffinity(hwnd, WDA_EXCLUDEFROMCAPTURE);
Enter fullscreen mode Exit fullscreen mode

This ensures that while the floating heads-up display is crisp and visible to you on your physical monitor, it is completely excluded from OS-level graphics capture buffers (Zoom, Microsoft Teams, Google Meet, or Discord screen shares).

4. Instant Answer Refinement Pills

Interviews are dynamic. Sometimes an answer is too lengthy; other times an interviewer interrupts with: "Can you give me a production example?"

WishPilot features four contextual quick-action pills that transform streamed answers with one click:

  • Make Shorter: Compresses the answer into an ultra-punchy 15-20 second spoken elevator pitch.
  • More Technical: Injects architectural trade-offs, concurrency locks, and Big-O complexity.
  • Give an Example: Weaves a concrete production case study with measurable quantitative outcomes.
  • Simpler Language: Translates the solution into plain English using intuitive analogies.

5. Multi-Industry Category Engine

Most interview tools assume everyone is doing LeetCode. But real-world interviews span diverse domains.

WishPilot ships with native, domain-specific evaluation frameworks across 9 professional streams:

  1. IT & Software: Distributed architectures, LeetCode trade-offs, and CAP theorem.
  2. BPO & Voice Ops: LAST framework (Listen, Apologize, Solve, Thank) with empathy protocols.
  3. Finance & Banking: 3-Statement financial modeling, DCF, and WACC calculations.
  4. Sales & BD: BANT / SPIN selling frameworks and objection-handling hooks.
  5. HR & Talent: STAR behavioral methodology and labor compliance.
  6. Product Management: CIRCLES and RICE prioritization scoring.
  7. Healthcare & Clinical: SBAR clinical handoffs and triage protocols.
  8. Core Engineering: Root Cause Analysis (5-Whys) and Six Sigma reliability.
  9. Universal: Adaptive first-person authentic delivery.

Try It Out & Get Involved

WishPilot is 100% free and open-source under the GNU General Public License v3.0.

As an independent software engineer, building this in the open has been an incredible journey. If you find the project useful or want to support my work, a star on the repo or a cup of coffee on Buy Me a Coffee or GitHub Sponsors goes a long way!

I’d love to hear your thoughts, feedback, and architecture questions in the comments below!

Top comments (0)