Stop Wrapping OpenAI: How We Built Native Offline Whisper Apps for macOS and Windows

Let's be completely honest: the world does not need another SaaS that charges $15/month just to forward an MP3 file to api.openai.com/v1/audio/transcriptions.

Besides the obvious margin compression and high churn of "API wrappers", there is an insurmountable architectural wall: Enterprise & Privacy Compliance.

If you're dealing with journalists holding source recordings, doctors logging telemedicine consultations, or legal teams transcribing depositions, you cannot touch AWS S3 or third-party cloud APIs. Period.

We spent the last few weeks building a native, 100% offline desktop transcription workstation for both macOS and Windows. Here is the technical breakdown of what worked, what blew up in our faces, and how we solved local model inference.

┌────────────────────────────────────────────────────────┐
│                   Native Desktop App                   │
├──────────────────────────┬─────────────────────────────┤
│      macOS Silicon       │      Windows x64 / CUDA     │
│   (Swift + WKWebView)    │   (PyWebView + WebView2)    │
├──────────────────────────┼─────────────────────────────┤
│ MLX-Whisper (Metal GPU)  │ Faster-Whisper (CUDA float16│
│ CAM++ Diarization        │ CAM++ Diarization (NVIDIA)  │
└──────────────────────────┴─────────────────────────────┘
          ▼                             ▼
   [Local Flash Storage Only — Zero Cloud Network Requests]

The macOS Stack: Embracing MLX

Running standard faster-whisper (CTranslate2) on Apple Silicon is a trap. CTranslate2 has no native Metal Performance Shaders (MPS) backend for Whisper, falling back to pure CPU computation. It pegs all performance cores to 100%, turns the fans into jet engines, and throttles.

We migrated the Mac pipeline to Apple's native mlx-whisper.

• Unified Memory Advantage: MLX loads the FP16 weights directly into Apple Silicon's unified memory pool. Both the CPU and GPU share the buffer with zero memory copies.
• Speed: An M2 Pro processes a 1-hour 16kHz audio file in under 3.5 minutes offline.
• Speaker Diarization: We paired Whisper with a local CAM++ acoustic embedding model. The app clusters audio segments by speaker voiceprints locally on disk.

The Windows Stack: Banishing Electron & Taming CUDA

On Windows, bundling Python machine learning apps usually ends up in one of two disasters:

1. An Electron monster that consumes 1.8GB of RAM before audio even loads.
2. A mess of terminal command prompts that terrifies non-technical users with CUDA_HOME not found.

Here's how we engineered the Windows native client:

• Zero-Electron Shell: We built the GUI with React, but render it using pywebview binding directly to Windows 11's built-in WebView2Loader.dll (Chromium Edge engine). The memory footprint dropped from ~1.5GB to <120MB.
• Bundled CUDA 12.4 Runtime: Standard pip install torch on Windows only installs CPU wheels. We baked in explicit PyTorch CUDA 12.4 bindings and dynamic fallback logic: cuda (float16) -> cuda (int8) -> cpu (int8).
• Single-Instance Mutex & SQLite WAL: When users double-click an .exe multiple times during the initial 15-second model load, PyInstaller child processes can race on SQLite tables. We enforced a Windows Named Mutex (Local\FreeAudioToText_SingleInstance) and enabled SQLite Write-Ahead Logging (PRAGMA journal_mode=WAL;).

What We Learned

Building "local-first" software is infinitely more frustrating than spinning up a serverless API route, but the user sentiment is on a completely different level. When users realize they can transcribe a confidential 3-hour audio file without an internet connection, their entire perception of AI software changes.

We packaged the binaries for both platforms at freeaudiototext.com. If you're building local-first ML tools, stop paying the cloud tax and let your users' local silicon do the heavy lifting.