For the last few months, I’ve been obsessed with a specific problem: the friction between privacy and utility in modern AI tools. Most "private" AI solutions still rely on a local LLM running on your CPU or GPU via a heavy desktop application. They require installation, constant background processes, and often struggle with performance on older hardware.
I wanted to see if we could do better. I wanted to see if we could run a capable language model entirely within the browser, using only the device’s hardware acceleration, with zero data leaving the machine.
The result is PrivateScribe, a tool I built to handle note summarization, email drafting, and rewriting. But more importantly, it’s an experiment in what’s possible when you treat the browser not just as a display layer, but as a compute engine.
The Wedge: WebGPU and True Offline
The core constraint that drove this project was simple: nothing leaves the device.
In the current landscape, "on-device AI" often means "installed on your device." This is fine for desktop apps, but it creates silos. You can’t easily share a workflow across a Chromebook, a Windows machine, and an iPad without installing three different native applications.
By leveraging WebGPU, PrivateScribe runs entirely in the browser. This unlocks a few critical advantages:
- Zero Installation: Users open a URL and start working. No downloads, no permission dialogs for file system access beyond what’s needed for the session.
- Hardware Acceleration: WebGPU allows the browser to tap directly into the GPU. This is crucial for inference speed. A small model that runs in your browser can process text significantly faster than a CPU-bound implementation, especially on modern laptops with integrated graphics.
- True Offline Capability: Because the model weights are loaded locally via WebAssembly and the inference happens on-device, the app works completely offline. If you lose your internet connection in the middle of drafting an email, the AI doesn’t stop. It continues to function because it isn’t waiting for an API response from a cloud server.
The Developer Experience
Building this presented several unique challenges. The most significant was managing memory. Browsers have strict memory limits, and loading a quantized model into the GPU memory requires careful handling. I had to ensure that the model could be loaded, used for inference, and then released without causing the tab to crash or the browser to become unresponsive.
Another challenge was latency. Even with WebGPU, inference isn’t instant. I implemented a streaming token output system that mimics the "typing" effect of cloud-based AI. This doesn’t just make the experience feel smoother; it also provides immediate feedback to the user, reducing the perceived wait time.
The codebase is relatively lean. It relies on a standard JavaScript/TypeScript stack for the frontend, with a custom inference engine that interfaces with the WebGPU API. There’s no backend server involved in the AI processing. The only server interaction is for authentication and saving your notes to your own storage if you choose to sync them.
Privacy by Design
Privacy isn’t a feature here; it’s the architecture. Since the AI processing happens locally, there is no data to leak. The model never sees your raw text in a way that can be logged or used for training. This is a significant shift for users who are wary of sending their personal notes and draft emails to third-party cloud providers.
This approach also means that the intelligence is tailored to the immediate context. The model doesn’t have access to a vast database of your previous interactions unless you explicitly provide them in the current session. This forces a cleaner, more focused interaction model.
Practical Usage
PrivateScribe is designed for two main workflows:
- Note Summarization: Paste a long article, meeting transcript, or document. The AI generates a concise summary, extracting key points and action items.
- Email Drafting and Rewriting: Draft a quick, rough email and ask the AI to refine the tone, fix grammar, or make it more concise. Alternatively, paste an incoming email and get a suggested reply.
The interface is minimal. There are no complex settings to tweak. You paste text, select a function, and get results. The goal is to reduce cognitive load, not add to it.
Pricing and Access
PrivateScribe is a paid tool, designed to be sustainable for long-term development. There is a 7-day trial that allows you to test the full functionality without commitment. For users who prefer to try before they buy, there are also free turns available in the associated AI games, which serve as a sandbox for the same underlying technology.
Why This Matters
The trend of moving AI to the edge is well-documented, but most implementations require native apps. By proving that a usable, fast, and private AI experience can be delivered via WebGPU in a browser, we open the door for more lightweight, cross-platform AI tools.
This isn’t just about avoiding cloud costs. It’s about control. It’s about having a tool that works when the internet is down, respects your data by design, and doesn’t require you to change your browser habits to use it.
If you’re curious about the technical implementation or
Top comments (0)