A voice notebook that can not phone home: on-device Whisper on Android

#android #productivity #opensource #ai

Most voice notes apps are recorders. You press a button, you get a file, and you never listen to it again.

I wanted the opposite: something that listens quietly, decides what was worth keeping, and writes it down. One file per session. No setup.

So I set one constraint before anything else: the app never gets the INTERNET permission. Not "we promise we don't upload." The manifest does not include it. If I want words out of it, they have to be made on the phone.

What it actually does

A foreground service keeps the mic hot, using Android's own microphone service type with an ongoing notification.
A small energy VAD runs over 16 kHz mono frames: a pre-roll so it catches the start of a sentence, a short hangover so it doesn't clip the end. Silence is never stored, and never leaves.
Speech segments go to whisper.cpp (base.en, q5_1, ~57 MB, bundled as an uncompressed asset, arm64).
A rules judge drops noise: fewer than four words, filler-only lines, near-duplicates by token overlap.
Kept lines append to one session-<date>.txt, and START/STOP produces a short local digest.

No cloud. No API key. No account. Works in airplane mode.

There is also a browser version with zero install if you just want the idea in ten seconds: https://tinyurl.com/22wmqo7q (that one uses the browser's speech engine, so it does need a network for the words).

Two things that actually broke

The first build used the system SpeechRecognizer. On Android 11+ the app cannot see the speech service unless the manifest declares a <queries> entry for android.speech.RecognitionService. Without it the recognizer never connects. It fails silently: no error, no exception, just nothing.
Distribution, not code, was the wall. A raw APK link on Android does not reliably open in a browser. It hands off to whichever app owns the link, and those apps cannot fetch a file. I ended up hosting a one-page site with a real button. That is not engineering, but it was the difference between "won't download" and "installed."

Why local-first

Speech models on a phone got good enough that the network stopped being necessary for accuracy. Once that is true, sending audio to a server is a choice, not a requirement. A voice notebook that can't phone home is a different product from one that merely says it won't.