Keep Platform Speech Recognition On-Device on Your First Mobile AI PR

You join the mobile AI squad on Tuesday with a Pixel 8a running Android 15 and a debug APK already installed. The dictation screen sits in the foreground with RECORD_AUDIO granted and SpeechRecognizer already listening. Nobody set the on-device recognizer extra, and a teammate still calls the platform API a local feature. You revoke microphone access from Settings, return without force-stopping, and watch for a leftover network socket.

That permission transition is the real first hour on this squad, not a demo after standup. You should not polish a stakeholder transcript before the recognizer is proven to stay local. Prove that speech cannot leave the device after a revoke, and that you can roll the flag back. A prior commit may already have opened the vendor cloud path without anyone noticing in review.

Treat the platform recognizer as a network API until proven otherwise

Android SpeechRecognizer and iOS SFSpeechRecognizer look on-device because the UI never leaves your activity. Both APIs can ship audio to a vendor cloud unless you opt into the on-device recognizer. Your first PR should make that opt-in explicit, testable, and reversible rather than burying it in a helper. Reviewers should reject the change without guessing which constructor the production binary actually calls.

A permission revoke does not always cancel work that already started on a background thread. Partial results, pending listeners, and cached PCM can outlive the Settings toggle by several seconds. You need a lifecycle experiment on hardware, not another prompt change in the dictation copy. Treat every "it is local by default" comment in Slack as unverified until a capture says so.

What on-device has to mean in the PR description

Write the PR so a reviewer can reject it without guessing:

• The production recognizer is created with the on-device constructor or the on-device request flag.
• If the device has no on-device model, the UI fails closed and never opens a network recognizer.
• A microphone revoke stops listening, drops in-memory PCM, and does not start cloud work.
• A feature flag can restore a disabled state in one release if the on-device pack is missing.

Those four bullets are the golden behavior for this merge. Anything else is a story about accuracy in a quiet room.

A proposed single-device experiment, not a benchmark

Do not publish latency numbers from this walkthrough because it is only a pass or fail check. Record the device, OS build, application state, and the exact Settings transition beside every note. Label the whole protocol as proposed until you have those fields filled on a physical phone. A missing speech pack is a valid outcome, and you should write it down before blaming the code.

Suggested environment

• Device: one physical phone you can revoke permissions on, not an emulator image.
• OS: Android 12+ so createOnDeviceSpeechRecognizer() exists, or iOS 17+ with on-device speech.
• App state: foreground dictation activity, process warm, screen on, charger connected.
• Network: Wi-Fi attached so a mistaken cloud path would have a route.
• Permission: RECORD_AUDIO or Microphone granted at start, then revoked from system Settings.
• Power: stay off battery saver so the OS does not silently throttle listeners.

Emulators often fake speech services and will lie about on-device availability. Skip them for this PR, even when the Dart or Kotlin unit tests already pass on CI.

Exact steps

1. Install a debug build whose application ID matches the one you will inspect with adb or Console.
2. Confirm the on-device speech pack is installed, or write down that it is missing before you start.
3. Open the dictation screen and start listening with a phrase you chose in advance.
4. Leave the process running, switch to Settings, and revoke microphone access for the app.
5. Return to the dictation screen without force-stopping, and wait thirty seconds.
6. Capture logs, a proxy session, and the UI state, then decide whether audio recovered, restarted, or vanished.

Expected observations if the PR is honest

Write the expected result before you touch Settings, then compare it with the capture. After revoke, listening should stop and the UI should show a permission error, not a spinner. The proxy should record no new TLS client hello to speech endpoints during those thirty seconds. Log lines should not print the audio sample, the partial transcript, or a cloud hostname.

If the buffer silently disappears with no error, that is still a defect for reviewers. The reviewer cannot tell a clean cancel from a swallowed upload without an explicit permission error. Killing and restarting the app must not replay the pre-revoke buffer either. Silent success is how leaked audio hides in a first-week merge.

Guard the Android constructor before you touch UI copy

The platform gives you two constructors, and the older factory is the network-friendly default path. Your first PR should call the on-device factory and refuse to fall back to the legacy one. Fail closed when the speech pack is missing, even if that makes the dictation button show an error. Do not hide a cloud recognizer behind a catch block that "keeps the feature working" on older phones.

// Proposed production path. Confirm on a physical device before you merge.
fun buildRecognizer(context: Context): SpeechRecognizer {
    if (!SpeechRecognizer.isOnDeviceRecognitionAvailable(context)) {
        error("on-device speech pack missing; fail closed")
    }
    return SpeechRecognizer.createOnDeviceSpeechRecognizer(context)
}

fun buildIntent(locale: Locale): Intent {
    return Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH).apply {
        putExtra(RecognizerIntent.EXTRA_LANGUAGE, locale.toLanguageTag())
        putExtra(RecognizerIntent.EXTRA_PREFER_OFFLINE, true)
        putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS, true)
    }
}

fun onPermissionRevoked(recognizer: SpeechRecognizer?, buffer: ByteArray?) {
    recognizer?.cancel()
    recognizer?.destroy()
    buffer?.fill(0)
}

Wire the revoke path to onStop and to a permission check when the activity resumes from Settings. Do not wait for a future onError from the recognizer, because that callback can arrive late. A late onError often means a network retry already started on a thread you do not control. Clear the PCM buffer in the same function that destroys the recognizer so logs cannot dump it later.

# Proposed checks for your first PR, run on the local debug APK.
unzip -p app-debug.apk classes*.dex | strings | grep -c createSpeechRecognizer
unzip -p app-debug.apk classes*.dex | strings | grep -c createOnDeviceSpeechRecognizer
adb shell dumpsys package com.example.dictation | grep RECORD_AUDIO
adb shell appops get com.example.dictation RECORD_AUDIO
adb logcat -s SpeechRecognizer:* ActivityManager:* | tee first-hour-speech.log

You want the default factory count at zero in release dex, and the on-device factory present. App ops should read deny after the Settings revoke, and the UI should match that denied state. If strings still contain a speech hostname, treat the PR as failed even when the UI looks correct. Run the same grep on the AAB you plan to ship, not only on the local debug APK.

Guard the iOS request flag the same morning

SFSpeechRecognizer can still hop to Apple servers when requiresOnDeviceRecognition stays false on the request. Set the flag first, then fail closed if the locale cannot run recognition on the device. Do not keep a second SFSpeechRecognitionRequest around for "when the user is on Wi-Fi." A Wi-Fi-only fallback is still a cloud path, and it will surprise you during the revoke test.

// Proposed production path. Confirm on a physical iPhone before you merge.
func makeRequest(locale: Locale) throws -> SFSpeechAudioBufferRecognitionRequest {
    let recognizer = SFSpeechRecognizer(locale: locale)
    guard let recognizer, recognizer.supportsOnDeviceRecognition else {
        throw SpeechGuardError.onDeviceModelMissing
    }
    let request = SFSpeechAudioBufferRecognitionRequest()
    request.requiresOnDeviceRecognition = true
    request.shouldReportPartialResults = true
    request.taskHint = .dictation
    return request
}

func dropSession(engine: AVAudioEngine, buffer: AVAudioPCMBuffer?) {
    engine.stop()
    engine.inputNode.removeTap(onBus: 0)
    if let channel = buffer?.floatChannelData?.pointee {
        channel.update(repeating: 0, count: Int(buffer!.frameLength))
    }
}

# Proposed Console filter while you revoke Microphone in Settings.
log stream --predicate 'subsystem CONTAINS "Speech"' --level debug

Juniors often dump debug WAV files into Documents during the first hour of pairing on a loaner phone. That directory can leave the device through a routine iCloud backup you will not see in Charles. Mark every debug recording excluded from backup, or write it only to a cache you delete on revoke. Never commit those WAV files to the repository, even when they contain only your own voice.

try FileManager.default.setAttributes(
    [URLResourceKey.isExcludedFromBackupKey: true],
    ofItemAtPath: debugWav.path
)

Keep eval phrases off the user's microphone

You still need a tiny golden set so reviewers can exercise the recognizer without recording coworkers. Draft twenty short phrases in the target locale, then speak them on the device at a known volume. Do not upload those recordings to a shared Drive folder, and do not commit WAV files into git. Keep the phrase list in the PR description as text, not as binary fixtures on a public fork.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you want a second pair of eyes while drafting that phrase list, MonkeyCode's free model access can help. Use the free server option on the eval laptop so the host never lives inside the mobile binary. Keep that hostname out of APK strings, out of plist files, and out of any runtime fallback URL. The phone's only job is on-device recognition plus a fail-closed error when the pack is missing.

Rollback if a prior commit already opened the network path

Rehearse rollback in the same PR, because on-device packs are missing on some retail SKUs. Ship a remote flag that chooses the constructor, defaulting to on-device, with a one-release kill switch. If you already shipped a network recognizer, do not fix forward by adding a second constructor. Cut a patch that forces disabled, wait until proxy traffic goes quiet, then ship the on-device flag.

Flag value	Constructor	When to use	Risk if stuck
on_device	createOnDeviceSpeechRecognizer / requiresOnDeviceRecognition = true	Default after this PR	Some devices show "model missing"
disabled	No recognizer created	Emergency rollback	Feature disappears, no audio leaves
legacy_network	Do not ship this value	Not for production	Audio can leave the device

If you already shipped legacy_network, record the store version that still contained the network constructor. Support needs that version number so they can tell users to update before the next privacy review. Do not leave both constructors in the binary behind an undocumented debug extra on the intent. Undocumented extras are how the default factory sneaks back in during a "temporary" support build.

Limitations, and who should not merge this as-is

This walkthrough does not claim packet captures, battery figures, or word-error rates from a device class. On-device packs vary by locale, OEM image, and whether the user downloaded a speech language model. Zeroing a PCM buffer does not erase copies already sitting in logd, a vendor HAL, or accessibility. Treat accessibility services and OEM voice assistants as separate trust boundaries you cannot control from app code.

Do not use this approach if you legally operate cloud STT with a consent screen and a DPA. Do not use it if you only have emulators, because they will not load the same speech packs. Flutter and React Native plugins often call the default Android constructor even when Dart says offline. Read the plugin source for createSpeechRecognizer before you trust a boolean named onDevice in Dart.

Tell the channel what you actually saw

Reply with the device model, OS build, and whether the on-device speech pack was actually installed. Include the exact Settings transition, and say whether you returned to the app without force-stopping. Say whether listening recovered, restarted after an error, or silently disappeared from the dictation UI. That field report is more useful to the next junior than another accuracy screenshot from a quiet office.

Comments

[雷笛歌]

I like the “network API until proven otherwise” framing. One Android gotcha is that a final/error callback can still arrive after the mic permission is revoked, so checking only onError can leave stale partials updating the UI. I’d invalidate a session token on stop and ignore callbacks from the old recognizer; that makes the revoke-and-return test much less ambiguous.