How I built a floating dictation orb on Android, and why the accessibility service is the whole app

Ayanda Phaketsi — Fri, 28 Aug 2026 03:57:49 +0000

I talk in two languages at once. Not alternately, in the same sentence. English and my home language, mixed, the way most people around me speak.

Every dictation tool I tried broke on that. Wispr Flow is genuinely good software and I am not pretending otherwise, but like the others it picks a language and pushes everything through it, and what comes back is something I have to retype. Gemini was the first model that gave me my actual sentences back. That was the trigger for building RMBLR, an Android app that puts a dictation orb over whatever you are typing in.

The interesting part turned out not to be the transcription. It was knowing when to appear.

The problem: which app owns the cursor?

I did not want another floating bubble living on the home screen. The orb should exist when a text field has focus and not otherwise.

Android gives you exactly one API that can answer "is there a caret in an editable field right now" for an app you did not write: an AccessibilityService. Nothing else sees another process's view tree.

private fun currentFieldIsEditable(): Boolean {
    val node = runCatching { findFocus(AccessibilityNodeInfo.FOCUS_INPUT) }.getOrNull()
        ?: return false
    val editable = node.isEditable || node.className?.contains("EditText") == true
    node.recycle()
    return editable
}

FOCUS_INPUT is the important constant. FOCUS_ACCESSIBILITY is where TalkBack's green box is, which is not the same thing.

isEditable alone is not enough in practice. Plenty of older views and WebView inputs report false while behaving like text boxes, so the EditText class-name check earns its place. Recycle the node: you run this on every event, and leaking AccessibilityNodeInfo will bite you.

In res/xml/accessibility_service_config.xml I subscribe to focus, click, text changed, text selection changed, window state and window content, with android:notificationTimeout="80". Higher and the orb visibly lags behind the keyboard. Lower and you re-walk the tree constantly for nothing.

The overlay must not want focus

Second half of the same problem. The orb is a TYPE_APPLICATION_OVERLAY window, and if it takes focus, the text field loses the caret, and now findFocus(FOCUS_INPUT) returns null and the orb hides itself. It fights itself out of existence.

WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE or
    WindowManager.LayoutParams.FLAG_LAYOUT_NO_LIMITS

FLAG_NOT_FOCUSABLE keeps the caret where it was. FLAG_LAYOUT_NO_LIMITS lets you park the orb under the status bar cutout instead of being shoved inside the safe area. When the overlay genuinely needs typed input, clear the flag, call updateViewLayout, and set it again afterwards.

Getting text back in uses the same service. Paste first, because it respects the caret and the host app's own input handling, and only rewrite the field if paste is refused:

clipboard?.setPrimaryClip(ClipData.newPlainText("RMBLR", text))
if (node.performAction(AccessibilityNodeInfo.ACTION_PASTE)) return true
// otherwise splice at textSelectionStart/textSelectionEnd and ACTION_SET_TEXT

ACTION_SET_TEXT replaces everything in the field, so it is the fallback and not the default. Half a draft message disappearing is a much worse bug than a failed dictation.

The gesture arc

Hold the orb and four tone chips fan out. There is no Composable for this and I would not use one anyway: the whole interaction is one uninterrupted finger press, so it is a single onTouch on the overlay root.

The rules that made it feel right:

Long press fires at 300ms.
Movement past 12dp before that cancels the long press and starts a drag instead. Without the slop, picking the orb up opens the menu every time.
Selection is by angle, not by quadrant or hit box. Take the angle from the touch origin to your finger, compare it against each chip's angle, pick the smallest difference. Distance stops mattering past a 46dp dead zone, so you can aim fast and sloppily and still land the chip you wanted.
A flick is travel over 90dp in under 300ms, and it runs the chip in that direction without the menu ever being drawn.

Four chips, not seven. Every chip you add shrinks the angle each one owns, and aiming gets worse for everyone.

Streaming, because latency lands in the worst place

The first version recorded the clip, then on release opened a socket, uploaded everything and waited. All of that latency landed after I stopped talking, which is exactly the moment I am staring at the screen waiting for words.

Now the socket opens on press. Audio goes up in 200ms chunks as the recorder produces them, and transcript deltas come back while I am still mid-sentence. By release there is usually nothing left to do but read the buffer. Total time barely moved. Perceived time is a different app, and that is the number people actually feel.

Two things cost me an evening each:

Server-side voice detection lies to you about completion. turnComplete and generationComplete arrive every time you pause for breath. They do not mean the user has finished. What ends the session is the input transcript going quiet for a fixed window after the microphone has stopped.

Keep the WAV anyway. I buffer the full audio even on the streaming path. Sockets die on flaky mobile data, and holding the bytes means a dead stream falls back to the batch endpoint instead of losing someone's dictation.

What it costs

Nothing, plus whatever your API provider charges. It is MIT, there is no account and no subscription, and you paste in your own key: Gemini, Groq, Mistral, OpenRouter, OpenAI, or any OpenAI-compatible endpoint you host. Nothing goes through a server I own, which is cheap for me and, more usefully, checkable by you.