DEV Community

Cover image for Past the Tap: Making Compose Screens Behave When the Mouse, Keyboard, and Stylus Show Up
Android Expert
Android Expert

Posted on Originally published at Medium

Past the Tap: Making Compose Screens Behave When the Mouse, Keyboard, and Stylus Show Up

Your app was tested with a thumb. Then a user opens it on a Chromebook, docks a foldable to a keyboard, or scribbles in a notes screen with a stylus, and a dozen assumptions fail quietly. Nothing crashes at first. The right-click does nothing, Tab skips the primary button, and holding Ctrl+S writes the file nine times.

This post walks through how input actually flows in Jetpack Compose, the idiomatic Kotlin for handling each device, and the failure modes that tend to reach production because nobody had a mouse on their desk during QA.

The mental model: two lanes, not one

Touch handling trains us to think in “taps.” Compose does not. It routes pointer events and key events through different machinery, and mixing them up is the root of most bugs below.

input-event-flow

Pointer events (touch, mouse, trackpad, stylus) are hit-tested against the nodes under the pointer, then delivered in passes: initial (parent to child, so a parent can intercept), main (child to parent, where most code lives), and final. Key events skip hit testing entirely. They are routed by focus. In the preview pass (onPreviewKeyEvent) the event travels from the root down to the focused node, so an ancestor can intercept it first. If nobody does, the regular pass (onKeyEvent) starts at the focused node and bubbles back up toward the root.

That one difference explains a large share of “my shortcut doesn’t fire” reports. A handler fires only when focus sits on its own node or on one of its descendants, so a handler on a subtree where nothing ever takes focus is dead code.

input-capability-matrix

Pointer input that knows what kind of pointer it is

Every PointerInputChange carries a type. Use it. A single gesture detector that treats all pointers alike is how you end up with a stylus that scrolls instead of draws.

@Composable
fun Whiteboard(
    onStroke: (Offset, Float) -> Unit,
    onContextMenu: (Offset) -> Unit,
    onZoomStep: (Float) -> Unit,
    modifier: Modifier = Modifier,
) {
    // Always call the newest lambdas, even though pointerInput(Unit) never restarts.
    val latestStroke by rememberUpdatedState(onStroke)
    val latestMenu by rememberUpdatedState(onContextMenu)
    val latestZoom by rememberUpdatedState(onZoomStep)

    Box(
        modifier.pointerInput(Unit) {
            awaitPointerEventScope {
                while (true) {
                    // Initial pass: we want the wheel before a scrollable ancestor swallows it.
                    val event = awaitPointerEvent(PointerEventPass.Initial)
                    // Pick the pointer we care about; changes[0] is not always the stylus.
                    val change = event.changes.firstOrNull { it.type == PointerType.Stylus }
                        ?: event.changes.firstOrNull()
                        ?: continue

                    when {
                        event.type == PointerEventType.Scroll &&
                            event.keyboardModifiers.isCtrlPressed -> {
                            latestZoom(if (change.scrollDelta.y < 0f) 1.1f else 0.9f)
                            change.consume()
                        }
                        event.type == PointerEventType.Press &&
                            event.buttons.isSecondaryPressed -> {
                            latestMenu(change.position)
                            change.consume()
                        }
                        change.pressed &&
                            change.type == PointerType.Stylus &&
                            (event.type == PointerEventType.Press ||
                                event.type == PointerEventType.Move) -> {
                            latestStroke(change.position, change.pressure)
                            change.consume()
                        }
                    }
                }
            }
        }
    )
}
Enter fullscreen mode Exit fullscreen mode

Imports note: isCtrlPressed here is the PointerKeyboardModifiers extension from androidx.compose.ui.input.pointer, not the KeyEvent one from androidx.compose.ui.input.key. On Android these flags come from the underlying MotionEvent's meta state, so verify them on your target devices. This sketch also draws on every stylus move; a production canvas would accumulate points into a path per stroke.

Three details are worth calling out. rememberUpdatedState lets the long-lived pointerInput(Unit) block see fresh callbacks. The initial pass is what lets Ctrl+wheel zoom win against a scrolling parent. And consume() is how you stop other handlers, such as a parent's gesture detector or a scrollable, from also reacting to the same event. Current clickable implementations respond only to the primary button, but that was not always true everywhere: Compose Multiplatform desktop reacted to any mouse button until a 2022 change restricted it. A custom right-click handler should therefore consume the press so parent pointer handlers don't also act on it, and you should confirm the behavior on the versions and platforms you ship.

Hover is state, so model it as state

@Composable
fun ToolbarButton(label: String, onClick: () -> Unit) {
    val interaction = remember { MutableInteractionSource() }
    val hovered by interaction.collectIsHoveredAsState()

    Text(
        text = label,
        modifier = Modifier
            .hoverable(interaction)
            .clickable(
                interactionSource = interaction,
                onClick = onClick,
            )
            .background(if (hovered) Color(0xFFEFF1FA) else Color.Transparent)
            .padding(horizontal = 12.dp, vertical = 8.dp),
    )
}
Enter fullscreen mode Exit fullscreen mode

On recent Foundation versions (1.7+) clickable picks up the default indication itself, so there's no need to pass LocalIndication.current; on older versions, pass an indication explicitly. Sharing one MutableInteractionSource between hoverable and clickable keeps hover, press, and focus visuals consistent instead of fighting each other.

Keyboard: focus first, shortcuts second

Shortcuts belong on a node that actually owns focus, or on an ancestor using the preview pass so it sees keys on the way down.

fun Modifier.shortcut(
    key: Key,
    ctrl: Boolean = true,
    action: () -> Unit,
): Modifier = onPreviewKeyEvent { e ->
    val isMatch = e.type == KeyEventType.KeyDown &&
        e.key == key &&
        e.isCtrlPressed == ctrl &&
        !e.isAltPressed && !e.isMetaPressed && !e.isShiftPressed &&
        e.nativeKeyEvent.repeatCount == 0   // Android-only: ignore auto-repeat
    if (isMatch) { action(); true } else false
}

@Composable
fun EditorScreen(onSave: () -> Unit) {
    val focus = remember { FocusRequester() }

    Box(
        Modifier
            .focusRequester(focus)
            .focusable()
            .shortcut(Key.S, action = onSave)
    ) { /* editor content */ }

    LaunchedEffect(Unit) { focus.requestFocus() }
}
Enter fullscreen mode Exit fullscreen mode

Checking Alt, Meta and Shift matters: without it, ctrl = false would match every plain key press, and Ctrl+S would also trigger a Ctrl+Shift+S binding. nativeKeyEvent ties this helper to Android; a multiplatform codebase needs its own repeat tracking instead.

Returning true marks the event handled. Returning false lets it keep travelling, which is exactly what you want for keys you don't care about.

key-event-repeat

The timeline above is the bug the repeatCount == 0 check prevents. Operating systems send repeated key-down events while a key is held, and a naive KeyDown handler treats each as a fresh press.

For traversal order, don’t fight Tab. Describe the order you want with focusProperties:

Modifier.focusProperties {
    next = nextFieldRequester
    previous = previousFieldRequester
}
Enter fullscreen mode Exit fullscreen mode

Stylus: pressure is data, palms are noise

A stylus reports pressure and can be an eraser. Treat both as first-class:

val strokeWidth = lerp(2f, 14f, change.pressure.coerceIn(0f, 1f))
val erasing = change.type == PointerType.Eraser
Enter fullscreen mode Exit fullscreen mode

When a stylus is down, ignore PointerType.Touch pointers in the same surface. A resting palm is just another finger as far as your gesture code is concerned, and it will pan the canvas mid-stroke if you let it.

Six failure modes that reach production

failure-modes

1. Stale lambdas inside pointerInput. pointerInput(Unit) captures the lambdas from the first composition. Your gesture keeps invoking an old callback after state has moved on. Either key the modifier on what it reads, or wrap callbacks with rememberUpdatedState as shown earlier.

2. Requesting focus before the node exists. FocusRequester.requestFocus() throws an IllegalStateException ("FocusRequester is not initialized") if no node is attached to that requester yet. It typically bites when the target sits behind a condition or inside a lazy item, and the LaunchedEffect lives somewhere the modifier is not guaranteed to have been applied. Waiting a frame is a guess, not a guarantee. Co-locate the request with the modifier instead:

@Composable
fun EditorHost(isEditorVisible: Boolean) {
    if (isEditorVisible) Editor()      // the effect is inside, with the modifier
}

@Composable
private fun Editor() {
    val focus = remember { FocusRequester() }
    Box(Modifier.focusRequester(focus).focusable()) { /* content */ }
    LaunchedEffect(Unit) { focus.requestFocus() }   // normally runs once Editor is in the tree
}
Enter fullscreen mode Exit fullscreen mode

This covers the common case, but effects and node attachment are not a hard contract. Reports exist of the same crash when focus is requested while an animation or a lazy subcomposition is still settling. If you hit that, request focus after the first layout instead, once, from a callback that only fires when the node is placed:

@Composable
private fun Editor() {
    val focus = remember { FocusRequester() }
    var requested by remember { mutableStateOf(false) }

    Box(
        Modifier
            .focusRequester(focus)
            .onGloballyPositioned {
                if (!requested) { requested = true; focus.requestFocus() }
            }
            .focusable()
    ) { /* content */ }
}
Enter fullscreen mode Exit fullscreen mode

Don’t reach for focusRestorer() as a substitute here. It restores focus to a previously focused child; it doesn't guarantee a requester is attached.

Inside lazy lists, request focus in response to an event or a restored-focus callback rather than at first composition.

3. Key-repeat double fire. Covered above. Filter on repeatCount, and decide deliberately whether any shortcut (say, arrow-key nudging) should repeat.

4. Click-only affordances. If a control is built from pointerInput { detectTapGestures }, it has no focus, no hover, no semantics, and no keyboard activation. Prefer clickable, toggleable, or selectable, which wire all of that for you. Reserve raw pointer handling for the canvas-like surfaces that truly need it.

5. Activity recreation on device attach. Plugging in a keyboard or mouse is a configuration change. Unless the Activity opts out, it is recreated and any state held outside rememberSaveable or a ViewModel disappears mid-typing. Declare what you handle:

<activity
    android:name=".MainActivity"
    android:configChanges="keyboard|keyboardHidden|navigation|screenSize|smallestScreenSize|screenLayout|orientation" />
Enter fullscreen mode Exit fullscreen mode

Opting out means you own the responsibility of reacting to the new configuration, so test it with real attach and detach, not just emulator defaults.

6. Gesture versus scroll fights. If your handler consumes a scroll event, the list behind it stops scrolling. If you never consume it, both react. Pick an owner per modifier combination: plain wheel scrolls, Ctrl+wheel zooms, and nothing else is consumed.

Test it without owning every device

Compose’s test APIs can simulate these inputs, so regressions get caught in CI:

@Test
fun ctrlS_savesOnce() {
    var saves = 0
    composeRule.setContent { EditorScreen(onSave = { saves++ }) }

    composeRule.onRoot().performKeyInput {
        keyDown(Key.CtrlLeft)
        pressKey(Key.S)
        keyUp(Key.CtrlLeft)
    }
    assertEquals(1, saves)
}

@Test
fun rightClick_opensMenu() {
    composeRule.setContent { /* screen under test */ }
    composeRule.onNodeWithTag("canvas").performMouseInput { rightClick() }
    composeRule.onNodeWithTag("context_menu").assertIsDisplayed()
}
Enter fullscreen mode Exit fullscreen mode

Pair automated tests with one manual pass on a real desktop-class device. Emulated input will never feel like a worn-out trackpad.

A short checklist before you ship

  • Every interactive element is reachable and activatable with Tab and Enter or Space.
  • Hover has a visual response, and right-click has a defined meaning or is deliberately ignored.
  • Shortcuts use the preview pass, check key-down only, and ignore repeats.
  • Stylus pressure and eraser are read, and touch is ignored while the pen is down.
  • Attaching and detaching hardware never loses user input.

Input support is rarely one big feature. It is a pile of small contracts between your UI and a device you didn’t picture. Test each contract once, and the screen feels native on hardware you never owned.


📱 Crack Your Next Android Interview

If you’re targeting a senior Android role at a FAANG or top-tier MNC, syntax knowledge alone won’t get you through the loop. I highly recommend “Cracking Top MNC Android Interviews” — a senior-level playbook covering framework internals, Jetpack Compose, coroutines, Android-specific system design, and DSA reframed around real mobile constraints, to help you walk into bar-raiser rounds ready.

Top comments (0)