DEV Community

StareBrain
StareBrain

Posted on

Why Your Android AI Agent Works in Dev and Breaks in Production

#ai #android #kotlin #a11y

There’s a gap between “it works on my device” and “it works on a user’s device” that Android accessibility-based agents fall into hard.

Here’s what it looks like, and what actually causes it.

The demo setup

In development, you’re running the agent on your own device. You’ve already granted BIND_ACCESSIBILITY_SERVICE. You know where the UI elements are. You’ve used the app enough that the accessibility tree is predictable.

Everything works.

What changes in production

  1. Accessibility permission isn’t granted by default

The user has to go to Settings → Accessibility → Installed Apps → enable your service manually. There’s no programmatic shortcut. You can deep-link them close, but they have to flip the toggle themselves.

Most users won’t do this unless the value is immediately obvious. Your onboarding has to earn that permission before asking for it.

`fun isAccessibilityServiceEnabled(context: Context): Boolean {
    val service = "${context.packageName}/${YourAccessibilityService::class.java.canonicalName}"
    val enabledServices = Settings.Secure.getString(
        context.contentResolver,
        Settings.Secure.ENABLED_ACCESSIBILITY_SERVICES
    ) ?: return false
    return enabledServices.contains(service)
}`
Enter fullscreen mode Exit fullscreen mode

Check this on every resume. Users revoke it without telling you.

  1. The accessibility tree varies by device and OEM

Samsung’s OneUI, Xiaomi’s MIUI, and stock Android expose the same apps differently in the accessibility tree. A node your agent finds reliably on a Pixel may not exist — or may have a different viewIdResourceName — on a Galaxy.

Don’t target by resource ID alone. Build a fallback chain:

`fun findSendButton(rootNode: AccessibilityNodeInfo): AccessibilityNodeInfo? {
    // Try resource ID first
    rootNode.findAccessibilityNodeInfosByViewId(
        "com.google.android.apps.messaging:id/send_message_button"
    ).firstOrNull()?.let { return it }

    // Fall back to content description
    rootNode.findAccessibilityNodeInfosByText("Send").firstOrNull()?.let { return it }

    // Fall back to class + position heuristic
    return findByClassAndBounds(rootNode, "android.widget.ImageButton")
}`
Enter fullscreen mode Exit fullscreen mode

Test on at least three OEMs before calling it production-ready.

  1. App updates break your agent silently

When WhatsApp or Gmail ships an update, resource IDs can change. Your agent doesn’t crash — it just can’t find the node and either fails silently or acts on the wrong element.

This is where the verification layer matters most. After every action:

`enum class ActionStatus {
    CONFIRMED,          // State verified — sent folder updated, etc.
    FAILED,             // State unchanged after action
    DENIED_UNRESOLVED   // State unverifiable — don't assume success
}`
Enter fullscreen mode Exit fullscreen mode

DENIED_UNRESOLVED is your safety net for the cases you can’t see coming. An app update ships at 2am, a node ID changes, your agent thinks it sent a message. Without this, that’s a silent success. With it, it’s a flagged unresolved action the user can check.

The honest summary

iOS doesn’t expose this layer the same way — what works on Android either isn’t available or requires entitlements Apple doesn’t grant easily. Android gives you the power, but the production surface is messier than the demo.

Build the permission check, build the fallback chain, build the verification layer. In that order.

StareBrain is an on-device AI agent for Android built on exactly this stack. Pre-launch — waitlist on the product page.

Top comments (0)