Vibe coding mobile apps with Compose Driver

#ai #android #kotlin #compose

I've been experimenting with AI coding assistants like Antigravity lately and, as an Android engineer, it's been quite fun using them for hobby Compose projects.

However, the feedback loop is often much tighter for web development, where these AI tools usually have browser instrumentation to inspect the DOM and verify their work instantly. As mobile developers, we feel a bit left out. Asking an AI to build an Android screen usually means generating code that can't be checked without manually running the app on a device or emulator.

So I built Compose Driver to bridge that gap.

What is it?

Compose Driver is a library and Gradle plugin that lets AI agents "drive" your Jetpack Compose app. It works by wrapping your UI in a test harness that listens for HTTP requests.

This means you can have an AI agent:

Query the UI to see what buttons or text are on the screen.
Interact by clicking, swiping, or typing.
Verify the result by printing the UI tree, taking a screenshot or recording a GIF.

It runs on the JVM headlessly, so it's very fast and runs anywhere, making it perfect for background or cloud agents. It supports both Desktop/Multiplatform Compose and Android Jetpack Compose (via Robolectric).

How it works

The core of the implementation is actually pretty simple and less than 300 lines. It starts a small local server that translates HTTP requests into standard ComposeUiTest actions.

For example, when the agent sends a request to click a button, the code looks something like this:

runComposeUiTest { uiTest ->
    // Set the application content. 
    // There is also a /reset endpoint to change this content at runtime.
    uiTest.setContent { MyApplication() }

    startServer {
        get("/click") { request ->
            val matcher = request.node()          // Find the node (e.g. by tag or text)
            uiTest.onNode(matcher).performClick() // Perform the click
            uiTest.waitForIdle()                  // Wait for animations to settle (advancing the virtual clock time)
            request.respondText("ok")             // Tell the agent it's done
        }
        get("/screenshot") { request ->
            val node = request.node()
            val image = uiTest.onNode(node).captureToImage()
            request.respondPng(image)
        }
        ...
    }
}

This simple loop allows the agent to navigate through the app just like a user would, but much faster since it can use a virtual clock to speed up animations.

This is all possible only because the incredible Jetpack Compose team has created very powerful testing APIs that are very well thought out. The fact that ComposeUiTest allows such fine-grained control over the UI clock and input injection, while being completely decoupled from the rendering platform, is what makes this tool feasible. Big credit goes to them for enabling this!

Playing with it

To test it out, I created a simple clone of an app most of you are probably familiar with: Instagram. It was surprisingly fun to prompt the agent to just "Build an Instagram UI clone" or "Improve the app and add missing features", and watch it navigate the menus, click the button, and confirm it is done adding 5 new screens in a single shot. Of course the app is far from being production ready, but I was pleasantly surprised by the result I got after playing with this in probably less than 1h and 10 prompts.

Here is a short video of what it looks like in action:

A word on reliability

Of course, Compose Driver doesn't magically solve all the challenges of building mobile apps with AI. For production workloads, it remains crucial to review the code and understand what the agent implemented (and why). But for hobby projects, prototypes, or just for the sake of experimenting with new workflows, it’s been a blast to use!

There is more and more evidence that the more tools an AI has to verify its work, the better it will perform. Giving agents the ability to close the feedback loop and iterate on their own can only improve the quality and result of the generated code.

Check it out

If you're interested in agentic workflows for mobile or just want to play around with it, the code is open source here: https://github.com/jdemeulenaere/compose-driver

I've included a sample/ project that you can open in your favorite AI editor to get started quickly. I hope that this will be useful to some of you :-) Let me know what you think!

DEV Community