When a coding agent changes Android XML, it is fairly confident about what it can read: it finds the layout, understands the constraints, and produces a diff. But there was almost always one more manual step afterwards. I would open the screen and decide whether the button, the text, or the card had actually ended up in the right place.
I wanted to remove that repeated switch. I was not trying to hand over responsibility for the product or to declare the emulator unnecessary. I wanted the result of the agent's changes to be available inside the same workflow in which it edited the code.
My first idea was a screenshot: the agent changes XML, gets an image, and looks at it. In practice, that turned out to be only half of the solution.
The setup
The loop I wanted for XML/View screens:
change XML → render → check the result → fix
I put the rendering into a local MCP server built with Kotlin/JVM and Robolectric. The agent supplies a layout and explicit data for it, and gets the rendered result back. MCP is not the interesting part here. It is just a way to take me out of a repetitive intermediate step: launching the render by hand and passing the result back to the agent.
The scope is deliberately narrow. This is for XML/View projects, not Compose. It is not a replacement for a device, it does not promise a pixel-for-pixel match with a physical phone, and it does not try to emulate hardware surfaces, video, or a camera.
The PNG worked, and the tool was still not finished
The screenshot made the result visible immediately. The agent could notice that text was clipped, that a block had unexpectedly disappeared, or that two elements were clearly overlapping. That was already better than editing XML blind.
At that point the project looked almost done. Screenshots were generated, the MCP server responded, and an inspect_view tool existed.
A closer review broke that impression. The screenshot was real, but the sidecar returned an artificial root node as the View tree. There was no real child View to query, so a tool called inspect_view was barely useful.
That showed how much an image alone leaves unresolved. It is poor at answering the questions that often decide whether a UI change is correct:
- What is the actual distance between two Views?
- Has an element gone beyond the screen boundary?
- Where exactly is the View with a particular
id?
A person can estimate these by eye. The agent needs verifiable data, not just an illustration.
A second wrong assumption: the warm JVM
Around the same time the coding agent pointed out another oversimplified idea in my design. You cannot just start the Robolectric runtime once and keep reusing it after changes to XML or Kotlin: you risk inspecting stale resources and an outdated classpath.
That observation changed the architecture. It removed the temptation to treat a "warm JVM" as a problem we had already solved. The default sidecar still is not the persistently warm JVM from my early architectural plan.
The fix: measure what you render
The turning point was simple to state: the PNG and the data must come from the very same layout, after layout has been performed.
After inflate → fixture → measure → layout → draw, the renderer saves both the image and the real View tree. Each node includes its identifier, type, text and state, padding, margins, and absolute bounds.
Three small operations make up the loop:
render_layout → creates a render session
get_view_tree → returns the tree of that session
inspect_view → returns a single node by id
The API names matter less than what they allow. The agent can move from "these look a little too close together" to a measurable fact, such as comparing the edge of a button with the edge of its container without estimating anything from a PNG.
That gave us a geometry feedback loop:
change → render → inspect → evidence → fix
The image helps reveal a problem, the tree provides the geometry, and the next change goes through the same render again.
Underneath there are also a source fingerprint and an allowlisted Gradle pipeline. They exist so the loop cannot silently render stale state or run arbitrary shell commands.
Testing it on a real layout
After these changes I connected the MCP server to an actual Android project I was working on. The coding agent used it to fix a real product-card layout with a required-items section.
Before the fix. The UI is in Russian.
After the fix. The UI is in Russian. The two screenshots show different items, so they are not a pixel-level comparison.
For me this was the key acceptance test. The tool had moved beyond its RFC and its own unit tests into a real UI task, and a coding agent used the loop in an actual UI change.
What the experiment did and did not prove
| What I wanted to check | Result |
|---|---|
| Can a real Android XML/View layout be rendered through MCP + Robolectric? | Yes |
| Can a real, machine-readable View tree be returned alongside the PNG? | Yes |
| Did a coding agent use this loop in a real UI change? | Yes |
| Can the agent's actions be fully reproduced from the saved data? | Not yet |
I do not consider this a replacement for an emulator or a device. Robolectric does not guarantee a pixel-perfect match with hardware. I have no compatibility matrix covering custom Views and complex themes. For the next experiment of this kind I want to keep the complete record: the prompt, the tool calls, render IDs, the XML diff, the measurements, and the final verdict.
These are reasonable limits for a first version. What matters more is that they can now be described as facts rather than as the impression left by a good screenshot.
Who did what
I did not give the agent the authority to "decide the UI." I defined a bounded problem, set the limits of support, refused to accept a PNG as sufficient evidence, and kept bringing the work back to verifiable criteria.
The coding agent was more than a code generator. It helped implement the tool, flagged the risk of a stale Robolectric runtime, and helped build and use the loop itself.
For me the main result is not that the agent now has an image. A screenshot makes an interface visible. A PNG together with a View tree makes it verifiable.
That is how manual visual review can leave the routine loop: not because I decided to trust the agent, but because it gets artifacts that can be measured and checked again.
The renderer is open source: android-ui-renderer-mcp on GitHub. The full case study is on my site: hram.github.io/en/articles/android-ui-renderer-mcp.
Top comments (4)
The part that usually breaks approaches like this is device density and font scale. Two configs at the same declared width can produce different pixel measurements once system font scale or display density changes, so a bounds check that passes on one setup fails on another for reasons that have nothing to do with the actual layout code. Worth running the same view tree capture across a couple of density buckets before trusting a single pass as ground truth. Pairing the rendered image with the measured tree instead of relying on either alone is what actually catches the kind of small overlap that only shows up in the numbers and not in the picture.
Agreed, and that's why density and font scale are explicit inputs here rather
than inherited from the environment: densityDpi, fontScale, locale, night mode
and orientation are applied to the Activity's resource configuration before
inflation, so a pass is always tied to a concrete config, not just "1280
wide".
What it doesn't do is sweep configs by itself. Running the same layout across
a few density/font-scale combinations is one render call per config, done by
the agent. A concrete example, same layout and width, two configs:
{ "layout": "item_cart_product", "widthPx": 722, "densityDpi": 212,"fontScale": 1.0 }
{ "layout": "item_cart_product", "widthPx": 722, "densityDpi": 212,
"fontScale": 1.3 }
For each render the agent calls inspect_view on productName and mandatory and
checks that the gap between their bounds stays positive. A check that holds at
1.0 can fail at 1.3 while the PNG still looks plausible. Changing densityDpi
works the same way. So a single pass is a data point, not ground truth.
On pairing: the PNG and the view tree come from the same laid-out hierarchy,
with per-view absolute bounds, padding and margins. A few-pixel overlap shows
up as a bounds comparison even when the image looks fine. It's still
Robolectric, so it doesn't replace a real device.
One case the positive gap check can miss: at fontScale 1.3 a TextView with maxLines=1 just ellipsizes, so the bounds never touch and the PNG looks fine, but half the product name is gone. getLayout().getEllipsisCount(0) > 0 catches it, could be a cheap flag in inspect_view output.
Also curious if Robolectric follows the nonlinear font scaling from Android 14. Since API 34 big text grows less than small text, so 1.3 on a 24sp title is not a clean 1.3x anymore. Would be interesting to compare those numbers with a real device.
Thanks, both points checked, and the first one led to a real change. Because of your comment, inspect_view now reports text truncation:
github.com/hram/android-ui-rendere...
Ellipsis: you're right, the gap check can't see it. Every TextView node now has a textLayout block: textSizePx, maxLines, lineCount, ellipsisCount (summed getEllipsisCount over all lines) and a truncated flag. The flag also covers maxLines clipping without ellipsize. Same item_cart_product at 722 px / 212 dpi with a 56-character product name: the right edge of
productName stays at 701 px at fontScale 1.0, 1.3 and 2.0, while ellipsisCount goes 23 → 27 → 36. The PNG at 1.3 looks completely normal.
Non-linear scaling: yes, Robolectric follows it. The renderer runs on SDK 34, and fontScale goes through Configuration, so the framework's FontScaleConverter is applied. Measured
textSize in px at 212 dpi:
12sp: 15.9 → 20.67 at 1.3 (×1.30) → 31.8 at 2.0
24sp: 31.8 → 34.98 at 1.3 (×1.10) → 47.7 at 2.0 (×1.5)
So a 24sp title at 1.3 is about 1.1x, not 1.3x, which matches the API 34 curves. One more detail: sizes coming from XML dimens go through getDimensionPixelSize, so they are rounded to whole pixels (20sp → 27 / 31 / 45 px).
I haven't compared against a real device yet. That's the next step, on the two tablet profiles we actually ship to.