DEV Community: krish pavuluri

Your Xbox Pad Is Not an Xbox Pad — Testing Controller Support on Real Android Devices

krish pavuluri — Fri, 04 Sep 2026 22:57:54 +0000

Press A on an Xbox pad. The game receives B. Nothing logs an error, nothing crashes, and every automated check still passes.

That was the bug, and it cost more time than the rest of the feature combined. We spent a chunk of this cycle forwarding a controller from your own machine into a real Android device in a browser tab. Most of it was straightforward. One part was not, and it is the part worth writing down, because it fails silently and it will bite anyone who tries this.

If you ship a mobile game with controller support you already know the testing story it replaces: a drawer of phones, a drawer of pads, and a Bluetooth pairing dance every time you want to check whether B still does what it did last sprint.

The easy half: input that the device believes

The naive version of remote input is a synthetic event: you take a tap in the browser, ship the coordinates, and inject them at the app layer. That works for a lot of testing and it falls apart for exactly the cases games care about. A synthetic event aimed at a focused view is not what a controller produces, and anything reading the actual input stack — which is most game engines — will not see it.

So the input has to arrive lower down. On Android, a forwarded controller can be presented to the system as a genuine input device: the kernel enumerates it, the input stack routes it, and a native game receives it the same way it would from a pad plugged into the phone. Nothing in the app has to be instrumented, and nothing has to know it is remote.

That distinction matters more than it sounds. It is the difference between "we can test the menus" and "we can test the game."

The iOS caveat, stated plainly. This is an Android claim. On iOS a forwarded pad can be presented to web content, but native apps do not see it. If you build an iOS game, remote controller testing is not solved by this and I am not going to pretend otherwise.

The hard half: an Xbox pad is not an Xbox pad

Android decides how to interpret a controller's report by looking at its vendor and product ids, then loading the matching key layout file. That layout is not cosmetic. It decides which bit in the report becomes which button.

The first version presented one fixed identity for every pad. Plug in an Xbox controller, and it arrived at the device claiming to be a PlayStation one. Everything looked fine — the device enumerated a controller, the buttons produced events, the sticks moved. But the two key layouts are near mirror images of each other. Read off a real device:

	DualShock	Xbox
Right stick	Z / Rz	Rx / Ry
Triggers	Rx / Ry	Z / Rz
Face buttons	rotated	—
Digital L2 / R2	present	absent

So the right stick arrived as a trigger, and the face buttons were rotated by one. A mismapped button still produces a perfectly valid event, so nothing anywhere reported an error — which is how you get back to pressing A and receiving B, with nothing to flag it but a human watching the screen going "that's not right."

The fix is that an identity is a descriptor and a button map together, never one alone. Present the ids, and you have committed to the layout that goes with them. Detection reads the vendor id the browser reports for the pad and falls back to the product name — and that fallback is not a nicety. A real DualShock 4 over Bluetooth on macOS reports itself to Chrome as Wireless Controller (STANDARD GAMEPAD), with no vendor or product id in the string at all. Match on vendor id alone and that pad never resolves.

Two things follow that are worth stealing if you are building anything similar:

An unrecognised pad should fall back to a real identity, not a neutral one. A neutral vendor id is what makes Chrome refuse to classify the pad, and then web content mislabels every button it guesses at. Better to be confidently one specific controller than vaguely none.
Swapping controllers mid-session has to tear down and re-create the device, because the identity is fixed at creation time. You cannot mutate a pad into a different pad.

That table came off a real device, confirmed bit by bit, because this is not an area where reading the documentation is enough.

And then an agent picked up the controller

The part we did not plan for.

The same controller surface is exposed as tool calls, so an AI agent can hold the pad. It sees the screen and sends button and axis state to the device.

The first attempt at this was useless, in an instructive way. An agent calling one tool per button press produces input separated by a full network round trip — seconds, not milliseconds. Every combo renders as a held button. The device is not wrong; the timing is simply gone.

What works is sending the sequence rather than the presses: a timed series of controller frames, played back on the device at thirty a second. The agent decides what the combo is, and the timing is preserved on the far end where it can actually be honoured.

What this is not: an agent with human reflexes. We measured it. A round trip through the tool interface is around six seconds. A purpose-built local loop — frame in, controller out, no agent — came in at 134 ms median, against a pre-registered bar of about 50 ms for anything that could claim reflex play. It missed. The verdict was to not build the reflex surface, and that verdict stands.

So: the agent can play. It cannot out-react you. Anyone claiming otherwise about their own setup should be asked for the number.

Why bother

The honest pitch is narrow. If your game has controller support and you currently test it by keeping physical pads next to physical phones, this removes the physical part: any device, any pad, from a browser, with the input arriving as real hardware.

That is it. It does not test your game for you, and on iOS it does not reach native apps at all.

If you want to try it, the devices are at robotactions.com — there is a free tier, and the tool interface installs with npx @robotactions/mcp@latest init.

And if you have hit the key-layout problem from the other side — writing the game rather than the harness — I would genuinely like to hear how you found it. My guess is "a tester said the buttons were wrong and nobody believed them for a week."

Your Mobile Suite Isn't Slow, It's Serial

krish pavuluri — Sun, 23 Aug 2026 18:25:18 +0000

A 400-test Appium suite that takes 90 minutes is not usually 90 minutes of slow tests. It's 90 minutes of one device doing 400 things in a row.

You can shave seconds off individual steps forever and never fix that. The only lever with real leverage is how many devices are running at once — and the reasons teams don't pull it are almost never technical.

The arithmetic nobody does

Take the suite you have. Total wall-clock is roughly:

wall_clock ≈ (sum of all test durations / devices) + slowest_single_test + setup_per_device

Two things fall out of that immediately.

One: past a certain point, adding devices stops helping. If your longest single test is 6 minutes, no amount of parallelism gets the suite under 6 minutes. Splitting that one test is worth more than the next four devices.

Two: setup_per_device is paid once per shard, not once per suite. If every shard installs the app, waits for a device to be handed to it, signs in and seeds fixtures, then a 12-way split pays that twelve times. Teams routinely go from 4 shards to 12, watch wall-clock barely move, and conclude parallelism doesn't work for mobile. What actually happened is that setup went from 20% of the run to 60% of it.

Measure both numbers before you buy anything.

Where to cut the suite

Sharding by "test file, alphabetically, into N buckets" is the default in most CI templates and it's the worst one available. Test durations on mobile are wildly uneven — a login test is 20 seconds, a checkout-through-payment test is four minutes — so alphabetical buckets give you one shard that finishes in three minutes and one that runs for twenty. Your suite is as slow as that last shard.

Two better options, in order of effort:

Balance by recorded duration. Keep last run's per-test timings in a JSON file in the repo, sort tests descending, and greedily assign each to whichever shard currently has the least work. Fifteen lines of code, and it typically beats alphabetical splitting by 30–40% on wall-clock:

// shard.mjs — longest-first bin packing
const timings = JSON.parse(fs.readFileSync("test-timings.json", "utf8"));
const tests = allTests.sort((a, b) => (timings[b] ?? 60) - (timings[a] ?? 60));

const shards = Array.from({ length: N }, () => ({ total: 0, tests: [] }));
for (const test of tests) {
  const lightest = shards.reduce((a, b) => (a.total <= b.total ? a : b));
  lightest.tests.push(test);
  lightest.total += timings[test] ?? 60;
}

Unknown tests default to a middle-of-the-road guess so a new test never lands in a shard that's already full.

Split by device requirement instead. Some tests only make sense on specific hardware — the biometric flow, the small-screen layout regression, the tablet split view. Those aren't interchangeable work units, so pin them and balance whatever's left.

The failure that shows up only in parallel

Serial suites hide shared state. Run them twelve at a time and it surfaces within a day.

One login account, twelve sessions. The classic. Test 7 signs in, test 31 signs the same account in elsewhere, the backend invalidates the first session, test 7 fails on a screen that has nothing to do with auth. Every re-run picks a different victim, so it reads as flakiness. Fix it with a pool of accounts leased per shard, not per suite.

Shared fixtures with mutable state. A test that edits the seeded user's profile and a test that asserts on that profile are fine in sequence and a coin flip in parallel. Either seed per shard or make tests create what they assert on.

Rate limits and quotas. Twelve shards hitting the same staging API multiply your request rate by twelve. Sandboxes for payments and SMS are especially quick to start throttling — and a 429 surfaces in your app as a generic error screen, not as "you are being rate limited."

Device state left behind. Cached logins, granted permissions, notification badges, a half-finished onboarding. Serially, test 40 inherits whatever test 39 left; in parallel it inherits whatever some other shard left. Decide explicitly whether each shard starts from a clean install or a warm known state, and enforce it in setup rather than hoping.

None of these are parallelism bugs. Parallelism just stops your suite from accidentally serialising around them.

Why "just add more parallel runs" is usually the expensive answer

On most device clouds, concurrency is the billed unit: you buy N parallel slots and the suite is shaped around N. That has two consequences that get felt long before the invoice does.

The first is queuing. When all slots are busy, jobs wait — so the number that matters isn't your test duration, it's your time-to-first-device during the 4pm rush when everyone merges.

The second is that you start optimising the wrong thing. Once concurrency is scarce, teams cut coverage to fit the slots they bought, and "which tests run on hardware" becomes a budgeting decision rather than a risk decision.

Our answer is a different unit. Devices here are dedicated to you and priced flat per device per month — not metered per parallel run, per minute, or per slot. A device you're paying for is yours whether it's mid-suite or idle at 3am, which means the shard count is something you tune for wall-clock, not something you ration.

It also means state persists between runs. A shard doesn't have to reinstall and re-sign-in against a freshly wiped machine every time, which cuts straight into that setup_per_device term that was eating your gains.

Wiring it into CI

The matrix is the easy part. This is GitHub Actions, but the shape is identical in GitLab CI or Jenkins:

jobs:
  e2e:
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4, 5, 6]
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: node shard.mjs --index ${{ matrix.shard }} --total 6 > shard-tests.txt
      - run: npx wdio run wdio.conf.js --spec-file shard-tests.txt
        env:
          GRID_URL: ${{ secrets.ROBOTACTIONS_GRID_URL }}
          GRID_TOKEN: ${{ secrets.ROBOTACTIONS_TOKEN }}
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: results-${{ matrix.shard }}
          path: reports/

Three details that matter more than they look:

fail-fast: false — otherwise the first red shard cancels the other five and you get one failure per run instead of the full picture. On a suite you're trying to stabilise, that turns a one-hour debugging loop into a one-day one.

if: always() on the artifact upload — results from failed shards are the ones you actually need.

And merge the reports. Six separate JUnit XMLs in six artifacts is not a test report; run them through a merge step so the PR gets a single pass/fail with a single list of failures.

What to do this week

Record per-test durations for one run. You need the data before any of this is a decision.
Find your longest single test. That's your floor — split it or accept it.
Measure setup_per_device. If it's over ~90 seconds, fixing it beats adding shards.
Move from alphabetical to duration-balanced sharding. It's an afternoon and it's free.
Then, and only then, raise the shard count.

Most suites have a comfortable 3–4× sitting in steps 2 through 4 before they need a single extra device.

Run your suite across dedicated devices →

Originally published at robotactions.com/blog/parallel-mobile-tests-sharding.

Flaky Mobile Tests: A Triage Order That Actually Converges

krish pavuluri — Sun, 23 Aug 2026 18:22:48 +0000

Most teams don't fix flakiness. They add a retry, watch the suite go green, and move on — and six months later nobody trusts a red build, because half of them are noise and telling which half is a coin flip.

The reason it never converges isn't a lack of effort. It's that "flaky" names five completely different problems with completely different fixes, and teams debug whichever one they thought of first.

Here's an order that terminates.

Step 0: measure, don't remember

You cannot triage from impressions. Before anything else, get one number per test: failure rate over the last 50 runs on unchanged code.

If you don't have that, the cheapest way to get it is a nightly job that runs the suite against main five times and appends results to a CSV. A week gets you 35 data points per test, which is enough to rank.

Two things fall out immediately and both are worth the week:

Usually fewer than 10% of tests produce more than half the flaky failures. Fix those five tests and the suite feels transformed. Without the data you'd have spent that time on whichever test annoyed someone loudly.
Some "flaky" tests aren't flaky at all — they fail 100% of the time on one device or one OS version and pass everywhere else. That's not flakiness, it's an unfiled bug wearing a disguise, and it's the highest-value thing in the list.

Rank by failure_rate × runs_per_day. Work top-down. Stop reading Slack for candidates.

Step 1: is it the test, or is it the app?

Before assuming a test bug, take the top offender and run it 20 times in isolation on one device.

Fails at a similar rate in isolation → it's the test, or a genuine app bug. Continue to step 2.
Passes 20/20 in isolation, fails in the suite → it's interference from the tests around it. Skip to step 3.

This one question splits the work in half, takes twenty minutes, and is skipped almost universally.

Step 2: the sync problem (most flakiness lives here)

Nearly all single-test flakiness is a race between your test and the app's rendering, and nearly all of it comes from the same three patterns.

Sleeping instead of waiting. sleep(2) is a bet that the app is ready in under two seconds on every device, every network, every CI load. It passes on your M-series laptop and fails on a mid-range Android under thermal load. Every fixed sleep is a flaky test with a delay fuse.

Waiting for presence, acting on interactivity. The element exists in the tree well before it's ready. A button that's rendered but still animating in eats the tap and reports success — the tap landed, it just didn't do anything, and you fail three assertions later on a screen that never changed. Wait for enabled and stable, not merely present.

Waiting for the wrong thing. The classic: waiting for a spinner to appear, when on a fast response it appears and vanishes before the poll interval catches it, and your wait times out on a screen that already loaded successfully.

// Flaky: passes when the spinner is slow, times out when the app is fast
wait.until(visibilityOfElementLocated(By.id("loading_spinner")));
wait.until(invisibilityOfElementLocated(By.id("loading_spinner")));

// Stable: wait for the post-condition, not the transition
wait.until(elementToBeClickable(By.id("results_list")));

The general rule: assert on the state you want, never on the transition into it. Transitions are racy by definition; end states are not.

One more that only shows up on real hardware — animations. A screen that has finished loading but is still sliding in will hand your tap to whatever pixel is under the finger mid-flight. Disabling system animations on the device (appium:disableWindowAnimation, or the developer options scale settings) removes an entire class of failure and speeds the suite up as a bonus.

Step 3: state leakage (the one that scales with your suite)

If the test passes alone and fails in the pack, something outside it is changing what it sees.

Work through the shared surfaces in this order — roughly most to least common:

Device state. Cached logins, granted permissions, dismissed onboarding, a notification still on screen, keyboard left open, app left backgrounded. Test 40 inherits whatever test 39 left behind.
Backend state. The account whose cart another test just emptied; the record another test renamed; the feature flag someone flipped.
Shared credentials. One login used by parallel shards, where a new session invalidates the old one and some unrelated test gets bounced to the login screen.
Time and clock. A test that passes except around midnight, or on the last day of a month, or in a CI runner on UTC when the assertion assumed local time.
Data accumulation. Test 1 creates an item, so test 12's "list should show 3 items" assertion has been quietly counting up all week.

The fix is always the same shape and it's rarely "clean up afterwards": make each test create what it asserts on. Teardown-based cleanup fails exactly when the test fails — which is when the leakage matters most — so the next test inherits the mess from the run that already went wrong, and one real failure cascades into six fake ones.

Step 4: environment (rule out before rewriting anything)

If steps 2 and 3 found nothing, stop editing the test and look at where it ran.

Device health. Low battery throttles CPU. Full storage makes installs and screenshots fail in creative ways. Thermal throttling after an hour of continuous runs slows everything enough to blow timeouts that were fine at run 1.
Network. A staging API with a p99 of eight seconds will produce flakiness no test-side fix can remove. Log the request timings; if the p99 moves with the failures, the fix is in the backend, not the suite.
Rate limits. Parallel shards multiply your request rate. A 429 usually surfaces in the app as a generic error screen, so it reads as an app bug.
OS-level interruptions. An OS update banner, a "storage almost full" alert, a carrier message landing on top of your flow. Rarer than people assume, but genuinely random when it happens — which makes it the most confusing category to debug from a stack trace alone.

This is also where inconsistent devices show up as inconsistent tests. If two runs of the same test get two differently-configured handsets — different locale, different OS patch level, different leftover state — the test is being blamed for the environment's variance. Running against a device that's dedicated and stable between runs doesn't fix a badly written test, but it does remove the variable, which is what lets you conclude anything at all from step 1.

Step 5: it's a real bug

Some flaky tests are correct. They're catching a genuine race in the app — a request that sometimes resolves after the screen is gone, a cache that's occasionally stale, a listener registered twice.

The tell: the failure is not at a findElement, the screenshot shows a plausible screen, and the app behaves wrongly rather than the test looking in the wrong place. Deleting or retrying this test is deleting a bug report that reproduces.

What to capture so triage is possible at all

Most of the above is impossible from a stack trace. On every failure, capture:

A screenshot at the moment of failure — separates "wrong screen" from "right screen, bad selector" instantly.
The element tree at that moment — tells you what locators actually existed, which is what you need to write the fix.
Video of the run — the only way to catch a dialog that appeared and disappeared, or a tap that landed mid-animation.
Device logs and network requests — turns "the app showed an error" into "the API returned 429."
Which device, OS version, and build — so the "fails only on one device" pattern can surface at all.

Without these you re-run locally to reproduce, which for genuinely flaky tests is the single biggest time sink in the process — you're trying to reproduce something that happens 1 in 8 times, and every attempt costs a full suite setup.

On retries

Retries are a measurement tool, not a fix. Configure them so that a test that passes on retry is still reported distinctly from one that passed first time — most runners support this, and it's the difference between a retry policy and a cover-up.

Then treat the retry-pass count as your flakiness metric, and hold it to a budget. A test that needs retries is on a clock, not on a list.

The quarantine rule

Quarantine works only with an expiry date. Without one it's a graveyard: tests nobody runs, nobody deletes, and nobody trusts.

A policy that survives contact with a real team:

A test over your flakiness budget gets quarantined — still runs, doesn't block the build.
It gets an owner and a two-week deadline, both written down.
At two weeks it is either fixed or deleted.

Deleting is a legitimate outcome. A test nobody trusts and nobody fixes has negative value: it costs runtime, it costs triage attention, and it trains everyone to ignore red.

Run your suite on dedicated devices with video, logs and the element tree on every failure →

Originally published at robotactions.com/blog/flaky-mobile-tests-triage.

Auto-Accept Permission Dialogs at Session Start — Then Turn It Off Mid-Test

krish pavuluri — Sun, 23 Aug 2026 18:19:44 +0000

Every mobile suite hits this on day one. The app launches, iOS asks for notifications, then location, then tracking, and every test that isn't about permissions dies on a dialog it never asked for.

So you set appium:autoAcceptAlerts and the suite goes green.

Then someone writes the one test that is about permissions — the one that asserts the location prompt appears, taps Don't Allow, and checks the empty state — and discovers that the capability that saved the suite is now the thing blocking it. The alert is gone before the test can see it.

Here's the accurate picture of what's session-scoped, what's runtime-toggleable, and what to reach for on each platform.

Layer 1: the session capabilities

These are set at session creation. They're the ones everyone reaches for first — and, as the next section shows, they are not the layer you actually want to be controlling.

iOS (XCUITest). appium:autoAcceptAlerts — "Accept all iOS alerts automatically if they pop up. This includes privacy access permission alerts (location, contacts, photos). Default is false." appium:autoDismissAlerts is the mirror image. Note what the wording covers: system alerts, including privacy prompts, not just your app's own UIAlertController.

Android (UiAutomator2). There's no autoAcceptAlerts equivalent, because Android runtime permissions aren't really alerts you accept — they're grants you can hand over before the dialog ever appears. That's appium:autoGrantPermissions: "Whether to grant all the requested application permissions automatically when a test starts."

Three constraints on it that bite in practice:

targetSdkVersion in the manifest must be ≥ 23 and the device must be Android 6+.
Apps with targetSdkVersion ≤ 22 "must be reinstalled to grant permissions" — e.g. with appium:fullReset.
Grants happen at install time. If your session reuses an already-installed app, the capability has nothing to act on. This is the single most common reason people report that "autoGrantPermissions doesn't work."

And for special permissions — notifications access, media recording — the plain grant path doesn't cover them; the docs point you at mobile: changePermissions with the appops target instead. More on that below.

One iOS caveat worth internalising: the appium:permissions capability, the one that lets you preset per-service permission states, "Allows to set permissions for the specified application bundle on Simulator only." On a real iPhone it isn't available. Real-device iOS permission state is something you handle through dialogs or through how the device was left.

The two settings that actually control it

The capabilities are not the mechanism. They're a convenience that seeds one, and the mechanism underneath is runtime-mutable — which means the mid-session toggle everyone assumes is impossible is in fact two lines of code.

When the driver builds its underlying iOS automation session, it does exactly this:

if (driver.opts.autoAcceptAlerts) {
  wdaCaps.defaultAlertAction = 'accept';
} else if (driver.opts.autoDismissAlerts) {
  wdaCaps.defaultAlertAction = 'dismiss';
}

So appium:autoAcceptAlerts is a front end for a setting called defaultAlertAction. And defaultAlertAction is registered in the settings map, so it can be written any time during the session. The driver forwards any setting key it doesn't handle specially straight to the agent's settings endpoint, so a plain setSettings call reaches it:

// Turn auto-accept off, mid-session
getAppiumDriver().setSettings(ImmutableMap.of("defaultAlertAction", ""));

// ... assert on the dialog, tap what you want ...

// Turn it back on
getAppiumDriver().setSettings(ImmutableMap.of("defaultAlertAction", "accept"));

You'll see "off" used for the disable value, and it works — but by accident rather than by design. The handling is a straight comparison against accept and dismiss; anything else falls through to a branch that logs 'off' default alert action is unsupported and does nothing. The behaviour you want, with a warning line per alert. An empty string is the value that's explicitly checked for and returns early with no log noise. (null works too, but ImmutableMap rejects null values, which is exactly why "off" tends to be what people reach for in Java.)

This is worth knowing precisely because none of it is in the driver's settings documentation — defaultAlertAction isn't listed there, and the Appium forum answer on this question is "remove autoAccept and manage all by yourself." That answer is out of date. The setting is real, it's readable back, and it's the supported shape of the thing.

`autoClickAlertSelector`, and why it's checked first

The second setting is autoClickAlertSelector:

Custom selector for an alert button. This can be used to automatically locate an element inside
an alert hierarchy, and tap it as soon as a new alert is detected.

This setting takes priority over the appium:autoAcceptAlerts and appium:autoDismissAlerts
capabilities.

"Takes priority" is literal: when an alert appears, the selector is consulted first, and if it's set the handler taps and returns without ever consulting defaultAlertAction — including when the tap fails. It's strictly more expressive than the boolean, because it names which button gets tapped rather than accepting whatever the OS considers the default:

await driver.updateSettings({
  autoClickAlertSelector: "**/XCUIElementTypeButton[`label CONTAINS[c] 'allow'`]",
});

To disable it, set it to an empty string. This isn't a trick — an empty value is special-cased ahead of any parsing, and it additionally tears down the alert monitor rather than merely failing to match:

getAppiumDriver().setSettings(ImmutableMap.of("autoClickAlertSelector", ""));

Don't disable it by supplying a valid selector that matches nothing. That leaves the monitor running and pays for a class-chain query against every alert for no benefit. And note that the documented "an error is thrown if the provided selector is invalid" applies only to non-empty values — the parse step is never reached for an empty one.

Putting the two together

Because the selector short-circuits the alert action, fully disabling auto-handling means clearing both, in that order:

private void pauseAlertHandling() {
  getAppiumDriver().setSettings(ImmutableMap.of("autoClickAlertSelector", ""));
  getAppiumDriver().setSettings(ImmutableMap.of("defaultAlertAction", ""));
}

private void resumeAlertHandling() {
  getAppiumDriver().setSettings(ImmutableMap.of("defaultAlertAction", "accept"));
}

Clearing only defaultAlertAction while a selector is still set changes nothing at all, because the selector is what's handling your alerts. That's the failure mode to watch for if you've configured both — the disable appears to do nothing, and it looks like the setting was ignored.

Wire that pair into @BeforeEach / @AfterEach and the permission-specific tests live in the same session and the same suite as everything else, with no second capability set.

You can also start the session with the setting instead of the boolean capability, using Appium's appium:settings[...] prefix:

{
  "platformName": "iOS",
  "appium:automationName": "XCUITest",
  "appium:settings[autoClickAlertSelector]": "**/XCUIElementTypeButton[`label CONTAINS[c] 'allow'`]"
}

Same effect as autoAcceptAlerts, but now the behaviour lives somewhere you can change without tearing down the session.

Layer 2, Android: revoke and re-grant at runtime

Android's runtime lever isn't about alerts at all — it's about the grant itself. The UiAutomator2 driver's mobile: changePermissions "Changes package permissions in runtime":

Argument	Values	Notes
`permissions`	permission name, array, or `all`	`all` works only with `target: pm`
`appPackage`	package name	defaults to the app under test
`action`	`grant` (default) / `revoke` for `pm`; `allow` / `deny` / `ignore` / `default` for `appops`
`target`	`pm` (default) or `appops`	`appops` requires the driver's `adb_shell` server security option

So the whole test starts with permissions granted, and the one test that needs the dialog revokes it first and lets the app ask:

// Start clean: revoke, so the next launch actually prompts
driver.executeScript("mobile: changePermissions", Map.of(
    "permissions", "android.permission.ACCESS_FINE_LOCATION",
    "action", "revoke"));

driver.executeScript("mobile: activateApp", Map.of("appId", "com.mycompany.myapp"));
// ... assert the system dialog, tap Deny, assert your empty state ...

// Hand it back for the rest of the suite
driver.executeScript("mobile: changePermissions", Map.of(
    "permissions", "all", "action", "grant"));

And to assert on state rather than guess at it, mobile: getPermissions takes a type of denied, granted or requested (the default) and returns a list of permission names. That turns "did the grant actually apply?" from a debugging session into an assertion.

Note the revoke-then-relaunch pattern. Revoking a permission that a running process already holds is not something the app observes gracefully — Android may kill the process. Do it before activating the app, not in the middle of a flow, unless killing the app is the thing you're testing.

Layer 3: handling one alert by hand

When you just need to deal with the dialog in front of you:

iOS — mobile: alert takes an action of accept, dismiss or getButtons, plus an optional buttonLabel. getButtons is underused and genuinely handy: assert on the exact button set the OS is offering before you commit to tapping one, which is how you catch iOS-version differences in prompt wording instead of failing on a hardcoded label.

const buttons = await driver.execute("mobile: alert", { action: "getButtons" });
// e.g. ["Don't Allow", "Allow Once", "Allow While Using App"]
await driver.execute("mobile: alert", { action: "accept", buttonLabel: "Allow Once" });

Android — mobile: acceptAlert and mobile: dismissAlert, both with an optional buttonLabel. The docs are refreshingly honest about the reliability: "This method might not always be reliable as there is no single standard for how Android alerts should look like within the Accessibility representation." Pass an explicit buttonLabel when you know it, and prefer changePermissions over dialog-tapping for anything permission-shaped.

Two more iOS settings worth knowing while you're in here:

acceptAlertButtonSelector / dismissAlertButtonSelector — class chains that change which button the standard W3C Accept Alert / Dismiss Alert commands press. For "handle accept buttons with arbitrary text," i.e. custom in-app dialogs whose buttons say Got it and Maybe later.
respectSystemAlerts — "Whether to automatically switch the active application to the system springboard if a native alert element is detected." Default false. If your alert handling works everywhere except when a system prompt overlays your app, turn this on before you start suspecting the driver.

A policy that holds up

Situation	Use
Permission prompts are noise in 95% of tests	iOS: `appium:settings[autoClickAlertSelector]`. Android: `appium:autoGrantPermissions`
One test must assert on the prompt	iOS: clear `autoClickAlertSelector` and `defaultAlertAction` to `""`, restore after. Android: `mobile: changePermissions` → `revoke`, then relaunch
Special permissions (notifications, recording)	Android: `mobile: changePermissions` with `target: appops`
Prompt wording varies by OS version	`mobile: alert` → `getButtons`, assert, then act
Custom in-app dialog, non-standard buttons	`acceptAlertButtonSelector` / `dismissAlertButtonSelector`
iOS Simulator, preset state before launch	`appium:permissions` (Simulator only)

The through-line: prefer the setting over the capability even when both would work today. The capability is a decision you make once per session and then live with; the setting is a decision you can revisit per test. Mobile suites always eventually contain the test that needs the opposite of the default, and discovering that after you've built 400 tests around a capability is an expensive time to find out the layer beneath it was writable all along.

What changes on a real device

Two things, and both cut in your favour once you know about them.

appium:permissions being Simulator-only means real-device iOS permission state comes from the device itself — whatever the last session left behind. On a shared pool that's a source of mystery failures: you don't know which state you inherited. On a dedicated device, that state is yours and it's stable between runs, so "granted at the start of the suite" is something you can establish once and rely on rather than re-derive every session.

The flip side is that stable state is still state. Make the starting condition explicit at suite setup — grant or revoke deliberately — rather than letting run N inherit run N−1 by accident.

And when a prompt does slip through in CI, the thing that ends the argument fastest is seeing it: a video of the run and the element tree at the moment of failure tell you in seconds whether the dialog appeared and went unhandled, or never appeared at all. Those two failures produce identical stack traces and completely different fixes.

Run your Appium suite on a dedicated real device →

Originally published at robotactions.com/blog/auto-accept-dismiss-alerts-appium.

How to Give an AI Agent Full Control of a Real iOS or Android Device (over MCP)

krish pavuluri — Wed, 05 Aug 2026 17:44:07 +0000

AI agents are getting good at writing code, browsing the web, and calling APIs. But there's one surface most of them still can't touch: a real phone. Not a simulator, not a screenshot fed back to the model — an actual iOS or Android device, driven with the fidelity of a human's fingers.

This guide shows you how to give an agent exactly that, over the Model Context Protocol (MCP). By the end, your MCP client — Claude Desktop, Cursor, Cline, or your own — will be able to open a real device, tap and type on it, see what the app is doing, capture and mock network traffic, spoof GPS, and even explore an app and generate test scripts. All of it as MCP tools the agent can call directly.

What you'll be able to do

Once connected, the agent has the same reach a human tester has on a real device:

Control — real taps, press-and-hold, swipes, drag, and typing (native, hardware-level input, not a laggy screen-share overlay)
See — live screenshots, on-device web inspection, the UI element tree / DOM
Inspect the network — capture every request the app makes, and mock any request or response to force error states and edge cases
Simulate — set GPS location anywhere on earth
Automate — explore an app, find issues, and generate runnable test scripts

Prerequisites

An MCP-capable client (Claude Desktop, Cursor, Cline, or any client that speaks MCP)
A free RobotActions account (that's what exposes the real devices as MCP tools)

Step 1 — Create a free account

That's all the setup you need: you'll authorize your MCP client in the next step and it fetches your access automatically, so there's no token to create, copy, or paste.

Step 2 — Connect your MCP client

You have two paths. Pick the one that matches your client.

Option A — Claude.ai / Claude Desktop (remote connector)

Add RobotActions as a remote MCP connector pointing at:

https://mcp.robotactions.com/mcp

Then click Connect and authorize. Signing in fetches your access automatically — no manual token entry.

Option B — Cursor, Cline, or a local MCP config

Use the connector package to wire it into your MCP config:

npx @robotactions/mcp@latest init

Or point your client's MCP config at the same endpoint https://mcp.robotactions.com/mcp, then authorize the same way when prompted. (RobotActions is also listed on the official MCP registry, Smithery, and glama if your client installs from there.)

Reload your client and you should see the RobotActions tools appear.

Step 3 — Your first commands on a real device

Now just talk to your agent. Try:

"List the available devices, open an Android one, and take a screenshot."

The agent will call the device-list and screenshot tools and hand you back a real frame from a real phone. From there:

"Open the Settings app, go to Wi-Fi, and turn it off."

The agent taps and swipes on the actual device — real touches, not a simulated overlay.

Step 4 — The part agents couldn't do before

This is where it gets interesting. Because the agent has full access — visual and programmatic — you can ask it to do things a screen-share bot never could:

Inspect and mock the network:

"Capture the network calls this app makes on launch, then mock the /profile response to return an empty state, and show me how the app renders it."

Force a location:

"Set the device's GPS to Tokyo and check that the store locator updates."

Explore and test:

"Walk through the checkout flow, flag anything that looks broken, and generate a test script that reproduces it."

The agent sees the screen, reads the network and the UI elements, and drives the input — so "test this app" becomes a single instruction instead of a week of setup.

Why this matters

Mobile test infrastructure was built for humans staring at a remote screen. As agents start doing real work, they need to operate real devices with human-level fidelity — and reach everything programmatically, not just the pixels. Exposing real-device control as first-class MCP tools is what turns "an agent that can see a phone" into "an agent that can actually use one."

Try it

You can do everything above right now on a live demo device — no cables, no local setup, no simulators.

Sign in with Google or GitHub and try it free →

How We Built a Chat AI Agent Into Live Device Testing Sessions

krish pavuluri — Tue, 10 Mar 2026 20:40:36 +0000

We ship a cloud device farm — real Android and iOS devices you can control from a browser. Our users are mostly SDETs and QA engineers running Appium tests.

The problem we kept hearing: finding the right locator wastes too much time.

The typical workflow looks like this:

Open a device session
Notice an element on screen
Switch to Appium Inspector
Inspect the element tree
Copy the locator
Paste it into your test
Run the test, fail, go back to step 3

We wanted to collapse that loop. So we built a Chat AI Agent that lives inside the device session.

What It Does

The agent can see the live device screen. You can ask it in plain English:

"What's the XPath for the equals button?"
"Give me a UIAutomator2 selector for the digit 7"
"What's the Accessibility ID of the login button?"

And it responds instantly with working locators — in whatever language you're using (Java, Python, Swift, Kotlin, WebDriverIO).

No switching tools. No Appium Inspector. Just ask.

How We Built It

Screen visibility

Our sessions already stream device screens via WebRTC. We grab frames from the stream at the point of the user's question — a single screenshot at query time. This keeps latency low and avoids sending a continuous video feed to the model.

The model

We send the screenshot + user message to a vision-capable LLM. The prompt is structured to return locators in a specific format — we parse the response and render it with syntax highlighting in the UI.

Locator formats

We support:

XPath
CSS Selector
UIAutomator2 (Android)
XCUITest (iOS)
Accessibility ID

The model is instructed to return all applicable formats for the visible element, not just one.

Code output

Users pick their language from a dropdown (Java, Python, Swift, Kotlin, WebDriverIO). We wrap the locator in idiomatic framework code for each:

# Python / Appium
driver.find_element(AppiumBy.XPATH, "//android.widget.Button[@content-desc='equals']")

// Java / Appium
driver.findElement(By.xpath("//android.widget.Button[@content-desc='equals']"));

UI integration

The panel sits alongside the device stream — it doesn't overlay the screen. Users can keep testing while asking questions. The conversation history stays within the session.

What We Learned

The hardest part wasn't the AI integration — it was the prompt engineering. Getting the model to return clean, parseable locator output (not prose with embedded code) required iteration.

We also found that grounding the model on the visible screen state (not a DOM or accessibility tree) made responses feel more natural. Users think in terms of what they see, not what's in the XML hierarchy.

Try It

The Chat AI Agent is live now in the RobotActions portal. Free trial available.

We'd love feedback from anyone doing Appium or mobile automation — especially if you've built similar tooling. Drop a comment or reach out directly.