DEV Community: Duchan

Most of a test phone's carbon is emitted before the first test runs

Duchan — Mon, 10 Aug 2026 14:23:14 +0000

There is a drawer in most mobile teams' offices. Mine had fourteen phones in it.

Two iPhones were frozen on old iOS versions with updates blocked, because you can't roll iOS back. An iPad nobody had charged in months. A few Android devices we kept around because one manufacturer had shipped an interesting bug.

And the drawer only helped people who could already build the app. Everyone else had to interrupt a mobile developer just to see a build running.

That's what we set out to fix. Carbon had nothing to do with it.

We looked at hosted device clouds first and stopped before signing up. The price scaled with the team, and our app binary would have to leave our network. So we built tapflow instead: an open-source, self-hosted tool that streams iOS simulators and Android emulators into the browser, so anyone on the team can test a build from a URL.

But while working on it, I realized that if the answer to "we don't have enough test devices" is "buy fewer test devices," there's another consequence besides the budget. Most of a phone's carbon footprint is already there before anyone starts using it.

Most of a device's carbon comes from manufacturing

Device	Total LCA	Production share
iPhone 17 (256 GB)	55 kg	76%
iPad (A16, 128 GB)	74 kg	74%
Mac mini (M4)	32 kg	74%
MacBook Pro 14" (M2 Pro)	243 kg	79%

Three quarters or more of each device's lifetime footprint comes from production.

That changes how I think about test hardware. Using a phone more often doesn't make its manufacturing footprint worse, and leaving it in a drawer doesn't make it better. That carbon was already emitted when the device was manufactured.

So a test phone used twice a month is particularly inefficient from a hardware-utilization perspective: almost the full manufacturing footprint, for very little actual use.

The two sides also age differently.

Test devices get replaced because of OS support windows and the need to validate new releases. That's the same reason two iPhones in my drawer had their updates permanently blocked.

A machine hosting simulators has less of that pressure. It's mostly memory-bound, runs headless, and can stay in service until its macOS version can no longer run the Xcode version you need.

Over eight years, replacing four test devices accounts for 444 kg of manufacturing carbon. One MacBook Pro accounts for 196.8 kg once.

So the question I became interested in wasn't how efficiently we test. It was how much hardware we had to buy to test at all.

The arithmetic

Take a team that needs coverage equivalent to four test devices — say three iPhones and an iPad.

They can buy those devices, or run four simulators on a Mac they already own. Four is about the practical ceiling on a 32 GB host.

For comparison, I annualised everything.

Manufacturing carbon comes from Apple's per-product environmental reports, divided by expected service life using Apple's own assumptions: three years for an iPhone, four years for an iPad and a Mac.

For electricity, I used Korea's grid factor of 417.3 gCO₂e/kWh.

The model assumes 1,000 session hours per year, with 30% of that time spent actively interacting with the app. The active power figure is measured rather than estimated; the commands to reproduce that measurement are near the end of this post.

	4 physical devices	tapflow (Mac already owned)
Manufacturing, annualised	55.5 kg	0 kg
Electricity	12.7 kg (30.4 kWh)	17.9 kg (42.9 kWh)
Total	68.2 kg/yr	17.9 kg/yr

That's 3.8× less in this model.

Almost all of the difference is the manufacturing row: 55.5 kg versus zero. Not because simulators use less electricity, but because the Mac already exists.

Electricity actually goes the other way

A Mac left powered on uses more electricity over a year than charging four phones.

I think that's important to keep in the comparison. If I removed it, the result would look better, but it would also hide one of the more useful findings from the measurement.

It also doesn't change the result much.

About 86% of the Mac's 42.9 kWh comes from the machine simply being powered on, not from simulator activity. Even if I scale the measured per-simulator cost linearly to four simulators, the annual total only moves by about a kilogram of CO₂e.

So this isn't really an energy-efficiency argument.

It's a hardware-utilization argument, and most of the carbon in this comparison comes from manufacturing the hardware.

I tried changing the weak assumptions

There are two inputs I'm least confident about: how per-simulator power scales beyond one simulator, and how often a test device actually gets charged.

I varied both.

Simulator increment	Device charging	tapflow	Devices	Ratio
×1	daily	16.0 kg	68.2 kg	4.3×
×2	daily	16.7 kg	68.2 kg	4.1×
×4 (linear, most conservative)	daily	17.9 kg	68.2 kg	3.8×
×4	3×/week	17.9 kg	60.9 kg	3.4×

The result stays between 3.4× and 4.3× across those cases.

The 3.8× headline number uses linear scaling for four simulators, which is the conservative assumption here.

There are also a few reasons the measurement may overstate tapflow's actual cost.

I measured with the browser decoding the stream on the same Mac, while in normal use decoding happens on the tester's machine. The active measurement was sixty seconds of continuous scrolling, which is much heavier than typical QA. And I treated multi-simulator power as linear even though the measurements are dominated by idle power.

An earlier version of this calculation didn't measure the active cost at all. I guessed 30 W. The measurement came back at 4.52 W — wrong by more than a factor of six, in the direction that made our own case look worse. That's why the number in the table is measured rather than estimated.

Where this stops being true

If you buy a Mac for this, you've bought hardware.

The zero in the manufacturing row only applies when the Mac already exists. If you buy a host specifically for tapflow, its manufacturing carbon has to be included. Against the four devices' 55.5 kg/year, a Mac mini breaks even in roughly six months and a MacBook Pro in about three and a half years.

The difference is large because a laptop includes a display and battery that a simulator host doesn't need. Its manufacturing footprint is roughly seven times that of a Mac mini. If you're buying a dedicated host, the Mac mini is the better choice.

It doesn't replace every device.

tapflow can't test features that depend on device hardware such as cameras, NFC, or biometrics. Teams that need those still need real devices. The question is how many devices you need, not whether you can eliminate them entirely.

Grid factors are regional.

These numbers use Korea's electricity mix. On a cleaner grid, both electricity rows shrink while the manufacturing comparison stays the same.

Old Macs have a floor.

An iOS simulator needs a recent Xcode, and Xcode needs a recent macOS. A Mac that can't run the required versions can't be used as an iOS simulator host, even if the hardware itself still works.

And tapflow is v0.x.

We use it every day and it's under active development. Plan accordingly.

What I'm deliberately not claiming

This is not an SCI score.

The Software Carbon Intensity specification, now ISO/IEC 21031:2024, measures emissions attributable to software: the energy it consumes plus the embodied carbon of the hardware allocated to it. It doesn't include a term for hardware that was never purchased.

Avoided emissions are outside that model, and the specification explicitly doesn't allow offsets or market measures to reduce the score. So the comparison in this post and an SCI score answer different questions.

If you run tapflow, its own SCI score isn't zero. There's still a Mac consuming electricity and carrying a share of its manufacturing footprint. Whether avoided emissions should count is a separate question, and I don't want to hide that distinction by folding them into a metric that wasn't designed for it.

I also can't give you a number against hosted device clouds.

It's an obvious comparison to want to make. A hosted service runs cloud infrastructure plus Mac hosts, while a self-hosted relay and agent can share one machine and stay on the LAN.

But the providers I looked at don't publish the information I'd need to calculate it: PUE, regional grid mix, or sessions per host. Without those numbers, a kilogram-for-kilogram comparison would be made up, so I'm leaving that one qualitative.

Measure it on your own host

The per-session power cost is the one input above that I measured directly.

I ran the test on a MacBook Pro 14" with an M2 Pro and 32 GB of memory, running macOS 26.5.2. One iOS simulator, the relay, and the agent were all running on the same host.

Each point uses 60 one-second samples from Combined Power (CPU + GPU + ANE):

sudo powermetrics --samplers cpu_power -i 1000 -n 60 \
  | grep 'Combined Power (CPU + GPU + ANE)'

Point	State	SoC power	Increment
A	No session	0.66 W (sd 0.14)	—
B	Session open, screen static	0.64 W (sd 0.29)	≈ 0
C	Session open, continuous scrolling	5.16 W (sd 1.99)	+4.52 W

Keeping a stream open adds effectively nothing in this measurement. A and B differ by −0.02 W, which is well inside the noise. That's consistent with how the H.264 stream behaves when the screen isn't changing: we had already measured roughly 2 MB/s for a still screen and 12–16 MB/s while scrolling.

During continuous interaction, the measured increment was 4.52 W at the SoC, or about 5.05 W at the wall after accounting for adapter efficiency.

powermetrics reports SoC power rather than the display, SSD, fans, and the rest of the system. That's useful here because I'm measuring the increment between states. The platform draw is present in A, B, and C, so it cancels out. For absolute draw, I used Apple's published wall-power figures instead.

If you already run a host, tapflow's built-in Mac resource monitoring also exposes CPU and memory series you can compare against your own workload.

The full calculation — inputs, sources, measurement conditions, sensitivity analysis, and the assumptions I discarded — is in sustainability-carbon-math.md.

There's also a shorter summary in the sustainability section of the docs.

npm install -g tapflow
tapflow start

Repo: https://github.com/jo-duchan/tapflow — MIT, self-hosted; builds and streams stay on your own network
Docs: https://www.tapflow.dev

The carbon in that drawer was spent years ago. Using those devices carefully can't undo it.

What we can change is what happens when those devices need replacing.

So if your team has a drawer like this, I'm curious: how many devices are in it, and how many days a month does each one actually get used?

I only have my own numbers.

A UI test that guesses is worse than one that fails

Duchan — Fri, 07 Aug 2026 07:22:10 +0000

The worst bug I've seen in a test suite wasn't a failure. It was a pass.

A login test tapped a button labeled "Continue". The app then shipped a second "Continue" inside a consent sheet, and the recorded test — which resolved its selector by taking the first match — started tapping the wrong one. It stayed green for two weeks, because the flow still ended up somewhere plausible, and a passing test is not something anyone goes back and reads.

So that became the rule for tapflow's flow runner: anywhere the tooling has to choose for you, it has to say so. Finding those places took longer than fixing them — there were four.

Last time I wrote about getting an accessibility tree out of a simulator that has no window, on both iOS and Android, in one schema. This post is about what that tree is for.

The automation axis here — the flow runner and the MCP server — is experimental. The manual browser QA path is the mature one.

The flow file

A flow is YAML, replayed by tapflow flow run with zero LLM calls. The same input produces the same steps in the same order, every time:

name: login-smoke
appId: com.example.app
steps:
  - clearState
  - launchApp
  - assertVisible: "Sign in"
  - tapOn: { id: "com.example.app:id/email" }
  - inputText: "user@example.com"
  - tapOn: "Sign in"
  - assertVisible: { label: "Orders", timeout: 15 }

There are ten steps in the entire vocabulary — clearState, launchApp, tapOn, inputText, pressKey, swipe, scroll, openUrl, assertVisible, and assertNotVisible. That's deliberate. Every step we add is another way for a flow to end up meaning something other than what it looks like it says.

Ambiguity is a bug in the selector

A bare string resolves in a fixed order — exact identifier, then exact label, then partial label — and once one of those stages matches, the later stages aren't tried. If more than one element survives that stage, a tapOn fails on the spot and tells you what it found:

2 elements match "New Orders" — add an index or a more specific role/label
  (candidates: button "New Orders" | text "New Orders")

This is the case from the top of the post. Taking matches[0] would let a suite silently retarget after a redesign, and there's no way to tell that apart from a suite that still works until something downstream breaks. The candidate list also turns a five-minute stare at a screenshot into a one-line edit.

Two disambiguators resolve it. role narrows by element kind, which handles the common case where a button and the text inside it carry the same label. index picks the Nth remaining match, zero-based, for rows that have neither a label nor an identifier:

- tapOn: { label: "New Orders", role: button }
- tapOn: { role: cell, index: 2 }

The object form needs at least one of id, label, or role. An index on its own would be a positional coordinate again, just written differently.

(assertVisible is checking presence rather than choosing a target, so it passes when at least one element matches.)

There is no sleep step

Timing was the next place the runner was guessing. A sleep records how slow CI happened to be on the day you wrote the test, and then every later run either wastes that time or fails because the machine was busier. So waiting is always a condition with a deadline instead: assertVisible with a timeout, which defaults to 10 seconds and can be set per selector.

That was the design, and it had a hole in it that only showed up in use.

Right after launchApp, the app isn't in the foreground yet, so a tree query legitimately throws. The first version of the poll loop treated any thrown query as a step failure, which meant a flow that began with a launch failed immediately. People worked around it by adding a long-press somewhere as a stand-in for a sleep. We had removed sleeps and handed everyone a worse one in return.

The loop now separates a failure worth waiting out from one that waiting cannot fix:

async function queryOrRetry(driver: FlowDriver, deadline: number) {
  try {
    // bound the query by the remaining deadline: a stalled response
    // must not block the loop past the step's own timeout
    return { tree: await driver.queryUITree(AbortSignal.timeout(deadline - Date.now())) }
  } catch (e) {
    if (e instanceof TransientQueryError) return { transient: e.message }  // keep polling
    throw e   // bad request, auth, missing session → fail now
  }
}

A foreground race, an idle timeout, or a network blip is retried until the deadline. A malformed request or a missing session fails immediately, because waiting won't fix either one. And when a wait does time out, the last transient error is included in the message, so "no element matched X within 10s" can't hide the fact that every query along the way was erroring.

Was that a test failure, or a dead runner?

The next one only shows up in CI, at 3am, when a red build tells you something broke but not whom to wake — the person who wrote the checkout screen, or the person who owns the Mac in the closet.

So the exit codes are a contract:

Code	Meaning
`0`	All flows passed
`1`	At least one flow failed
`2`	Environment/config error (parse failure, relay unreachable, no device)

- name: Run flows
  run: |
    tapflow flow run .tapflow/flows/*.yaml \
      --relay "$TAPFLOW_RELAY_URL" \
      --device "iPhone 16 Pro" \
      --build "$BUILD_ID" \
      --junit report.xml

--junit writes one testcase per flow, and a failure drops a screenshot from the moment it failed into .tapflow/artifacts/. The launchApp step takes no argument and launches whatever --build installed, which keeps build ids out of the flow file so the same flow can run against a fresh build every time.

Where a model is allowed to guess

The last one came later, and it's the reason the runner does no inference at all.

An agent can drive a session through the MCP tools — tap, type, read the tree — and author a flow out of what it just did. That part is exploratory and non-deterministic, and that's fine. What it produces is then replayed by the runner with no model in the loop, and the replay is what decides whether the flow was any good. Generation is allowed to be fast and occasionally wrong, because the deterministic pass is the gate it has to clear.

The same engine backs the MCP run_flow tool, so an agent can author a scenario once and replay it afterward instead of re-deriving it on every run. Replay costs nothing in API calls, which is also why it's cheap enough to run on every commit.

What this is really for

Flow Capture is the reason any of this exists. A person tests a build by hand in the browser, the way they already do, and every tap is recorded as a selector read from the tree instead of a coordinate. The QA someone already performed becomes the flow that runs on every build after it.

It isn't built yet, and I don't want to oversell it. What is done is the part it stands on: the tree, the shared schema, and a runner that reports an ambiguous situation instead of resolving it silently.

Limits worth knowing

Selector matching is the roughest edge here, especially in the seconds right after a launch, and the automation axis as a whole is experimental. If your app has screens where nothing carries an identifier or a label, role and index will get you through them, but a flow written that way is more brittle than one written against a screen with accessibility identifiers. That's its own argument for adding them.

Try it

npm install -g tapflow
tapflow start
tapflow flow run .tapflow/flows/login-smoke.yaml

Repo: https://github.com/jo-duchan/tapflow
Docs: flow reference · MCP in CI/CD

If your suite currently resolves selectors by taking the first match, it's worth going and looking at what it's tapping today.

Coordinate-based UI tests break. So we read the accessibility tree instead — from inside the simulator.

Duchan — Thu, 30 Jul 2026 09:30:18 +0000

Every recorded mobile test I have ever inherited died the same way: someone moved a button.

The recording said "tap at (340, 712)". The redesign moved that button up by one row, and the test kept tapping — now on empty space, or whatever happened to land there instead. It didn't fail right away. Three sprints later, it started failing in confusing ways, and by then nobody trusted the suite anymore.

The fix isn't a better recorder. It's recording a different thing: not where you tapped, but what you tapped. That needs an element tree, and for a while we didn't have one.

tapflow is an open-source, self-hosted tool that streams iOS simulators and Android emulators into a browser, so a whole team can test builds without installing anything. Until now, everything it moved was pixels in one direction and taps in the other. This post is about getting an element tree out of a simulator with no window, on both platforms. What we do with that tree — replaying flows that survive a redesign — is the next post in this series.

The automation axis this feeds — the flow runner and the MCP server — is experimental. The manual browser QA path is the mature one.

The constraint: no WebDriverAgent, and no simulator window

tapflow already injects touches into the iOS simulator without WebDriverAgent — it loads CoreSimulator.framework and pushes HID events through SimDeviceLegacyHIDClient (that story is ep.1). Streaming reads the framebuffer IOSurface directly. Neither path needs Simulator.app on screen, and that's deliberate: an agent Mac in a closet running four simulators shouldn't be babysitting four windows.

So whatever we used for the tree had to follow the same rule. No WDA to install and keep in sync with Xcode. No simulator window on screen.

Our first attempt ran into exactly that limitation. macOS exposes an accessibility API (AXUIElement), and Simulator.app publishes its content through it. We wrote a helper around it, and it worked perfectly on a developer's laptop. On the headless path it returned nothing, because the AX bridge only exists while the simulator window is being rendered. A simulator started with simctl boot in current Xcode doesn't open a window at all.

So we ended up with a tree reader that only worked in the one situation where we didn't actually need it.

A resident XCUITest runner, living inside the simulator

XCUITest reads the tree of any app by bundle id — that's what it's designed to do — and it runs inside the simulator, so no window is involved. The catch is that xcodebuild test is built around "run a test, print results, exit," while we need something that can answer queries for hours.

So the test target doesn't actually test anything. It starts an HTTP server and blocks:

// TreeRunner — the UI test that never finishes
func testServeTree() {
    let server = try! TreeServer(port: port)
    server.start()
    // block forever: the process stays resident and keeps serving
    RunLoop.current.run()
}

The server is about 100 lines of Network.framework with two routes: GET /health for readiness, and GET /tree?bundleId=<id>, which returns XCUIApplication(bundleIdentifier:).debugDescription — the full element subtree as text, including role, label, identifier, and frame for everything on screen. It's not a documented API contract, and I'll come back to that in the limits section.

The network binding was the detail I most underestimated. The iOS simulator shares the host's network stack, so a default NWListener bind listens on all interfaces. That would expose every screen of the app under test to the local network with no authentication, which is exactly what a self-hosted setup is supposed to avoid. requiredInterfaceType isn't enough. You have to pin the local endpoint:

params.requiredLocalEndpoint = NWEndpoint.hostPort(
    host: NWEndpoint.Host("127.0.0.1"), port: nwPort
)

A failed bind also exits the process immediately, because a listener quietly sitting in .failed looks alive to the parent process and turns a stale-port collision into a two-minute mystery after a 90-second readiness poll.

On the Node side, XCUITreeReader builds the runner once with build-for-testing (cached, so only the first query pays for it), launches it detached with test-without-building, polls /health, then serves queries from the resident process. It starts lazily, on the first tree query, so manual QA — the primary path, where a designer just wants to check a build — never pays the startup cost for a runner it isn't going to use.

Shutdown kills the detached process group and then runs simctl terminate on the in-simulator host app too, since that process isn't part of the group and would otherwise keep holding the port:

// detached spawn → the child leads its own group; kill the whole group
if (pid) process.kill(-pid, 'SIGTERM')
execFile('xcrun', ['simctl', 'terminate', udid, RUNNER_HOST_BUNDLE], () => {})

One rule matters more than anything else: a tree read never returns an empty array to mean "something went wrong." A garbage response body, a wrong bundle id, or an app that's no longer in the foreground all produce explicit errors. "The screen has no elements" should only ever mean exactly that. If it can also mean the infrastructure failed, every test built on top of it becomes ambiguous.

Android was the easy one

uiautomator dump already gives you an XML hierarchy, and the parser is straightforward. The only thing worth watching for is that uiautomator waits for the window to go idle before dumping, so an app with a continuous animation — a spinner or a looping splash screen — never reaches that state and the dump hangs.

The timeout has to run on the device, not on the host:

adb exec-out timeout 10 uiautomator dump /dev/tty

That timeout comes from Android's toybox. The same rule applies as on iOS: infrastructure failures should never look like valid UI output.

One schema, and why frame normalization matters

Both backends parse into the same element shape:

interface UIElement {
  role: 'button' | 'text' | 'input' | 'cell' | 'image' | ...
  label: string
  identifier: string
  frame: { x: number; y: number; width: number; height: number } // 0–1
  enabled: boolean
}

role normalizes two vocabularies into one: iOS XCUIElementType and Android class names (AppCompatButton, AutoCompleteTextView, RecyclerView), ordered so composite names resolve before their generic substrings — ToggleButton has to match switch before Button.

The biggest simplification comes from normalizing every frame into the same 0–1 coordinate space that the touch path already uses. iOS debugDescription frames arrive in points and are divided by the window frame. Android bounds are divided by the root node, which already spans the display, so landscape mode works without an extra wm size query.

After that, the whole pipeline becomes:

tree query → element.frame center → tap(x, y) → the same HID path a human tap uses

No second connection to the device, no separate driver, and no extra coordinate conversion.

What this enables

A test used to have exactly one way to describe where to tap:

(340, 712)

Now it can simply say:

tapOn: "Sign in"

That resolves against the tree — exact identifier first, then exact label, then partial label — and the matched element's frame center goes through the same tap path shown above.

The tree isn't only for the flow runner, either:

GET /api/v1/sessions/:sessionId/ui-tree     # REST
query_ui_tree                               # MCP tool

An LLM agent driving a session reads exactly the same schema a flow does. ep.4 gave an agent eyes through screenshots. This gives it names for what it's looking at.

Limits worth knowing

The iOS tree comes from debugDescription. It's a text format, not a stable API. The parser is a pure function with fixture tests, so an Xcode format change fails a unit test before production — but the format can still change.
enabled is approximate on iOS. debugDescription doesn't expose it, so elements default to enabled: true.
One resident runner at a time. It listens on a fixed port (xcodebuild doesn't propagate host environment variables into the in-simulator runner, so per-device ports aren't available that way). Concurrent multi-device tree queries are deferred.

Try it

npm install -g tapflow
tapflow start          # → http://localhost:4000

Repo: https://github.com/jo-duchan/tapflow
Docs: https://www.tapflow.dev
Earlier in this series: why we built it and how the touch path works, driving down streaming latency.

Next in the series: what you can build once a test can name the thing it taps, and the four places we made the runner refuse to guess.

If you've solved the headless tree problem another way, especially something more stable than debugDescription that doesn't require WebDriverAgent, I'd be interested to hear about it.

Your React Native build is done. Now your whole team can test it in the browser.

Duchan — Fri, 03 Jul 2026 08:12:44 +0000

If your team ships a React Native app, you already know the build is the easy part. The friction shows up after the build — when everyone who isn't a mobile developer needs to actually look at it.

The PM wants to compare two versions of a flow. The designer wants to check layout across screen sizes. The backend engineer wants to confirm what the staging build actually does with their API change. And every one of them ends up in the same back-and-forth:

"Can you share the build with me?" — and then, "wait, how do I install this?"

Because installing and running a build means having Xcode or Android Studio and a simulator set up, which non-developers don't. So
a mobile developer becomes a human deployment pipeline — installing builds for people, or booting the app on their own machine to s
how it, a dozen times a day.

We built tapflow — an open-source, self-hosted tool that streams iOS simulators and Android emulators straight to the browser — to get rid of that interruption. This post is the whole picture, told from a React Native team's point of view.

Why this fits React Native so naturally

tapflow isn't a React Native tool. It's framework-agnostic — and that's exactly why it covers React Native so completely.

tapflow works one layer below the framework. It doesn't care whether your app was built with RN, Expo, or bare native code. It operates on the built artifact — the .app or .apk — and the simulator running it:

RN / Expo / native        ← your framework
      │ build
      ▼
  .app / .apk              ← tapflow starts here
      │
      ▼
  simulator / emulator  ──stream──►  browser (your whole team)

Since a React Native build compiles down to an ordinary native app, it lands inside tapflow's umbrella with nothing special required — an Expo/EAS build or a bare RN build, tapflow treats them all as the native artifacts they are. It never needs to know it's React Native. (More on wiring up EAS below.)

And because React Native teams are cross-platform by nature — one codebase, iOS and Android — the payoff is sharper than for a single-platform native team. One dashboard, both platforms, anyone on the team, no local setup.

npm install -g tapflow
tapflow start
# → http://localhost:4000

That's the whole install. Anyone opens the dashboard, picks a device, and starts interacting.

Works with Expo — and bare React Native

Turn every EAS build into a browser-based mobile preview.

If you're on Expo, add one profile to your eas.json:

{
  "build": {
    "tapflow": {
      "ios": { "simulator": true },
      "android": { "buildType": "apk" }
    }
  }
}

eas build --profile tapflow --platform all
# iOS → .tar.gz (simulator .app inside)   ·   Android → .apk

That one profile produces exactly what tapflow runs: for iOS, a simulator build — which EAS hands you as a .tar.gz — not the default device .ipa (which can't run on the Simulator); for Android, an .apk. tapflow takes both formats as-is, so there's nothing to repackage:

  .tar.gz (iOS sim) · .apk (Android)
        │  CI uploads it as-is
        ▼
     tapflow
        │
        ▼
  browser preview — whole team, no install

Your CI (or a small webhook receiver) downloads the finished artifact and uploads it to tapflow automatically, so wiring it to an EAS Build webhook turns every green build into a browser-testable preview the whole team can open, before TestFlight or an internal APK share.

On bare React Native, there's no EAS step to think about: point tapflow at the simulator .app / .apk your existing react-native run-ios or Gradle build already produces.

This is a complement, not a replacement. EAS builds, signs, and ships your app; tapflow slots in right after the build so everyone can see it without installing anything.

Metro stays. This is for the build, not the dev loop

React Native developers will reasonably ask: does this replace Metro / Fast Refresh?

No — and it isn't trying to. Your inner dev loop (edit → Fast Refresh → repeat on your own machine) stays exactly as it is. tapflow covers the other half: the built artifact that the rest of the team needs to review — the staging build, the EAS build, the "does the sandbox behave right" check.

Metro is for the developer writing the code. tapflow is for the ten other people who need to see the result without becoming React Native developers first.

Why self-hosted (and why we didn't use a cloud lab)

Cloud simulator services solve the browser-access problem too. We evaluated them seriously before building anything, and stopped at two blockers:

Cost. Per-seat, per-minute pricing scales badly for a whole team just reviewing builds.
Data. They require uploading your app binary to an external service. For a proprietary app, that's a non-starter.

tapflow runs on Macs you already own. The architecture keeps everything in your network:

Browser (your team)  ←─ WebSocket ─→  Relay  ←─ WebSocket (outbound) ─→  Mac Agent
                                    (Linux/Mac)                        (iOS · Android)

The Mac Agent connects outbound to the relay, so there's no firewall or NAT setup. The relay runs on the Mac you already own — or another machine on the same network — so there's nothing extra to buy, and your app binary and test data stay on infrastructure you control. It's MIT-licensed and fully self-hosted.

What React Native teams actually reach for: deep links

The feature that comes up most in real QA sessions maps directly onto something React Native leans on heavily — deep links.

RN navigation, Linking, notification payloads, OAuth redirects — verifying a specific app state usually means firing a deep link. The old workflow always pulled in a developer: trigger it from their machine, or build a debug menu into the app just for testing.

In tapflow you fire a deep link straight from the QA session toolbar. Hit ⌘K, paste the URL, and it runs on the active device:

Browser ──open-url──► Relay ──open-url──► Mac Agent
                                             │
                          iOS:     xcrun simctl openurl booted <url>
                          Android: adb shell am start -a VIEW -d <url>
Browser ◄──open-url:done/error── Relay ◄─────┘

Under the hood it's a symmetric openUrl(url) on the device interface, so iOS and Android behave the same way and the dashboard shows a toast either way. A PM can now reproduce the exact screen behind a myapp://product/42 link without touching a terminal.

The same session toolbar covers the rest of the repetitive QA actions with keyboard shortcuts:

Shortcut	Action
`⌘K`	Open deep link dialog
`⌘S`	Take screenshot
`⌘⇧Y`	Start / stop recording
`⌘⇧O`	Rotate simulator

Screenshots are also exposed as an API, so a designer doing layout review across screen sizes can capture without asking anyone.

Where it's headed

There's an experimental MCP server that gives an LLM agent eyes and hands on the simulator — it can see the screen and drive taps directly. It's opt-in and completely separate from the manual testing path, so it doesn't change anything above. If that's interesting, it has its own write-up.

Getting started

npm install -g tapflow
tapflow start

On Expo, produce a tapflow-ready build with the profile from earlier:

eas build --profile tapflow --platform all

Then drop the build in and share the dashboard URL with your team.

Repo: https://github.com/jo-duchan/tapflow
Docs: https://www.tapflow.dev
If you want the deeper cuts: how we built the browser-streaming touch pipeline, how we drove down streaming latency, and one-command host setup.

If your team ships React Native, the build was never the hard part. Letting everyone see it was. That's the part tapflow removes.

The DX we wanted for tapflow setup: a host-ready Mac in one command

Duchan — Wed, 17 Jun 2026 13:49:47 +0000

tapflow streams iOS simulators and Android emulators into a browser, so a whole team can test an app without installing anything. The simulators run on a Mac that hosts the tapflow agent, and that Mac needs the mobile toolchain: Xcode, a simulator runtime, the Android SDK, an emulator, AVDs.

tapflow already gives the people who use it a zero-install experience. We wanted the person who hosts it to get the same one. So tapflow setup brings the whole host environment up in as close to one command as the toolchain allows.

Here are the DX decisions behind it.

`doctor` diagnoses, `setup` installs and configures

We split diagnosis from the steps that install and configure.

tapflow doctor is read-only: it checks the prerequisites — Xcode, simctl, a simulator runtime; the SDK, adb, an AVD — and reports. It never changes your machine, so it's safe to run anywhere, anytime. A clean run looks like this:

tapflow doctor

  ✓  Node v20.11.0

  iOS
  ✓  Xcode 16.2
  ✓  xcrun simctl
  ✓  Simulator available (8)

  Android
  ✓  Android SDK: ~/Library/Android/sdk
  ✓  adb found: ~/Library/Android/sdk/platform-tools/adb
  ✓  AVD available: tapflow-phone

  All checks passed.

When something's missing, each failing line carries the exact fix — ⚠ AVD → No AVD found. Run: tapflow setup android — so doctor always points straight at setup.

tapflow setup is the one command allowed to install and configure. The two mirror each other, so the mutating verb lives in exactly one place. Run setup with no argument and it reads the environment:

if (process.platform === 'darwin') platforms.push('ios')
if (resolveAdb() !== null) platforms.push('android')

macOS implies iOS; an existing adb implies you care about Android. If neither signal is there, it asks instead of guessing.

iOS: installed ≠ usable

Xcode can only come from the App Store, so setup doesn't fake it — it opens the right page and waits.

The part we kept getting wrong was that a freshly installed Xcode isn't a working one. Three steps stand between "the app exists" and "xcodebuild runs": point the active developer directory at Xcode (xcode-select -s), accept the license, and finish first launch. setup runs them for you, after asking, since they need sudo. The check that matters isn't "does Xcode.app exist" — it's whether xcodebuild -version actually runs.

Android: a self-contained SDK

This is the decision we're happiest with.

The obvious path is "install Android Studio." We didn't. The host doesn't need a GUI IDE, and depending on one means fighting whatever SDK location, ANDROID_HOME, and AVDs the user already has. Instead setup builds a self-contained SDK under one path we own:

sdkmanager --sdk_root=~/Library/Android/sdk \
  "cmdline-tools;latest" "platform-tools" "emulator" \
  "system-images;android-35;google_apis;arm64-v8a"

After that, every Android binary tapflow touches comes from inside that directory. A couple of details that make it reliable:

sdkmanager needs a JDK or it won't run, so a Temurin check happens first.
The system image is google_apis, not the Play Store one, which is unstable the way we drive it.

The one thing that has to outlive the process is ANDROID_HOME on your PATH. setup writes it into your shell rc inside a marker block, and only if it isn't already there — so re-running never duplicates it:

# >>> tapflow android sdk >>>
export ANDROID_HOME="$HOME/Library/Android/sdk"
export PATH="$ANDROID_HOME/platform-tools:$ANDROID_HOME/emulator:$PATH"
# <<< tapflow android sdk <<<

The catch it can't remove: the variable isn't in your current shell, so setup tells you to open a new terminal before doctor.

setup prepares; the relay boots

For both platforms, setup stops at "a bootable device exists." It never boots a simulator or emulator. That's the relay's job — it boots the right device on demand when a teammate joins a QA session. Two components owning device lifecycle would just race each other.

One rail under all of it

Every step that changes the machine asks first, and only auto-runs in an interactive terminal. Run setup in CI and instead of curling an install script as root, it prints guidance and exits clean. Nothing in tapflow deletes your data — the only teardown command is reset, which shuts down running simulators.

Honest limitations

Xcode is still a manual App Store download. There's no API for it; setup automates everything around it.
The first run usually needs a new shell for the Android PATH to take effect.
The agent host is a Mac, since that's the only place iOS simulators run. (The relay itself runs on Linux.)
Still v0.x, so the steps will keep moving as the toolchain shifts.

Takeaway

The downloads were never the hard part. The DX lives in the glue around them — Xcode activation, the PATH that needs a fresh shell, knowing to stop at "bootable" and let the relay boot the rest. Automating a dev environment means automating everything that isn't the install.

Try it

tapflow is MIT licensed.

npm install -g tapflow
tapflow doctor     # what's missing?
tapflow setup      # set it up
tapflow start

🔗 GitHub: https://github.com/jo-duchan/tapflow
📖 Docs: https://www.tapflow.dev

We switched simulator streaming to H.264 and it felt worse. Here's how we fixed the latency.

Duchan — Wed, 10 Jun 2026 07:06:31 +0000

In an earlier post I described how tapflow streams iOS simulators to the browser: pull frames off the simulator's IOSurface, JPEG-encode them on the Mac, push them over WebSocket at ~30fps.

JPEG has one great property for interactive streaming: every frame is independent and decodes instantly. There's no buffer, no inter-frame dependency. On localhost it feels like you're touching the simulator directly.

It also has one terrible property: size. A full-frame JPEG of a scrolling screen is ~590KB. On a LAN that's 12–16 MB/s, and our relay started dropping 16–27 frames a second under backpressure — visible tearing.

So we did the obvious thing and moved to H.264. Bandwidth dropped roughly 140× on a still screen and 5× while scrolling. Drops nearly vanished.

And the stream felt worse.

This post is about why, and the two fixes that got H.264 back to "feels like direct touch."

The bar: localhost JPEG

Before touching anything I needed a number, not a vibe. So I instrumented the pipeline end to end — a per-stage panel that reports decode→present and glass→glass (capture timestamp to on-screen) latencies live.

One caveat I'll repeat throughout: glass→glass absolute values are only valid on localhost, where capture and display share one clock. decode→present is a same-machine delta and valid anywhere, so I'll lean on it for the cross-environment claims.

Here's the baseline that mattered, measured on localhost:

Path	decode→present p50/p95 (ms)
JPEG still	12.4 / 15.4
JPEG scroll	9.4 / 11.6
H.264 (WebCodecs) still	267 / 274

H.264 decode was ~20× slower than JPEG. On a hardware decoder. That made no sense — until I looked at what the decoder was actually doing.

Fix 1: the decoder was buffering 8 frames for no reason

The transport was clean (~1ms), the input queue was empty. The latency was entirely inside the decoder: it was holding ~8 frames before emitting the first one.

That's a DPB (decoded picture buffer). A decoder reorders frames when B-frames are present — it has to wait for future frames to arrive before it can output the current one in display order. So it buffers up to the level's maximum.

But our encoder is baseline H.264, B-frames off. There is no reordering. The actual reorder depth is zero. The decoder was buffering anyway because the bitstream never told it the reorder depth was zero.

The signal lives in the SPS (sequence parameter set), in the bitstream_restriction flags inside VUI. Our VideoToolbox encoder wasn't setting them, so the decoder fell back to the worst case for the level — max_dec_frame_buffering of ~8 frames at Level 5.0.

The fix is to rewrite the SPS and inject the missing declaration:

max_num_reorder_frames = 0
max_dec_frame_buffering = num_ref_frames

We do this in the agent, on the keyframe SPS, before the frame ever leaves the Mac — so every decoder downstream benefits, not just one browser path:

// agent-core/utils/sps.ts — rewrite the SPS to declare zero reordering
function rewriteLowLatencySps(sps: Uint8Array): Uint8Array {
  const bits = new BitstreamWriter(parseSps(sps))
  bits.vui.bitstreamRestriction = true
  bits.vui.maxNumReorderFrames = 0
  bits.vui.maxDecFrameBuffering = bits.numRefFrames
  return serialize(bits)
}

Result on localhost:

Path	decode→present p50/p95 (ms)
H.264 WebCodecs still (before)	267 / 274
H.264 WebCodecs still (after)	2.5 / 4
H.264 WebCodecs scroll (after)	2.1 / 3.9

267 → 2.5ms, roughly 100×. The encoder was lying to the decoder by omission, and the decoder defended itself by buffering. One declaration fixed it.

The browser confirms it's receiving the rewrite — the SPS now reports bitstreamRestriction: true, maxNumReorderFrames: 0.

Fix 2: MSE is a buffer you can't turn off

Fix 1 only helps the WebCodecs path. And WebCodecs has a hard constraint: it only runs in a secure context — HTTPS or localhost.

A team using tapflow over their LAN hits it at plain http://<mac-ip>:4000. That's a non-secure context, so the browser can't use WebCodecs. The fallback at the time was MSE (Media Source Extensions): feed the H.264 into a <video> element through a muxer.

The problem is that <video> is a buffer. It's designed for media playback, where a jitter buffer is a feature. For interactive streaming it's structural latency you can't remove. I measured it on localhost by forcing the MSE tier:

Path	decode→present p50/p95 (ms)
H.264 MSE still	239 / 254
H.264 MSE scroll	229 / 244

~235ms, on the same reorder=0 stream that WebCodecs decoded in 2.5ms. The SPS fix can't reach this — it's the media-element buffer, not the decoder's DPB. I'd already set the muxer's flushingTime to 0. There was nothing left to shave.

So I stopped trying to make MSE fast and removed it.

The decoder layer is now two tiers, picked automatically per environment:

// pickDecoder — secure → WebCodecs, otherwise WASM
export function pickDecoder(): Decoder | null {
  if (isSecureContext && 'VideoDecoder' in window) {
    return new WebCodecsDecoder()      // HW, lowest latency
  }
  if (webgl2Available && wasmSupported) {
    return new WASMDecoder()           // tinyh264, zero-buffer
  }
  return null                          // → fall back to JPEG
}

On non-secure LAN-HTTP, we decode H.264 in WASM (tinyh264). It's a software decoder, so it costs CPU — but it has no media-element buffer at all. That's the whole point: it gives you JPEG's immediacy with H.264's bandwidth, on plain HTTP.

Measured on localhost (the worst case — encoder and decoder share one Mac):

Path	decode→present p50/p95 (ms)
H.264 WASM still	8.7 / 30.4
H.264 WASM scroll	14.3 / 37.9

That's on par with the localhost-JPEG baseline (12.4 / 9.4) — the bar we set at the start. Removing MSE also let us drop the muxer dependency entirely.

One constraint this introduces: tinyh264 only decodes baseline H.264. iOS already encodes baseline. For Android we pin scrcpy to baseline (profile:int=1) so both platforms share the exact same HTTP→WASM path. High profile is still available on the WebCodecs (secure) tier.

One more thing: dropping H.264 isn't like dropping JPEG

There's a subtlety the switch exposed. With JPEG, every frame is a keyframe, so dropping a frame under backpressure is harmless — the next one stands alone. With H.264, if you drop a P-frame, every following P-frame references something the decoder never received. A zero-buffer decoder like WASM tinyh264 shears until the next IDR arrives.

So the relay had to become keyframe-aware: once it starts dropping under backpressure, it drops the whole GOP until the next keyframe, rather than handing the decoder a broken reference chain. The keyframe flag rides in our frame envelope, so this needs zero NAL parsing on the relay.

// relay — once dropping, drop until the next keyframe
if (backpressured) {
  if (!frame.isKeyframe) return       // skip P-frames in a broken GOP
  dropping = false                    // keyframe resets the chain
}

Honest limitations

WASM decode is CPU-bound. At high resolution × fps it hits a CPU ceiling. We mitigate by downscaling the encode resolution — the display is small, so it's a triple win on bandwidth, CPU, and latency.
The localhost numbers are best-case for latency and worst-case for CPU. On a real LAN the decoder runs on a separate machine. In our cross-machine measurements, scroll p95 climbs to ~50ms on both decoders — at that point the bottleneck is load/transport, not the codec. The decode→present deltas above hold; the glass→glass absolutes do not transfer across two clocks.
Still v0.x. The decoder tiers and SPS rewrite are in agent-core; expect them to keep moving.

Takeaway

Two bugs, same symptom ("H.264 feels laggy"), completely different causes:

The decoder's DPB buffered 8 frames because the SPS didn't declare reorder=0. Fix: rewrite the SPS at the encoder.
The media-element buffer in MSE added ~235ms that no encoder flag can reach. Fix: remove MSE, decode in WASM on non-secure contexts.

The lesson I keep relearning: when streaming feels slow, measure each stage before you change the codec. The codec usually isn't the problem — the buffer you didn't know you had is.

Try it

tapflow is MIT licensed.

npm install -g tapflow
tapflow start

🔗 GitHub: https://github.com/jo-duchan/tapflow
📖 Docs: https://www.tapflow.dev

Giving an LLM Eyes and Hands on a Mobile Simulator

Duchan — Sat, 30 May 2026 08:23:03 +0000

Mobile QA has a scaling problem.

Unit tests and API tests run in CI automatically. But the thing that actually matters to most users — does tapping this button do the right thing, does this screen look right after this flow, does the deeplink open the correct state — none of that runs automatically. Someone has to open the simulator, walk through the steps, and verify. Every time.

The usual answer is Appium or XCUITest. But those require engineers to write and maintain test code that mirrors the UI, breaks whenever the screen changes, and only runs against builds developers already have locally.

We had a different idea. tapflow already lets humans control a simulator through a browser. What if we gave an LLM the same interface?

The interface a human uses

When a person does QA in tapflow, the loop is:

Look at the simulator screen
Decide what to do (tap, swipe, type)
Do it
Look again

This is exactly the perception-action loop that vision-capable LLMs are built for. The model sees a screenshot, reasons about what it shows, decides what action to take, and calls a tool to execute it.

We didn't need to build a new automation layer. We just needed to expose tapflow's existing WebSocket and REST APIs as MCP tools.

What the MCP server does

@tapflowio/mcp-server connects to a running tapflow relay and registers 13 tools that any MCP-compatible client can call:

list_devices       — see all simulators registered on the relay
connect_device     — join a device session
boot_device        — boot a simulator (waits up to 30s for ready state)
screenshot         — capture the current screen
tap                — tap at a pixel coordinate
swipe              — swipe between two coordinates
type_text          — type into the focused field
press_key          — press a keyboard key (Return, Delete, Escape...)
press_button       — press a hardware button (home, lock)
install_app        — install a build from App Center
launch_app         — launch an installed app
list_builds        — list available builds on the relay
disconnect_device  — end the session

Setup is two environment variables:

TAPFLOW_RELAY_URL=wss://your-relay-url
TAPFLOW_TOKEN=your-pat-token
npx @tapflowio/mcp-server

Add it as an MCP server in your client config, and those tools appear in the model's tool list.

How the tools are implemented

Screenshot — the model's eyes

The screenshot tool calls the REST endpoint we added in v0.3.0 (GET /api/v1/sessions/:id/screenshot), gets back a PNG or JPEG buffer, base64-encodes it, and returns it as MCP image content alongside the pixel dimensions:

return {
  content: [
    { type: 'image', data: buf.toString('base64'), mimeType },
    { type: 'text', text: `Screenshot saved: ${filePath} (${width}×${height}px)` },
  ],
}

The model receives the actual image. It can read text on screen, identify UI elements, notice error states — the same things a human would.

Tap and swipe — normalized coordinates

Here's the part that took a few iterations to get right. The simulator's logical coordinate space is different from screenshot pixel coordinates, and it changes with screen resolution, device type, and scale factor.

Rather than exposing logical coordinates (which the model can't reason about without device-specific knowledge), we have the model work entirely in screenshot pixel space. The tap tool takes pixel coordinates plus the screenshot dimensions, then normalizes internally:

// tools.ts
client.tap(sessionId, x / screenshotWidth, y / screenshotHeight)

The model calls screenshot first, reads the dimensions from the response, then uses those same dimensions when calling tap. This means the model can identify "the button is at roughly pixel 200, 450" from the image and tap it directly — no coordinate system translation required.

Swipe works the same way, with 8 interpolated touch:move events across the duration to simulate a natural gesture:

// client.ts — swipe interpolation
const STEPS = 8
const interval = durationMs / STEPS

this.send({ type: 'input:touch:start', sessionId, payload: { x: startX, y: startY } })
for (let i = 1; i < STEPS; i++) {
  await delay(interval)
  const t = i / STEPS
  this.send({
    type: 'input:touch:move',
    sessionId,
    payload: {
      x: Math.round(startX + (endX - startX) * t),
      y: Math.round(startY + (endY - startY) * t),
    },
  })
}

Async operations over WebSocket

Several tools involve async operations — booting a device, installing an app — where the relay sends a confirmation back over WebSocket after the operation completes.

The client uses a waitFor pattern: register a predicate against incoming messages, return a promise that resolves when a matching message arrives, and reject if a timeout fires first.

// client.ts — waitFor
private waitFor(predicate: (msg) => boolean, timeoutMs: number): Promise<RelayMsg> {
  return new Promise((resolve, reject) => {
    const timer = setTimeout(() => {
      this.waiters.splice(this.waiters.findIndex(w => w.resolve === resolve), 1)
      reject(new Error('Request timed out'))
    }, timeoutMs)
    this.waiters.push({ predicate, resolve, reject, timer })
  })
}

boot_device waits up to 30 seconds. install_app waits 60 seconds. Each resolves on the confirmation message or rejects with the error payload.

What a session looks like

A model running a login flow might do this:

1. list_devices → pick a session
2. connect_device
3. list_builds → find the build to test
4. boot_device
5. install_app
6. launch_app
7. screenshot → see the login screen
8. tap(email field coordinates) → focus the input
9. type_text("test@example.com")
10. tap(password field coordinates)
11. type_text("password")
12. tap(login button coordinates)
13. screenshot → verify the home screen loaded
14. disconnect_device

Each screenshot gives the model a chance to verify state before proceeding. If step 13 shows an error message instead of the home screen, the model knows something went wrong.

Where we are: experimental

The version says 0.3.1-experimental.1 for a reason. The tools work, but the layer needs more hardening before we'd call it reliable.

The core issue is consistency. The same sequence of tool calls should produce predictable behavior every time. Right now it doesn't always — there are timing edge cases where an action fires before the UI has fully settled, device state can drift between steps without the model noticing, and error recovery when something unexpected happens mid-flow is rough.

These are solvable problems, but we want to solve them before presenting this as something teams should build pipelines on.

Where we're going: CI/CD without a QA script

The direction we're aiming at is using the MCP server as the foundation for LLM-driven smoke tests in CI.

The scenario: a new build passes unit tests and gets uploaded to App Center. A CI step spins up the MCP server, points it at the relay, and gives a model a natural-language test spec:

"Install the latest build. Log in with test credentials. Navigate to the cart, add an item, and confirm the checkout screen shows the correct total. Take a screenshot at each step."

The model does the steps, captures evidence, and reports what it saw. No automation code to write. No selectors to maintain when the UI changes. The spec is just a description of what a human would do.

This isn't production-ready yet. The stability work comes first. But the pieces — browser-controllable simulators, screenshot REST endpoint, MCP tool layer — are in place. The question is whether the model can run a flow reliably enough to be trusted in CI without a human verifying each run.

We think it can. That's what we're building toward.

Try the MCP server (experimental)

npm install -g @tapflowio/mcp-server@experimental

You'll need a running tapflow relay and a PAT token with viewer scope. Configure it in your MCP client:

{
  "mcpServers": {
    "tapflow": {
      "command": "npx",
      "args": ["@tapflowio/mcp-server"],
      "env": {
        "TAPFLOW_RELAY_URL": "wss://your-relay-url",
        "TAPFLOW_TOKEN": "your-pat-token"
      }
    }
  }
}

If you try it and hit rough edges, open an issue — that feedback is exactly what's shaping the stability work.

🔗 GitHub: https://github.com/jo-duchan/tapflow
📖 Docs: https://www.tapflow.dev/guide/mcp-server

tapflow v0.3.x: Deeplinks, Keyboard Shortcuts, Screenshot API, and an Experimental MCP Server

Duchan — Fri, 29 May 2026 07:38:22 +0000

tapflow started as a simple idea: stream iOS simulators and Android emulators to the browser so anyone on the team can do mobile QA without touching Xcode or Android Studio. v0.2.x got the core working — streaming, touch input, App Center, session recording.

v0.3.x is about filling in the gaps that matter during actual QA sessions. This post covers what shipped and ends with something we're still figuring out: an experimental MCP server that lets LLM agents control simulators directly.

Deeplink execution from the browser

The one that came up most in real usage: testers frequently need to trigger deeplinks to verify specific app states — product detail pages, notification payloads, OAuth redirects. The old workflow always involved a mobile developer — either having them trigger it on their machine or building a debug menu inside the app specifically for this purpose.

In v0.3.0 you can now fire a deeplink directly from the QA session toolbar. Click the link icon (or ⌘K), enter the URL, and it executes on the active device.

Under the hood it's a new open-url WebSocket message type that routes browser → relay → agent:

Browser ──open-url──► Relay ──open-url──► Mac Agent
                                              │
                           iOS: xcrun simctl openurl booted <url>
                           Android: adb shell am start -a VIEW -d <url>
Browser ◄──open-url:done/error── Relay ◄──────┘

The DeviceAgent interface got a new openUrl(url) method, so both iOS and Android agents implement it symmetrically. The relay routes it and returns either open-url:done or open-url:error with the failure reason. The dashboard shows a toast either way.

Keyboard shortcuts for simulator controls

QA sessions are repetitive. Reaching for the toolbar icons on every screenshot or rotation adds up. v0.3.0 adds keyboard shortcuts to all the common actions:

Shortcut	Action
`⌘K`	Open deeplink dialog
`⌘S`	Take screenshot
`⌘⇧Y`	Start / stop recording
`⌘⇧O`	Rotate simulator
`⌘⇧U`	iOS: press Home
`⌘⇧K`	iOS: toggle software keyboard

Tooltips now show the shortcut hint inline, so they're discoverable without reading docs. One implementation detail worth noting: key detection uses e.code instead of e.key. This matters for IME input — Korean, Japanese, and Chinese users composing text would otherwise trigger shortcuts mid-composition.

Screenshot REST endpoint

This one unlocks a new class of CI usage.

GET /api/v1/sessions/:sessionId/screenshot returns a PNG or JPEG of the current simulator screen. You can call it with a PAT token from any CI step — before asserting a visual state, during an automated flow, after a build install.

The tricky part was the request/response pattern. The relay communicates with agents over WebSocket (long-lived, multiplexed), but HTTP is request/response. Screenshots are taken on the Mac, not the relay.

We introduced a requestId-based pending map: the relay generates a unique ID, sends a take-screenshot message to the agent over WebSocket, registers a promise keyed by requestId, and resolves it when screenshot:result comes back. The HTTP handler awaits that promise and sends the binary payload:

GET /api/v1/sessions/:id/screenshot
    │
    ▼
Relay: generate requestId, push to pending map
    │
    ├──screenshot-request──► Mac Agent
    │                            │ simctl io screenshot (iOS)
    │                            │ ADB screencap (Android)
    ◄──screenshot:result─────────┘
    │
    ▼
HTTP 200 (binary image)

iOS supports both PNG and JPEG via --type. Android returns PNG regardless — ADB doesn't offer format selection at this layer.

PAT scope enforcement

Personal Access Tokens existed before v0.3.0, but the scope field wasn't actually enforced on API routes. A developer scoped token could call any endpoint.

v0.3.0 adds proper scope checks to all builds endpoints. PATs are now enforced at the middleware layer: a token issued for builds access can upload and manage builds, but can't touch team settings or session data. This makes it safe to issue narrow tokens for CI pipelines without giving them broader access than they need.

Frame performance instrumentation

For anyone debugging streaming latency: v0.3.x adds per-frame hop timestamps via a binary header (TFFE — tapflow frame envelope). Each frame now carries the capture time, relay-received time, and client-received time in an 8-byte prefix before the JPEG/H.264 payload.

The dashboard can surface a live performance overlay showing frame latency broken down by segment (agent → relay, relay → browser). Useful when diagnosing whether a slowdown is in the network leg or the browser decode path.

Experimental: an MCP server

v0.3.x also ships @tapflowio/mcp-server (0.3.1-experimental.1) — it exposes tapflow's WebSocket/REST APIs as MCP tools so an LLM agent can drive a simulator the same way a human does in the browser: screenshot → reason → tap/type → screenshot again.

It's early (the experimental suffix is literal — consistency and error-recovery still need work), and it's a big enough topic to have its own write-up: Giving an LLM Eyes and Hands on a Mobile Simulator covers the full tool list, the normalized-coordinate tap/swipe, and where this is headed (LLM-driven smoke tests in CI).

npm install -g @tapflowio/mcp-server@experimental

Try it

npm install -g tapflow
tapflow start
# http://localhost:4000

🔗 GitHub: https://github.com/jo-duchan/tapflow
📖 Docs: https://www.tapflow.dev

Your whole team can now run mobile QA from the browser. Here's how we built it.

Duchan — Wed, 27 May 2026 05:08:18 +0000

If you work on a mobile product, you've probably seen this.

Physical devices are never enough. Covering every OS version is even harder — iOS doesn't support downgrading, so maintaining a range of versions means managing a pool of locked devices, which is overhead nobody wants.

But the bigger friction is access. Simulators only run on a developer's Mac, behind complex toolchains. Anyone on the team who isn't a mobile developer has to ask one every single time they need to verify something:

Server / FE developer — "How do I install the sandbox build to check what was deployed?"

Product manager — "I keep having to install and remove different versions just to compare behavior."

Designer — "I need to check the layout across screen sizes, but I don't have the right devices."

Cloud simulator services exist. But uploading internal app builds to an external service — and paying monthly fees for simulators already running on Macs you own — was never something we wanted to do.

So we built tapflow: an open-source, self-hosted tool that streams iOS simulators and Android emulators to the browser. Anyone on your team opens the dashboard, picks a device, and starts interacting — no Xcode, no Android Studio, no setup.

npm install -g tapflow
tapflow start
# → http://localhost:4000

This post is about how we built it — specifically the parts that weren't obvious.

Why we didn't just use Appetize or BrowserStack

Both services solve the browser access problem. We evaluated them seriously. Before signing up, we hit two blockers:

Cost. Appetize starts at $59/month and scales with team size.
Data. Both require uploading your app binary to external servers. For anything with sensitive business logic, that's a non-starter.

We already had Macs in the office. So we built tapflow instead.

Architecture

Browser (your team)  ←─ WebSocket ─→  Relay Server  ←─ WebSocket (outbound) ─→  Mac Agent
                                     (Linux / Mac)                           (iOS · Android)

The Mac Agent connects outbound to the relay — no firewall or NAT configuration needed. The relay can run on a small Linux server (a ~$5/month Fly.io instance handles it). App data never leaves your infrastructure.

iOS touch — without WebDriverAgent

WebDriverAgent was the obvious starting point. We didn't use it.

The problems: WDA breaks on Xcode updates, requires provisioning profiles, needs the app to be in the foreground, and adds a layer of process management complexity we didn't want to own.

Instead, we load CoreSimulator.framework dynamically via dlopen in a Swift binary (touch-helper), then inject HID events directly through SimDeviceLegacyHIDClient and IndigoHID:

// touch-helper — HID event injection into the simulator
let client = SimDeviceLegacyHIDClient(device: device)
let event = IndigoHIDEvent.touch(x: x, y: y, phase: .began)
client.send(event)

This bypasses WDA entirely. It works independently of the app lifecycle and doesn't break on Xcode updates.

The tradeoff: these are private APIs. They've been stable across Xcode versions in our testing, but Apple could remove them. We think that's a better bet than WDA's reliability track record.

iOS streaming — IOSurface

xcrun simctl io screenshot works, but the latency is too high for interactive use.

Instead, we access IOSurface directly through SimulatorKit, pulling frames straight from the simulator's GPU surface. ~~Frames are JPEG-encoded on the Mac and streamed over WebSocket at ~30fps.~~

For slow clients, we drop frames rather than buffering — backpressure is handled at the WebSocket layer to prevent memory accumulation on the relay when a client can't keep up.

Update: JPEG was the first version. The default is now H.264 with a buffer-free 2-tier browser decoder (WebCodecs on secure contexts, WASM on plain HTTP). The full teardown — why H.264 first felt worse, and the two fixes that solved it — is a separate post: We switched simulator streaming to H.264 and it felt worse.

Android — scrcpy H.264 → WebGL

Android was cleaner. scrcpy already does the hard work of capturing the emulator display as an H.264 stream.

We receive the H.264 Annex B stream from scrcpy over a local TCP socket, relay it through WebSocket, then decode and render it in the browser. Android now shares the same buffer-free 2-tier decoder as iOS (see the update above).

scrcpy server (emulator)
    → TCP socket
    → Mac Agent
    → WebSocket
    → Browser (WebGL2)

Pinch gestures

scrcpy's INJECT_TOUCH_EVENT supports multiple pointer IDs. Pinch is implemented by sending two simultaneous touch events:

// ScrcpyControl — multi-touch injection
pinchStart(x1: number, y1: number, x2: number, y2: number): void {
  this.touchDown(0, x1, y1)
  this.touchDown(1, x2, y2)
}

What's included

Beyond streaming and input:

App Center — upload .app.zip (iOS) or .apk (Android), manage build status (Backlog / In Progress / Done / Rejected), REST API + Personal Access Tokens for CI/CD integration
Session recording — record and share QA sessions, kept for ~72 hours before automatic cleanup
Team management — invite links, role-based access (Admin / Developer / QA / Viewer)
Mac resource monitoring — CPU and RAM time-series charts per agent

Honest limitations

iOS simulators require macOS — Apple's constraint, not ours
One Mac typically handles 2–4 simultaneous simulators depending on RAM; connect multiple Macs to pool devices
Still v0.x — breaking changes may appear before v1.0

Try it

tapflow is MIT licensed.

npm install -g tapflow
tapflow start
tapflow init  # create the first admin account

For team deployments with a shared relay:

# Relay server (Linux/macOS)
JWT_SECRET=$(openssl rand -hex 32) tapflow relay start

# Each Mac agent
tapflow agent start --relay wss://your-relay-url

🔗 GitHub: https://github.com/jo-duchan/tapflow
📖 Docs: https://www.tapflow.dev

DEV Community: Duchan

Most of a test phone's carbon is emitted before the first test runs

Most of a device's carbon comes from manufacturing

The arithmetic

Electricity actually goes the other way

I tried changing the weak assumptions

Where this stops being true

What I'm deliberately not claiming

Measure it on your own host

A UI test that guesses is worse than one that fails

The flow file

Ambiguity is a bug in the selector

There is no sleep step

Was that a test failure, or a dead runner?

Where a model is allowed to guess

What this is really for

Limits worth knowing

Try it

Coordinate-based UI tests break. So we read the accessibility tree instead — from inside the simulator.

The constraint: no WebDriverAgent, and no simulator window

A resident XCUITest runner, living inside the simulator

Android was the easy one

One schema, and why frame normalization matters

What this enables

Limits worth knowing

Try it

Your React Native build is done. Now your whole team can test it in the browser.

Why this fits React Native so naturally

Works with Expo — and bare React Native

Metro stays. This is for the build, not the dev loop

Why self-hosted (and why we didn't use a cloud lab)

What React Native teams actually reach for: deep links

Where it's headed

Getting started

The DX we wanted for tapflow setup: a host-ready Mac in one command

doctor diagnoses, setup installs and configures

iOS: installed ≠ usable

Android: a self-contained SDK

setup prepares; the relay boots

One rail under all of it

Honest limitations

Takeaway

Try it

We switched simulator streaming to H.264 and it felt worse. Here's how we fixed the latency.

The bar: localhost JPEG

Fix 1: the decoder was buffering 8 frames for no reason

Fix 2: MSE is a buffer you can't turn off

One more thing: dropping H.264 isn't like dropping JPEG

Honest limitations

Takeaway

Try it

Giving an LLM Eyes and Hands on a Mobile Simulator

The interface a human uses

What the MCP server does

How the tools are implemented

Screenshot — the model's eyes

Tap and swipe — normalized coordinates

Async operations over WebSocket

What a session looks like

Where we are: experimental

Where we're going: CI/CD without a QA script

Try the MCP server (experimental)

tapflow v0.3.x: Deeplinks, Keyboard Shortcuts, Screenshot API, and an Experimental MCP Server

Deeplink execution from the browser

Keyboard shortcuts for simulator controls

Screenshot REST endpoint

PAT scope enforcement

Frame performance instrumentation

Experimental: an MCP server

Try it

Your whole team can now run mobile QA from the browser. Here's how we built it.

Why we didn't just use Appetize or BrowserStack

Architecture

iOS touch — without WebDriverAgent

iOS streaming — IOSurface

Android — scrcpy H.264 → WebGL

Pinch gestures

What's included

Honest limitations

Try it

`doctor` diagnoses, `setup` installs and configures