Benchmark a Mobile Agent Across Airplane Mode and Your First Config Rollback

You sit down with a Pixel 8a on Android 15 and a signed staging build already on the home screen. A half-finished agent turn still paints the chat, with a tool-call spinner frozen after two seconds. You enable Airplane Mode before the HTTP response arrives, then lock the device for thirty seconds. That lifecycle jump is the first thing you should reproduce, because generated agent code usually assumes the radio never drops.

This is a proposed test, not a measured result from a private lab that you should copy blindly. You should run the steps on your own phone, then write down whether the turn recovered, restarted, or silently disappeared. Juniors joining a mobile AI repo in 2026 will meet agent scaffolding on day one, and that scaffolding is cheap to generate. The expensive part is proving the agent fails closed when cellular vanishes during an in-flight tool call.

What you prove in the first hour

You are not proving that the model sounds clever during a hallway demo for stakeholders. You are proving three boring properties that reviewers can re-run without a staging VPN. Write them on the ticket before you touch a prompt file or a tool mapping.

• The in-flight tool call has a generationId and a configHash, not just a spinner.
• Airplane Mode plus a lock screen cannot create a later surprise retry with a stale schema.
• A config rollback cannot resurrect queued work that was produced by the previous agent bundle.

If your repo already ships an on-device runtime, keep that path in the loop during hour one. The remote model is only an oracle for JSON shape, error codes, and fail-closed messaging. Do not treat the oracle as the shipping product for your first pull request.

Record the environment before you flip the radio

Capture versions and radio state before Airplane Mode, or your notes will be useless by next week. Use this list as the header of the pull request so reviewers can repeat the jump. Skip any row and the experiment stops being comparable across phones.

1. Device and OS: Pixel 8a on Android 15, or your iPhone and the exact iOS build.
2. Application state: signed in, chat visible, tool call in flight, screen on, then locked.
3. Framework pins: React Native, NetInfo, HTTP client, and the on-device or edge runtime.
4. Network and power: Wi-Fi versus LTE, Battery Saver off, no VPN, no private DNS surprises.
5. Permissions: location or microphone only if that tool truly needs them; otherwise leave them denied.
6. Expected observation: a user-visible paused state, no duplicate side effect, no missing transcript.

If you cannot name the runtime that executes the tool, you are not ready to change the prompt. Spend the rest of hour one grepping for AsyncStorage, WorkManager, URLSession, and any retry interceptor. Those four hits usually hide the silent duplicate that will show up after rollback.

Proposed experiment: one device, one transition

Follow these steps on a physical device rather than a simulator with fake radios. Emulators lie about radio loss and about process death after you lock the screen. Keep the chat on screen so the lifecycle starts from an in-flight tool call, not from a cold start.

1. Install the staging build and open a new chat with a disposable account.
2. Send a prompt that forces a network tool, such as a store lookup, not a local rewrite.
3. When the spinner appears, enable Airplane Mode within one second, then lock the phone.
4. Wait thirty seconds without peeking. Unlock and screenshot the transcript, banner, and logs.
5. Disable Airplane Mode, wait for the radio, and watch for a silent retry you did not request.
6. Force-stop the app, relaunch, and check whether a queued tool call fires on its own.
7. Record one outcome: recovered with the same generationId, restarted as a new turn, or disappeared.

You should expect a visible paused or failed-closed state, not a second store lookup after the radio returns. A disappeared spinner with a later duplicate side effect is a failing result, even when the prose looks fine. Attach logcat or Console.app lines that show the same generationId across the lock, because screenshots alone will not survive review.

The state machine your first PR should ship

Keep the first pull request small enough to roll back in one commit on the same afternoon. Model the turn as a state machine, then persist (generationId, configHash, state) before any HTTP write. If you cannot persist that tuple, you cannot prove rollback safety later.

Proposed states:

• idle — no work, nothing queued
• tool_pending — request on the wire
• queued_offline — only if the tool is idempotent and the user opted into retry
• cancelled — fail closed and show a pause chip
• rolled_back — config hash no longer matches, so you drop the queue

Default to cancelled when Airplane Mode fires during tool_pending. queued_offline is an explicit product choice, not a courtesy of your HTTP client. Juniors get this backward because interceptors retry by default and generated glue copies that habit.

Proposed React Native sketch

// Proposed example, not production code. Pin versions in the PR header.
import NetInfo from '@react-native-community/netinfo';
import { AppState } from 'react-native';
import AsyncStorage from '@react-native-async-storage/async-storage';

type AgentState =
  | 'idle'
  | 'tool_pending'
  | 'queued_offline'
  | 'cancelled'
  | 'rolled_back';

type TurnRecord = {
  generationId: string;
  configHash: string;
  state: AgentState;
  createdAt: number;
};

const TURN_KEY = 'agent.turn.v1';

export async function persistTurn(record: TurnRecord): Promise<void> {
  await AsyncStorage.setItem(TURN_KEY, JSON.stringify(record));
}

export async function failClosedIfStale(
  currentHash: string
): Promise<TurnRecord | null> {
  const raw = await AsyncStorage.getItem(TURN_KEY);
  if (!raw) return null;
  const turn = JSON.parse(raw) as TurnRecord;
  if (turn.configHash !== currentHash) {
    const rolled: TurnRecord = { ...turn, state: 'rolled_back' };
    await persistTurn(rolled);
    return rolled;
  }
  return turn;
}

export function watchRadio(onDrop: () => void): () => void {
  const unsubNet = NetInfo.addEventListener((state) => {
    if (!state.isConnected || state.isInternetReachable === false) {
      onDrop();
    }
  });
  const unsubApp = AppState.addEventListener('change', (next) => {
    if (next !== 'active') {
      onDrop();
    }
  });
  return () => {
    unsubNet();
    unsubApp.remove();
  };
}

Wire onDrop so it moves tool_pending to cancelled and paints a pause chip the user can read. Do not call fetch again from the NetInfo listener when the radio returns a few seconds later. That listener is how silent duplicates are born, and reviewers should reject any PR that retries from connectivity callbacks.

Contract-test the tool JSON before production credentials exist

Your first hour should also freeze the tool schema beside the client, not inside a chat screenshot. Save one golden request and one golden response, then assert names, required fields, and error shape. Leave prose quality out of this test, because wording will drift while the contract must not.

A practical comparison oracle without production credentials is MonkeyCode's free model access on its free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Point the oracle at the same JSON schema your on-device or edge path must emit, and keep user transcripts off that box. Stand the oracle up for hour one, then delete the instance once the contract test stays green.

// Proposed Jest contract. Replace ORACLE_URL with your comparison endpoint.
type ToolCall = { name: string; args: Record<string, unknown> };

export function assertToolShape(call: ToolCall): void {
  const allowed = new Set(['lookupStore', 'cancelTurn']);
  if (!allowed.has(call.name)) {
    throw new Error(`unexpected tool ${call.name}`);
  }
  if (call.name === 'lookupStore' && typeof call.args.query !== 'string') {
    throw new Error('lookupStore.query must be a string');
  }
}

test('oracle tool call matches the mobile contract', async () => {
  const res = await fetch(process.env.ORACLE_URL!, {
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify({
      prompt: 'find a store',
      schemaVersion: 'store-lookup-v1',
    }),
  });
  const body = await res.json();
  assertToolShape(body.toolCall);
});

If the oracle and the phone disagree on tool names, fail the PR before anyone debates latency. Schema drift is cheaper to catch in Jest than in a store listing that double-books a reservation. Hash the golden files in CI so a prompt edit cannot land without a contract bump.

First rollback without resurrecting queued work

Rollback is the third hour-one drill, and most teams skip it until a bad prompt ships. You revert the agent config or the prompt bundle, reinstall the previous binary, then launch into the same chat. The question is not whether the old copy builds. The question is whether yesterday's queue still fires.

# Proposed checks after you reinstall the rolled-back build.
adb shell dumpsys package com.example.staging | grep versionName
adb exec-out run-as com.example.staging cat files/agent.turn.v1
adb logcat -d | grep generationId

On iOS, dump the same record from the app container and look for configHash after the restore. If the hash from the newer config is still sitting in storage, the next foreground will try a tool the old binary does not understand. Your PR should treat hash mismatch as rolled_back and show a one-line system message, not a reconstructed answer that pretends the turn finished.

Decision table for the review

Condition	Tool in flight	Stored hash	Required next state	User-visible result
Airplane Mode during tool_pending	yes	matches	cancelled	pause chip, no retry
Radio returns, user did not ask	no	matches	idle	no new tool call
Config rollback, queue present	maybe	mismatch	rolled_back	drop queue, system line
Idempotent tool, user opted in	yes	matches	queued_offline	explicit resume affordance

If a cell is missing from your ticket, the reviewer should bounce the PR without debating model quality. Generated agent glue loves the empty cell called just retry, and that cell is how a junior's first rollback creates a duplicate side effect. Fill the table in the description, then paste the screenshots that match each row you claim to handle.

Native and Flutter notes for the same tuple

The same tuple belongs in DataStore on Android, or in a small Room row if you already have one. On iOS, put generationId and configHash in the application support directory, not in a defaults file that backup may scoop up. Flutter teams can store the record with shared_preferences only if you remember that plugin is not a queue and not a lock.

WorkManager and BGTaskScheduler are for opted-in idempotent tools, not for resurrecting cancelled agent turns after Airplane Mode. If a native retry worker already exists, your first PR should make it read state before it touches the network. Otherwise the cross-platform client will fail closed while the platform worker quietly completes the stale call.

Limitations, and who should not use this

This article is a proposed onboarding workflow, not a claim about any phone's battery curve or token throughput. The write-up does not attach lab numbers, and you should not copy anyone else's milliseconds into your review. NetInfo isInternetReachable can lag behind the modem, and iOS will freeze listeners in ways Android will not.

Do not send production traffic, paid user chats, precise location, audio, or account tokens to a comparison endpoint. Teams with a locked staging mesh already have an oracle, so do not add a second one for novelty. Voice interruption, Doze completions, and backup of prompt caches are separate drills, and this workflow does not replace them.

Skip fail-closed cancellation if your product legally must complete a payment tool once the user confirms. In that case you need an idempotency key on the server of record, which is a different review with different failure language. Juniors should still persist generationId, but the next state is queued_offline with an explicit resume, never a hidden interceptor retry.

Ask your device, then tell us what happened

Run the Airplane Mode lock on your own build this week and keep the notes next to the PR. Comment with the device, OS, framework pins, and the exact transition you used. Say whether the turn recovered, restarted, or silently disappeared after rollback, because that single word is the evidence the next junior needs.