Give the Fallback a Deadline: A Junior's First Mobile AI PR for Remote-to-On-Device Handoff

The scenario (illustrative, not a measured run)

You join a mobile AI repo on your first week, and the feature you are handed already has two inference paths in it. The app calls a remote model for long answers and a small on-device model when the device is offline, and nobody on the team can tell you which path a given tap actually took. The example below uses placeholder versions on purpose: substitute your own device, OS build, framework, and SDK versions before you write anything down.

• Device: one physical phone you own, not an emulator (emulators lie about network and backgrounding).
• OS: whatever build your repo's README pins; record the exact build string.
• Framework: React Native or Flutter, with the version your lockfile resolves to.
• Network: Wi-Fi you control, plus a way to disable it without touching the app.

The problem worth your first PR is not "add a model." It is that the remote path has no deadline, so a cold server, a captive portal, or a dropped radio can leave the user staring at a spinner while a perfectly usable on-device model sits idle. Your job is to give the handoff a budget, make it observable, and prove it degrades instead of hanging.

Step 1: Write the budget before you write the code

A fallback without a number is a hope. Decide, in one table, how long the remote path may hold the user before the app switches engines. Keep the values small enough that a user notices an answer rather than an outage, and write down that they are placeholders you will tune on a real device.

Condition	On-device model ready?	Remote reachable?	Engine to use	Expected user-visible behavior
Steady Wi-Fi	yes	yes, answers under budget	remote	normal answer, no badge
Radio off / airplane mode	yes	no	on-device	answer appears after local latency, "on-device" badge
Captive portal or stalled TCP	yes	connection opens, never answers	on-device after deadline	one short delay, then local answer
Cold remote start	yes	yes, slow	on-device after deadline	same as above, reason logged
On-device model not downloaded	no	no	none	explicit "unavailable" state, not a spinner

The last row is the one juniors forget. If both engines are unavailable, "fallback" is not a fallback, and the UI must say so in words rather than leave a disabled button.

Step 2: An illustrative deadline wrapper

The code below is illustrative and was not executed for this article; treat it as a shape to adapt, not a tested library. Its single idea is that every remote call is raced against an abort timer, and every failure path converges on the on-device call.

// Illustrative only — not executed or tested. Rename to fit your project.
export type Engine = "remote" | "on-device";

export type InferenceResult = {
  engine: Engine;
  text: string;
  elapsedMs: number;
  degraded: boolean;
  reason?: string; // why we left the remote path, for your logs
};

const REMOTE_DEADLINE_MS = 1500; // placeholder: set from your own budget table
const TOTAL_DEADLINE_MS = 2500;  // placeholder: what the user will tolerate

export async function inferWithDeadline(
  prompt: string,
  deps: {
    runRemote: (prompt: string, signal: AbortSignal) => Promise<string>;
    runOnDevice: (prompt: string) => Promise<string>;
    now?: () => number;
  },
): Promise<InferenceResult> {
  const now = deps.now ?? (() => Date.now());
  const startedAt = now();
  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), REMOTE_DEADLINE_MS);

  try {
    const text = await deps.runRemote(prompt, controller.signal);
    return { engine: "remote", text, elapsedMs: now() - startedAt, degraded: false };
  } catch (err) {
    // Offline, abort, DNS failure, 5xx and cold-start timeouts all land here.
    const reason = err instanceof Error ? err.name : "unknown";
    const text = await deps.runOnDevice(prompt);
    const elapsedMs = now() - startedAt;
    return {
      engine: "on-device",
      text,
      elapsedMs,
      degraded: elapsedMs > TOTAL_DEADLINE_MS,
      reason,
    };
  } finally {
    clearTimeout(timer);
  }
}

Three details matter more than the code. First, runRemote must actually honor the AbortSignal, or the timer buys you nothing and the socket keeps burning battery. Second, reason belongs in your logs and your telemetry, because "why did we fall back" is the question your first on-call rotation will ask. Third, the wrapper has an honest gap: it does not cancel the on-device call, so the total budget is advisory rather than enforced — say that in the PR description instead of letting a reviewer discover it.

Where a free endpoint fits on day one

You need something to point runRemote at while you build this, and standing up inference infrastructure is not a good first-week task. This is the narrow place where MonkeyCode's free model access and free server option are genuinely useful: a staging endpoint for your own device tests, and a nightly contract check in CI rather than a production dependency.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The operator describes both a free model tier and a free server option, and I am deliberately not stating model names, quotas, hardware, or uptime here because those change and I have not verified them. Read the project's own documentation for current limits before you put a number in a README, and keep real user prompts out of any shared free endpoint.

Step 3: A four-condition test plan you can run in an hour

Run these in order on one physical device, and write down what you observed rather than what you expected. Each condition targets a different failure mode, so a single pass does not let you claim the fallback is safe.

1. Airplane mode on, model already downloaded. Expect an on-device answer and zero remote attempts in the log. Fail if you see retry storms or a spinner.
2. Wi-Fi on, endpoint pointed at a black-hole host. Expect the answer after roughly your remote deadline, with reason naming an abort or network error. Fail if the delay exceeds the deadline by more than your on-device latency.
3. Network flips mid-request. Start on Wi-Fi, disable it two seconds in, and watch whether the promise rejects, resolves late, or hangs. Record which one; this is the case most teams have never tested.
4. Background the app during the remote call. Bring it back and check whether the result still arrives, is discarded, or double-fires the on-device model. A double answer is worse than a slow one.

For each condition, record device, OS build, framework version, permission and network state, the deadline you configured, the observed engine, and whether the UI recovered, restarted, or silently did nothing. That last column is what turns a demo into evidence.

Step 4: The review checklist for your first PR

Keep the diff small enough that a reviewer can hold it in their head, and put the reasoning in the description where it will outlive the branch.

• The budget table is in the PR description, with placeholders clearly marked as unverified.
• Both engines are logged with an engine tag, so a later provenance badge costs nothing.
• The "both unavailable" path has explicit copy, not a disabled control.
• Abort is wired through the transport layer, not just the wrapper.
• No free-tier endpoint is referenced in production configuration or committed secrets.

Limitations and who should skip this

This approach buys graceful degradation, not correctness, and it cannot tell you whether the remote answer was better than the local one. If your product has a hard latency SLA, regulated data that must not leave the device, or a remote path that must be consistent across users, a fallback budget is necessary but nowhere near sufficient. Do not adopt a free shared endpoint for production traffic, and do not cite this article's placeholder numbers as measurements — they are starting points you replace with your own device data.

If you run the four conditions above, I would like the comparable evidence: device, OS build, the transition you performed, and whether the result recovered, restarted, or disappeared. That is a more useful comment than agreement.

If you want to try the deadline wrapper before you own any infrastructure, the project's free model access and free server option will get you a staging endpoint in an afternoon; confirm today's limits on its documentation page first.