Your First Mobile AI PR: Point Inference at a Dev Backend, Then Cut the Network

A new engineer joined our mobile AI repo on a Tuesday, and by 10:40 she had the transcript feature working on her desk Wi-Fi. At 11:05 she walked into the stairwell, the request never returned, and the UI sat on a spinner for ninety seconds before she force-quit the app. Nobody had told her that the remote inference path was the part of the system most likely to fail first, and nobody had given her a backend she was allowed to break.

That is the onboarding gap this post closes. Everything below is a test plan plus code you can run yourself, not a benchmark report: the measurement columns stay empty until you fill them in on your own device, OS, and framework versions.

Why the remote path is the wrong first debugging target

When a feature calls a hosted model, the first hour of debugging usually happens against production credentials and production latency. That is a bad classroom, because you cannot make the endpoint fail on purpose, and every retry costs somebody money or quota.

Worse, the failure modes you need to see are not the ones a laptop reproduces. Airplane mode, Doze, a revoked microphone permission, and a backgrounded app all change what the client does before the network is even involved.

So the first task for a junior engineer should be split in two: get a dev inference endpoint that is disposable, then deliberately break the network around it.

Step 1: Give the new engineer one dev endpoint they cannot break

The practical blocker on day one is procurement: an API key, a spending limit, and a staging slot can take longer than the PR itself. For this repo we route the dev build at MonkeyCode's free model access and free server option, which the operator describes as including a token allowance (currently stated as 10 million tokens) plus a hosted server, so a new engineer gets a working endpoint without a purchase order. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Free-tier terms and allowances change, so verify the current numbers on the project's own page before you write them into a README.

Two rules keep this honest and useful:

1. The dev endpoint is a development backend, never a production dependency; no release build may point at it.
2. Only synthetic or published sample audio goes to it. A free endpoint is the wrong place for real user recordings, even during onboarding.

The goal is not to compare providers. The goal is to let a new engineer send one hundred deliberately broken requests before lunch without asking anyone for permission.

Step 2: Pin the endpoint in exactly one place

Most onboarding failures I have seen are configuration failures, not model failures. If the base URL appears in three files, the junior will edit the wrong two.

// lib/inference_config.dart
class InferenceConfig {
  // Inject at build time so no credential or URL lives in source control.
  static const baseUrl = String.fromEnvironment(
    'INFERENCE_BASE_URL',
    defaultValue: 'http://10.0.2.2:8787', // emulator loopback to your laptop proxy
  );

  static const requestTimeout = Duration(seconds: 6);
}

Run the app with the endpoint injected, and keep the default pointing at your laptop so the first build already talks to the fault proxy below:

flutter run --dart-define=INFERENCE_BASE_URL=http://10.0.2.2:8787

Note that 10.0.2.2 is the Android emulator's alias for the host machine. On a physical device you need the laptop's LAN address instead, and the phone and laptop must be on the same network.

Step 3: The artifact — a small fault proxy you run on your laptop

This Node script (Node 18 or newer, because it uses global fetch) sits between the app and the real dev endpoint. It can pass traffic through, inject a 500, add latency, or never answer at all so you can exercise the client timeout.

// tools/fault-proxy.mjs
// MODE=ok|500|delay|hang UPSTREAM=https://your-dev-endpoint node tools/fault-proxy.mjs
import http from 'node:http';

const UPSTREAM = process.env.UPSTREAM;
const MODE = process.env.MODE ?? 'ok';
const DELAY_MS = Number(process.env.DELAY_MS ?? 8000);
const PORT = Number(process.env.PORT ?? 8787);

if (!UPSTREAM) {
  console.error('UPSTREAM is required');
  process.exit(1);
}

http
  .createServer(async (req, res) => {
    if (MODE === 'hang') return; // never respond: exercise the client timeout
    if (MODE === '500') {
      res.writeHead(500, { 'content-type': 'application/json' });
      return res.end('{"error":"injected_failure"}');
    }
    if (MODE === 'delay') await new Promise((r) => setTimeout(r, DELAY_MS));

    const chunks = [];
    for await (const chunk of req) chunks.push(chunk);

    const upstream = await fetch(UPSTREAM + req.url, {
      method: req.method,
      headers: { ...req.headers, host: new URL(UPSTREAM).host },
      body: req.method === 'GET' ? undefined : Buffer.concat(chunks),
    });

    res.writeHead(upstream.status, Object.fromEntries(upstream.headers));
    res.end(Buffer.from(await upstream.arrayBuffer()));
  })
  .listen(PORT, () => console.log(`fault-proxy MODE=${MODE} on :${PORT}`));

Start it in one terminal, run the app in another, and switch MODE between sessions. If the app does something different for hang than for 500, you have already found the first review comment.

Step 4: The device matrix, with your numbers instead of mine

Fill this in for your own hardware; do not copy a row you have not run. Keep the evidence column literal, because "looked fine" is not a result.

State	How to enter it	Expected client behavior	Evidence to record
Airplane mode	adb shell cmd connectivity airplane-mode enable	Fallback to the on-device path or a clear offline state	outcome, elapsed ms
Injected timeout	proxy MODE=hang	Timeout at the configured deadline, then fallback	timeout value, fallback flag
Injected 500	proxy MODE=500	No infinite retry, no crash, user-visible state	retry count, error code
Background 30s	Home gesture, wait, return	No orphaned request, state restored	result after resume
Mic revoked	adb shell pm revoke <pkg> android.permission.RECORD_AUDIO	Feature degrades without a silent no-op	permission state in log
Doze	adb shell dumpsys deviceidle force-idle	Work is deferred or explicitly queued	queued vs dropped

On iOS, the simulator will not fake airplane mode reliably, so run the network rows on a physical device and use Network Link Conditioner for the latency rows. Record the device, OS build, framework version, and permission state next to every row, because a result without those fields cannot be reproduced next sprint.

Step 5: The fallback decision table your PR has to satisfy

Before the junior writes feature code, agree on what each outcome means. A short table in the PR description prevents a long argument in review.

• Success: remote transcript returned within the deadline; store it, do not re-upload.
• Timeout: fall back to on-device inference once, log outcome=timeout, and surface the source of the transcript.
• HTTP 4xx: do not retry; this is a request problem and retrying hides a client bug.
• HTTP 5xx: retry once with jitter, then queue or fall back, and never loop.
• Permission revoked: block the action with a clear message; do not silently return an empty transcript.

The client side that implements the timeout row is small and reviewable:

Future<Transcript> transcribe(File audio) async {
  try {
    return await remote.transcribe(audio, timeout: InferenceConfig.requestTimeout);
  } on TimeoutException {
    log('infer_fallback', {
      'reason': 'timeout',
      'endpoint': InferenceConfig.baseUrl,
      'elapsed_ms': stopwatch.elapsedMilliseconds,
    });
    return local.transcribe(audio); // on-device path
  }
}

Step 6: Log fields that make the review reviewable

A fallback that works but cannot be observed is not finished. Ask for these fields in the first PR, and reject logs that contain the transcript text itself.

1. request_id — correlates client and endpoint without shipping audio.
2. endpoint — which base URL answered, without credentials.
3. mode — pass-through, injected failure, or delay.
4. elapsed_ms — from send to final outcome.
5. outcome — success, timeout, error, or fallback_local.
6. permission_state and foreground_state — the two fields most often missing.

Limitations and who should not do this

This workflow assumes one developer, one device, and a disposable dev endpoint. It does not prove anything about production traffic, and it will not catch model-quality regressions, because a free dev endpoint and your production model may not be the same thing.

Skip this approach if your app handles regulated audio or health data, since even synthetic-looking payloads on a free backend can violate your own policy. Skip it as a staging replacement too; a free server is not an availability guarantee, and routing release builds through it is how a demo becomes an incident. If you need deterministic outputs for regression tests, record fixtures once and replay them offline instead.

Who it does suit is the engineer on day one: someone who needs a real request, a real device, and permission to break both. If that is your onboarding problem, MonkeyCode's free models and free server are one way to get an endpoint in the first hour instead of the first week, and you can verify the current offer yourself before pointing a build at it.

What I want back from you is comparable evidence, not agreement. Tell me your device, OS build, framework version, the lifecycle transition you forced, and whether the feature recovered, restarted, or silently disappeared.