Your first week on a mobile AI repo usually starts with a task that sounds small: point the app at a different inference backend so you can test without touching production credentials. On a mid-range Android phone in a debug build, with the screen off for thirty seconds and Wi-Fi dropping to LTE mid-request, that "small" task is where juniors either learn the repo or quietly break it. The drill below turns that task into a repeatable swap with evidence, and it fits the first hour, first PR, first rollback shape of onboarding rather than a production migration.
Everything here is a proposed walkthrough, not a benchmark report. The numbers in the evidence table are yours to fill from your own device, and the code blocks are starting points you should adapt to your framework's actual versions.
Why the upstream is the right first task
An inference upstream is the seam between your app and every model you will ever call, so it attracts every bad habit at once. Juniors hardcode URLs in screens, paste keys into the bundle, and assume a request either succeeds or throws a clean error. A swap drill exposes all three before you write anything that ships.
- It is reversible. One flag flips the upstream back, which means a mistake costs minutes, not a release.
- It is measurable. You can timestamp request start, first byte, completion, and UI paint without touching model code.
- It is mobile-specific. Backgrounding, network changes, and permission loss all change the outcome, and none of them appear in a desktop test.
Record the environment before you start, because a lifecycle result without versions is folklore. Note the device model, OS build, framework version, and the exact lockfile entries for any HTTP or AI SDK you touch.
Step 1: Define the adapter before you touch the network
Start by writing the interface your screens will call. The rule for the first PR is that no screen imports a vendor SDK directly, and the compiler enforces it.
// src/ai/upstream.ts
export type InferenceRequest = { prompt: string; maxTokens: number };
export type FailureReason = 'timeout' | 'network' | 'server' | 'cancelled';
export type InferenceResult =
| { ok: true; text: string; upstream: string; elapsedMs: number }
| { ok: false; reason: FailureReason; upstream: string; elapsedMs: number };
export interface InferenceUpstream {
readonly name: string;
infer(req: InferenceRequest, signal: AbortSignal): Promise<InferenceResult>;
}
A discriminated result type is the part juniors skip and later regret, because a thrown error loses the reason at the call site. With ok: false you can route timeout, offline, and server failure to different UI without string matching. Keep upstream in the payload so every log line names the backend that produced it.
The remote adapter with a hard deadline
// src/ai/remoteUpstream.ts
import type { InferenceUpstream, InferenceRequest } from './upstream';
const TIMEOUT_MS = 8000;
export function createRemoteUpstream(baseUrl: string, name: string): InferenceUpstream {
return {
name,
async infer(req, outerSignal) {
const started = Date.now();
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), TIMEOUT_MS);
const onAbort = () => controller.abort();
outerSignal.addEventListener('abort', onAbort);
try {
const res = await fetch(`${baseUrl}/v1/infer`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(req),
signal: controller.signal,
});
const elapsedMs = Date.now() - started;
if (!res.ok) return { ok: false, reason: 'server', upstream: name, elapsedMs };
const data = (await res.json()) as { text?: string };
if (typeof data.text !== 'string') {
return { ok: false, reason: 'server', upstream: name, elapsedMs };
}
return { ok: true, text: data.text, upstream: name, elapsedMs };
} catch (e) {
const elapsedMs = Date.now() - started;
const reason = outerSignal.aborted ? 'cancelled' : controller.signal.aborted ? 'timeout' : 'network';
return { ok: false, reason, upstream: name, elapsedMs };
} finally {
clearTimeout(timer);
outerSignal.removeEventListener('abort', onAbort);
}
},
};
}
Two details matter more than the code style. First, the timer is cleared in finally, because a leaked timer keeps a wake lock alive and drains battery after the screen is off. Second, Date.now() is not monotonic, so if your runtime exposes a monotonic clock, use it for the deltas and keep wall time only for log correlation.
Step 2: Put the key on a server, not in the bundle
Mobile clients cannot keep a secret, and any build-time variable prefixed for bundler inlining ends up readable in the shipped app. So the app talks to a thin proxy, and the proxy holds the upstream credential.
// config.ts — URLs are fine to inline; secrets are not
const BASE = process.env.EXPO_PUBLIC_AI_BASE_URL ?? 'http://10.0.2.2:8787';
const FLAG = process.env.EXPO_PUBLIC_AI_UPSTREAM ?? 'remote-free';
The free-server option is what makes this drill practical for a new hire, since it gives you a disposable host for that proxy instead of a staging cluster that requires three approvals. MonkeyCode's free model access and free server option are the availability claims the operator supplied for this workflow. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
A minimal proxy sketch, unexecuted here and deliberately boring:
// proxy/server.js — sketch, not production code
import express from 'express';
const app = express();
app.use(express.json());
app.post('/v1/infer', async (req, res) => {
const { prompt, maxTokens } = req.body ?? {};
if (typeof prompt !== 'string' || !prompt.length) return res.status(400).json({ error: 'bad_request' });
try {
const upstream = await fetch(process.env.MODEL_ENDPOINT, {
method: 'POST',
headers: { 'content-type': 'application/json', authorization: `Bearer ${process.env.MODEL_KEY}` },
body: JSON.stringify({ prompt, max_tokens: Math.min(maxTokens ?? 256, 512) }),
});
if (!upstream.ok) return res.status(502).json({ error: 'upstream_failed' });
const data = await upstream.json();
return res.json({ text: data.text ?? '' });
} catch {
return res.status(504).json({ error: 'upstream_timeout' });
}
});
app.get('/healthz', (_req, res) => res.json({ ok: true }));
app.listen(process.env.PORT ?? 8787);
Keep the proxy stateless for this exercise, log only timings and status codes, and never log prompts containing real user data. That single rule prevents the most common privacy incident in a first PR.
Step 3: Run the lifecycle matrix, not one happy path
Before you claim the swap works, run each row below on one physical device and write the observed result next to the expectation. Airplane mode and the app switcher are your two most valuable test tools in week one.
| # | Transition | Expected observation | Recorded |
|---|---|---|---|
| 1 | Cold start, Wi-Fi, tap once |
ok: true, upstream matches flag |
|
| 2 | Background 30s mid-request, return | Request either completes or reports cancelled, never both |
|
| 3 | Wi-Fi to LTE during request | One terminal result, reason is network or success |
|
| 4 | Airplane mode on, tap |
ok: false, reason: network, UI offers retry |
|
| 5 | Proxy stopped, tap |
ok: false, reason: server, no crash |
|
| 6 | Flag flipped to fallback, tap |
upstream changes, old path never called |
Row 2 is the one that catches real bugs, because backgrounding can suspend timers and leave an AbortController that fires long after the user returned. If you see a success and a timeout for the same tap, you have two owners for one request and need to fix that before the PR review.
Capture the timings you can defend
Log four timestamps per request: tap, request dispatch, response received, and first frame after state update. Export the log with device, OS build, framework version, and network generation, then paste the raw numbers into the PR description instead of an average. Averages hide the tail, and the tail is what users on a train feel.
Step 4: Rehearse the swap and the revert
The rollback here is configuration, so practice it in this order and keep the steps in the PR body.
- Flip the flag to the fallback upstream and confirm the old backend receives no traffic in its logs.
- Restore the flag and confirm the fallback stops being called, with the same install and no reinstall.
- Rotate the proxy credential once, to prove the app never depended on an embedded key.
- Delete the proxy instance, rebuild, and record the user-visible failure message as your limitation note.
If step 1 cannot be verified from server logs, your evidence is a feeling rather than a measurement. Write that down too.
Limitations and who should skip this
This drill proves your app handles upstream substitution and failure on one device under a few transitions. It does not prove throughput, cost, model quality, or behavior across device classes, and a free shared endpoint is not a benchmark environment. Do not send real transcripts, health data, or anything user-identifying through a proxy you stood up for an exercise, and do not treat this setup as a production path with an availability promise.
Skip this approach if your feature requires a contractual latency or uptime target, if your data cannot leave the device at all, or if your team already has a staging inference service with real credentials. In those cases, the adapter and the lifecycle table still apply, but the disposable upstream does not.
If you are onboarding someone this month, hand them the adapter contract and the six-row matrix instead of a wiki page. If you want a throwaway upstream for that first swap, MonkeyCode's free model access and free server option are enough to stand one up before lunch.
What did your device actually do on row 2? Post the device, OS build, the transition steps, and whether the request recovered, restarted, or silently disappeared, because that comparison is the part no tutorial can give you.
Top comments (0)