Keep the Model Key Off the Phone: A First-Hour Proxy Test for Mobile AI

A junior engineer joins your mobile repo on a Tuesday, and the first ticket looks harmless: point the new summary screen at a hosted model and ship it before Friday. They do the obvious thing, which is to read the key from an environment file, call the endpoint, and watch the text appear on the emulator. It works, the demo lands, and nobody notices that the credential now lives inside a release artifact that anyone can download and unzip.

This post is the first-hour version of that story that ends differently. You will verify the leak from the built artifact, move the call behind a thin server-side proxy, and then run four device tests that decide whether the proxy is real or decorative. Everything below is a proposed harness you can reproduce on your own device; I am not reporting benchmark results, and the code is deliberately thin so you can read it in one sitting.

Why the first hour, not the first sprint

A key that reaches an app bundle has a blast radius measured in app-store review latency, not in minutes. Once the artifact is public you cannot un-publish it, you can only rotate the credential and wait for users to update. That asymmetry is why the cheapest moment to fix the architecture is the first hour, before the screen has a design review and a product manager attached to it.

There is a second reason that is specific to mobile AI work. On a phone, the "model" is usually three things pretending to be one: a packaged on-device model, a remote endpoint, and a cached fallback string. If the remote path holds the credential, the fallback ladder becomes a security decision as well as a reliability decision, and junior engineers inherit it without being told.

Step 1: Prove the credential is reachable before you refactor

Do not start with the fix. Start with evidence, because a reproducible failure is what makes the pull request reviewable. Build a release artifact the way CI does, then inspect what actually shipped.

# Android: the dex files are inside the APK, so extract and scan
unzip -p app-release.apk 'classes*.dex' | strings | grep -Ei 'sk-|api[_-]?key|bearer '

# iOS: the binary inside the IPA is plain Mach-O
unzip -o MyApp.ipa -d /tmp/ipa
strings /tmp/ipa/Payload/App.app/App | grep -Ei 'sk-|bearer '

# Repo side: catch build-time injection points, not just literals
git grep -nE 'UPSTREAM_KEY|API_KEY|Authorization' -- ':!*.md' ':!*.lock'

Expected observation is a hit in the artifact scan and a definition in the repo scan. Record the exact command, the artifact hash, and the tool versions in the PR description, because a reviewer will ask whether you scanned a debug build by accident. If your scan comes back empty, do not celebrate yet: check whether the key arrives through a generated BuildConfig, a plist template, or a bundler plugin that inlines process.env at build time.

Step 2: Put one thin proxy between the phone and the model

You do not need a platform team for this. You need one process that holds the upstream credential, authenticates the install, and refuses to be a general-purpose relay. Here is a dependency-free Node 20 sketch that you can run on any small host, including the free server option that some hosted coding tools provide for staging endpoints.

// server/proxy.mjs — Node 20+, zero dependencies, put TLS in front of it
import { createServer } from "node:http";
import { randomUUID } from "node:crypto";

const UPSTREAM = process.env.UPSTREAM_BASE_URL; // full URL, from provider docs
const UPSTREAM_KEY = process.env.UPSTREAM_KEY;   // never leaves this process
const MAX_REQ_PER_INSTALL = Number(process.env.MAX_REQ_PER_INSTALL ?? 200);

const installs = new Map(); // installId -> { issuedAt, used }

const server = createServer(async (req, res) => {
  const send = (code, obj) => {
    res.writeHead(code, { "content-type": "application/json" });
    res.end(JSON.stringify(obj));
  };

  if (req.method === "GET" && req.url === "/healthz") return send(200, { ok: true });

  if (req.method === "POST" && req.url === "/v1/install") {
    const id = randomUUID();
    installs.set(id, { issuedAt: Date.now(), used: 0 });
    return send(201, { installId: id });
  }

  if (req.method !== "POST" || req.url !== "/v1/summarize") {
    return send(404, { error: "not_found" });
  }

  const rec = installs.get(req.headers["x-install-id"]);
  if (!rec) return send(401, { error: "unknown_install" });
  if (rec.used >= MAX_REQ_PER_INSTALL) {
    return send(429, { error: "install_budget_exhausted" });
  }

  let raw = "";
  for await (const chunk of req) raw += chunk;
  const body = JSON.parse(raw || "{}");
  rec.used += 1;

  try {
    const upstream = await fetch(UPSTREAM, {
      method: "POST",
      headers: {
        "content-type": "application/json",
        authorization: `Bearer ${UPSTREAM_KEY}`,
      },
      body: JSON.stringify({ input: String(body.text ?? "").slice(0, 4000) }),
      signal: AbortSignal.timeout(20_000),
    });
    if (!upstream.ok) return send(502, { error: "upstream_error", status: upstream.status });
    const data = await upstream.json();
    return send(200, { text: data.text ?? null, used: rec.used });
  } catch {
    return send(504, { error: "upstream_timeout" });
  }
});

server.listen(Number(process.env.PORT ?? 8080));

Three details matter more than the code. The request body is truncated and shaped on the server, so the phone cannot smuggle arbitrary prompts into your bill. Every response carries used, which gives the client a reason to show a real quota message instead of a spinner. And the ledger is in memory, which means it resets on restart; that is fine for a first-hour experiment and unacceptable for production, so plan the persistence before you ship.

If you want to stand this up without provisioning anything first, MonkeyCode's operator advertises free model access (with a token allowance they quote on the project page) and a free server option, which is enough to run the proxy above as a staging endpoint. Treat both as operator-supplied availability claims rather than as guarantees: check the current allowance and terms yourself, and do not hard-code a number from any blog post, including this one.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Step 3: Rotate a key and watch old builds fail

This is the test I would put in the first PR, because it converts a security claim into an observable behavior. Rotate UPSTREAM_KEY on the server, restart the proxy, and then run a build you produced before the rotation against it.

1. Confirm the server returns 502 upstream_error with the old key and 200 with the new one.
2. Confirm the pre-rotation app build still gets a response, which proves the phone never held the credential.
3. Confirm an unknown x-install-id receives 401 unknown_install rather than a model response.
4. Confirm a fresh install receives a new installId after app data is cleared, and that the exhausted ledger returns 429.

Write the observed status codes into the PR, not just a checkmark. A reviewer signing off on a mobile AI change wants to see which surface answered, under which condition, and what the client rendered when it did not.

Step 4: Four lifecycle tests that break naive clients

A proxy on a server is only half the problem; the other half is what the phone does when the network is not cooperating. These four tests are cheap, and each one has failed for me in real projects.

• Backgrounding mid-request. Send the request, background the app for thirty seconds, return, and record whether the response is rendered, dropped, or duplicated by a retry.
• Network flip. Start on Wi-Fi, switch to cellular during the request, and check that the client surfaces a single terminal state rather than both a timeout and a success.
• Airplane mode after submit. The user submits text and loses the network; the honest outcome is a queued item with a visible status, not a silent disappearance.
• Forced upstream failure. Flip a FORCE_FAIL=1 environment variable and confirm the client falls back to on-device output or a cached summary, with copy that tells the user what happened.

For each test, log the device model, OS version, framework and dependency versions, network state, permission state, and the recovery outcome. Without that record, the next engineer cannot tell whether a fix worked or whether the device changed.

The decision table I use before choosing a path

Option	Where the credential lives	Works offline	Radio cost per user action	Pick it when
On-device model only	Nowhere	Yes	None	The task is short, the model fits, and privacy is the product
Direct call from the app	In the artifact	No	One round trip	Never for a shipped build
Thin proxy (this post)	Server process	No, but gives clean fallback	One round trip plus a small hop	You need a hosted model and a user-visible quota
Proxy plus own server	Server process	No	Same, plus your own uptime	You need logs, per-install limits, and rotation

What this does not tell you, and who should skip it

I have not measured latency added by the proxy hop, token consumption, or battery impact on any specific handset, and you should not infer those numbers from this article. The ledger counts requests, not tokens, and an in-memory Map will happily forget everything on a restart. Hosting terms, allowances, and rate limits change, so verify them at the source before you put a capacity plan in front of a stakeholder.

Skip this approach if your feature must work entirely offline, if your team has no capacity to run and monitor even one process, or if you cannot rotate a credential without a release cycle. In those cases a packaged on-device model plus a cached fallback is the more honest architecture, and the proxy becomes a later optimization rather than a foundation.

If you run this harness, I want the boring details: device, OS version, the lifecycle transition you tested, the status codes you saw, and whether the client recovered, restarted, or silently dropped the request. Those answers are worth more than another architecture diagram, and they are the ones a junior engineer can produce on their first day.

Comments

[Alex Shev]

Keeping the provider key off the device is only the first boundary. The proxy should also authenticate the user, constrain model and token budgets per feature, log request lineage without storing sensitive prompts, and revoke abuse quickly. That makes the server-side key an enforceable policy point rather than just a hidden copy of the secret.