Your Fallback Copy Is a Guess Until You Shadow-Test It Against Free Models

You work on a habit-tracker app. A user denies the notification permission. Your app shows one line of explanation. That line is a static string today. A model could write a better sentence. But every API call costs money on the phone. So you built a fallback path. Your fallback triggers when the network fails. Your tests pass on Wi-Fi. Your simulator shows the fallback. The reviewer approves. Real users see your guess.

The fallback assumption that quietly breaks

Your fallback only triggers on network errors. Most mobile AI clients are written that way. Real mobile users fail differently. The OS suspends your app mid-request. The user revokes a permission while the prompt is in flight. The endpoint returns 429 when quota runs out. The phone goes offline between retry and response.

None of this appears in a happy-path demo. All of it appears in production support tickets. Shadow testing is how you find it before users do.

What a shadow test is

Shadow mode runs a model in parallel with real product flows. It never renders the model output to the user. The product path still shows your deterministic fallback. The shadow path records what the model would have said, how long it took, and how many tokens it consumed. You compare those records later in a log file. The fallback keeps running until the model earns trust.

Free models and a free server from MonkeyCode support this exact pattern: one endpoint for the model output, one place for the logs. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Use the trial allowance as a staging budget, not a production SLA. Keep your production client pointed at your own endpoint. The shadow runner alone points at the free endpoint.

A shadow runner you can copy

This script sends one real prompt through an OpenAI-compatible endpoint, records the measurements, and compares the output against your fallback string. It captures the exact conditions your fallback logic cares about: HTTP status, latency, token usage, and output length. No model output is written back to the app.

// shadow-runner.mjs — one prompt, one free model, one trace
// Usage:
//   SHADOW_BASE_URL=https://... \
//   SHADOW_API_KEY=test-key \
//   SHADOW_MODEL=free-model \
//   node shadow-runner.mjs ./prompt.txt
import { readFile } from "node:fs/promises";

const prompt = await readFile(process.argv[2] ?? "./prompt.txt", "utf8");
const fallback = "We need notifications to remind you about your streaks.";

const t0 = performance.now();
const res = await fetch(`${process.env.SHADOW_BASE_URL}/chat/completions`, {
  method: "POST",
  headers: {
    "content-type": "application/json",
    authorization: `Bearer ${process.env.SHADOW_API_KEY}`,
  },
  body: JSON.stringify({
    model: process.env.SHADOW_MODEL,
    messages: [{ role: "user", content: prompt }],
    temperature: 0.3,
    max_tokens: 200,
  }),
  signal: AbortSignal.timeout(12_000),
});

const latencyMs = Math.round(performance.now() - t0);
const data = await res.json();
const modelText = data.choices?.[0]?.message?.content ?? "";

const fallbackWins =
  res.status >= 400 || modelText.trim().length < 10;

console.log(JSON.stringify({
  httpStatus: res.status,
  latencyMs,
  promptTokens: data.usage?.prompt_tokens ?? null,
  completionTokens: data.usage?.completion_tokens ?? null,
  modelText: modelText.slice(0, 280),
  fallbackWins,
}, null, 2));

Node 18 or newer is enough. Export the three environment variables from a local shell, never from the app binary. The prompt file should contain only synthetic or redacted copy during experiments.

A sample trace looks like this. It shows the shape of the record, not a benchmark result.

{
  "httpStatus": 200,
  "latencyMs": 1841,
  "promptTokens": 86,
  "completionTokens": 17,
  "modelText": "We need notifications to remind you about your streaks. Want to keep them?",
  "fallbackWins": false
}

fallbackWins is a length heuristic, not a quality judgment. A long hallucination can still win under this rule. Use the trace to decide which scenarios deserve a human review pass.

The failure drill for your fallback

The runner alone proves the endpoint works. It does not prove the lifecycle survives. Run the same request through these six scenarios on a real device and log each result.

1. Wi-Fi baseline. Run the runner three times and record the latency spread. This becomes your timeout budget reference.
2. Airplane mode mid-flight. Start the request, toggle airplane mode before it settles. Check whether the fallback renders or the spinner hangs forever.
3. Background kill. Start the request, background the app, force-quit it, and relaunch. Inspect the retry queue state.
4. Permission revoke. Revoke notification permission while the request is in flight. The feature must disable itself and never send a blank prompt.
5. Endpoint failure. Redirect SHADOW_BASE_URL to a stub that returns 429. Confirm the fallback triggers and the retry loop backs off.
6. Retry storm. Fail the endpoint five consecutive times. Measure battery impact and data usage from the platform energy monitor.

Each scenario maps to one log entry. Each log entry either confirms your fallback design or contradicts it. That contradiction is the review comment, not a suspicion.

When the fallback should win

Use this decision table when you read the traces. It keeps the judgment consistent across devices and reviewers.

Scenario	Trust model output	Use fallback	Trace signal
HTTP 200, latency under 2s	Yes, in staging only	No	fallbackWins: false
HTTP 429 or 5xx	No	Yes	fallbackWins: true
Request timeout	No	Yes	AbortError in the trace
Background kill	No	Yes, on next launch	retry queue state
Permission revoked	No	Yes, feature disabled	no prompt sent
Output shorter than 10 chars	No	Yes	fallbackWins: true
Consistent output across 5 runs	Yes, with versioned prompt	No	identical modelText shape

The table is not a replacement for product judgment. It prevents the most common mistake: trusting the model in exactly the cases where it fails hardest.

Who should skip this workflow

Shadow testing assumes your feature already has a deterministic fallback and a network path to a model. That premise does not fit everyone.

• On-device model teams. There is no endpoint quota and no network failure to simulate. You need an offline evaluation set instead.
• Content-critical surfaces. Medical, legal, or safety copy should not be shadow-tested into production. Use a human review loop.
• Production 24/7 services. The free staging environment is a test budget, not a high-availability tier.
• Teams with strict data residency. A shadow run from a developer laptop may route prompts outside your control region. Verify the endpoint location first.

Also keep the shadow endpoint away from real user messages. Synthetic prompts and redacted copy keep the experiment honest and the privacy review short.

The next time you write a fallback

Your fallback is a product decision, not a placeholder. It deserves the same evidence as a screen design or a network timeout. Free models make that evidence cheap. A free server gives you one place to collect it.

The next time your PR description says "added a fallback," add one more line: Shadow-tested against free models in staging. Your reviewer may still ask questions. A trace answers them in milliseconds.