The Health Check Said 200 While the Model Said 401: Debugging a Silent Endpoint Fallback

I have watched the same failure three times this month, and it never looks like the same failure at first glance.

Someone wires an AI coding assistant to an endpoint they copied out of a README, sends one prompt, and gets a spinner that turns for thirty seconds before collapsing into the words "Request failed." The console is clean, the network tab has a response worth reading, and the UI tells them none of it.

Is it the key? The URL? The model id? The proxy? The stream flag? The browser? Six suspects, one vague sentence on screen, and no way to bisect them from inside the app.

The walkthrough below is a reconstruction of that class of bug, written so you can execute every step against your own endpoint. It is not a benchmark and it is not a product review; it is a debugging retrospective with a runnable harness at the end.

Why this failure got common in the last few months

The number of AI endpoints a frontend developer is expected to juggle went up, and the amount of configuration discipline per endpoint went down. A single app now resolves a base URL from a UI field, a shell variable, a .env file, and a hard-coded default, and whichever one wins is usually decided by accident.

I have been testing this with MonkeyCode, an open-source AI coding project whose operator states that you can use free model access (up to 10 million tokens) and a free server option. Those two paths hand you two different endpoints, which is exactly the condition that produces the bug below.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

To be clear about scope: the free tier is operator-supplied availability, not a guarantee, and I am not quoting any quota, model list, or uptime number I could not re-verify on the project's own page at publication time. Re-check the current terms before you build a workflow on top of any free tier.

The symptom, written as a state table

Before touching code, write down what each layer reports. In the incident I keep seeing, the layers disagree in a very specific way:

Layer	What it reports	What it means
GET /health	200 OK in 40 ms	The gateway process is alive
POST /v1/chat/completions	401 with a JSON body	Your key is not valid for this route
App fetch wrapper	ok === false	Truth
App UI	"Request failed."	Useless
User	Reloads the page and tries again	Wasted afternoon

The health endpoint answered a different question than the one the user was asking. That is the whole bug, and everything below is mechanical: prove it, isolate it, then make the UI incapable of lying that way again.

Step 1: Stop trusting the app and run the request yourself

Open a terminal and send the exact request the app sends. A three-rung curl ladder separates reachability, authorization, and first-byte timing, which are three completely different failures that the browser collapses into one.

export MC_BASE="https://your-endpoint.example.com"
export MC_KEY="sk-..."        # from a local .env, never committed
export MC_MODEL="your-model-id"

BASE="${MC_BASE%/}"          # strip one trailing slash, not all of them

# Rung 1: DNS, TLS, and whether the host answers at all
curl -sS -o /dev/null -w 'health: %{http_code} in %{time_total}s\n' "$BASE/health"

# Rung 2: auth and path, non-streaming, one token of output
curl -sS -D /tmp/headers.txt -o /tmp/body.txt \
  -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $MC_KEY" \
  -H 'Content-Type: application/json' \
  -d "{\"model\":\"$MC_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"ping\"}],\"max_tokens\":1}"
head -n 1 /tmp/headers.txt; head -c 300 /tmp/body.txt; echo

# Rung 3: streaming on, measure time to first byte
curl -sS -N -o /dev/null -w 'ttfb: %{time_starttransfer}s total: %{time_total}s\n' \
  -X POST "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $MC_KEY" -H 'Content-Type: application/json' \
  -d "{\"model\":\"$MC_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"ping\"}],\"stream\":true,\"max_tokens\":1}"

In the case I keep seeing, rung 1 returns 200 and rung 2 returns 401. That single pair of lines ends the guessing phase, and it costs you about eight seconds.

Reading rung 3 without over-reading it

-N disables curl's own buffering, so a fast ttfb with a slow total means streaming genuinely works end to end. A slow ttfb means something between you and the model is holding bytes back, and no amount of client-side work will fix it. Keep those two conclusions separate, because mixing them is how people end up rewriting a perfectly good render loop.

Step 2: Dump the config the app actually resolved

The 401 was not a credential problem in the end. The app had read OPENAI_BASE_URL from a shell profile it inherited before the user typed anything into the settings panel, and the explicit value lost the precedence fight silently.

Fix the resolution order first, then log it exactly once at startup:

// config.js — loud, ordered, and logged once
const sources = [
  ['ui',    process.env.MC_BASE_URL],
  ['shell', process.env.OPENAI_BASE_URL],
  ['file',  readConfigFile()?.baseUrl],
];

const resolved = sources.find(([, value]) => typeof value === 'string' && value.length > 0);
if (!resolved) {
  throw new Error(`No base URL configured. Checked: ${sources.map(([name]) => name).join(', ')}`);
}

console.info(`[config] base=${resolved[1]} source=${resolved[0]}`);

A silent fallback is worse than a missing value, because a missing value stops you at boot and a fallback follows you into production. If the user set something explicitly, a lower-precedence source should never override it, not even when the explicit value looks malformed.

The trailing /v1 trap

Most endpoints accept https://host/v1/chat/completions, and a large number of copy-pasted configs store https://host/v1 as the base. Join those two naively and you request https://host/v1/v1/chat/completions, which typically returns a 404 that reads like a model problem and is not one.

const joinUrl = (base, path) => {
  const b = base.replace(/\/+$/, '');
  const p = path.replace(/^\/+/, '');
  // if the base already ends in /v1, drop the duplicate segment from the path
  const deduped = b.endsWith('/v1') && p.startsWith('v1/') ? p.slice(3) : p;
  return `${b}/${deduped}`;
};

Route every request through one joiner. One function is testable; twelve template literals scattered across a codebase are not.

Step 3: Replace the health check with a readiness probe

A boolean fetch('/health').then(r => r.ok) answers "is a process listening," which nobody has ever needed to know. What you want is a probe that performs a minimal real completion and classifies the failure into something the UI can say out loud.

const STATUS_REASONS = {
  401: 'auth', 403: 'auth',
  404: 'path',
  429: 'rate',
  500: 'server', 502: 'server', 503: 'server', 504: 'server',
};

export async function probe({ baseUrl, apiKey, model, timeoutMs = 8000 }) {
  const ac = new AbortController();
  const timer = setTimeout(() => ac.abort(), timeoutMs);
  const started = performance.now();

  try {
    const res = await fetch(`${baseUrl.replace(/\/+$/, '')}/v1/chat/completions`, {
      method: 'POST',
      signal: ac.signal,
      headers: { 'content-type': 'application/json', authorization: `Bearer ${apiKey}` },
      body: JSON.stringify({
        model,
        messages: [{ role: 'user', content: 'ping' }],
        max_tokens: 1,
        stream: false,
      }),
    });

    const ms = Math.round(performance.now() - started);
    const text = await res.text();

    if (!res.ok) {
      return { ok: false, reason: STATUS_REASONS[res.status] ?? 'server', status: res.status, ms, detail: text.slice(0, 200) };
    }

    let json;
    try { json = JSON.parse(text); }
    catch { return { ok: false, reason: 'server', status: res.status, ms, detail: 'non-JSON body' }; }

    const content = json?.choices?.[0]?.message?.content;
    if (typeof content !== 'string' || content.trim() === '') {
      return { ok: false, reason: 'empty', status: res.status, ms };
    }

    return { ok: true, status: res.status, ms };
  } catch (err) {
    const ms = Math.round(performance.now() - started);
    return { ok: false, reason: ac.signal.aborted ? 'timeout' : 'network', status: 0, ms, detail: String(err) };
  } finally {
    clearTimeout(timer);
  }
}

Two notes on honesty rather than cleverness. A probe spends tokens, so keep max_tokens at 1 and run it on session start or an explicit "Test connection" click, never per keystroke. If your provider enforces a larger floor than 1, raise it to the smallest accepted value and say so in a code comment, because the next maintainer will assume 1 works everywhere.

The empty branch is the one that saves the most time in practice. A proxy that rewrites your model alias happily returns 200 with choices: [], and an app that only checks res.ok renders a blank bubble with no error at all.

Step 4: Map every transport outcome to one UI state

Once the probe can classify, the client should not be allowed to improvise. This table is the whole contract between transport and screen, and I keep a copy of it in the repository next to the fetch wrapper:

Probe result	Visible UI	Screen reader	Retry policy
ok	normal composer	silent	n/a
auth	"This key was rejected by host" + link to settings	assertive alert once	blocked until edited
path	"Endpoint path looks wrong (404)"	assertive alert once	blocked
rate	"Rate limited — try again shortly"	polite status once	enable after Retry-After if present
server	"Endpoint error, one automatic retry"	polite status once	one automatic retry, then manual
timeout	"No response in 8s" + Cancel	polite status once	manual
empty	"The model returned an empty reply"	assertive alert once	manual

The accessibility rules that keep that table honest

Three rules matter more than the table itself, because a table is easy to write and easy to violate in a status component.

1. Announce the transition, not the timer. A live region that updates every second while waiting turns a slow request into a screen-reader hostage situation.
2. Only alert for states the user must act on. Auth and path failures block everything, so they earn assertiveness; slow networks do not.
3. Put the host and the next action in the text. "Request failed" costs a debugging session; "This key was rejected by api.example.com" costs nothing.

Step 5: Wrap it in a typed state machine

Types are the cheapest way to stop the UI from inventing a seventh state nobody designed. Four phases cover the whole lifecycle, and the blocked phase carries the reason as a union rather than a string.

type BlockReason = 'auth' | 'path' | 'rate' | 'server' | 'timeout' | 'empty' | 'network';

type Session =
  | { phase: 'idle' }
  | { phase: 'probing' }
  | { phase: 'ready'; endpoint: string; model: string }
  | { phase: 'blocked'; reason: BlockReason; host: string; retryAfterMs?: number };

const timeoutBudget = { connectMs: 3000, firstTokenMs: 8000, totalMs: 60000 };

In practice I use the same object to drive both the visual state and the announcement copy, so a new reason cannot ship without a message. If you add a variant and the build still passes, you forgot an exhaustive switch somewhere, and TypeScript will tell you where.

The timeout budget is deliberately three numbers instead of one. A connect timeout tells the user their network is the problem, a first-token timeout points at buffering or model queueing, and a total timeout points at a runaway generation you should let them cancel rather than silently truncate.

A verification matrix you can run in an afternoon

Reproducing a bug is half the job; proving you fixed the whole class is the other half. These six rows are the ones that have caught real regressions for me:

Scenario	Setup	Expected probe	Expected UI
Direct connection	no proxy	ok	composer enabled
Corporate proxy	HTTPS_PROXY set, TLS intercepted	network or server	error names the host, no infinite spinner
Wrong key	key from another project	auth	alert + settings link, retry blocked
Doubled path	base ends in /v1	path	alert, no auto-retry loop
Idle free endpoint	first request after a quiet period	slow ok	one polite status, then success
Empty rewrite	proxy strips choices	empty	explicit empty reply message

Run the matrix with the network panel open and once with a screen reader on. The two passes catch different bugs, and the screen-reader pass is the one that surfaces double announcements you will never notice visually.

Limitations, and who should not do this

This workflow is a debugging aid, not an architecture. A single probe proves one path at one moment, and it will happily pass while a streaming request through the same endpoint still stalls on a buffering proxy.

Please do not build on it if any of these apply:

• You handle regulated data. A shared free endpoint is not the place to send it, regardless of how good the ergonomics are.
• You need reproducible evaluation runs. Probe-and-retry logic hides model drift instead of pinning it.
• You need an availability commitment. Free tiers are operator-supplied and can change; keep a second configured endpoint and a documented switch.
• You only want a demo that looks fast. Skip the probe and accept that your first real user becomes your health check.

If you want a second endpoint to run the curl ladder against, the free model access and free server option in MonkeyCode are a low-friction way to get one, and having two configs is precisely what makes precedence bugs visible. Run the three rungs against both, and tell me which rung fails first.

When you report a reproduction, include your Node version, curl version, whether a proxy sits in the path, and the exact transition that broke — probe to first token, or first token to done. That detail is what turns "it doesn't work" into a bug somebody can actually close.