Keep Streaming, Abort, and Announcements Intact When You Move a Free Model Behind Your Own Server

The useful test is not whether a free model endpoint can stream. It is whether the browser can still cancel it, retry it, and announce its state changes after you insert your own server in the middle.

That question matters because most streaming AI prototypes start with a hosted endpoint and then graduate to a proxy you control. The browser should not notice the difference in the transport; it should only notice the same observable contract.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator describes MonkeyCode as an open-source project with a free model access path and a free server option, and the current promotion lists a 30M-token allowance. I am treating that number as a current, verify-before-you-build number rather than a permanent guarantee, and I am not using a specific model name because quotas and model availability change.

The browser contract is small but unforgiving. When a user asks for a stream, the first token should arrive incrementally; when the user presses cancel, the upstream fetch should stop; when the stream fails, the failure should arrive as a single stable state that the interface can announce; and when the user retries, the stream should be a fresh sequence rather than a replay of stale tokens. A proxy change is the mechanical equivalent of swapping the engine on a moving car: the passengers should feel no difference unless something was already loose.

The smallest harness that proves both paths behave identically reads SSE events, aborts mid-stream, and records how many tokens arrived before the cancellation. Run it against the hosted URL first, then against your own server URL.

// smoke-equal.mjs
// Usage: HOSTED_URL=... OWN_SERVER_URL=... node smoke-equal.mjs

const hosted = process.env.HOSTED_URL;
const own = process.env.OWN_SERVER_URL;

if (!hosted || !own) {
  throw new Error('Set HOSTED_URL and OWN_SERVER_URL');
}

async function* sseTokens(url, signal) {
  const response = await fetch(url, {
    signal,
    headers: { accept: 'text/event-stream' }
  });

  if (!response.ok || !response.body) {
    throw new Error(`HTTP ${response.status}`);
  }

  const reader = response.body.getReader();
  const decoder = new TextDecoder();
  let buffer = '';

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    buffer += decoder.decode(value, { stream: true });
    let boundary = buffer.indexOf('\n\n');

    while (boundary !== -1) {
      const block = buffer.slice(0, boundary);
      buffer = buffer.slice(boundary + 2);

      for (const line of block.split('\n')) {
        if (line.startsWith('data:')) {
          yield line.slice(5).trim();
        }
      }

      boundary = buffer.indexOf('\n\n');
    }
  }
}

async function measureAbort(url, label, abortAfterMs = 900) {
  const controller = new AbortController();
  const events = [];
  let finishReason = 'completed';

  const timer = setTimeout(() => {
    controller.abort();
    finishReason = 'abortRequested';
  }, abortAfterMs);

  try {
    for await (const token of sseTokens(url, controller.signal)) {
      events.push(token);
    }
  } catch (error) {
    finishReason = error.name || 'Error';
  } finally {
    clearTimeout(timer);
  }

  console.log(JSON.stringify({ label, finishReason, eventCount: events.length }));
}

await measureAbort(hosted, 'hosted');
await measureAbort(own, 'own-server');

The script deliberately does not assert success. It asserts transport equivalence: if the hosted path and your own server path print a similar event count and finish with AbortError after the abort timer fires, you have not accidentally introduced response buffering. If the own-server path completes before the abort timer or finishes with an error like TypeError, you are probably parsing the body too early or losing the AbortSignal in a middleware layer.

Contract check	Why it catches	What to fix if it fails
First event arrives before abort	A buffering proxy hides streaming from assistive tech	Use response.body.getReader() instead of res.json()
Abort reaches for await as AbortError	The browser's cancel gesture must stop upstream work	Pass signal through to fetch
Non-200 status becomes a named error	Screen-reader announcements need a stable failure state	Don't swallow upstream status
Retry starts a fresh SSE stream	Focus and live region should not replay old tokens	Close the previous reader before retry

That table is the part most people skip. The visible text can arrive correctly while the cancellation path is broken, because both paths can still put words on the screen. Screen-reader and keyboard users notice the broken path first: a cancel gesture that appears to work but leaves the network connection running, or an error that is logged on the server but never announced to the live region.

When you proxy behind your own server, add logging, caching, or retry middleware under the same test file. The important rule is that middleware may inspect a stream, but it must not decide success for the client. If your proxy turns an upstream 429 into its own retry loop and keeps the connection open, the browser sees only silence; the accessible pattern is to surface the status as a named failure and let the UI state machine offer a retry after a visible interval. The same is true for abort: a server-side fetch without signal forwarding is a quiet resource leak that eventually shows up as a rate-limit surprise.

This approach has real limits. It does not prove the free allowance will stay at 30M tokens, and it does not prove that a self-hosted server will have enough memory or upstream quota for your traffic. Use the conformance harness as a pre-commit check, not as a load test, and re-check the promotional numbers against the current project page before relying on them. It also does not replace screen-reader testing; transport equivalence only removes one class of regression before you sit down with a keyboard and a screen reader.

Who should not copy this approach into production? Teams that need fixed capacity, hardware guarantees, or controlled data handling should not treat a free tier or a quick proxy as an operational plan. Teams that cannot forward abort signals without hiding errors should keep their UI on the hosted endpoint until they can make failure states observable. If any of those constraints applies, the honest move is to qualify the path switch on a test environment first.

If you want a starting point for this harness, check MonkeyCode's current project page for the token and server limits before you wire the UI. Qualify both paths with the same observable contract, and your accessible streaming interface will survive the move from hosted endpoint to your own server.