Abort-Forwarding Proxy for a Free Model Endpoint: Stop Losing Tokens After the User Leaves

A streaming model UI usually pauses when the user hits Cancel, but the cancel button only stops the local reader. The upstream generation can keep running unless the browser's AbortSignal travels through your server to the model provider. That gap is how free-tier token budgets leak: a user closes a tab, the client fetch is aborted, and the provider keeps producing tokens that nobody sees. This article builds a small AbortController-aware proxy that closes that gap. MonkeyCode's operator describes a free model endpoint and a free server option; the server is a natural place to run this proxy, but the pattern works with any streaming HTTP runtime. The operator also lists a 30M-token allowance, which I'm treating as an unverified availability claim rather than a benchmark or guarantee. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Think of local cancellation as hanging up a phone by putting your handset down while the other line keeps talking. The call is not over until the network tears down the circuit. A browser fetch can be aborted, but that abort only breaks the socket between the browser and your backend. If your backend has already opened a separate connection to a model provider, that upstream request may happily continue because nobody told it to stop. This matters most with streaming APIs, where a long response is intentionally produced over time and where the client may disappear at any moment: a page refresh, a route change, a suspended tab, or a human pressing Escape.

The fix is to make cancellation a network path rather than a UI-only convention. When the browser aborts, the server should detect that close, abort its upstream fetch, and stop forwarding any remaining bytes. The result is not just cleaner state on the client. It is also a budgeting mechanism, because an aborted upstream request should stop consuming tokens shortly after the client goes away.

browser (AbortController)
   | POST /api/chat + stream: true
   v
your free server proxy (auth + forward + abort)
   | POST upstream with an AbortSignal
   v
model endpoint (SSE)

The core proxy is deliberately small. It accepts a JSON body from your own client, injects the provider auth header on the server side, forwards the request, and watches for the client socket closing. The important detail is that the upstream call receives its own AbortController signal. When the local request ends prematurely, that signal fires and the upstream fetch is cancelled. This version uses only Node's built-in HTTP and Fetch APIs, so there is no dependency to audit.

// proxy.mjs
import http from 'node:http';

const UPSTREAM_URL = process.env.UPSTREAM_URL;
const UPSTREAM_AUTH = process.env.UPSTREAM_AUTH;

const server = http.createServer(async (req, res) => {
  if (req.method !== 'POST' || req.url !== '/api/chat') {
    res.writeHead(404);
    res.end('Not found');
    return;
  }

  const chunks = [];
  for await (const chunk of req) chunks.push(chunk);
  const body = Buffer.concat(chunks);

  const upstreamController = new AbortController();
  let upstreamFinished = false;

  req.on('close', () => {
    if (!res.writableEnded && !upstreamFinished) {
      upstreamController.abort();
    }
  });

  try {
    const upstream = await fetch(UPSTREAM_URL, {
      method: 'POST',
      headers: {
        'content-type': 'application/json',
        authorization: `Bearer ${UPSTREAM_AUTH}`
      },
      body,
      signal: upstreamController.signal
    });

    res.writeHead(upstream.status, {
      'content-type': upstream.headers.get('content-type') || 'text/event-stream',
      'cache-control': 'no-cache'
    });

    const reader = upstream.body.getReader();

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      res.write(value);
    }

    upstreamFinished = true;
    res.end();
  } catch (error) {
    if (upstreamController.signal.aborted) {
      console.log('client disconnected; upstream aborted');
    } else {
      console.error(error);
    }
    if (!res.writableEnded) res.end();
  }
});

server.listen(process.env.PORT || 3000);

On the client, cancellation should be visible state, not a silent side effect. The AbortController is created once per conversation attempt. A cancel button calls its abort method, and the page also aborts when the document becomes hidden, because the user may have closed or backgrounded the tab without clicking anything. The live region announces the state change so a screen reader user is not left wondering whether a response is still loading.

<button type="button" id="cancel" hidden>Cancel</button>
<div role="status" aria-live="polite" id="status">Idle</div>
<div aria-live="polite" id="output"></div>

<script type="module">
const controller = new AbortController();
const output = document.querySelector('#output');
const status = document.querySelector('#status');
const cancelButton = document.querySelector('#cancel');

cancelButton.addEventListener('click', () => {
  controller.abort();
  status.textContent = 'Cancelled; asking server to stop upstream.';
});

document.addEventListener('visibilitychange', () => {
  if (document.hidden) controller.abort();
});

async function streamPrompt(prompt) {
  status.textContent = 'Streaming...';
  cancelButton.hidden = false;
  try {
    const response = await fetch('/api/chat', {
      method: 'POST',
      headers: { 'content-type': 'application/json' },
      body: JSON.stringify({ prompt, stream: true }),
      signal: controller.signal
    });

    const reader = response.body.getReader();
    const decoder = new TextDecoder();

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      const text = decoder.decode(value, { stream: true });
      output.textContent += text;
    }
    status.textContent = 'Done.';
  } catch (error) {
    if (error.name === 'AbortError') {
      status.textContent = 'Cancelled. You can start a fresh prompt.';
    } else {
      status.textContent = 'Connection lost; start again.';
    }
  } finally {
    cancelButton.hidden = true;
  }
}

window.streamPrompt = streamPrompt;
</script>

To verify the abort end to end, start the proxy with the upstream URL and auth token in environment variables, then issue a streaming request with curl. Terminate curl after the first few frames rather than waiting for the response to finish. The server should log that the client disconnected and that the upstream was aborted. Next run the same request from the actual browser, click Cancel while text is still arriving, and listen with a screen reader to confirm that the status region announces the cancelled state. Finally, run a longer generation and close the tab at the midpoint. If your provider exposes a usage view, watch it after a short delay. The point is not to claim a precise zero-token outcome, because network propagation and server-side batching add noise, but you should see the allowance stop climbing much sooner than it would with a local-only abort.

curl -N -X POST http://localhost:3000/api/chat \
  -H 'content-type: application/json' \
  -d '{"prompt":"Stream a long answer","stream":true}'

This approach has real limits. A request close does not always arrive immediately on every platform, and some serverless runtimes buffer or disconnect sockets in ways that can delay the signal. If you deploy this proxy on a free server with its own request duration, memory, or concurrency constraints, test it under actual stream lengths instead of assuming the browser's abort will always reach upstream instantly. The proxy also has no user authentication, request logging, retry queue, or provider-specific error mapping, so it is not a replacement for a production API gateway. It keeps the provider key out of browser JavaScript, but it still trusts the server environment with that secret. Teams handling sensitive or regulated data, or teams that need replayable requests and multi-user rate limiting, should evaluate a more complete gateway rather than this small forwarder.

The useful part of this pattern is not the proxy's size. It is that cancellation becomes measurable. Once the browser, server, and upstream share one abort path, you can start asking concrete questions about how many tokens a cancelled conversation actually costs. MonkeyCode's free server option is a convenient place to run that experiment if you don't want to configure a VPS for a tiny HTTP forwarder.