An Accessible Chat UI Needs a Batch Policy, Not Just a Free Token Endpoint

A free endpoint with a large token allowance can still make an accessible chat interface fail in ways that have nothing to do with model quality, so the first test I would run is not whether the responses are clever. The question that matters more is how many live-region updates a raw stream produces and whether the batch policy keeps a screen reader from being flooded before you ever show the first full answer. MonkeyCode's operator describes free model access with a 30-million-token allowance and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I have not independently audited that allowance or the current limits, so I am not presenting the number as a guarantee; the point is to give you a reproducible test you can point at any compatible endpoint before that budget becomes a design assumption.

A screen reader does not read every change to an aria-live region, and when it does read, it can drop, merge, or interrupt updates. A fast model endpoint is therefore not automatically an accessible one. If you write every streaming delta into the live region, the stream becomes a wall of speech and the listener loses the beginning of the sentence; if you withhold everything until the end, the user sits in silence wondering whether the request is alive. The boundary condition is a batch policy: an interval that collects small text deltas and announces them at a cadence a human can follow. Think of it as watering a plant with a soaker hose instead of a fire hose. The same amount of water arrives, but the plant is far more likely to stay upright.

Because the project is open source, you can check the exact stream route and envelope instead of guessing. The probe below assumes the common OpenAI-compatible /v1/chat/completions shape and leaves authentication out deliberately, so a missing key or wrong path shows up as a visible error instead of a silent blank. Change the path or JSON fields if the repository uses a different contract; the useful part is the batching instrument around it.

The probe is a single HTML page. It reads the stream with fetch and ReadableStream, records the first token time, counts chunks and approximate space-separated tokens, and flashes the latest batch into a polite live region. You can change the endpoint, the model, and the batch interval without touching the code.

<form id='probe'>
  <label>Endpoint <input name='endpoint' value='http://localhost:11434/v1/chat/completions'></label>
  <label>Model <input name='model' value='mistral'></label>
  <label>Batch ms <input name='batchMs' type='number' value='700'></label>
  <button>Stream</button>
  <button id='stop' type='button' disabled>Stop</button>
</form>
<div id='status' aria-live='polite'></div>
<pre id='log'></pre>
<script>
const $ = (s) => document.querySelector(s);
const form = $('#probe');
const live = $('#status');
const log = $('#log');
const N = String.fromCharCode(10);
let controller;
let batchTimer;
let pending = '';
let firstTokenAt = 0;
let chunks = 0;
let tokens = 0;

function announce(text) {
  live.textContent = '';
  requestAnimationFrame(() => { live.textContent = text; });
}

function flush() {
  if (!pending) return;
  const text = pending;
  pending = '';
  announce(text.slice(-160));
}

form.addEventListener('submit', async (event) => {
  event.preventDefault();
  controller?.abort();
  controller = new AbortController();
  $('#stop').disabled = false;
  const data = new FormData(form);
  const endpoint = data.get('endpoint');
  const model = data.get('model');
  const batchMs = Number(data.get('batchMs') || 700);
  chunks = 0;
  firstTokenAt = 0;
  log.textContent = 'connecting';
  try {
    const res = await fetch(endpoint, {
      method: 'POST',
      signal: controller.signal,
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        model,
        stream: true,
        messages: [{ role: 'user', content: 'Explain a progress bar to a screen-reader user in four sentences.' }]
      })
    });
    if (!res.ok || !res.body) throw new Error('HTTP ' + res.status);
    const reader = res.body.getReader();
    const decoder = new TextDecoder();
    let buffer = '';
    while (true) {
      const { value, done } = await reader.read();
      if (done) break;
      buffer += decoder.decode(value, { stream: true });
      const frames = buffer.split(String.fromCharCode(10, 10));
      buffer = frames.pop() ?? '';
      for (const frame of frames) {
        for (const line of frame.split(String.fromCharCode(10))) {
          if (!line.startsWith('data:')) continue;
          const payload = line.slice(5).trim();
          if (payload === '[DONE]') continue;
          let json;
          try { json = JSON.parse(payload); } catch { continue; }
          const delta = json.choices?.[0]?.delta?.content ?? '';
          if (!delta) continue;
          if (!firstTokenAt) firstTokenAt = performance.now();
          chunks += 1;
          tokens += delta.split(String.fromCharCode(32)).filter(Boolean).length;
          pending += delta;
          clearTimeout(batchTimer);
          batchTimer = setTimeout(flush, batchMs);
        }
      }
    }
    flush();
    const elapsed = performance.now() - firstTokenAt;
    announce('Done. First token at ' + firstTokenAt.toFixed(0) + ' ms, ' + chunks + ' chunks, about ' + tokens + ' tokens, batched every ' + batchMs + ' ms.');
    log.textContent += N + 'first_token_ms=' + firstTokenAt.toFixed(0) + N + 'chunks=' + chunks + N + 'tokens_approx=' + tokens;
  } catch (err) {
    if (err.name === 'AbortError') {
      announce('Stream stopped.');
      log.textContent += N + 'aborted';
    } else {
      announce('Error: ' + err.message);
      log.textContent += N + 'error=' + err.message;
    }
  } finally {
    clearTimeout(batchTimer);
    $('#stop').disabled = true;
  }
});

$('#stop').addEventListener('click', () => controller?.abort());
</script>

Run it against the free endpoint first with a 700 ms batch, then with a 2000 ms batch, then with a 0 ms value so each delta goes through the timer path unchanged. If the first token takes more than around 800 ms on a cold run, run the same probe again before blaming the model; you want to separate endpoint latency from browser scheduling and server cold starts. A high chunk count with a low approximate token count means the endpoint is sending many tiny deltas, so your batch interval is doing most of the work; a low chunk count with a large token count means the endpoint is already buffering, and your live region can be updated less often.

You can host this file as a static page, including on the free server option if it serves static assets. The fetch runs in the visitor's browser, so the server only needs to deliver the HTML. If CORS blocks direct browser access to the model endpoint, put the same fetch behind a minimal local proxy and keep the markup unchanged.

The probe is deliberately narrow. It measures one stream from one browser tab, so it will not expose a free server's cold start under concurrent traffic, a shared quota queue, or an upstream outage. The token count is an approximate whitespace split, not a billing token count, so do not use it to audit an allowance. It also does not judge factual quality, cutoff behavior, safety filters, or how the model handles tool calls. If your product depends on a strict response time or a guaranteed finish reason, you need a much heavier harness and a paid or self-hosted path; this is a first-pass smoke test for accessible streaming cadence, not a load test.