Streaming an AI Agent's Progress to the Browser in TypeScript

#typescript #node #agents #streams

Book: AI That Acts
The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here
My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
Me: xgabriel.com | GitHub

A chat completion streams tokens, so the user sees progress immediately. An
agent does not: it thinks, calls a tool, waits on an API, thinks again, and
produces its first user-visible token forty seconds later.

From the browser, that is indistinguishable from a crash. Users reload. The
reload starts a second run. Now you are paying twice and the first run is
still going.

The fix is to stream what the agent is doing, not just what it eventually
says.

Make the loop a generator

The cleanest change is turning the agent from a function that returns into one
that yields.

export type AgentEvent =
  | { type: "turn_started"; turn: number }
  | { type: "thinking"; text: string }
  | { type: "tool_started"; id: string; name: string; summary: string }
  | { type: "tool_finished"; id: string; ok: boolean; ms: number }
  | { type: "answer_delta"; text: string }
  | { type: "done"; text: string; costUsd: number; turns: number }
  | { type: "failed"; reason: string };

export async function* runAgent(
  task: string, ctx: Ctx,
): AsyncGenerator<AgentEvent> {
  const state = init(task);

  while (state.turns < MAX_TURNS) {
    yield { type: "turn_started", turn: ++state.turns };

    const res = await client.messages.create({ /* ... */ });
    state.cost += costOf(MODEL, res.usage);

    if (res.stop_reason !== "tool_use") {
      const text = textOf(res.content);
      yield { type: "answer_delta", text };
      yield { type: "done", text, costUsd: state.cost, turns: state.turns };
      return;
    }

    for (const b of res.content.filter(isToolUse)) {
      yield { type: "tool_started", id: b.id, name: b.name,
              summary: describe(b) };
      const started = performance.now();
      const out = await execute(b, ctx);
      yield { type: "tool_finished", id: b.id, ok: !out.is_error,
              ms: Math.round(performance.now() - started) };
    }
  }
  yield { type: "failed", reason: "turn_limit" };
}

A discriminated union rather than loose objects, because the client will
switch on it and you want exhaustiveness checking on both ends.

summary: describe(b) is deliberate. Never stream raw tool arguments — they
contain user data, ids, and occasionally secrets. Send something renderable:

const describe = (b: ToolUseBlock) => ({
  search_docs: `Searching documentation`,
  get_order: `Looking up an order`,
  send_email: `Preparing an email`,
}[b.name] ?? `Running ${b.name}`);

The SSE endpoint

app.post("/agent/stream", async (req, res) => {
  res.writeHead(200, {
    "Content-Type": "text/event-stream",
    "Cache-Control": "no-cache, no-transform",
    "X-Accel-Buffering": "no",
    Connection: "keep-alive",
  });
  res.flushHeaders();

  const ac = new AbortController();
  res.on("close", () => { if (!res.writableEnded) ac.abort(); });

  const beat = setInterval(() => {
    if (!res.writableEnded) res.write(": ping\n\n");
  }, 15_000);

  let id = 0;
  try {
    for await (const ev of runAgent(req.body.task, { ...ctx, signal: ac.signal })) {
      res.write(`id: ${++id}\n`);
      res.write(`event: ${ev.type}\n`);
      res.write(`data: ${JSON.stringify(ev)}\n\n`);
    }
  } catch (err) {
    if (!ac.signal.aborted) {
      res.write(`event: failed\ndata: ${JSON.stringify({ reason: "error" })}\n\n`);
      logger.error("agent stream failed", { err });
    }
  } finally {
    clearInterval(beat);
    res.end();
  }
});

X-Accel-Buffering: no and no-transform are what stop a proxy from
buffering the whole response — the failure that presents as "streaming doesn't
work" with nothing in your logs.

The heartbeat keeps the connection non-idle during a slow tool call. : ping
is an SSE comment: clients ignore it, proxies count it as traffic.

Aborting on close matters more here than in a chat endpoint, because an
agent that keeps running after the user left is still calling paid APIs and
still causing side effects.

The client

export function useAgentStream(task: string) {
  const [events, setEvents] = useState<AgentEvent[]>([]);
  const [answer, setAnswer] = useState("");

  useEffect(() => {
    const es = new EventSource(`/agent/stream?task=${encodeURIComponent(task)}`);

    const on = <T extends AgentEvent["type"]>(t: T,
      f: (e: Extract<AgentEvent, { type: T }>) => void) =>
      es.addEventListener(t, (m) => f(JSON.parse((m as MessageEvent).data)));

    on("tool_started",  (e) => setEvents((p) => [...p, e]));
    on("tool_finished", (e) => setEvents((p) => [...p, e]));
    on("answer_delta",  (e) => setAnswer((p) => p + e.text));
    on("done",          () => es.close());
    on("failed",        () => es.close());

    return () => es.close();
  }, [task]);

  return { events, answer };
}

EventSource reconnects automatically, which is convenient and dangerous: on
reconnect it will hit your endpoint again and, with the code above, start a
second agent run.

That is the bug worth designing around.

Decouple the run from the connection

The fix is to make the stream a view of a run rather than the run itself.

app.post("/agent", async (req, res) => {
  const runId = crypto.randomUUID();
  void startRun(runId, req.body.task, ctx);   // writes events to a store
  res.status(202).json({ runId });
});

app.get("/agent/:runId/events", async (req, res) => {
  const from = Number(req.headers["last-event-id"] ?? 0);
  // ... SSE headers ...
  for await (const ev of eventStore.subscribe(req.params.runId, from)) {
    res.write(`id: ${ev.seq}\nevent: ${ev.type}\ndata: ${JSON.stringify(ev)}\n\n`);
  }
});

Two things fall out. A reconnect resumes from Last-Event-ID instead of
restarting the agent, which is what the id: field was always for. And a
second browser tab can watch the same run, which is how you get a shareable
progress link.

The event store can be Redis streams, a Postgres table, or an in-memory map if
you run one instance. What matters is that events outlive the socket.

What to actually show

Restraint helps here. Users do not want a debugger.

Show the current activity as one line — "Searching documentation…" — replaced
as it changes, with completed steps collapsed into a count. Show elapsed time
after about five seconds, because that is when people start wondering.

Do not stream raw thinking text to end users. It is long, it contradicts
itself mid-stream, and it reads as the system being confused. Keep it for an
internal debug view.

{events.at(-1)?.type === "tool_started" && (
  <Row spinner>{(events.at(-1) as ToolStarted).summary}…</Row>
)}
{done && <Row muted>{toolCount} steps · {seconds}s</Row>}

The one-line reason to do this

An agent that shows nothing for forty seconds gets reloaded. Every reload is a
duplicate run, duplicate cost, and duplicate side effects, so progress
streaming is not polish, it is the thing that stops your users from
accidentally launching your agent twice.

If this was useful

AI That Acts covers the agent loop and
its surface — turning the loop into events, streaming them safely, and keeping
tool arguments out of anything a browser can see.