The Zombie Stream Problem: Hardening SSE Failure Boundaries for ChatGPT Web Relays

It is 3:00 AM, your APM dashboards are glowing green with 200 OK status codes, and yet your support queue is lighting up because users are staring at half-generated code blocks frozen mid-token. The upstream TCP socket died, intermediate reverse proxies swallowed the teardown, and your front-end chat state is still waiting for chunks that will never arrive. Your application is officially lying to your users.

When auditing miuuyy/codex-chatgpt-web as an external systems engineer, this failure boundary stands out as the single greatest architectural liability. The repository provides immense utility: it wraps ChatGPT Web—including ChatGPT Pro workflows—into a drop-in upstream model endpoint for Codex-style developer environments. But Codex clients demand a rock-solid, unambiguous transport contract: strictly ordered incremental tokens, an explicit terminal delimiter, and immediate abort propagation. When you place a reverse-engineered web session behind an API proxy, you inherit every network instability, token refresh hiccup, and gateway timeout along the path.

Here is how to design and build an unbuffered Server-Sent Events (SSE) failure boundary that keeps your streaming chat interface honest when upstream connections collapse.

---

Decouple Transport Mechanics from UI State

Front-end teams routinely blame streaming freezes on React rendering bottlenecks or state updates. In reality, the root issue is poor boundary separation. A robust streaming interface requires three decoupled architectural tiers:

1. The Transport Adapter: Manages raw HTTP fetch lifecycles, HTTP headers, SSE byte framing, and bidirectional cancellation propagation.
2. The SDK Protocol Layer: Converts raw chunk streams into typed structures compatible with @ai-sdk/react or custom dispatchers.
3. The Presentation Layer: Manages optimistic message updates, loading states, manual retry triggers, and Markdown token rendering.

A dropped TCP connection is a network transport failure, not a state-management bug. Similarly, an upstream rate-limit event should never blow away the local conversation tree.

Your transport adapter must also guard internal credentials. Keep provider keys, session tokens, and relay endpoints strictly server-side. The Next.js Route Handler below enforces strict unbuffered passthrough, explicit upstream abort signal binding, non-2xx status propagation, and clean socket teardown when the browser disconnects:

// app/api/chat/route.ts
import { NextRequest } from "next/server";

export const runtime = "nodejs";

export async function POST(request: NextRequest) {
  const body = await request.text();
  const upstream = new AbortController();
  const disconnect = () => upstream.abort();

  request.signal.addEventListener("abort", disconnect, { once: true });

  try {
    const response = await fetch(process.env.BLOST_SSE_URL!, {
      method: "POST",
      headers: {
        "content-type": "application/json",
        authorization: `Bearer ${process.env.BLOST_API_KEY}`,
        accept: "text/event-stream",
        "cache-control": "no-cache",
      },
      body,
      signal: upstream.signal,
    });

    if (!response.ok || !response.body) {
      const detail = await response.text().catch(() => "upstream unavailable");
      return new Response(detail, {
        status: response.status || 502,
      });
    }

    return new Response(response.body, {
      status: 200,
      headers: {
        "content-type": "text/event-stream; charset=utf-8",
        "cache-control": "no-cache, no-transform",
        connection: "keep-alive",
        "x-accel-buffering": "no",
      },
    });
  } finally {
    request.signal.removeEventListener("abort", disconnect);
  }
}

Do not write a bespoke SSE parser in this intermediary handler. If the relay emits OpenAI-compatible framing, leave stream deserialization to the consumer's client SDK. Maintaining a second, ad-hoc parser in an edge handler introduces subtle parsing bugs—such as dropping split multi-byte UTF-8 sequences or choking on multiline data: fields—leading to silent truncation.

---

Treat Abort and Interruption as First-Class States

When an upstream connection drops, never discard the tokens already displayed on screen. Partial assistant output represents compute already spent and context the user has already parsed.

Instead of collapsing the bubble back into an error banner, transition the assistant message to an interrupted status and render an inline "Retry from here" control. However, never trigger automated, silent retries on network dropouts. If the conversation has triggered external tools or agent side effects, a silent replay risks re-executing non-idempotent operations and burning downstream quotas.

To eliminate race conditions when users hit "Stop" and immediately send a revision, track request lifecycles using an incremental generation ref:

• Increment the generation ID on every dispatch.
• Discard incoming chunks from any stream whose generation ID does not match the active ref.
• On abort, immediately terminate the active controller and lock the partial message state.

Beyond client code, beware of the proxy buffering trap. If Cloudflare, NGINX, or AWS ALB sits between your relay and your Next.js app, default proxy buffering rules will queue chunks until a 4KB buffer fills, destroying perceived TTFT (Time to First Token). Setting Cache-Control: no-cache, no-transform and X-Accel-Buffering: no is essential, but you must verify end-to-end delivery in the browser's Network DevTools by inspecting chunk arrival timestamps.

---

Production Observability: Structured Telemetry Over Token Dumps

When a stream fails in production, raw payload dumps violate data governance and clog log aggregators. Capture high-signal operational metrics instead:

• Correlation Request ID and active model name.
• Upstream HTTP status code.
• Time to First Event (TTFE) and total event counts.
• Terminal termination reason (completed, client_abort, upstream_reset, framing_error).

If you encounter a malformed chunk, log the exact byte offset and parser error, then terminate the stream cleanly with a structured error frame. Pushing forward through broken frames risks outputting hallucinated or out-of-order text directly into the user's workspace.

---

The Core Engineering Dilemma

Integrating tools like miuuyy/codex-chatgpt-web forces an architectural trade-off: raw latency versus state durability.

A direct, unbuffered SSE passthrough yields minimal latency and zero persistent state overhead, but shifts the entire burden of handling socket dropouts and CDN buffering onto your frontend architecture. Introducing durable event queues or transactional streaming brokers solves reconnection headaches, but turns a snappy real-time chat interface into a heavy distributed pipeline.

For Codex workflows, honesty beats clever recovery: surface partial output, expose explicit stream states, and treat network failure as a standard UI condition.

What does your team's streaming gateway topology look like under production load? Are you relying on native edge streaming, or running dedicated terminating proxies with fallback buffers? Share your architecture and hard-learned battle scars in the comments below.

---

Disclosure: Multi-model API relays and compute for this evaluation are sponsored by b-lost.com — an AI gateway offering 0.8x official pricing, native prompt caching, and zero user-data retention. All observations reflect independent developer testing.