DEV Community

Sukhpinder Singh
Sukhpinder Singh

Posted on

Your JSON Array Was Streaming All Along

I set out to prove a code-review comment right. The stopwatch had other plans.

Code review last week: a progress endpoint for a long import job, returning IAsyncEnumerable<Step> from a minimal API. The comment under it said what I'd have written myself a month ago: "JSON responses buffer, the client won't see anything until the job finishes. Use SignalR." I've repeated that advice for years without once pointing a stopwatch at it. So before approving, I did.

The rig

One minimal API, one fake job that yields a step every 400 ms:

app.MapGet("/steps/json", (CancellationToken ct) => Produce(padding: 0, ct));

async IAsyncEnumerable<Step> Produce(int padding, [EnumeratorCancellation] CancellationToken ct = default)
{
    for (var i = 1; i <= TotalSteps; i++)
    {
        await Task.Delay(DelayMs, ct);
        yield return new Step(i, $"step {i}/{TotalSteps}", padding == 0 ? "" : new string('x', padding));
    }
}
Enter fullscreen mode Exit fullscreen mode

The probe is deliberately dumb: HttpCompletionOption.ResponseHeadersRead, then raw stream.ReadAsync in a loop, logging elapsed time and byte count for every read. No JSON parsing, no framework help on the client side. I only want to know when bytes hit the wire.

Conditions, since I'm about to quote numbers: .NET 10 (SDK 10.0.302), Kestrel and the client in the same Linux container, localhost. Not a lab. I care about the arrival pattern, not the milliseconds.

If the folklore holds, the log should show silence for three seconds and then one fat read.

The folklore loses

GET /steps/json
  headers      675 ms   200 application/json
  read  1      680 ms       40 B   [{"number":1,"name":"step 1/8","pad":""}
  read  2     1066 ms       40 B   ,{"number":2,"name":"step 2/8","pad":""}
  read  3     1465 ms       40 B   ,{"number":3,"name":"step 3/8","pad":""}
  read  4     1866 ms       40 B   ,{"number":4,"name":"step 4/8","pad":""}
  read  5     2266 ms       40 B   ,{"number":5,"name":"step 5/8","pad":""}
  read  6     2667 ms       40 B   ,{"number":6,"name":"step 6/8","pad":""}
  read  7     3068 ms       40 B   ,{"number":7,"name":"step 7/8","pad":""}
  read  8     3469 ms       41 B   ,{"number":8,"name":"step 8/8","pad":""}]
  done        3471 ms   321 B in 8 read(s)
Enter fullscreen mode Exit fullscreen mode

Eight reads. Forty bytes each. One every 400 ms, landing the moment each element was yielded. The array was streaming the whole time: opening bracket first, elements as they came, closing bracket three seconds later.

I assumed tiny payloads were a fluke, so I re-ran it with ~4 KB per element. Same rhythm, ~4.1 KB per tick. System.Text.Json's async path flushes pending output when your producer goes off to await something, and minimal APIs have been quietly good at this for a while now. The warning I kept repeating does have an ancestor, to be fair: MVC's Newtonsoft.Json path really does buffer IAsyncEnumerable to the end. I didn't retest that path here. But on minimal APIs with System.Text.Json, on current .NET, it's simply not your problem.

One small detail from the logs I hadn't thought about: the response headers didn't leave until the first item did, in every variant. Your TTFB is your first yield, not your return.

So the new SSE support is pointless?

That was my second wrong take of the afternoon. The server was never the problem; the consumer is. What arrives is a JSON array with its closing ] missing until the very end. If the caller is another .NET service, that's fine, because the deserializer streams too:

await foreach (var step in JsonSerializer.DeserializeAsyncEnumerable<Step>(stream, JsonSerializerOptions.Web))
    Console.WriteLine($"item {step!.Number}   {sw.ElapsedMilliseconds} ms");
Enter fullscreen mode Exit fullscreen mode
item 1     413 ms
item 2     803 ms
item 3    1204 ms   ...usable as they arrive, same endpoint, no protocol change
Enter fullscreen mode Exit fullscreen mode

A browser is a different story. fetch(...).json() resolves when the body ends, so the dashboard renders nothing for the whole job and then everything at once, which is exactly the symptom that convinced all of us the server was buffering. You could hand-roll an incremental parser over a half-open array. Nobody does. They install SignalR, for a one-way progress feed.

.NET 10 finally hands that job to the right tool. Same producer, one different return type:

app.MapGet("/steps/sse", (CancellationToken ct) =>
    TypedResults.ServerSentEvents(ProduceSse(ct)));

async IAsyncEnumerable<SseItem<Step>> ProduceSse([EnumeratorCancellation] CancellationToken ct = default)
{
    await foreach (var step in Produce(padding: 0, ct))
        yield return new SseItem<Step>(step, eventType: "step") { EventId = step.Number.ToString() };
}
Enter fullscreen mode Exit fullscreen mode

curl -N shows classic text/event-stream framing, one event per yield, same 400 ms heartbeat:

Enter fullscreen mode Exit fullscreen mode

And the browser side is two lines, no package, no hub, no negotiation handshake:

const source = new EventSource("/steps/sse");
source.addEventListener("step", e => render(JSON.parse(e.data)));
Enter fullscreen mode Exit fullscreen mode

EventSource reconnects on its own and sends a Last-Event-Id header when it does — that's why I bothered setting EventId. Resuming from that header is still your code to write, but the protocol carries the bookkeeping for free.

Where I landed

The framing isn't free: 520 B for eight events versus 321 B for the plain array. Sixty-ish percent overhead on comically small payloads, rounding error on real ones.

Where I wouldn't use SSE: service-to-service calls, since the plain array plus DeserializeAsyncEnumerable is already streaming; anything needing client-to-server messages on the same channel; fan-out to huge audiences where you want groups and a backplane. That's SignalR's turf and it earns it there. Two more things worth knowing: on HTTP/1.1 browsers allow roughly six connections per origin and every open EventSource holds one, so serve this over HTTP/2. And buffering middleware — response compression, some reverse proxies — can still flatten either approach into one blob at the end. The folklore isn't dead; it just moved up a layer.

My take, stated as such: for one-way progress and dashboard feeds on .NET 10, SSE should be the default and SignalR the exception you argue for. A return statement beat a hub for this endpoint.

The actual lesson cost me an afternoon: the advice I nearly left in that review was years stale. Measure the folklore once in a while. It goes off.

Full runnable sample: https://github.com/ssukhpinder/dev-to-code-samples/tree/main/004-json-streaming-vs-sse

What's a piece of .NET folklore you've caught being stale? Tell me in the comments and I'll point the stopwatch at it.

— Sukhpinder, still pointing stopwatches at endpoints nobody complained about

Top comments (0)