A resolved top-level null was dropped from streamPartialJson (stream_struct 0.3.2)

stream_struct takes a stream of text deltas from a model and turns it into a stream of parsed JSON values. The entry point is streamPartialJson:

Stream<Object?> streamPartialJson(Stream<String> deltas)

It accumulates the deltas into a buffer and, after each one, tries to parse the buffer so far. When the buffer parses, it emits the value. It skips deltas that leave the buffer unparseable (a half-written token or object) or whose parsed value matches the last one emitted. A listener sees the object grow as the deltas arrive.

That much worked. The bug was in how the loop decided a delta had produced a value.

The bug

streamPartialJson was built on the exported helper parsePartialJson:

Object? parsePartialJson(String buffer)

parsePartialJson returns Object?, and it returns null in two different situations:

1. Nothing is decodable yet. The buffer is a partial token like "nu", or a half-written object.
2. The buffer decoded to the JSON literal null, which is a real, resolved value.

The old streaming loop collapsed both into one check:

final value = parsePartialJson(buffer.toString());
if (value == null) continue;

So a resolved top-level null hit the same branch as "not ready yet" and was skipped. If the whole answer was null, the stream completed having emitted nothing.

Reproducing it

Here is a model that streams the JSON literal null across two deltas:

import 'package:stream_struct/stream_struct.dart';

Future<void> main() async {
  // A model that streams the JSON literal null across two deltas.
  Stream<String> deltas() async* {
    yield 'nu';
    yield 'll';
  }

  final frames = <Object?>[];
  await for (final value in streamPartialJson(deltas())) {
    frames.add(value);
  }
  print(frames);
  // stream_struct 0.3.1: []      the resolved null was dropped, no frame emitted.
  // stream_struct 0.3.2: [null]  the null is emitted once, as a resolved value.
}

On 0.3.1 the first delta "nu" does not parse, so parsePartialJson returns null and the loop skips it, which is correct. The second delta completes the buffer to null, which parses to null, which the loop reads as "not ready" and also skips. The stream ends with an empty list.

Why it matters

A model's structured answer can legitimately be a top-level null: a nullable result, an optional tool output, or an explicit "no value". A consumer waiting for a frame for that answer waits forever, because the one frame that would carry the value is the one the loop skipped.

Scope: this was only ever the top level

A null nested inside an object or array was never affected. The container parses to a non-null value, so the loop emits it and the nested null rides along inside:

Stream<String> obj() async* {
  yield '{"a":';
  yield 'null}';
}
// streamPartialJson(obj()) emits: [{}, {a: null}]

The partial buffer {"a": produces an empty map as the first frame. The second delta completes the object to {"a": null}, which parses to {a: null} and is emitted as the second frame. Both frames are non-null maps, so the value == null check never fired. The bug required the top-level value itself to be null.

The fix

0.3.2 adds an internal record type that carries presence separately from the value. parsePartialJsonResult returns a record (bool hasValue, Object? value). "Nothing decodable yet" is hasValue false. "Decoded to null" is hasValue true with value null.

The diff in lib/src/streaming.dart:

// before
final value = parsePartialJson(buffer.toString());
if (value == null) continue;
final encoded = jsonEncode(value);
if (encoded == lastEncoded) continue;
lastEncoded = encoded;
yield value;
// after
final result = parsePartialJsonResult(buffer.toString());
if (!result.hasValue) continue;
final encoded = jsonEncode(result.value);
if (encoded == lastEncoded) continue;
lastEncoded = encoded;
yield result.value;

The loop now checks hasValue to decide whether a delta produced a value, and reads result.value for the value itself. A decoded null has hasValue true, so it passes the check, flows through the change detection, and reaches the yield.

The public API did not change

parsePartialJson keeps its exact Object? contract. The new parsePartialJsonResult and its PartialJsonResult type are @internal and not exported. Nothing in the public surface changed, which is why 0.3.2 is a patch bump from 0.3.1.

Tests that shipped

The regression tests in 0.3.2 cover:

• a top-level null streamed across two deltas emits a single [null] frame,
• a top-level null arriving in one delta also emits [null],
• ordinary object growth and a never-resolving buffer still behave as before.

The root cause: null doing double duty

null was carrying two meanings on the same value. As the return of parsePartialJson, null meant "not ready yet"; as a decoded JSON value, null meant "the answer is null". An Object? cannot tell those apart, because they are the same runtime value, so any caller that has to distinguish them is stuck.

The fix gives presence its own field. A (bool hasValue, Object? value) record has a hasValue field for "is there a value at all" and a separate value field for "what is it", so "no value yet" and "the value is null" stop sharing one representation. That is the same shape as an Option, or any two-field result: worth reaching for whenever a sentinel you return can also be a legitimate value. Here both were null, which is why they collided.