Beyond Token Streaming: Solon AI 4.1's Semantic Chat Events

Streaming an LLM response looks simple until the response contains more than text.

Depending on the provider, a modern stream may include reasoning fragments, tool-call arguments, citations, media updates, safety decisions, usage snapshots, status messages, and errors over the same connection. Treating every frame as “another partial response” pushes provider-specific parsing into the UI and makes it difficult to distinguish a transient fragment from the final answer.

Solon AI 4.1 addresses that problem with a semantic event stream:

Before 4.1:       Flux<ChatResponse>
Starting with 4.1: Flux<ChatEvent>

The change is more than a generic type replacement. ChatEvent represents what is happening while a response is in flight. ChatResponse represents an aggregated result. The distinction gives application code a vocabulary for text, reasoning, tools, lifecycle, usage, and failure without requiring it to understand every provider's raw SSE dialect.

This article builds on the official streaming event guide and checks the behavior against the Solon AI 4.1 source and core tests. One important qualification: the event API is marked @Preview("4.1") in the current source, so treat it as a preview contract that may evolve in future releases.

call() and stream() now answer different questions

The synchronous API still asks for the completed answer:

ChatResponse response = chatModel.prompt("Explain semantic streaming").call();
String text = response.getText();

The streaming API asks to observe the response lifecycle:

Flux<ChatEvent> events = chatModel
        .prompt("Explain semantic streaming")
        .stream();

That difference prevents an intermediate frame from pretending to be a complete response.

For a text-only projection, select the exact semantic event you need:

Flux<String> text = chatModel.prompt(message).stream()
        .filter(event -> event.is(ChatEventType.TEXT_DELTA)
                && event.hasText())
        .map(ChatEvent::getText);

Two details are intentional:

1. isDelta() is too broad for a typewriter projection. It also covers reasoning, tool arguments, media partials, and refusal deltas.
2. getText() is nullable. hasText() keeps a Reactor map from receiving a null result.

This is the first practical rule of the new model: project by meaning, not by transport shape.

Nine groups provide the current routing layer

The current source defines 31 event types organized into nine groups:

Group	What it represents	Representative events
LIFECYCLE	Whole-response state	RESPONSE_START, STATUS, HEARTBEAT, RESPONSE_END, ABORT
STEP	One model round	STEP_START, STEP_END
TEXT	User-visible answer text	TEXT_START, TEXT_DELTA, TEXT_END
THINKING	Reasoning-related output	THINKING_START, THINKING_DELTA, THINKING_END
TOOL_CALL	Client-executed tools	TOOL_CALL_START, TOOL_CALL_ARGS_DELTA, TOOL_CALL_END, TOOL_RESULT
SERVER_TOOL	Provider-side tools	SERVER_TOOL_START, SERVER_TOOL_ARGS_DELTA, SERVER_TOOL_RESULT
MEDIA	Citations and generated media	CITATION, MEDIA_PARTIAL, MEDIA_DONE
SAFETY	Refusals and filtering	REFUSAL_DELTA, CONTENT_FILTER
META	Usage, errors, raw and custom data	USAGE, ERROR, RAW, CUSTOM

These groups are useful when building a generic router. Specific event types can evolve while the application keeps stable top-level destinations:

void route(ChatEvent event) {
    switch (event.getGroup()) {
        case TEXT:
            if (event.is(ChatEventType.TEXT_DELTA) && event.hasText()) {
                ui.appendAnswer(event.getText());
            }
            break;
        case THINKING:
            if (event.is(ChatEventType.THINKING_DELTA) && event.hasText()) {
                ui.appendReasoning(event.getText());
            }
            break;
        case TOOL_CALL:
            toolPanel.accept(event);
            break;
        case SERVER_TOOL:
            serverToolPanel.accept(event);
            break;
        case MEDIA:
            mediaPanel.accept(event);
            break;
        case SAFETY:
            safetyPanel.accept(event);
            break;
        case STEP:
            timeline.accept(event);
            break;
        case LIFECYCLE:
            lifecycle.accept(event);
            break;
        case META:
            diagnostics.accept(event);
            break;
        default:
            log.debug("Unhandled event type: {}", event.getType());
    }
}

Keeping a default branch is still sensible for preview APIs and future versions.

An event-phase END is not necessarily the end of the stream

Events also have phases such as START, DELTA, END, and NONE. The word END is local to an event scope.

For example:

• TEXT_END closes a text block;
• THINKING_END closes a reasoning block;
• TOOL_CALL_END closes one tool call;
• STEP_END closes one model round;
• RESPONSE_END closes a successful response lifecycle.

Use event.isTerminal() when you mean a whole-stream semantic terminal. In the current enum, it is true for:

RESPONSE_END
ABORT
ERROR

Do not replace that check with event.getPhase() == END. ERROR is terminal but has phase NONE, while several local boundary events have phase END without terminating the response.

Recover the final response from RESPONSE_END

If a caller needs the final aggregate while using the streaming path, it should select RESPONSE_END first:

ChatResponse response = chatModel.prompt(query).stream()
        .filter(event -> event.is(ChatEventType.RESPONSE_END))
        .map(ChatEvent::getResponse)
        .blockFirst();

The asynchronous form can expose the first matching terminal response as a Mono<ChatResponse>:

Mono<ChatResponse> response = chatModel.prompt(query).stream()
        .filter(event -> event.is(ChatEventType.RESPONSE_END))
        .map(ChatEvent::getResponse)
        .next();

Calling blockFirst() on the unfiltered stream is a classic migration mistake. The first event is normally RESPONSE_START, not the final answer.

The caller does not need to concatenate every TEXT_DELTA to reconstruct the official final response. Solon AI accumulates the response internally and attaches the aggregate to the terminal event. Local concatenation can still be useful for immediate rendering, but it should be treated as a UI projection rather than the source of truth for tool calls, usage, and the complete assistant message.

The normalizer creates consistent stateful event boundaries

Provider stream formats differ in how consistently they expose boundaries. A provider may send a text delta without an explicit text-start frame. Tool-call fragments may omit a stable identifier after the first frame. Reasoning and answer text may alternate. A stream can terminate while a content block is still open.

ChatEventNormalizer sits between parsed provider semantics and the subscriber. Its job is to make event boundaries usable:

TEXT_START -> TEXT_DELTA* -> TEXT_END
THINKING_START -> THINKING_DELTA* -> THINKING_END
TOOL_CALL_START -> TOOL_CALL_ARGS_DELTA* -> TOOL_CALL_END

For text and thinking blocks, the normalizer uses stricter tracking:

• a bare delta can cause a matching start to be synthesized;
• duplicate starts for the same block are dropped;
• an unmatched end is dropped;
• switching between thinking and text closes the previously open block;
• completion closes remaining open blocks.

Tool calls use a deliberately looser policy. An arguments delta can synthesize a start when no call is open, and an unmatched end is preserved. This is not a malformed-stream validator. It is a compatibility layer for providers whose later argument fragments do not repeat a complete tool-call ID.

That difference is important when writing application code. The normalizer provides compatibility-oriented boundary completion, not strict tool-call identity validation. Use normalized events for rendering and correlation, but read the complete ToolCall objects from a completed step or final response rather than assuming that one argument delta contains valid JSON.

Tool execution introduces steps

An automatic tool workflow is not a single model round. It commonly looks like this:

RESPONSE_START
  STEP_START (0)
    TOOL_CALL_START
    TOOL_CALL_ARGS_DELTA ...
    TOOL_CALL_END
    TOOL_RESULT
  STEP_END (0)
  STEP_START (1)
    TEXT_START
    TEXT_DELTA ...
    TEXT_END
  STEP_END (1)
RESPONSE_END

The response lifecycle remains one unit, while each provider request becomes a step. The current implementation starts step numbering at zero and increments it for recursive model calls.

A normally completed STEP_END event carries the terminal snapshot and usage for that step. RESPONSE_END carries the whole successful response aggregate and cross-step usage total. This lets an observability system answer two different questions:

• What did this model round produce and cost?
• What did the complete tool-assisted response produce and cost?

Tool argument fragments may be interleaved when multiple calls are in flight. Correlate them by toolCallId when one is available; do not build a parser that assumes all fragments for one call arrive as a single contiguous JSON document.

Usage is a two-level aggregation problem

Usage data is easy to overcount.

Within one provider step, usage frames are commonly cumulative snapshots. Adding every frame would count the same tokens repeatedly. Across steps, however, each step is a separate model request, so completed-step totals need to be added.

Solon AI's stream session reflects that distinction:

within a step: keep/merge the provider snapshot
across steps:  add completed-step usage

Consequently:

• STEP_END.getUsage() describes the current completed step;
• RESPONSE_END.getUsage() describes the accumulated successful response;
• a standalone USAGE event is an observation, not an instruction to blindly add every value it contains.

The source also preserves provider usage metadata across steps. Numeric fields such as token and server-tool counts are accumulated, while label-like values use different merge behavior. The larger lesson is portable: billing telemetry needs a scope model, not just a counter.

Errors, aborts, and cancellation are different

A production subscriber must distinguish three mechanisms.

ERROR and Reactor onError

For failures that enter the main reactive error path, Solon AI attempts to expose an ERROR event before terminating the Reactor stream with an error. The two channels serve different purposes:

• the event channel can carry semantic context such as a previously completed response snapshot or accumulated usage;
• Reactor onError drives retry, fallback, timeout, and recovery operators.

They describe one failure, not two failures. Also, do not assume that ERROR.getResponse() contains the partially streamed text from the currently failing step. The aggregate may contain only previously completed steps, or it may be null when the first step fails early.

A custom event filter can suppress a META event such as ERROR, while Reactor onError still arrives. Keep an error consumer even if the UI also handles error events:

chatModel.prompt(query).stream().subscribe(
        this::route,
        error -> recoverOrReport(error),
        () -> markTransportComplete());

ABORT

ABORT is an upstream semantic event. It is not the same thing as the subscriber cancelling its subscription. Application code can route it as a lifecycle signal, but should not infer Reactor cancellation from it.

Reactor cancellation

Operators such as take(...) may cancel the subscription. Once cancellation occurs, the downstream cannot expect extra events. In particular, it should not wait for synthetic TEXT_END, STEP_END, ABORT, or RESPONSE_END events after it has cancelled.

Cleanup belongs in Reactor lifecycle hooks such as doFinally, not in a handler that assumes every path ends with RESPONSE_END:

chatModel.prompt(query).stream()
        .doFinally(signal -> releaseUiResources(signal))
        .subscribe(this::route, this::recoverOrReport);

Event filtering is delivery policy, not aggregation policy

The default filter rejects high-volume HEARTBEAT and unmodeled RAW events. Diagnostic or gateway code can request more:

chatModel.prompt(query)
        .eventFilter(ChatEventFilter.all())
        .stream();

Or it can extend the default policy:

ChatEventFilter filter = ChatEventFilter.DEFAULT.or(
        ChatEventFilter.of(ChatEventType.RAW));

chatModel.prompt(query)
        .eventFilter(filter)
        .stream();

There is a subtle source-level detail worth knowing. Runtime filtering is guarded so lifecycle and step groups survive a non-null custom filter. Therefore, this:

.eventFilter(ChatEventFilter.of(ChatEventType.TEXT_DELTA))

does not mean the subscriber will receive only text deltas. Lifecycle and step events remain available for aggregation and boundaries. In the current preview implementation, the interaction between the default filter, guarded custom filters, and HEARTBEAT is nuanced because heartbeat itself belongs to the lifecycle group.

The robust approach is:

• use eventFilter to control broad delivery volume;
• still use an exact downstream filter when projecting one semantic channel;
• use ChatEventFilter.all() only when raw protocol visibility is genuinely needed.

Filtering happens after Solon AI's internal normalization and aggregation. Hiding an event from a subscriber does not undo the internal final response.

A provider dialect translates frames; it should not duplicate semantics

A custom provider integration implements the streaming parser entry point:

void parseResponseJson(ChatStreamContext ctx, String respJson);

Conceptually, there are two output paths:

1. Put primary content such as answer text, reasoning, and client tool calls into the accumulator so the core can generate standard events and aggregates.
2. Emit semantic events directly for provider capabilities such as citations, status, server-side tools, or specialized media.

The following is illustrative pseudocode; readProviderResponseId, readTextDelta, readCitation, and buildCitation are provider-specific helpers:

@Override
public void parseResponseJson(ChatStreamContext ctx, String json) {
    ctx.setProviderResponseId(readProviderResponseId(json));

    String text = readTextDelta(json);
    if (text != null && !text.isEmpty()) {
        ctx.getAccumulator().addContentItem(
                new AssistantMessage(text));
    }

    String citationUrl = readCitation(json);
    if (citationUrl != null) {
        ctx.emit(ctx.event(ChatEventType.CITATION)
                .block(buildCitation(citationUrl))
                .build());
    }
}

Do not send the same semantic payload through both paths. If text is added to the accumulator and also emitted as a text delta, consumers can see duplicate deltas and the aggregate can be corrupted.

The unified layer is valuable precisely because provider frames and application events are not one-to-one. A single frame may update usage and text; one semantic tool call may span many frames. The dialect should translate provider protocol, while the core owns the cross-provider event contract.

A practical migration checklist

When moving pre-4.1 streaming code to the preview event model:

• Replace Flux<ChatResponse> declarations with Flux<ChatEvent>.
• Render answer text only from TEXT_DELTA plus hasText().
• Route reasoning separately from answer text.
• Do not use isDelta() as a synonym for user-visible text.
• Select RESPONSE_END before calling blockFirst() or next() for the final response.
• Read completed tool calls from STEP_END or RESPONSE_END, not from one arguments fragment.
• Treat STEP_END usage and RESPONSE_END usage as different scopes.
• Keep Reactor error handling even when consuming ERROR events.
• Treat cancellation as a transport/control action, not as ABORT.
• Revisit custom dialects for the parseResponseJson(ChatStreamContext, String) contract.
• Re-check the exact 4.1 preview API before upgrading future versions.

What the tests establish

The core source contains focused tests for the event model. I ran these three deterministic suites against the reviewed checkout:

ChatEventNormalizerTest       20 tests
ChatEventFilterTest            6 tests
ChatStreamSessionUsageTest     9 tests
---------------------------------------
Total                         35 tests

Result:

Tests run: 35, Failures: 0, Errors: 0, Skipped: 0
BUILD SUCCESS

They provide a useful regression net for boundary normalization, filter composition, and cross-step usage aggregation. They do not prove every arbitrary event sequence, every provider's behavior, deep immutability of every payload, or every cancellation race. Provider capabilities also differ: an application must not expect every dialect to emit reasoning, citations, media, safety, server-tool, or usage events.

The architectural payoff

The most useful part of Solon AI 4.1's streaming redesign is not the number of event types. It is the separation of responsibilities:

provider SSE / JSON frames
          ↓
provider dialect parsing
          ↓
semantic accumulation and event emission
          ↓
boundary normalization
          ↓
application routing and protocol adapters

A UI can focus on rendering. An agent can focus on tool steps. An observability layer can focus on usage and failure. A gateway can opt into raw events without forcing ordinary consumers to pay that cost.

Token streaming answers, “What bytes arrived next?” Semantic streaming answers, “What happened next?” For applications that combine reasoning, tools, citations, safety, and multiple provider calls, the semantic model is generally easier to extend.

Further reading:

• Official Solon AI streaming event guide
• Solon AI source repository