Implementing Streaming RAG Responses in Flutter with Server-Sent Events (SSE)

Why Standard REST Calls Fail for Mobile AI UX
When integrating Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) into mobile apps, waiting 5 to 10 seconds for a complete JSON payload blocks the user interface and makes the application feel unresponsive. By consuming Server-Sent Events (SSE) over a Dart Stream, we can render tokens in real time as soon as the vector retrieval step finishes.

1. Creating the Dart SSE Stream Client Using the standard http package, we can send a http.Request and listen directly to the chunked byte stream instead of awaiting http.post:

import 'dart:convert';
import 'package:http/http.dart' as http;

class RagStreamService {
final http.Client _client = http.Client();

Stream streamQuery({
required String prompt,
required List retrievedContextIds,
}) async* {
final request = http.Request(
'POST',
Uri.parse('https://api.example.com/v1/rag/stream'),
);
request.headers['Content-Type'] = 'application/json';
request.headers['Accept'] = 'text/event-stream';
request.body = jsonEncode({
'prompt': prompt,
'context_ids': retrievedContextIds,
'stream': true,
});

final response = await _client.send(request);

if (response.statusCode != 200) {
  throw Exception('Failed to connect to RAG stream: ${response.statusCode}');
}

await for (final chunk in response.stream
    .transform(utf8.decoder)
    .transform(const LineSplitter())) {
  if (chunk.startsWith('data: ')) {
    final data = chunk.substring(6).trim();
    if (data == '[DONE]') break;

    final decoded = jsonDecode(data) as Map<String, dynamic>;
    final token = decoded['token'] as String?;
    if (token != null) {
      yield token;
    }
  }
}

}
}

1. Consuming the Token Stream Efficiently in the UI To avoid rebuilding the entire screen on every token arrival, isolate the streaming text area inside a dedicated StreamBuilder or a scoped ValueNotifier:

class StreamingAnswerCard extends StatefulWidget {
final Stream tokenStream;
const StreamingAnswerCard({super.key, required this.tokenStream});

@override
State createState() => _StreamingAnswerCardState();
}

class _StreamingAnswerCardState extends State {
final StringBuffer _buffer = StringBuffer();

@override
Widget build(BuildContext context) {
return StreamBuilder(
stream: widget.tokenStream,
builder: (context, snapshot) {
if (snapshot.hasData) {
_buffer.write(snapshot.data);
}
return Padding(
padding: const EdgeInsets.all(16.0),
child: SelectableText(
_buffer.toString(),
style: const TextStyle(fontSize: 16, height: 1.5),
),
);
},
);
}
}

Key Production Takeaway
Always decouple your vector search retrieval latency from token generation by sending an initial status event over the stream, and ensure your http.Client is properly closed on widget disposal to prevent memory and socket leaks when users navigate away mid-generation.