Streaming LLM-Generated UI in React 19 with OpenUI Lang

Every generative UI demo I have shipped hits the same wall: the model emits a JSON component tree, and nothing paints until the closing brace arrives. For a 2,000-token dashboard payload that is several seconds of blank screen followed by a layout jump. You can skeleton it, but you cannot partially render an object that has not been closed yet.

OpenUI (thesysdev/openui) takes a different position. Instead of JSON, the model emits OpenUI Lang: a line-oriented DSL where each statement binds one component to a name. Because each line parses independently, the renderer can paint as tokens land. In our integration with thesysdev/openui, that changed the perceived latency profile far more than any streaming text typewriter I have built.

The library is the contract

OpenUI does not let the model invent components. You define a Zod-schema'd library, generate a system prompt from it, and the parser validates streamed output against those signatures. Layout components can come from the prebuilt @openuidev/react-ui set; you usually only author leaves.

// src/library.ts
import { defineComponent, createLibrary } from '@openuidev/react-lang';
import { z } from 'zod';

const StatCard = defineComponent({
  name: 'StatCard',
  description: 'Single KPI tile: label, value, and trend direction',
  props: z.object({
    label: z.string().describe('Metric name'),
    value: z.string().describe('Formatted value, e.g. $1.2M'),
    trend: z.enum(['up', 'flat', 'down']),
  }),
  component: ({ label, value, trend }) => (
    <div className='rounded border p-3'>
      <div className='text-xs uppercase text-neutral-500'>{label}</div>
      <div className='text-2xl font-semibold'>{value}</div>
      <div data-trend={trend} className='text-xs'>{trend}</div>
    </div>
  ),
});

export const library = createLibrary({ components: [StatCard] });
export const systemPrompt = library.prompt({
  preamble: 'You are a dashboard assistant.',
  additionalRules: ['Write layout statements before data statements.'],
});

The CLI writes that prompt to src/generated/system-prompt.txt, and the output the model produces looks like this:

root = Stack([header, stats])
header = TextContent("Q4 Dashboard", "large-heavy")
stats = Grid([s1, s2])
s1 = StatCard("Revenue", "$1.2M", "up")
s2 = StatCard("Users", "450k", "flat")

Note that root references header and stats before either exists. That is legal. Forward references render as skeletons, which is exactly the progressive behaviour you want.

Stream text, not JSON

The single most common failure I see is people leaving the upstream SSE frames glued to the browser. Unwrap them server-side so the client receives one plain text stream it can concatenate blindly.

// src/app/api/chat/route.ts
import { readFileSync } from 'node:fs';
import { join } from 'node:path';

const systemPrompt = readFileSync(
  join(process.cwd(), 'src/generated/system-prompt.txt'),
  'utf-8',
);

export async function POST(req: Request) {
  const { messages } = await req.json();
  const upstream = await fetch(`${process.env.MODEL_BASE_URL}/chat/completions`, {
    method: 'POST',
    headers: {
      'content-type': 'application/json',
      authorization: `Bearer ${process.env.MODEL_API_KEY}`,
    },
    body: JSON.stringify({
      model: process.env.MODEL_ID,
      stream: true,
      messages: [{ role: 'system', content: systemPrompt }, ...messages],
    }),
    signal: req.signal,
  });

  const reader = upstream.body!.getReader();
  const dec = new TextDecoder();
  const enc = new TextEncoder();

  const body = new ReadableStream({
    async start(controller) {
      let tail = '';
      for (;;) {
        const { value, done } = await reader.read();
        if (done) break;
        tail += dec.decode(value, { stream: true });
        const lines = tail.split('\n');
        tail = lines.pop() ?? '';
        for (const line of lines) {
          if (!line.startsWith('data:')) continue;
          const raw = line.slice(5).trim();
          if (raw === '[DONE]') continue;
          const delta = JSON.parse(raw).choices?.[0]?.delta?.content;
          if (delta) controller.enqueue(enc.encode(delta));
        }
      }
      controller.close();
    },
    cancel() {
      void reader.cancel();
    },
  });

  return new Response(body, {
    headers: {
      'content-type': 'text/plain; charset=utf-8',
      'cache-control': 'no-store',
      'x-accel-buffering': 'no',
    },
  });
}

x-accel-buffering: no is not optional. Nginx and most managed edge proxies buffer by default, and your beautifully incremental stream will arrive in one lump two seconds late. Localhost will never reproduce it.

Client: one string, one renderer

'use client';
import { useRef, useState } from 'react';
import { Renderer } from '@openuidev/react-lang';
import { library } from '@/library';

export function ChatPane() {
  const [response, setResponse] = useState('');
  const [isStreaming, setStreaming] = useState(false);
  const abortRef = useRef<AbortController | null>(null);

  async function send(prompt: string) {
    abortRef.current?.abort();
    const ac = new AbortController();
    abortRef.current = ac;
    setResponse('');
    setStreaming(true);
    try {
      const res = await fetch('/api/chat', {
        method: 'POST',
        headers: { 'content-type': 'application/json' },
        body: JSON.stringify({ messages: [{ role: 'user', content: prompt }] }),
        signal: ac.signal,
      });
      const reader = res.body!.getReader();
      const dec = new TextDecoder();
      for (;;) {
        const { value, done } = await reader.read();
        if (done) break;
        setResponse(prev => prev + dec.decode(value, { stream: true }));
      }
    } catch (err) {
      if ((err as Error).name !== 'AbortError') throw err;
    } finally {
      setStreaming(false);
    }
  }

  return (
    <>
      <button onClick={() => send('Q4 revenue dashboard')}>Generate</button>
      <button onClick={() => abortRef.current?.abort()}>Stop</button>
      <Renderer response={response} library={library} isStreaming={isStreaming} />
    </>
  );
}

Five things that actually broke

1. Multibyte splits. Cursor symbols and CJK labels routinely straddle chunk boundaries. new TextDecoder() without { stream: true } turns them into replacement characters on both ends of the pipe.
2. meta.unresolved mid-stream is not an error. Forward references are by design. Only inspect it after the stream closes; treating it as a failure mid-flight will flash error states on every valid render.
3. Positional arguments follow Zod key order. Reordering two properties in a schema silently changes what every generated statement means. Regenerate the prompt file whenever the library changes and diff it in review.
4. Re-rendering per token. setResponse(prev => ...) on every chunk is fine at 40 tok/s and janky at 300. Coalesce with requestAnimationFrame or wrap the append in a React 19 transition.
5. Aborts must propagate upstream. signal: req.signal on the server fetch is what stops you paying for tokens a user cancelled by closing the tab.

Verdict

Line-oriented output is a genuine architectural win: partial paint is free, and the repo's own token benchmarks put a contact-form payload at 294 tokens against 893 for a comparable JSON tree. The cost is discipline. Your component library becomes an API contract, and the generated system prompt becomes a build artifact that has to be regenerated and diffed like any other. Treat it that way and the streaming side stops being the hard part.