Building High-Performance Generative Studio UIs with React, Tailwind, and Live SSE Telemetry

Studio surfaces for generative video work live and die by their interaction model. Every click, every keyframe drag, every framing adjustment has to feel native, even when the underlying pipeline is still grinding away on remote GPUs. Over the past six months I have been building the frontend for a tool I will call Shadow, and the three pieces that defined whether the product felt "alive" or "laggy" were the timeline stitcher, the shot framing controls, and the SSE synchronisation layer. This is the playbook for that work.

The Core Architectural Tension

A generative video studio is unusual. You have:

• Long-running jobs that stream progress through Server-Sent Events.
• A dense editing surface that needs 60fps interaction locally.
• A shared state model where multiple panels (timeline, gallery, framing, prompt) must reflect the same truth without rerendering each other into oblivion.

The mistake I made early on was treating the SSE stream as a Redux-style event log. It is not. It is a continuous, unordered, occasionally backfilling signal that mixes control events, progress telemetry, and artefact references. Treating it as a single stream forces everything into one reducer and creates the exact render storms you are trying to avoid.

The fix is to split the responsibilities:

┌──────────────────────────────────────────────────────────────┐
│  Client                                                       │
│  ┌────────────┐   ┌──────────────┐   ┌─────────────────────┐ │
│  │ Timeline   │   │ Shot Framing │   │ Gallery             │ │
│  │ Canvas     │   │ Panel        │   │ (artefact viewer)   │ │
│  └─────┬──────┘   └──────┬───────┘   └──────────┬──────────┘ │
│        │                │                       │            │
│        └────────────────┼───────────────────────┘            │
│                         │                                    │
│                  ┌──────▼───────┐                            │
│                  │ Store (Zust) │  ← domain slices            │
│                  └──────┬───────┘                            │
│                         │                                    │
│              ┌──────────▼──────────┐                         │
│              │ SSE Adapter (split) │                         │
│              └──────────┬──────────┘                         │
└─────────────────────────┼────────────────────────────────────┘
                          │
                  ┌───────▼────────┐
                  │ /api/telemetry │
                  │  (text/event-  │
                  │   stream)      │
                  └────────────────┘

Three domains, one store, one adapter. Let me walk through each piece.

The SSE Adapter: Splitting a Stream by Channel

Shadow's backend multiplexes several signals onto a single SSE connection: progress percentage, keyframe completion notices, gallery artefacts, framing metadata, and system warnings. Pushing every event through one path means a single React tree rerender decision.

Instead, the SSE adapter acts as a router. Each incoming event: header tags the channel:

// sse/adapter.ts
type SSEChannel =
  | 'job.progress'
  | 'keyframe.completed'
  | 'artefact.ready'
  | 'framing.adjusted'
  | 'system.warning';

interface SSEEnvelope<T> {
  id: string;
  channel: SSEChannel;
  jobId: string;
  payload: T;
  ts: number;
}

export function createSseAdapter(url: string) {
  const handlers = new Map<SSEChannel, Set<(msg: SSEEnvelope<unknown>) => void>>();
  let es: EventSource | null = null;

  const subscribe = <T>(
    channel: SSEChannel,
    handler: (msg: SSEEnvelope<T>) => void,
  ) => {
    if (!handlers.has(channel)) handlers.set(channel, new Set());
    handlers.get(channel)!.add(handler as (m: SSEEnvelope<unknown>) => void);
    return () => handlers.get(channel)!.delete(handler as never);
  };

  const dispatch = (raw: MessageEvent) => {
    try {
      const env = JSON.parse(raw.data) as SSEEnvelope<unknown>;
      const set = handlers.get(env.channel);
      if (!set) return;
      // Batched flush inside a single microtask to coalesce bursts.
      queueMicrotask(() => {
        for (const fn of set) fn(env);
      });
    } catch (err) {
      console.warn('[sse] dropped malformed envelope', err);
    }
  };

  const connect = () => {
    es = new EventSource(url, { withCredentials: true });
    es.onmessage = dispatch;
    es.onerror = () => {
      // Reconnect with capped exponential backoff.
    };
  };

  return { subscribe, connect };
}

The microtask flush is intentional. SSE bursts (a keyframe completing often fires four to six events in under 10ms) should hit the store as one commit, not six. Tailwind projects feel especially prone to jank here because theme recompiles and class hashing amplify render costs when commits are uneven.

Domain Slices: Why Zustand Beat Redux Toolkit Here

I prototyped this with RTK. The result was correct, but a 24-keyframe timeline with progress telemetry rerendered the entire gallery panel during drag operations. The overhead of cloning nested state, even with Immer, became the bottleneck.

Zustand with selector subscriptions gave me the surgical rerenders I needed:

// store/timeline.ts
interface Keyframe {
  id: string;
  index: number;
  shotId: string;
  startMs: number;
  durationMs: number;
  status: 'pending' | 'rendering' | 'done' | 'failed';
  framing: ShotFraming;
}

interface TimelineSlice {
  keyframes: Record<string, Keyframe>;
  selectedId: string | null;
  upsert: (k: Keyframe) => void;
  select: (id: string | null) => void;
}

export const useTimeline = create<TimelineSlice>((set) => ({
  keyframes: {},
  selectedId: null,
  upsert: (k) =>
    set((s) => ({ keyframes: { ...s.keyframes, [k.id]: k } })),
  select: (id) => set({ selectedId: id }),
}));

// store/artefacts.ts
interface Artefact {
  id: string;
  jobId: string;
  thumbUrl: string;
  fullUrl: string;
  width: number;
  height: number;
  framing: ShotFraming;
}

interface ArtefactSlice {
  byKeyframe: Record<string, Artefact[]>;
  append: (kfId: string, art: Artefact) => void;
}

export const useArtefacts = create<ArtefactSlice>((set) => ({
  byKeyframe: {},
  append: (kfId, art) =>
    set((s) => ({
      byKeyframe: {
        ...s.byKeyframe,
        [kfId]: [...(s.byKeyframe[kfId] ?? []), art],
      },
    })),
}));

Notice that each slice owns one concern. The timeline owns the structure; the gallery owns the artefacts. Cross-slice reads happen through composed selectors, never through subscription fan-out.

The Interactive Keyframe Timeline Stitcher

The timeline is a horizontal strip of clips, each representing a keyframe in the generated sequence. The user can:

• Reorder keyframes by drag and drop.
• Trim durations by dragging the edge of a clip.
• Insert new keyframes between existing ones.
• Right click for actions (regenerate, duplicate, delete).

The render surface is a single absolutely-positioned <canvas> inside a Tailwind container, with a React overlay for selection chrome. Drawing the clips themselves on canvas keeps DOM count flat regardless of how many keyframes exist.

Hit Testing Without React

A common bug in timeline editors is rerendering the canvas on every state change. We avoid that by separating the visual model from the interaction model:

// timeline/Stitcher.tsx
export function Stitcher({ jobId }: { jobId: string }) {
  const canvasRef = useRef<HTMLCanvasElement>(null);
  const keyframes = useTimeline((s) =>
    Object.values(s.keyframes).filter((k) => k.shotId.startsWith(jobId)),
  );
  const selectedId = useTimeline((s) => s.selectedId);
  const select = useTimeline((s) => s.select);
  const upsert = useTimeline((s) => s.upsert);

  // Imperative draw: only runs when keyframes array reference changes.
  useEffect(() => {
    const ctx = canvasRef.current?.getContext('2d');
    if (!ctx) return;
    drawTimeline(ctx, keyframes, { width, height });
  }, [keyframes]);

  // Pointer interactions stay on the canvas; no React state per move.
  const onPointerDown = (e: React.PointerEvent<HTMLCanvasElement>) => {
    const id = hitTestKeyframe(e, keyframes);
    if (id) select(id);
  };

  return (
    <div className="relative h-32 w-full bg-slate-950">
      <canvas
        ref={canvasRef}
        width={width}
        height={height}
        onPointerDown={onPointerDown}
        className="absolute inset-0"
      />
      <SelectionOverlay selectedId={selectedId} />
    </div>
  );
}

drawTimeline paints clips, their status colours (pending grey, rendering amber, done emerald, failed rose), and the stitching band between adjacent keyframes. The stitching band is critical: when two keyframes transition, the visual needs to suggest the interpolation, so we draw a tapered gradient between the end of clip N and the start of clip N+1.

Drag, Trim, Insert: Pointer Capture

For drag and drop, I use the native Pointer Events API rather than a library. The reason is tight control over pointer capture, which prevents dropped events when the cursor leaves the canvas mid-drag:

// timeline/dragController.ts
export function attachDragController(
  canvas: HTMLCanvasElement,
  keyframes: Keyframe[],
  onReorder: (from: number, to: number) => void,
) {
  let dragState:
    | { kind: 'move' | 'trim-left' | 'trim-right'; id: string; startX: number }
    | null = null;

  canvas.addEventListener('pointerdown', (e) => {
    const hit = hitTestWithEdges(e, keyframes);
    if (!hit) return;
    canvas.setPointerCapture(e.pointerId);
    dragState = { kind: hit.kind, id: hit.id, startX: e.clientX };
  });

  canvas.addEventListener('pointermove', (e) => {
    if (!dragState) return;
    const dx = e.clientX - dragState.startX;
    if (dragState.kind === 'move') {
      const preview = computeMovePreview(keyframes, dragState.id, dx);
      paintPreview(canvas, preview);
    } else if (dragState.kind === 'trim-left') {
      const preview = computeTrimLeftPreview(keyframes, dragState.id, dx);
      paintPreview(canvas, preview);
    }
  });

  canvas.addEventListener('pointerup', (e) => {
    if (!dragState) return;
    const dx = e.clientX - dragState.startX;
    onCommit(dragState.kind, dragState.id, dx, onReorder);
    canvas.releasePointerCapture(e.pointerId);
    dragState = null;
  });
}

The preview is painted imperatively each pointermove, while the canonical state only updates on pointerup. That gives smooth 60fps drags even on a 60-keyframe sequence, because React never sees the intermediate frames.

Stitching Across Multiple Shots

The word "stitcher" in the title is not just a flourish. When a job spans multiple shots, the timeline has to visually concatenate them. Each shot contributes a contiguous range of keyframes. The stitcher treats each shot boundary as a thick vertical marker and applies different padding rules.

function drawShotBoundary(ctx: CanvasRenderingContext2D, x: number, h: number) {
  ctx.fillStyle = '#1e293b'; // slate-800
  ctx.fillRect(x - 1, 0, 2, h);
  ctx.fillStyle = '#0ea5e9'; // sky-500
  ctx.fillRect(x - 1, 0, 2, 4);
}

The result is a timeline that reads as scenes rather than as one undifferentiated strip. Users find cut points at a glance.

Dynamic Shot Framing Controls

Framing is the second big interactive surface. The five canonical framings I support are:

• Extreme Close-Up (ECU): tight on eyes or hands.
• Medium (MED): waist up, dialogue standard.
• Wide (WD): full body plus environment.
• Over-The-Shoulder (OTS): reverse angle across another character's shoulder.
• Macro (MAC): extreme detail, often texture or object.

Each framing has parameters the user can dial: focal length proxy, subject offset, depth of field, and aspect behaviour. The panel renders a small preview that re-renders locally as the user adjusts sliders, while the canonical state only commits on debounce.

The Framing State Model

// types/framing.ts
export type FramingKind = 'ECU' | 'MED' | 'WD' | 'OTS' | 'MAC';

export interface ShotFraming {
  kind: FramingKind;
  focalMm: number;        // 14 to 200
  subjectOffset: { x: number; y: number }; // -1 to 1
  dof: number;            // 0 to 1
  aspectLock: '16:9' | '9:16' | '1:1' | 'free';
}

// Defaults per kind; UI reads these to seed sliders.
export const FRAMING_PRESETS: Record<FramingKind, ShotFraming> = {
  ECU: { kind: 'ECU', focalMm: 85, subjectOffset: { x: 0, y: 0.1 }, dof: 0.2, aspectLock: '16:9' },
  MED: { kind: 'MED', focalMm: 50, subjectOffset: { x: 0, y: 0 }, dof: 0.4, aspectLock: '16:9' },
  WD:  { kind: 'WD',  focalMm: 24, subjectOffset: { x: 0, y: 0 }, dof: 0.7, aspectLock: '16:9' },
  OTS: { kind: 'OTS', focalMm: 50, subjectOffset: { x: -0.2, y: 0 }, dof: 0.3, aspectLock: '16:9' },
  MAC: { kind: 'MAC', focalMm: 100, subjectOffset: { x: 0, y: 0 }, dof: 0.1, aspectLock: '1:1' },
};

The Framing Panel

// framing/FramingPanel.tsx
export function FramingPanel({ keyframeId }: { keyframeId: string }) {
  const keyframe = useTimeline((s) => s.keyframes[keyframeId]);
  const upsert = useTimeline((s) => s.upsert);
  const [local, setLocal] = useState<ShotFraming | null>(null);
  const current = local ?? keyframe.framing;

  // Reset local buffer when keyframe changes.
  useEffect(() => setLocal(null), [keyframeId]);

  const update = (patch: Partial<ShotFraming>) => {
    const next = { ...current, ...patch };
    setLocal(next); // immediate UI feedback
  };

  const commit = useDebouncedCallback(() => {
    if (!local) return;
    upsert({ ...keyframe, framing: local });
    setLocal(null);
  }, 220);

  return (
    <aside className="w-80 bg-slate-900 p-4 space-y-4">
      <FramingKindPicker value={current.kind} onChange={(k) => update({ ...FRAMING_PRESETS[k] })} />
      <FocalSlider value={current.focalMm} onChange={(focalMm) => { update({ focalMm }); commit(); }} />
      <SubjectOffsetPad value={current.subjectOffset} onChange={(subjectOffset) => { update({ subjectOffset }); commit(); }} />
      <DofSlider value={current.dof} onChange={(dof) => { update({ dof }); commit(); }} />
      <AspectLockPicker value={current.aspectLock} onChange={(aspectLock) => { update({ aspectLock }); commit(); }} />
      <FramingPreview framing={current} />
    </aside>
  );
}

The local-state-with-debounce-commit pattern is the single biggest win for perceived performance. The slider thumb tracks the cursor instantly because we never round-trip through the store, but other panels (timeline highlight, preview thumbnail) only update 220ms after the user stops fiddling.

The Live Preview Thumbnail

The framing preview is a small <canvas> that renders a stylised mock of the composition rules. For ECU, the subject rect is large; for WD, it is small and centred with environment bands. It is intentionally schematic, not photorealistic, because the point is to show framing geometry, not to consume generation credits.

function renderFramingPreview(canvas: HTMLCanvasElement, f: ShotFraming) {
  const ctx = canvas.getContext('2d')!;
  const { width: w, height: h } = canvas;
  ctx.clearRect(0, 0, w, h);
  ctx.fillStyle = '#020617';
  ctx.fillRect(0, 0, w, h);

  const subjectRatio = SUBJECT_SIZE_BY_KIND[f.kind]; // ECU 0.55, WD 0.18 etc.
  const sw = w * subjectRatio;
  const sh = h * subjectRatio;
  const sx = (w - sw) / 2 + f.subjectOffset.x * (w / 4);
  const sy = (h - sh) / 2 + f.subjectOffset.y * (h / 4);

  ctx.fillStyle = '#475569';
  ctx.fillRect(sx, sy, sw, sh);
  // Focal length as ring blur approximation.
  ctx.filter = `blur(${(1 - f.dof) * 4}px)`;
  ctx.fillStyle = '#94a3b8';
  ctx.fillRect(sx + sw * 0.3, sy + sh * 0.3, sw * 0.4, sh * 0.4);
  ctx.filter = 'none';
}

This preview recomputes only when the framing prop changes, not on every parent rerender. We achieve that by memoising the component with a custom equality function on the framing object:

export const FramingPreview = memo(
  ({ framing }: { framing: ShotFraming }) => {
    const ref = useRef<HTMLCanvasElement>(null);
    useEffect(() => {
      if (ref.current) renderFramingPreview(ref.current, framing);
    }, [framing]);
    return <canvas ref={ref} width={240} height={135} />;
  },
  (a, b) =>
    a.framing.kind === b.framing.kind &&
    a.framing.focalMm === b.framing.focalMm &&
    a.framing.subjectOffset.x === b.framing.subjectOffset.x &&
    a.framing.subjectOffset.y === b.framing.subjectOffset.y &&
    a.framing.dof === b.framing.dof,
);

Zero-Jank Live State Synchronisation

This is the part I am proudest of and the part that took the longest to land. The challenge is that SSE messages can arrive in the middle of a user interaction. If the SSE handler synchronously calls set(...) on the timeline store while the user is mid-drag, React's concurrent mode can produce flicker or worse, lost updates.

The Three Rules I Landed On

1. Pointer interactions own the local optimistic state. SSE updates for the keyframe being dragged are buffered, not committed.
2. SSE updates during a drag are coalesced into a single post-drag flush. When the drag ends, we read the buffered messages and apply only the diffs that did not conflict with the user's final state.
3. Render critical paths never touch the store directly. The timeline draw reads from a useSyncExternalStore subscription that returns the same reference if nothing relevant changed.

Implementation of the Buffer

// store/sseBuffer.ts
type Buffered = Map<string, Partial<Keyframe>>; // keyframe id -> patch

let buffer: Buffered = new Map();
let draggingId: string | null = null;
let flushScheduled = false;

export function beginDrag(id: string) {
  draggingId = id;
}

export function endDrag() {
  draggingId = null;
  scheduleFlush();
}

export function bufferPatch(id: string, patch: Partial<Keyframe>) {
  if (draggingId === id) {
    buffer.set(id, { ...buffer.get(id), ...patch });
    return;
  }
  // Outside a drag, apply immediately.
  useTimeline.getState().upsertPatch(id, patch);
  scheduleFlush();
}

function scheduleFlush() {
  if (flushScheduled) return;
  flushScheduled = true;
  queueMicrotask(() => {
    flushScheduled = false;
    const drain = buffer;
    buffer = new Map();
    for (const [id, patch] of drain) {
      useTimeline.getState().upsertPatch(id, patch);
    }
  });
}

The upsertPatch method on the timeline slice is a partial merge, so we do not blow away local optimistic edits the user made during the drag.

Synchronising Across Panels

The gallery, the framing panel, and the timeline must agree. With per-slice stores, this is easy if every component reads through a stable selector:

function TimelineRuler({ jobId }: { jobId: string }) {
  const totalMs = useTimeline((s) => {
    let total = 0;
    for (const k of Object.values(s.keyframes)) {
      if (k.shotId.startsWith(jobId)) total += k.durationMs;
    }
    return total;
  }, shallow);
  // shallow comparison for the computed value; same numeric total => no rerender.
  return <div className="text-xs text-slate-400">{formatDuration(totalMs)}</div>;
}

The shallow equality function (from Zustand's shallow util) is essential for derived values. Without it, every store update would rerender the ruler.

Dealing With Stale Thumbnails

When a keyframe is regenerated, the new artefact arrives over SSE with a new id. The gallery must swap out the old thumbnail without flashing. We do that by:

1. Keeping a single map artefactByKeyframe keyed by keyframe id.
2. On artefact.ready, replacing the entry but retaining a CSS transition class that cross-fades the old image into the new.

function ArtefactCard({ artefact }: { artefact: Artefact }) {
  return (
    <div className="relative aspect-video overflow-hidden rounded-md bg-slate-800">
      <img
        src={artefact.thumbUrl}
        alt=""
        className="absolute inset-0 h-full w-full object-cover transition-opacity duration-300"
      />
      <span className="absolute bottom-1 right-1 rounded bg-black/60 px-1 text-[10px]">
        {artefact.framing.kind}
      </span>
    </div>
  );
}

The thumbnail transition is a Tailwind utility combo; no JS animation library is needed. Keeping the same key on the parent (the keyframe id) lets React reuse the DOM node, so the transition fires correctly.

Tailwind Choices That Mattered

A few Tailwind decisions had outsized effects:

• bg-slate-950 for the timeline canvas container. Pure black at #000 tends to bleed into OLED and causes perception issues with the stitching bands. Slate-950 gives enough contrast without halation.
• transition-opacity over transition-all. When dozens of cards animate on artefact swap, transition-all is a perf footgun because it forces layout invalidation. transition-opacity is GPU-composited and free.
• will-change-transform on the framing preview canvas only. I tried putting it on the entire studio shell and observed a regression on lower-end hardware. Apply it narrowly.
• Avoiding arbitrary values where possible. w-[237px] and friends do not get purged, but they also bypass the design system. For a tool like Shadow, where the UI is dense and bespoke, I allow a small set of arbitrary values for timeline widths and stick to the scale everywhere else.

Practical Failure Modes and Fixes

Symptom: Timeline stutters when artefacts arrive

Cause: The SSE adapter was committing to the store synchronously on every artefact.ready event.
Fix: Microtask flush in the adapter (shown above). Five artefact events now produce one commit.

Symptom: Framing sliders feel laggy on Linux

Cause: The FocalSlider was a controlled component tied to the store. Each input event rerendered the preview canvas.
Fix: Local state inside the slider, debounced commit to the store, preview reads the local state via useState directly.

Symptom: Drag and drop occasionally misses the drop target

Cause: pointerup was firing outside the canvas because setPointerCapture was not called.
Fix: Capture on pointerdown, release on pointerup. Standard pattern but easy to forget.

Symptom: Stale closure on keyframe selection during regenerate

Cause: The regenerate handler captured the keyframe object at click time. By the time the network round-trip returned, the SSE update had replaced the object.
Fix: Read fresh state via useTimeline.getState() inside async handlers, never capture it in the closure.

Takeaways

• Split SSE by channel and coalesce within a microtask.
• Local-state-with-debounced-commit is the right pattern for any control where the user wants immediate visual feedback.
• Canvas for high-density timelines, React for chrome around them.
• Buffer SSE updates that target a keyframe the user is currently dragging, and flush on drag end.
• Treat derived values with shallow equality to avoid rerenders in adjacent panels.

If you are building a generative studio UI, the temptation is to make every panel a React tree that subscribes to a global store. Resist it. The product feels alive when each panel is doing one job and talking to the others through narrow, typed contracts. That is the whole architecture. The rest is tuning.

---

Written autonomously via Shadow