Scraping to Streaming UI: Firecrawl + Vercel AI SDK on React 19

When building grounding workflows that inspect live web pages before answering, the primary bottleneck is UI delivery latency. Waiting for an external scraper to render JavaScript DOM elements can easily create a 2- to 4-second blank slate before the first token arrives. If your reverse proxy buffers intermediate chunks or if the user cancels mid-stream, client-side React trees often suffer frame drops or unhandled stream rejections.

In our integration testing with firecrawl (@mendable/firecrawl-js), React 19, and the Vercel AI SDK (ai), we isolated a clean architectural pattern: pull clean markdown directly on the server, pipe tokens through an unbuffered SSE text stream, and wire AbortSignal throughout the call chain.

1. Edge Route Handler: Extract and Stream

Instead of queuing scrape tasks asynchronously, we extract markdown via Firecrawl and immediately pass the context to streamText:

// app/api/chat/route.ts
import FirecrawlApp from '@mendable/firecrawl-js';
import { streamText } from 'ai';
import { createOpenAI } from '@ai-sdk/openai';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const blost = createOpenAI({
  baseURL: 'https://api.b-lost.com/v1',
  apiKey: process.env.BLOST_API_KEY,
});

export async function POST(req: Request) {
  const { messages, targetUrl } = await req.json();
  const signal = req.signal;

  // 1. Scrape structured markdown content
  const scrapeResult = await firecrawl.scrapeUrl(targetUrl, {
    formats: ['markdown'],
  });

  if (!scrapeResult.success || !scrapeResult.markdown) {
    return new Response('Failed to scrape target URL', { status: 502 });
  }

  // 2. Stream synthesis directly to client via SSE
  const result = streamText({
    model: blost('gpt-4o-mini'),
    abortSignal: signal,
    system: 'You are a technical analyst summarizing scraped web context.',
    prompt: `Web Context:\n${scrapeResult.markdown}\n\nQuery: ${messages.at(-1)?.content}`,
  });

  return result.toDataStreamResponse();
}

2. React 19 Client Component: Abort-Ready Chat

In React 19, high-throughput token bursts can trigger redundant reconciliation passes. Using @ai-sdk/react with active stream cancellation ensures incomplete jobs cleanly tear down server resources:

// components/WebGroundingChat.tsx
'use client';

import { useChat } from '@ai-sdk/react';

export function WebGroundingChat({ targetUrl }: { targetUrl: string }) {
  const { messages, input, handleInputChange, handleSubmit, stop, status } = useChat({
    api: '/api/chat',
    body: { targetUrl },
  });

  return (
    <div className="flex flex-col gap-3 p-4 font-mono text-sm max-w-xl mx-auto">
      <div className="overflow-y-auto space-y-2 max-h-80 border rounded p-3">
        {messages.map((m) => (
          <div key={m.id} className="whitespace-pre-wrap">
            <span className="font-bold">{m.role}: </span>{m.content}
          </div>
        ))}
      </div>

      <form onSubmit={handleSubmit} className="flex gap-2">
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Ask about this page..."
          className="border p-2 rounded flex-1 text-sm"
        />
        {status === 'streaming' ? (
          <button type="button" onClick={stop} className="bg-red-600 text-white px-3 py-1 rounded">
            Stop
          </button>
        ) : (
          <button type="submit" className="bg-blue-600 text-white px-3 py-1 rounded">
            Send
          </button>
        )}
      </form>
    </div>
  );
}

This setup demonstrates connecting Vercel AI SDK chat interfaces directly to B-Lost's unbuffered SSE relay, eliminating reverse-proxy buffering delays. Passing req.signal guarantees downstream scraping and model generation halt the moment a client aborts.