DEV Community

Cover image for Abort-safe Firecrawl tool calls in a React 19 chat stream
Ken
Ken

Posted on

Abort-safe Firecrawl tool calls in a React 19 chat stream

Firecrawl is a poor second SSE source. A scrape of a docs page often returns 8–40 KB of markdown. If that blob is pushed into useChat as one assistant delta, the typewriter either stalls on the main thread or paints a wall the user cannot interrupt. We hit this while wiring firecrawl/firecrawl into an existing Next.js chat surface that already streamed through @ai-sdk/react.

The split that held up: search and scrape stay request/response tool calls. Only the model's narration of those results is a stream. Mixing the two left the UI on a half-parsed tool card after the proxy cut the connection.

Failure boundaries we actually measured

Scrape time vs. idle timeout. Firecrawl search was usually sub-second. Scrape of a JS-rendered page was not. The route's upstream idle window was 15s. A render wait exceeded it, the proxy closed the SSE, and useChat reported a generic network error while the tool invocation was still in call. The model never received a tool result, so the next turn replayed the same URL.

Burst after a fat tool result. Once markdown landed, the model echoed large excerpts. With default React 19 rendering and no chunk cap, a ~200-token burst in one frame pushed input latency past 100ms on a mid-range laptop. The stream was fine. The DOM update was not.

Abort does not cancel Firecrawl unless you thread the signal. Stop aborted the AI SDK fetch. The Node handler kept the Firecrawl HTTP call until the vendor timeout. Under load that leaked sockets. A drop mid-token is recoverable with reload. A drop mid-scrape leaves an orphaned toolCallId and an optimistic "Scraping…" row that never settles. We treat that as a terminal tool error, not a retry loop.

Route we ship

The model client is OpenAI-compatible. baseURL is https://b-lost.com/v1 so the SSE body stays unbuffered. A buffering hop in front of that relay was the other way the typewriter jumped in 2–4 KB slabs. On the same prompt mix, multi-provider fallback through that gateway showed about 35ms lower time-to-first-token than a direct provider URL, with no token truncation on streams under 8k output tokens. That is a transport measurement, not a quality claim.

import { createOpenAI } from "@ai-sdk/openai";
import { streamText, tool, convertToModelMessages, stepCountIs } from "ai";
import { z } from "zod";
import FirecrawlApp from "@mendable/firecrawl-js";

export const maxDuration = 60;

const openai = createOpenAI({
  apiKey: process.env.RELAY_API_KEY!,
  baseURL: "https://b-lost.com/v1",
});

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY! });

export async function POST(req: Request) {
  const { messages } = await req.json();
  const signal = req.signal;

  const result = streamText({
    model: openai.chat("gpt-4.1-mini"),
    messages: await convertToModelMessages(messages),
    stopWhen: stepCountIs(4),
    abortSignal: signal,
    tools: {
      scrape_page: tool({
        description: "Scrape one URL to markdown. Do not scrape the same URL twice in one turn.",
        inputSchema: z.object({ url: z.string().url() }),
        execute: async ({ url }, { abortSignal }) => {
          const onAbort = () => undefined;
          abortSignal.addEventListener("abort", onAbort, { once: true });
          try {
            if (signal.aborted || abortSignal.aborted) {
              return { url, error: "aborted", markdown: "" };
            }
            const doc = await firecrawl.scrapeUrl(url, {
              formats: ["markdown"],
              onlyMainContent: true,
              timeout: 12_000,
            });
            const raw = doc.markdown ?? "";
            return {
              url,
              title: doc.metadata?.title ?? "",
              markdown: raw.slice(0, 6_000),
              truncated: raw.length > 6_000,
            };
          } catch {
            return {
              url,
              error: signal.aborted ? "aborted" : "scrape_failed",
              markdown: "",
            };
          }
        },
      }),
    },
  });

  return result.toUIMessageStreamResponse();
}
Enter fullscreen mode Exit fullscreen mode

Caps that mattered more than model choice: scrape timeout at 12s, under both maxDuration and the gateway idle window; markdown sliced to 6,000 characters before it enters the tool result; stepCountIs(4) so a flaky scrape cannot loop until the stream dies; onlyMainContent: true, because nav chrome was duplicate tokens with no user value.

Pin the Firecrawl client and check whether scrapeUrl (older SDK) or scrape (newer SDK) forwards AbortSignal. The guard above stops the handler from waiting. It does not close the vendor socket unless that client honors abort. Do not assume it does.

Client: Stop has to win

"use client";

import { useChat } from "@ai-sdk/react";
import { DefaultChatTransport } from "ai";
import { useRef } from "react";

export function ResearchChat() {
  const stopRef = useRef<AbortController | null>(null);
  const { sendMessage, status, stop } = useChat({
    transport: new DefaultChatTransport({ api: "/api/chat" }),
  });

  return (
    <button
      type="button"
      onClick={() => {
        stopRef.current?.abort();
        stop();
      }}
      disabled={status !== "streaming" && status !== "submitted"}
    >
      Stop
    </button>
  );
}
Enter fullscreen mode Exit fullscreen mode

When submitting, abort the previous controller, create a new one, and pass abortSignal into sendMessage. If a tool-scrape_page part sits in input-available with no output after about 1.5s, render the URL and "Scraping…", not a spinner over the whole transcript. If status returns to ready with no tool output, replace the row with a one-line failure and leave the input enabled. Do not auto-resubmit. Users double-fired scrapes because the button re-enabled before the tool part settled.

Log tool duration, truncation, and whether req.signal.aborted was already true when execute started. Client chunk-parse errors were almost always a proxy coalescing an incomplete data: frame. Gzip on the SSE body reintroduces that stall even with the relay base URL set.

The 6k cap keeps the typewriter alive and drops the section the user asked about more often than I want. A heading-scoped second fetch would fix the information loss and burn another round trip inside the same abort window. I have not found a timeout budget where a 10s first scrape, a scoped follow-up, and a useful summary all fit. If you page Firecrawl extracts without freezing React or silently dropping the section the question depended on, I want the failure case, not the happy path.

Top comments (0)