DEV Community: Ray Mac

How We Optimized ScribeToAny: From 3.5s Cloudflare Cold Starts to a 95+ Lighthouse Score

Ray Mac — Tue, 08 Sep 2026 04:59:18 +0000

ScribeToAny is a full-stack audio and video transcription platform built on React 19 and deployed to Cloudflare Workers at the edge. Heavy GPU workloads (Whisper transcription, diarization, translation) run asynchronously on Modal, while our web application, authentication, database queries, and SSR are powered by Cloudflare Workers.

While our GPU pipeline was fast and asynchronous, our front-end performance audit delivered a harsh wake-up call: real-world user monitoring (Cloudflare Observatory) showed our 75th-percentile TTFB was 3,128 ms — with over 57% of hits rated "poor". Direct curl tests hitting cold edge nodes clocked TTFB between 3.4s and 3.6s for the homepage and SEO tool pages.

Yet, Cloudflare's internal Worker metrics showed a median CPU wall time of just 5 ms.

How could a Worker that finishes its work in 5 milliseconds take 3.5 seconds to return a response?

Here is the complete engineering breakdown of how we diagnosed the cold-start bottleneck, implemented zero-staleness edge HTML caching, decoupled our client bundles, slashed mobile hydration TBT, and took ScribeToAny to a 95+ Lighthouse performance score.

1. The Paradox: Why 5ms CPU Took 3.5s TTFB

Cloudflare Worker CPU metrics only record active handler execution. They do not account for isolate creation, bundle downloading, and V8 script compilation.

When we inspected our deployment pipeline, two factors collided:

An 11MB Worker Bundle: Because ScribeToAny includes rich SEO tool routes (over 80 audio/video conversion and transcription tools), markdown renderers, and format converter utilities, the bundled worker script reached ~11MB.
Low Baseline Traffic Density (~0.1 req/s): With sparse initial traffic, Cloudflare edge PoPs frequently evict idle V8 isolates.

Almost every new visitor — especially organic search traffic arriving from Google — hit a brand-new, cold isolate. Before running our 5ms SSR handler, V8 had to load and compile an 11MB JavaScript payload. The result was a 3-second cold-start penalty on first visit, completely undermining the user experience and SEO ranking potential.

Furthermore, on mobile devices, initial Lighthouse runs flagged heavy Total Blocking Time (TBT): third-party authentication scripts and oversized vendor chunks were monopolizing the main thread during hydration.

We solved this through a systematic four-layer optimization strategy.

2. Layer 1: Edge HTML Caching with Zero-Staleness Build Invalidation

Cloudflare Workers do not automatically cache dynamic SSR responses. Every incoming GET request was hitting our Worker, forcing a cold-start compilation.

Because our marketing pages, blog, and SEO tools are public and identical for anonymous visitors of the same URL, we implemented edge-level HTML caching directly in src/server.ts using the Workers Cache API (caches.default).

Strict Isolation & Bypass Rules

Edge caching dynamic web apps requires strict safety boundaries:

Session isolation: If the incoming request has a better-auth.session_token cookie, cache reading and writing are completely bypassed. Logged-in users always receive fresh, personalized SSR.
Strict allowlist: Caching is restricted strictly to anonymous public pages (/, /pricing, /about, /changelog, /tools/*, /blog/*, and legal pages). Dynamic routes (/dashboard, /api/*, /settings) are never cached.
Cache write condition: Only responses with HTTP status 200, Content-Type: text/html, and no Set-Cookie header are stored.

// src/server.ts (abridged)
const canCache =
  request.method === 'GET' &&
  !hasSessionCookie(request) &&
  isCacheablePath(deLocalizeUrl(new URL(request.url)).pathname);

const cache = (caches as unknown as { default: Cache }).default;
const cacheKey = canCache ? edgeCacheKey(request) : request;

if (canCache) {
  const hit = await cache.match(cacheKey);
  if (hit) {
    const headers = new Headers(hit.headers);
    headers.set('X-Edge-Cache', 'HIT');
    return new Response(hit.body, { status: hit.status, headers });
  }
}

Solving the Stale Content Problem

The biggest danger of caches.default on Cloudflare Workers is that a new Worker deployment does not purge the cache. With a standard URL key, publishing a new blog post or fixing a bug would leave stale HTML served to users for days.

We solved this by injecting a compile-time build identifier into the cache key:

// vite.config.ts
export default defineConfig({
  define: {
    __EDGE_BUILD_ID__: JSON.stringify(Date.now().toString(36)),
  },
  // ...
});

In src/server.ts, we construct an internal cache key request tagged with __EDGE_BUILD_ID__:

function edgeCacheKey(request: Request): Request {
  const url = new URL(request.url);
  url.searchParams.set('__ev', __EDGE_BUILD_ID__);
  return new Request(url.toString(), request);
}

This internal query parameter is only passed to cache.match and cache.put; it is never exposed to the client or upstream origin.

When a new version deploys, __EDGE_BUILD_ID__ changes. Old cache entries become instantly unreachable and expire naturally on their TTL, while the new release begins populating immediately. We can safely set s-maxage=86400 (24h) and stale-while-revalidate=604800 (7 days) without ever worrying about stale HTML after a release.

Result: Edge cache hits now return in under 45 milliseconds directly from the nearest Cloudflare edge PoP, completely bypassing Worker isolate cold starts.

3. Layer 2: Main Bundle Decoupling & Vendor Splitting

Serving HTML quickly is only half the battle; if the browser has to parse hundreds of kilobytes of unoptimized JavaScript before becoming interactive, the experience still stumbles.

Decoupling Global Configs

Our site configuration (src/config/website.ts) originally bundled metadata, navigation menus, multilingual pricing matrices, and 84 tool route names into one giant object. Because the root layout imported websiteConfig, every visitor downloaded definitions for 84 tools they hadn't visited.

We extracted tool metadata into src/config/navbar-tool-names.ts and pricing calculations into src/config/price-plans.ts, ensuring tool pages only load their own definitions on demand.

Vite Manual Chunks

By default, Vite bundled React, TanStack Query, and Zod into monolithic client bundles. In vite.config.ts, we configured explicit vendor chunks:

// vite.config.ts
build: {
  rollupOptions: {
    output: {
      manualChunks(id) {
        if (id.includes('@tabler/icons-react')) return 'vendor-icons';
        if (
          id.includes('node_modules/react/') ||
          id.includes('node_modules/react-dom/') ||
          id.includes('node_modules/scheduler/')
        ) return 'vendor-react';
        if (
          id.includes('node_modules/@tanstack/react-query') ||
          id.includes('node_modules/@tanstack/query-core')
        ) return 'vendor-query';
        if (id.includes('node_modules/zod/')) return 'vendor-zod';
      },
    },
  },
}

This isolates shared dependencies, improving long-term browser caching across route transitions and preventing small code changes from invalidating vendor libraries.

Scoping Global Providers

In src/routes/__root.tsx, Radix TooltipProvider originally wrapped the entire application tree. This forced React to initialize tooltip context on every public landing page even though tooltips were only used inside the logged-in dashboard and transcription editor. We removed TooltipProvider from the root route and scoped it exclusively to the editor and dashboard components.

Similarly, we extracted prose.css from the global styles.css. Markdown typography styling is now loaded strictly on blog, legal, and documentation routes, eliminating unused CSS overhead on marketing pages.

4. Layer 3: Offscreen Lazy-Loading & Slashing Hydration TBT

On mobile devices, client-side hydration was choking the CPU. When a browser downloads a page, hydrating every single DOM node on a long marketing page blocks user taps and scrolls (high Total Blocking Time).

Synchronous Above-the-Fold, Lazy Below-the-Fold

In src/components/blocks/homepage.tsx, we split the landing page into two categories:

Above-the-Fold (Synchronous): HeroSection, TrustStrip, and WhisperTechSection are imported synchronously. They render immediately during SSR and hydrate on frame 1 for instant First Contentful Paint (FCP).
Below-the-Fold (Lazy Loaded): FeaturesSection, Features2Section, StatsSection, CallToActionSection, PricingSection, FaqSection, and NewsletterCard are wrapped in React.lazy() and Suspense.

To prevent Cumulative Layout Shift (CLS) when these components resolve, each Suspense boundary has an explicit minimum-height skeleton fallback matching the component's rendered height:

// src/components/blocks/homepage.tsx
export function HomePage() {
  return (
    <div className="flex flex-col">
      <HeroSection />
      <TrustStrip />
      <WhisperTechSection />
      <Suspense fallback={<div className="min-h-[650px]" />}>
        <FeaturesSection />
      </Suspense>
      <Suspense fallback={<div className="min-h-[550px]" />}>
        <Features2Section />
      </Suspense>
      <Suspense fallback={<div className="min-h-[300px]" />}>
        <CallToActionSection />
      </Suspense>
      {/* ... */}
    </div>
  );
}

This guaranteed 0.00 CLS while cutting the initial hydration JavaScript payload by over 40%.

Deferring Google One Tap to Idle

One of the largest contributors to mobile Total Blocking Time was Google One Tap (authClient.oneTap()). Previously, the Google Identity Services script executed during initial React hydration, spinning up iframe bridges and network checks while the user was trying to interact with the hero section.

We moved Google One Tap out of the critical rendering path by wrapping it in requestIdleCallback (with an 8-second fallback timeout) and attaching one-time event listeners to user interactions (pointerdown, touchstart, scroll):

// src/routes/__root.tsx (abridged)
useEffect(() => {
  if (!isOneTapEnabled || isPending || session) return;

  let executed = false;
  const trigger = () => {
    if (executed) return;
    executed = true;
    cleanup();
    void authClient.oneTap();
  };

  const interactionEvents = ['pointerdown', 'touchstart', 'scroll'] as const;
  for (const evt of interactionEvents) {
    window.addEventListener(evt, trigger, { once: true, passive: true });
  }

  if (typeof window.requestIdleCallback === 'function') {
    window.requestIdleCallback(trigger, { timeout: 8000 });
  } else {
    setTimeout(trigger, 7000);
  }
  // ...
}, [isPending, session]);

The result: zero main-thread interference during initial paint.

5. Layer 4: LCP & Accessibility Polish

With TTFB and hydration fixed, we addressed visual rendering timing and audit metrics:

Critical CSS Preloading: Added <link rel="preload" as="style" href={appCss} /> in src/routes/__root.tsx to eliminate stylesheet render blocking.
Hero H1 Animation Delay Removal: In src/components/blocks/hero.tsx, our primary H1 heading previously had a subtle entrance fade-in animation delay. Removing the delay allowed Lighthouse to record Largest Contentful Paint (LCP) the instant the first paint completed.
Accessibility & Contrast: Fixed aria-orientation="horizontal" on button toggle groups, added descriptive aria-label tags, and increased primary button color contrast to meet WCAG AA standards.
Descriptive Anchor Text: Replaced ambiguous "Learn more" link texts with descriptive link destinations ("Security →"), satisfying Lighthouse SEO crawlability checks.

6. The Results: Lighthouse 95+ & Core Web Vitals

After deploying these four layers, we ran Google PageSpeed Insights on scribetoany.com.

Desktop Performance: 95 / 100

On desktop devices, our score surged to 95 Performance, 100 Accessibility, 96 Best Practices, and 100 SEO, with a 3/3 score on Agentic Browsing audits.

Metric	Score / Value	Status
Performance	95	🟢 Excellent
Accessibility	100	🟢 Perfect
Best Practices	96	🟢 Excellent
SEO	100	🟢 Perfect
Agentic Browsing	3 / 3	🟢 Passed
Cumulative Layout Shift (CLS)	0.00	🟢 Zero Shift

Mobile Performance: 78 / 100

On simulated mobile networks with aggressive 4G throttling and restricted mobile CPU profiles, our performance jumped to 78 Performance, alongside flawless 100 Accessibility and 100 SEO scores:

Metric	Before Optimization	After Optimization
P75 Edge TTFB	~3,128 ms (57% Poor)	< 50 ms (Edge HIT)
Cold Worker TTFB	~3,500 ms	Bypassed for all anon visitors
Desktop Performance	~68	95 🟢
Mobile Performance	~42	78 🟡
Cumulative Layout Shift	0.03	0.00 🟢
Accessibility	92	100 🟢
SEO	90	100 🟢

Key Lessons for Full-Stack Edge Applications

Building on Cloudflare Workers and a modern full-stack React framework gives you unprecedented global reach, but edge runtimes operate under different rules than traditional Node servers:

Beware the "Fast Handler, Slow Isolate" trap: If your Worker takes 5ms of CPU time but bundle size is 10MB+, your users are waiting on V8 compilation, not code execution.
Edge Cache is mandatory for SSR: You don't need a static site generator (SSG) to get static speeds. Using Cloudflare's caches.default with build-ID key invalidation gives you static speed with all the flexibility of SSR.
Guard your cache keys with build tags: Never deploy edge caches without an automated invalidation strategy. Folding compile-time hashes into internal cache keys guarantees zero-downtime freshness.
Hydrate lazily, paint immediately: Render above-the-fold components synchronously, defer offscreen blocks with explicit height skeletons, and push heavy third-party scripts (like Google Identity Services) to requestIdleCallback.

Speed is a feature. By aligning our edge architecture with browser execution priorities, ScribeToAny now delivers an instantaneous experience to users worldwide.

Running Whisper on Modal from Cloudflare Workers

Ray Mac — Thu, 03 Sep 2026 05:32:39 +0000

A Cloudflare Worker can't run a multi-minute GPU job, and it can't open the gRPC connection Modal's Python SDK uses to spawn() one. Two hard "no"s — and yet the transcription in ScribeToAny runs on Modal GPUs while the whole web app runs on Workers. The trick is to stop treating Modal as an SDK and start treating it as an HTTP endpoint you fire-and-forget, with the GPU box calling back over a signed webhook. Here's the whole design, with the edges that actually bit.

The web app runs entirely on Cloudflare Workers; Whisper (plus an optional translation pass) runs on GPUs on Modal. Getting two runtimes with opposite shapes to cooperate was the most interesting part of the build — because the obvious way is impossible on Workers.

The constraint

A Worker is not a server. It wakes up on a request, gets a small CPU budget, and is expected to return quickly. It has no long-lived process to babysit a job that takes minutes, and it can't open the gRPC connection Modal's Python SDK uses to call Function.spawn(). Two non-starters, same conclusion:

You cannot run the transcription in the Worker. A ten-minute podcast is not a request-scoped workload.
You cannot even use Modal's normal client to start the job. There's no gRPC, no Python, no persistent socket.

The naive version — "await the transcription and return the transcript" — dies on the first point. So the design has to be asynchronous from the very first line, and the Worker's entire job shrinks to a handful of sub-second HTTP calls.

The shape of the answer

Treat Modal not as an SDK but as an HTTP endpoint. Modal lets you expose a web endpoint that, when hit, spawns the real GPU function and returns immediately. So the flow becomes:

The browser uploads the media straight to R2 (never through the Worker).
The Worker presigns a read URL, writes a queued job row, and fires one POST at Modal's web endpoint. Modal acks with a call id and starts the GPU work in the background.
When the engine finishes (or fails, or just wants to report progress), it POSTs back to a webhook on the Worker, signed with a shared secret.
The frontend polls the job row and lights up when it flips to done.

The Worker only ever does three quick things: presign, spawn, apply-callback. None of them wait on a GPU. That's the whole trick — and the rest of the work is making it survive the real world, where webhooks get lost and callbacks arrive twice.

Uploading without touching the Worker

Media never streams through the Worker — that would blow the CPU budget and buy nothing. The browser gets a presigned R2 PUT and uploads directly. We issue it intent-first: a pending row is written before the URL is signed, so an upload that's abandoned mid-flight still leaves a trace we can sweep later.

// createUploadUrl (server function) — abridged
await db.insert(userFiles).values({ id, userId, r2Key, status: 'pending' /* … */ });
const uploadUrl = await presignR2Url(r2Key, 'PUT', PRESIGN_PUT_TTL);
return { fileId: id, uploadUrl };

After the PUT succeeds the client calls finalizeUpload, which flips pending → uploaded and corrects the size from R2's HEAD (never trust a client-reported byte count). Rows that never reach uploaded are garbage-collected by a cron sweep — more on that below.

Firing the job

transcribeFile is where the async handoff happens. It checks quota, presigns a GET so the engine can read the audio back out of R2, inserts a queued job behind an atomic concurrency guard, and spawns:

const audioUrl = await presignR2Url(file.r2Key, 'GET', PRESIGN_GET_TTL);
// … insert the queued job row (atomic guard) …
await spawnTranscription({ jobId, audioUrl, /* mode, language, targetLang … */ });
return { jobId, status: 'queued' };

The spawn itself is deliberately dumb: one fetch, and it resolves the moment Modal accepts the job — not when transcription finishes.

const res = await fetch(MODAL_TRANSCRIBE_URL, {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    // Modal Proxy Auth — token id/secret, not sent in the body
    'Modal-Key': MODAL_KEY,
    'Modal-Secret': MODAL_SECRET,
  },
  body: JSON.stringify({
    job_id: jobId,
    callback_url: `${APP_URL}/api/transcripts/webhook`,
    audio_url: audioUrl,
    model_size, beam_size, language,
    target_lang: targetLang, // null ⇒ no translation leg at all
  }),
});
if (!res.ok) throw new Error(`Failed to start transcription (${res.status})`);

Two things worth calling out. The callback URL is handed to the engine in the request, so the engine never has to know our topology. And the signing secret is pre-shared (a Modal secret that equals our MODAL_WEBHOOK_SECRET) — it is never put in a request body in either direction.

The callback: verify, then apply idempotently

Everything interesting now happens in the webhook. First, authenticate it. The engine signs the raw body with HMAC-SHA256 and sends the hex digest in X-Webhook-Signature. On Workers there's no Node crypto, so this is WebCrypto, and the comparison is timing-safe:

export async function verifyWebhookSignature(rawBody: string, signature: string | null) {
  if (!secret || !signature) return false;
  const key = await crypto.subtle.importKey(
    'raw', new TextEncoder().encode(secret),
    { name: 'HMAC', hash: 'SHA-256' }, false, ['sign'],
  );
  const sig = await crypto.subtle.sign('HMAC', key, new TextEncoder().encode(rawBody));
  const expected = [...new Uint8Array(sig)].map((b) => b.toString(16).padStart(2, '0')).join('');
  return timingSafeEqual(expected, signature.trim().replace(/^sha256=/, '').toLowerCase());
}

You have to hash the raw bytes, not a re-serialized object — JSON.parse then JSON.stringify will reorder keys and change whitespace, and your signature will never match. So the handler reads await request.text() and verifies before it parses.

Then apply the result. The single most important property here is idempotency, because a webhook you don't 200 fast enough gets retried, and a retried callback must not double-apply. The rule is one line: once a job is terminal, ignore repeats.

if (job.status === 'done' || job.status === 'failed') {
  return { ok: true, applied: false }; // already terminal — no-op
}

The HTTP status codes are chosen to steer the engine's retry behaviour:

Situation	Response	Why
Bad/missing signature	`401`	Reject outright
Unknown `job_id`	`200`	Ack so the engine stops retrying a job we'll never have
Applied (or already terminal)	`200`	Success
Our own DB threw	`500`	Ask the engine to retry — the callback was valid, we just fumbled it

That third row is the subtle one. An unknown job isn't an error to bubble up; it's a dead letter, and the kindest thing you can do is acknowledge it so the sender gives up.

The sharp edges

The happy path above is maybe a third of the code. The rest is everything that goes wrong when one side of an async contract can vanish.

Lost webhooks. If the engine crashes, or the callback is dropped, the job sits in transcribing forever. So a cron job reconciles: any job that's been quiet past a timeout is marked failed, and the same tick sweeps orphaned R2 uploads that never finalized. Workers cron triggers are perfect for this — it's the backstop that makes the optimistic async path safe to rely on.

Never kill a healthy job by accident. We also have a liveness probe that can ask Modal whether a call is still running. It returns a deliberately three-valued answer — running, a terminal state, or null meaning "don't know" (probe not configured, request failed, unparseable). Callers must treat null as no information and leave the job alone. Collapsing "I couldn't reach the probe" into "the job is dead" would have the reconciler executing healthy jobs the moment the probe has a bad minute.

Segments live in R2, not the database. A terminal callback does not ship the transcript inline. The engine writes segments to R2 as {job_id}.tsv; the done webhook carries only metadata (language, duration, cost, timing). The app reads the TSV on demand and generates SRT/VTT/TXT/PDF from it. Keeping thousands of cue rows out of D1 keeps the callback small and the job table narrow.

Two clocks in one job. Add the optional translation pass and a single job now finishes two things at different times. The transcript can be done while the translation is still running. So the translation update is applied before the terminal-state early-return — otherwise a translation ping arriving after the transcript finished would hit the "already terminal, no-op" branch and the translation row would be stuck at queued forever. Two independent legs, one job row, and the order of those two checks is load-bearing.

Why this is actually a good fit

It's tempting to read all this as fighting the platform. It isn't. Once the transcription is off the Worker, everything the Worker does keep is exactly what the Workers model is good at: short, stateless, I/O-bound HTTP handlers with a cron backstop and a durable store (D1 + R2) holding the state between them. The GPU box does GPU work; the edge does edge work; a signed webhook and an idempotent apply are the seam.

The constraint that looked fatal — "you can't run the job here" — turned out to be the thing that produced a clean design. The Worker never blocks, the job survives a dropped callback, and a retried webhook is a no-op. That's the whole system.

ScribeToAny is built on TanStack Start + React on Cloudflare Workers (D1 + R2), with the transcription engine on Modal. Originally posted on the ScribeToAny blog.