Running Whisper on Modal from Cloudflare Workers

Ray Mac — Thu, 03 Sep 2026 05:32:39 +0000

A Cloudflare Worker can't run a multi-minute GPU job, and it can't open the gRPC connection Modal's Python SDK uses to spawn() one. Two hard "no"s — and yet the transcription in ScribeToAny runs on Modal GPUs while the whole web app runs on Workers. The trick is to stop treating Modal as an SDK and start treating it as an HTTP endpoint you fire-and-forget, with the GPU box calling back over a signed webhook. Here's the whole design, with the edges that actually bit.

The web app runs entirely on Cloudflare Workers; Whisper (plus an optional translation pass) runs on GPUs on Modal. Getting two runtimes with opposite shapes to cooperate was the most interesting part of the build — because the obvious way is impossible on Workers.

The constraint

A Worker is not a server. It wakes up on a request, gets a small CPU budget, and is expected to return quickly. It has no long-lived process to babysit a job that takes minutes, and it can't open the gRPC connection Modal's Python SDK uses to call Function.spawn(). Two non-starters, same conclusion:

You cannot run the transcription in the Worker. A ten-minute podcast is not a request-scoped workload.
You cannot even use Modal's normal client to start the job. There's no gRPC, no Python, no persistent socket.

The naive version — "await the transcription and return the transcript" — dies on the first point. So the design has to be asynchronous from the very first line, and the Worker's entire job shrinks to a handful of sub-second HTTP calls.

The shape of the answer

Treat Modal not as an SDK but as an HTTP endpoint. Modal lets you expose a web endpoint that, when hit, spawns the real GPU function and returns immediately. So the flow becomes:

The browser uploads the media straight to R2 (never through the Worker).
The Worker presigns a read URL, writes a queued job row, and fires one POST at Modal's web endpoint. Modal acks with a call id and starts the GPU work in the background.
When the engine finishes (or fails, or just wants to report progress), it POSTs back to a webhook on the Worker, signed with a shared secret.
The frontend polls the job row and lights up when it flips to done.

The Worker only ever does three quick things: presign, spawn, apply-callback. None of them wait on a GPU. That's the whole trick — and the rest of the work is making it survive the real world, where webhooks get lost and callbacks arrive twice.

Uploading without touching the Worker

Media never streams through the Worker — that would blow the CPU budget and buy nothing. The browser gets a presigned R2 PUT and uploads directly. We issue it intent-first: a pending row is written before the URL is signed, so an upload that's abandoned mid-flight still leaves a trace we can sweep later.

// createUploadUrl (server function) — abridged
await db.insert(userFiles).values({ id, userId, r2Key, status: 'pending' /* … */ });
const uploadUrl = await presignR2Url(r2Key, 'PUT', PRESIGN_PUT_TTL);
return { fileId: id, uploadUrl };

After the PUT succeeds the client calls finalizeUpload, which flips pending → uploaded and corrects the size from R2's HEAD (never trust a client-reported byte count). Rows that never reach uploaded are garbage-collected by a cron sweep — more on that below.

Firing the job

transcribeFile is where the async handoff happens. It checks quota, presigns a GET so the engine can read the audio back out of R2, inserts a queued job behind an atomic concurrency guard, and spawns:

const audioUrl = await presignR2Url(file.r2Key, 'GET', PRESIGN_GET_TTL);
// … insert the queued job row (atomic guard) …
await spawnTranscription({ jobId, audioUrl, /* mode, language, targetLang … */ });
return { jobId, status: 'queued' };

The spawn itself is deliberately dumb: one fetch, and it resolves the moment Modal accepts the job — not when transcription finishes.

const res = await fetch(MODAL_TRANSCRIBE_URL, {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    // Modal Proxy Auth — token id/secret, not sent in the body
    'Modal-Key': MODAL_KEY,
    'Modal-Secret': MODAL_SECRET,
  },
  body: JSON.stringify({
    job_id: jobId,
    callback_url: `${APP_URL}/api/transcripts/webhook`,
    audio_url: audioUrl,
    model_size, beam_size, language,
    target_lang: targetLang, // null ⇒ no translation leg at all
  }),
});
if (!res.ok) throw new Error(`Failed to start transcription (${res.status})`);

Two things worth calling out. The callback URL is handed to the engine in the request, so the engine never has to know our topology. And the signing secret is pre-shared (a Modal secret that equals our MODAL_WEBHOOK_SECRET) — it is never put in a request body in either direction.

The callback: verify, then apply idempotently

Everything interesting now happens in the webhook. First, authenticate it. The engine signs the raw body with HMAC-SHA256 and sends the hex digest in X-Webhook-Signature. On Workers there's no Node crypto, so this is WebCrypto, and the comparison is timing-safe:

export async function verifyWebhookSignature(rawBody: string, signature: string | null) {
  if (!secret || !signature) return false;
  const key = await crypto.subtle.importKey(
    'raw', new TextEncoder().encode(secret),
    { name: 'HMAC', hash: 'SHA-256' }, false, ['sign'],
  );
  const sig = await crypto.subtle.sign('HMAC', key, new TextEncoder().encode(rawBody));
  const expected = [...new Uint8Array(sig)].map((b) => b.toString(16).padStart(2, '0')).join('');
  return timingSafeEqual(expected, signature.trim().replace(/^sha256=/, '').toLowerCase());
}

You have to hash the raw bytes, not a re-serialized object — JSON.parse then JSON.stringify will reorder keys and change whitespace, and your signature will never match. So the handler reads await request.text() and verifies before it parses.

Then apply the result. The single most important property here is idempotency, because a webhook you don't 200 fast enough gets retried, and a retried callback must not double-apply. The rule is one line: once a job is terminal, ignore repeats.

if (job.status === 'done' || job.status === 'failed') {
  return { ok: true, applied: false }; // already terminal — no-op
}

The HTTP status codes are chosen to steer the engine's retry behaviour:

Situation	Response	Why
Bad/missing signature	`401`	Reject outright
Unknown `job_id`	`200`	Ack so the engine stops retrying a job we'll never have
Applied (or already terminal)	`200`	Success
Our own DB threw	`500`	Ask the engine to retry — the callback was valid, we just fumbled it

That third row is the subtle one. An unknown job isn't an error to bubble up; it's a dead letter, and the kindest thing you can do is acknowledge it so the sender gives up.

The sharp edges

The happy path above is maybe a third of the code. The rest is everything that goes wrong when one side of an async contract can vanish.

Lost webhooks. If the engine crashes, or the callback is dropped, the job sits in transcribing forever. So a cron job reconciles: any job that's been quiet past a timeout is marked failed, and the same tick sweeps orphaned R2 uploads that never finalized. Workers cron triggers are perfect for this — it's the backstop that makes the optimistic async path safe to rely on.

Never kill a healthy job by accident. We also have a liveness probe that can ask Modal whether a call is still running. It returns a deliberately three-valued answer — running, a terminal state, or null meaning "don't know" (probe not configured, request failed, unparseable). Callers must treat null as no information and leave the job alone. Collapsing "I couldn't reach the probe" into "the job is dead" would have the reconciler executing healthy jobs the moment the probe has a bad minute.

Segments live in R2, not the database. A terminal callback does not ship the transcript inline. The engine writes segments to R2 as {job_id}.tsv; the done webhook carries only metadata (language, duration, cost, timing). The app reads the TSV on demand and generates SRT/VTT/TXT/PDF from it. Keeping thousands of cue rows out of D1 keeps the callback small and the job table narrow.

Two clocks in one job. Add the optional translation pass and a single job now finishes two things at different times. The transcript can be done while the translation is still running. So the translation update is applied before the terminal-state early-return — otherwise a translation ping arriving after the transcript finished would hit the "already terminal, no-op" branch and the translation row would be stuck at queued forever. Two independent legs, one job row, and the order of those two checks is load-bearing.

Why this is actually a good fit

It's tempting to read all this as fighting the platform. It isn't. Once the transcription is off the Worker, everything the Worker does keep is exactly what the Workers model is good at: short, stateless, I/O-bound HTTP handlers with a cron backstop and a durable store (D1 + R2) holding the state between them. The GPU box does GPU work; the edge does edge work; a signed webhook and an idempotent apply are the seam.

The constraint that looked fatal — "you can't run the job here" — turned out to be the thing that produced a clean design. The Worker never blocks, the job survives a dropped callback, and a retried webhook is a no-op. That's the whole system.

ScribeToAny is built on TanStack Start + React on Cloudflare Workers (D1 + R2), with the transcription engine on Modal. Originally posted on the ScribeToAny blog.

DEV Community: Ray Mac