Building an AI Video App on Cloudflare Workers with TanStack Start

songxiaoman — Fri, 30 Oct 2026 03:12:46 +0000

I run a small AI video web app: users upload a character photo and a short motion clip, and a hosted video model renders the character performing those moves. The whole thing (UI, API, auth, payments, storage and the job pipeline) runs as one Cloudflare Worker. No servers, no containers, no separate queue service.

This post walks through the architecture and the handful of problems that turned out to be interesting. If you're considering Workers for anything that handles large files or long-running third-party jobs, most of this transfers.

The stack

TanStack Start + React 19 for SSR and file-based routing, built with Vite and the Cloudflare Vite plugin
A single Worker entry (src/server.ts) that hands requests to TanStack Start's handler
D1 (SQLite) through Drizzle ORM for users, credits and jobs
R2 for uploads and finished videos
KV as a cache
Workers AI for a vision-model safety check on uploaded photos
Better Auth for email/password and Google sign-in
Cloudflare Email for transactional mail, behind a small provider interface
GitHub Actions deploying with wrangler deploy on every push to main, plus per-PR preview URLs

TanStack Start's server functions (createServerFn with a Zod validator and middleware chain) replace a separate API layer. The client calls them like ordinary async functions, and they run inside the same Worker with direct access to the bindings.

Problem 1: large uploads without blowing up memory

Motion clips can be tens of megabytes. Buffering that in a Worker is a bad idea. Instead, the upload route takes the raw file as the request body and streams it straight into R2:

// simplified
const key = `uploads/${userId}/${crypto.randomUUID()}.mp4`;
await env.BUCKET.put(key, request.body, {
  httpMetadata: { contentType: request.headers.get('content-type') ?? '' },
});
return Response.json({ key });

The client then passes that key to the "start job" server function. Uploads are deleted as soon as a job has used them, and an R2 lifecycle rule removes anything older than a day as a safety net. That keeps storage costs flat and means user-provided material doesn't sit around.

Problem 2: pricing a video by length without downloading it

Pricing is per second of motion video, so the server needs the clip's duration before charging. Reading the entire file just to get one number defeats the point of streaming.

MP4 and MOV files store their duration in the mvhd box inside moov. So the server uses R2 range reads to walk the top-level box headers (8â€“16 bytes each), skips over mdat without reading it, fetches only the moov box, and parses timescale and duration from mvhd. Typically that's a few kilobytes read instead of the whole file. When the job is submitted, the video body goes to the model provider as a stream, again never held whole in memory.

Problem 3: jobs that outlive the request

Video rendering takes a few minutes, far longer than a request should stay open. Queues or Durable Objects would work, but a simpler pattern is enough:

Start: one request validates input, runs the photo check, charges credits, submits to the provider and writes a row to an effect_job table with the provider's request ID.
Advance on poll: the client polls a status server function. Each poll asks the provider for status and moves the job forward. When it's done, the result is copied into R2 under the user's files. When it fails, the credits are refunded.
Notify: on completion, an email goes to the user ("your video is ready") so they can close the tab.

Because the status call is idempotent (it only moves a job forward from its stored state), duplicate polls from multiple tabs are harmless.

Problem 4: refunds that happen minutes later

The codebase has a withCredits helper that charges, runs a callback and refunds if the callback throws. That's perfect for synchronous AI calls but useless here: the job can fail five minutes after the request that charged for it has finished.

The fix was to store the spend on the job row. When a poll discovers a failure, it refunds exactly what that job cost and marks the row as refunded, so the refund can't happen twice. Treat the job row as the source of truth for money, not the call stack.

Problem 5: safety checks before charging

Every photo passes through a Workers AI vision model via the AI binding before any credits move. It flags sexual content and recognizable public figures (both always refused) and applies extra rules for photos that appear to include minors. If the check itself errors, the photo is refused rather than let through. Running it first means users are never charged for a request that would be rejected anyway.

Problem 6: private files, shareable thumbnails

Finished videos are private. They're served from a route that checks the owner's session. But generating a first-frame poster image uses Cloudflare Media Transformations, which needs to fetch the video by URL. The answer was short-lived signed links: the video route accepts either the owner's session or a valid signature, and the poster pipeline uses the latter, then stores the WebP through the Images binding.

Local development against remote resources

One opinionated choice: local dev and end-to-end tests connect to the remote D1, R2 and KV ("remote": true on each binding in wrangler.jsonc) rather than local emulators. That removes a whole class of "works locally" bugs around SQLite versions and R2 behaviour, at the cost of needing a dev database. For a solo project that trade has been worth it.

Would I do it again?

Yes. The single-Worker model keeps operations close to zero: one deploy, one log stream, costs that scale with usage, and no idle servers. Its constraints (no Node-only APIs, careful memory use, request time limits) pushed the design toward streaming and stateless polling, which ended up simpler than a queue-based design.

If you want to see the result, the app is live at MotionControlAI. It's an independent project, not affiliated with Kling AI or Kuaishou, even though it runs their motion control models. There's a plain-English guide to how Kling Motion Control works if you're curious about the model side.

DEV Community: songxiaoman