A video generation request can time out after the provider has accepted it. If you treat that timeout as permission to submit again, you can end up paying for two generations while your user waits for one.
That's a failure case I wanted to account for while building Crazy Frog AI Video Generator. The interface takes two photos, assigns a rider and a chaser, and creates an invisible motorcycle chase. Behind it, I use TypeScript on Cloudflare Workers, D1 for job state, and R2 for the finished media.
The part I want to share is how I separate a browser retry, a provider status check, and a new paid task. They need different rules.
Keep the browser request attached to one job
The client supplies a request ID. On the server, I validate the inputs and use that ID as the job ID. I store the validated inputs and the provider request body before contacting the video service.
When the browser repeats a request with the same ID, the server checks the owner and stored inputs. Matching requests return the existing job. Reusing the ID with different inputs produces an error.
This handles repeated HTTP requests on our side. The remote submission still needs its own recovery path: our database and the provider don't share a transaction.
Record submission intent before the network call
Before creating a provider task, the worker writes a submission timestamp to the job row. It stores the provider's task ID when the response arrives.
A later worker can make a decision from those two fields:
| Submission marker | Provider task ID | Next action |
|---|---|---|
| Absent | Absent | Claim the job and attempt the first submission |
| Present | Present | Check that existing task |
| Present | Absent | Pause for reconciliation; don't submit another task |
The third row includes an uncomfortable case. A worker might save the marker and stop before sending the request. We then pause a job that the provider never received.
I accept that tradeoff in the current implementation. Without a verified way to deduplicate a provider submission, I can't distinguish that case from a provider accepting the request and losing the response. For an ambiguous submission, the app keeps the video allowance reserved and shows a support-review message. A confirmed rejection follows a separate failure path that restores the allowance.
If I add a provider-supported idempotency key later, I'll need to verify its scope and retention window before changing this behavior.
A lease needs protection against late writes
A worker claims a pending job with a two-minute lease. Another worker can take over after that lease expires.
That creates a second problem: the first worker may still be waiting on a provider or storage operation. When it resumes, it must not overwrite the newer worker's state.
I use an incrementing claim token, stored in the attempts column. Claiming a job increments it. State updates check both that token and the lease. This is the guard used by the update helper, with the SET clause omitted:
WHERE id = ?
AND attempts = ?
AND lease_until > ?
AND status IN ('queued', 'processing', 'composing', 'saving')
RETURNING *
I also include the claim token in output object names. If an older worker finishes an upload late, it writes to a separate object rather than replacing the newer output.
These checks protect local state and files. They don't make an external API call exactly-once, so the submission marker still matters.
Recover each stage on its own
The workflow has separate stages:
queued -> processing -> composing -> saving -> succeeded
The video provider generates silent output. A separate composition task adds the selected scene's audio. The app then downloads and validates the result and stores it in R2 before marking the job as succeeded.
If storage fails after composition, recovery can continue from the existing task and result. Starting video generation over would repeat the most expensive part of the workflow.
Audio composition also has a submission marker and task ID. Moving a paid operation into a later stage doesn't remove the lost-response problem.
For callbacks, I verify the request, identify the stored job, and wake processing. The worker asks the provider for current task state rather than accepting the callback's output as final. A reconciliation pass checks pending jobs too, with backoff and jitter between attempts.
What I'd check before changing the retry policy
The cases I care about include a repeated client request, a timeout after provider acceptance, a worker resuming after its lease expires, and an R2 write failing after composition. Each case needs a different recovery action.
I haven't removed the manual work from ambiguous submissions. For now, I prefer that visible operational limit to creating a second paid task because the first response went missing.
You can see the product this supports at crazy-frog-ai.org. If you've handled ambiguous submissions with a provider that offers limited task lookup, I'd be interested in how you reconcile them.
Top comments (0)