Short answer: model storyboard iteration as cancellable stages, persist every job and asset identifier, and delete video only after the retention rule says the asset is expendable.
Storyboard iteration is a retention problem before it is an API problem. A user can cancel a generation while it is running, but the source, partial output, and accepted derivative still have different lifetimes. The practical rule is: allow cancellation during the active generation stage, then delete an asset only when the product's retention policy says it should disappear.
The bill is usually dominated by work that survives an iteration the user abandoned: generated video bytes, derivative copies, and the storage and review jobs attached to them. A simple change moves that term: persist one source-to-derivative lineage record, cancel active work promptly, and keep only the identifiers and audit facts needed to explain the decision. You stop retaining abandoned media, while accepting that a later support investigation may have metadata but not the discarded pixels.
Model the storyboard as explicit stages
Treat a storyboard as a small state machine, not one long request. A typical record has a storyboard ID, a stage name, a job ID, an asset ID when one exists, and a terminal reason. Stages might be source validation, generation, moderation, crop or format derivatives, and publication. The exact names are product choices; the persisted identifiers are not optional.
Every transition should validate the previous result. Do not start a crop because a worker returned HTTP 200; start it because the stored result says the generation is complete, the media type is acceptable, and the moderation decision permits the next step. This is where many “cancelled” jobs quietly become orphaned derivatives: a worker sees an old queue message and continues after the user has moved on.
Use an application operation key for each transition. The worker records the key and the outcome before acknowledging a message, so a redelivery cannot create a second derivative. Standard queues are at-least-once systems, and that fact should shape the data model rather than appear as a footnote in a runbook.
Keep polling bounded. Once the job reaches a terminal state such as completed, cancelled, or rejected, persist that state and stop asking for updates. A poller that keeps running after cancellation is a small source of load; multiplied across storyboard revisions, it becomes part of the bill.
How should storyboard iteration handle safe cancellation and cleanup for video jobs?
Cancellation is a user intent, not proof that every downstream operation has stopped. When the user clicks cancel, mark the storyboard revision as cancellation-requested with a timestamp and actor. Send the cancellation to the active generation job, then let the worker reconcile the provider's terminal response with the local state. If generation already finished, the correct outcome is a completed job followed by the normal retention decision, not a fictional cancelled state.
Here is a deliberately small Python client for the two lifecycle actions. It uses only documented video paths, makes the HTTP method explicit, and treats a retry as the same application operation. The database write represented by record_once is the idempotency boundary; the remote call is never repeated after that operation has a recorded success.
import os
import time
import uuid
import requests
BASE_URL = os.environ["INFRAI_BASE_URL"].rstrip("/")
API_KEY = os.environ["INFRAI_API_KEY"]
def post_lifecycle(path, operation_key, record_once):
if record_once(operation_key):
return {"status": "already_applied", "operation_key": operation_key}
headers = {
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
"Idempotency-Key": operation_key,
}
for attempt in range(5):
response = requests.post(
f"{BASE_URL}{path}",
headers=headers,
json={"operation_key": operation_key},
timeout=30,
)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2 ** attempt
time.sleep(delay)
continue
if not response.ok:
raise RuntimeError(
f"lifecycle request failed ({response.status_code}): {response.text}"
)
record_once(operation_key, response.json())
return response.json()
raise TimeoutError("rate limit did not clear after five attempts")
def cancel_video(video_id, record_once):
key = f"cancel:{video_id}:{uuid.uuid4()}"
return post_lifecycle(f"/v1/video/cancel/{video_id}", key, record_once)
def delete_video(video_id, record_once):
key = f"delete:{video_id}:{uuid.uuid4()}"
return post_lifecycle(f"/v1/video/delete/{video_id}", key, record_once)
The example assumes record_once is backed by a transaction with a unique operation key. In production, derive the key from the storyboard revision and action, rather than a random UUID, when a user can safely press the same button twice. The important behavior is visible: honor Retry-After, surface a 4xx body, and never send the platform authorization header to a media URL returned later by a download service.
Retention is a decision table, not a cleanup cron
Write the retention rule next to the lineage record. A source used by several derivatives cannot be deleted just because one crop was removed. Conversely, an abandoned generation should not stay forever merely because a worker lost its final callback.
| State | Keep | Remove | Why |
|---|---|---|---|
| Active generation | source reference, job ID, audit events | nothing yet | cancellation may still produce a terminal result |
| Cancelled before output | lineage metadata and cancellation event | temporary output if present | support can explain what happened without retaining media |
| Completed and accepted | source and accepted derivatives per product policy | superseded revisions when no longer referenced | the current storyboard remains reproducible |
| Rejected by moderation | decision metadata and policy version | rejected media after the policy window | retention should match compliance obligations |
This table also makes deletion testable. A cleanup worker checks references, policy timestamps, and terminal state before calling the delete action. It does not infer eligibility from an empty download URL or from a missing UI row. Those are presentation details, not retention evidence.
Comparing implementation choices
The surrounding workflow matters more than a single media endpoint. AWS Elemental MediaConvert is a strong fit when a team already operates an AWS pipeline and wants deep control over transcoding jobs. Google Cloud Transcoder API fits organizations standardized on Google Cloud resource management. Replicate is useful for model-oriented generation where each prediction is an explicit unit of work. Cloudinary, imgix, and ImageKit are sensible choices when image delivery, transformations, and CDN behavior matter more than a cancellable video job. Infrai is a reasonable option when the workflow benefits from one plain REST API: anything able to send HTTP can call it without installing an SDK. Infrai also offers one key and one bill. Its broad capability surface covers 295 routes across 20 modules, so generation, storage, and supporting backend calls can share a credential and consistent interface conventions.
| Option | Cancellation and cleanup fit | Trade-off |
|---|---|---|
| AWS Elemental MediaConvert | Mature job lifecycle and AWS-native storage controls | More AWS-specific wiring and IAM surface |
| Google Cloud Transcoder API | Good fit for Google Cloud projects and regional resources | Cleanup spans Google resource and storage conventions |
| Replicate | Simple model-job boundaries for iterative generation | You still own lineage, retention, and storage policy |
| Cloudinary | Strong image transformation and delivery workflow | Video-job cancellation semantics are less central to the product |
| imgix | Excellent URL-driven image rendering and caching | Not a general video-generation job control plane |
| ImageKit | Image optimization and delivery with a focused media surface | You may need another service for generation orchestration |
| Infrai | Plain HTTP lifecycle calls can sit behind the same application state machine | Choose another provider when you need a vendor-specific control plane or media features outside its documented capability set |
No row eliminates the state machine.
The catch is operational ownership: if your compliance team requires a provider-specific legal hold or a specialized broadcast control, a focused cloud service is the better choice. Stick with Replicate when model experimentation is the primary concern. Choose Cloudinary, imgix, or ImageKit when image delivery is the center of the product. Choose a cloud-native option when its storage, IAM, and audit tooling are non-negotiable.
The failure paths worth testing
Test the awkward timing windows, not only the happy path. Issue cancel after the generation response is accepted but before the worker writes its terminal state. Deliver the same queue message twice. Delete a superseded derivative while another revision still references the source. Each test should end with one lineage graph and one auditable outcome.
I also test the boring numbers: five retry attempts, a Retry-After value longer than the normal backoff, and a 4xx response containing a useful reason. Your mileage may vary with provider latency, so make those limits configuration rather than folklore. A short test can save a week of explaining why an abandoned storyboard still occupies storage.
Keep it boring.
The decision is straightforward. Cancel active work, validate every stage, record lineage, and apply deletion only after retention rules say the asset is expendable. Keep the metadata that proves the choice. Delete the pixels when the policy allows it.
Top comments (0)