Routing Between Video Models Without Leaking the Abstraction

leojxu — Fri, 24 Jul 2026 19:02:37 +0000

If you build on a single video generation model, you will rewrite your integration when it gets deprecated. If you build on several, you spend your time writing adapters. Here is what I learned choosing the second option.

The parameters do not line up

Every provider has a different idea of the same concept:

Duration: some take seconds, some take frame counts, some only accept a fixed enum
Aspect ratio: "16:9" vs {width, height} vs a named preset
Seeds: supported, ignored, or silently non-deterministic
Image conditioning: URL, base64, or a pre-uploaded asset ID

A lowest-common-denominator interface is tempting and wrong — you lose the capabilities you are paying for. The alternative that held up: a common core plus a typed passthrough.

type VideoRequest = {
  prompt: string
  durationSec: number        // normalised, adapter converts
  aspect: '16:9' | '9:16' | '1:1'
  image?: ImageInput
  providerOptions?: Record<string, unknown>  // escape hatch, typed per adapter
}

The core covers what every provider supports. The escape hatch means using a provider-specific feature does not require redesigning the interface.

Async is the actual hard part

Video generation takes minutes, so every provider is asynchronous — and each has invented its own async:

Polling with a job ID
Webhook callbacks
Server-sent events
A queue position field that sometimes goes backwards

Normalising this matters more than normalising parameters. What worked: adapters expose a single poll(jobId): Promise<JobState> and internally hide whatever the provider does. JobState is a small discriminated union — queued | running | done | failed — with a normalised progress when available and undefined when not.

Do not fabricate progress. A spinner that lies is worse than a spinner that says "still working."

Failure modes deserve normalising too

Providers fail differently: content policy rejections, transient 5xx, quota exhaustion, silent truncation. Mapping these into a common taxonomy is what makes retry logic possible:

retryable — backoff and retry
policy — do not retry, surface to user
quota — do not retry, switch provider or fail loudly

Getting this wrong means retrying a policy rejection 5 times and burning credits on a request that will never succeed.

Credits across providers

If you expose one balance over several backends with different pricing, you need a normalisation table and you need to be honest that it is an approximation. I run this in production at HyperFrames — text or image in, routed across Kling, Veo, Wan and others, one balance.

Caveat