Routing Between Video Models Without Leaking the Abstraction

If you build on a single video generation model, you will rewrite your integration when it gets deprecated. If you build on several, you spend your time writing adapters. Here is what I learned choosing the second option.

The parameters do not line up

Every provider has a different idea of the same concept:

• Duration: some take seconds, some take frame counts, some only accept a fixed enum
• Aspect ratio: "16:9" vs {width, height} vs a named preset
• Seeds: supported, ignored, or silently non-deterministic
• Image conditioning: URL, base64, or a pre-uploaded asset ID

A lowest-common-denominator interface is tempting and wrong — you lose the capabilities you are paying for. The alternative that held up: a common core plus a typed passthrough.

type VideoRequest = {
  prompt: string
  durationSec: number        // normalised, adapter converts
  aspect: '16:9' | '9:16' | '1:1'
  image?: ImageInput
  providerOptions?: Record<string, unknown>  // escape hatch, typed per adapter
}

The core covers what every provider supports. The escape hatch means using a provider-specific feature does not require redesigning the interface.

Async is the actual hard part

Video generation takes minutes, so every provider is asynchronous — and each has invented its own async:

• Polling with a job ID
• Webhook callbacks
• Server-sent events
• A queue position field that sometimes goes backwards

Normalising this matters more than normalising parameters. What worked: adapters expose a single poll(jobId): Promise<JobState> and internally hide whatever the provider does. JobState is a small discriminated union — queued | running | done | failed — with a normalised progress when available and undefined when not.

Do not fabricate progress. A spinner that lies is worse than a spinner that says "still working."

Failure modes deserve normalising too

Providers fail differently: content policy rejections, transient 5xx, quota exhaustion, silent truncation. Mapping these into a common taxonomy is what makes retry logic possible:

• retryable — backoff and retry
• policy — do not retry, surface to user
• quota — do not retry, switch provider or fail loudly

Getting this wrong means retrying a policy rejection 5 times and burning credits on a request that will never succeed.

Credits across providers

If you expose one balance over several backends with different pricing, you need a normalisation table and you need to be honest that it is an approximation. I run this in production at HyperFrames — text or image in, routed across Kling, Veo, Wan and others, one balance.

Caveat

This abstraction is worth it above roughly three providers. Below that, adapters cost more than they save — just write the integrations directly and move on.