I build an image-to-video tool. The user uploads a photo, describes the motion, picks a model, and gets an MP4 back. Six models sit behind that picker: Kling O3, Kling 2.6, Wan 2.6, MiniMax H3, Seedance 2.5 and Seedance 2.0.
All six go through one aggregator, one endpoint: POST /api/v1/jobs/createTask, with a model slug and an input object.
Same endpoint, same auth, same response envelope. So I assumed the input object was roughly the same shape too, with a few extra knobs per model.
That assumption cost me about two days. Here is every place the contract actually diverged, and what I changed structurally so it stopped hurting.
1. The same field, two different types
duration is the one field every model needs. Every model wants it differently:
// Kling O3, Kling 2.6, Wan 2.6
duration: String(durationSeconds), // "5"
// MiniMax H3, Seedance 2.5, Seedance 2.0
duration: durationSeconds, // 5
There's no pattern to infer from. It doesn't even split by vendor — Wan and Kling come from different companies and both want strings; MiniMax and ByteDance are different companies and both want integers.
The part that burned the most time: the playground docs for Wan 2.6 showed an integer. The live API rejected it with
duration it must be a string
A malformed error message, contradicting the published docs, on a field I had already "verified" by reading them.
The lesson: for a third-party API, the docs describe intent and the live endpoint describes the contract. When they disagree, the endpoint wins. I now record the date I live-tested each field, right next to the field:
// Live API rejects integers ("duration it must be a string") — like the
// Kling family, despite the playground docs. Verified 2026-08-19.
duration: String(durationSeconds), // "5" | "10" | "15"
That comment is worth more than the code. Six months from now the only question that matters is "when did we last confirm this, and against what?"
2. Defaults that quietly spend your money
This is the one I'd want someone to tell me before I shipped.
Three of the six models have a billing-relevant field with a non-cheapest default:
| Model | Field | Upstream default | Effect if you omit it |
|---|---|---|---|
| Wan 2.6 | resolution |
1080p |
~50% more per second than 720p |
| MiniMax H3 | resolution |
2K |
~63% more per second than 768P |
| Seedance 2.5 / 2.0 | generate_audio |
true |
extra render work, on by default |
None of these are documented as "this is the expensive default." They're documented as defaults.
If you build the naive wrapper — send only the fields the user actually chose, let the provider fill in the rest — you get a working product that silently bills at a higher tier on every request where the user picked the cheap option and you didn't forward it.
The fix is trivial and the rule is absolute:
For any field that affects billing, always send it explicitly — including when the value you want is the documented default.
Defaults are the vendor's business decision, not a stable part of your contract. They can change in a release note you don't read.
3. One concept, two magic words, both mandatory
Both Kling O3 and the Seedance family need to be told that the output ratio follows the source photo. They spell it differently, and both reject everything else:
// Kling O3, single-image image-to-video
aspect_ratio: "auto",
// → anything else: "aspect_ratio must be auto for image-to-video
// without custom multi-shot"
// Seedance 2.5 / 2.0, first-frame tasks
aspect_ratio: "adaptive",
// → "first-frame and first-last-frame tasks only support adaptive
// aspect ratio"
The trap underneath: the playground pages for both models list a full menu of ratios — 16:9, 9:16, 1:1. That menu is real. It applies to text-to-video. In image-to-video the legal set collapses to exactly one value, and the docs don't split the two modes.
Generalizes to: a capability list in vendor docs is the union across all modes. The legal set for your mode is a subset, and it's usually only discoverable by getting rejected.
I stopped building a ratio selector for these models entirely. The UI hides the row and the output follows the photo — which is what the product promises anyway, so the constraint and the feature happened to agree. That's luck, not design.
4. The same toggle, three different cost curves
Audio is one checkbox in my UI. Behind it:
- Kling O3 — a per-second surcharge at 720p and 1080p, and free at 4K. The cost of the toggle is non-monotonic in quality.
- Kling 2.6 — audio doubles the per-clip price.
- Seedance 2.5 / 2.0 — billed by resolution only. Audio is free.
- Wan 2.6 — audio is built in and can't be turned off.
- MiniMax H3 — silent output, no audio at all.
Five behaviours across six models, for one boolean.
My first implementation had a single audioSurcharge constant. That's wrong in five of six cases. What it has to be is a per-model, per-resolution-tier value plus a capability enum:
audio: "none" | "builtin" | "toggle";
"builtin" renders a badge, "none" hides the control, "toggle" renders a switch and reads the surcharge off the selected tier. The UI stopped having opinions about audio; it renders whatever the spec declares.
5. HTTP 200 is not success
Both endpoints return 200 OK for business-logic failures. The real status is in the body:
if (resp.code !== 200 || !resp.data?.taskId) {
throw new KieError(`createTask failed: ${resp.msg}`, undefined, resp);
}
And the result URL is nested one layer deeper than you'd expect — inside resultJson, which is a JSON string, whose field name varies by model. So the extractor has to try several shapes:
const obj = JSON.parse(info.resultJson);
const candidates = [obj.resultUrls, obj.imageUrls, obj.images, obj.urls, obj.output];
There's one guard in there worth calling out, because getting it wrong is expensive in a specific, silent way. Every candidate has to be checked for startsWith("http") — including the branch that digs a url out of an array of objects:
if (typeof u === "string" && u.startsWith("http")) return u;
Without that predicate, a relative path or an odd scheme gets returned as a "URL". It then flows into the persist step, fails the fetch, and gets stored as the result by the fallback path. The user is charged, the link is dead, and no refund fires — because the "success but no URL" guard never trips. A URL was extracted. It just wasn't one.
That's the shape of bug I now actively hunt for: a validation gap that converts a loud failure into a paid-for silent one.
What generalizes
Every one of these differences ended up in the same place: a registry, not a branch.
export interface VideoModelSpec {
id: string;
slug: string;
durations: number[];
resolutions: ResolutionTier[];
audio: "none" | "builtin" | "toggle";
supportsLastFrame?: boolean;
buildInput: (opts: BuildInputOpts) => Record<string, unknown>;
}
Each model owns its own buildInput. The string-vs-integer duration, the "auto" vs "adaptive" magic word, the explicit resolution — all of it lives inside the closure for that one model. Nothing above it knows or cares.
The payoff is the property I was actually after: adding a model is a data edit, never a UI edit. The studio renders duration buttons, resolution tiers and the audio control straight off the spec. The cost calculator reads the same spec. A new model is one array entry.
The mistake I made first was trying to normalize the providers into one shared shape and handle exceptions with if (model === "wan"). That works for two models. At four it's unreadable, and at six every new model turns into a bug hunt through unrelated code paths.
Don't normalize away differences that are real. Encode them as data, and push them down to the leaf that owns them.
Three things I'd tell myself at the start
- Live-test every field before you trust the docs, and write the date in the comment. Two of my worst bugs were fields I'd "verified" by reading a playground page.
- Every billing-relevant field gets sent explicitly. Never let a vendor default decide what you pay.
- Grep your own fallbacks for the failure mode "charged, broken, no refund." Fallbacks get written to make things resilient, and they're very good at converting a crash into a silent charge.
The system this came from is Stivio, which turns a photo into a short video. The model registry is about 250 lines and it's the file I'm least embarrassed by — mostly because everything above it spent a week being embarrassing first.
If you're wrapping any multi-vendor AI API right now: start with the registry. You'll need it by the third provider, and retrofitting it is much worse than over-engineering it on day one.
Top comments (0)