I built a small internal tool to generate short product demo clips from a text description and a couple of reference product photos — the kind of thing a small e-commerce team could use instead of booking a video shoot for every SKU. The video model doing the actual generation was ByteDance's Seedance 2.0, and the first week of building this taught me more about prompt structure than any guide I read beforehand, mostly because my first dozen prompts produced technically fine but oddly generic clips.
Here's what actually moved the needle, and the small API client I ended up with once I stopped fighting the async job pattern these video generation APIs use.
Seedance 2.0, briefly
Seedance 2.0 is ByteDance's multimodal video generation model, released earlier this year, built on a dual-branch diffusion transformer that generates video and audio together rather than bolting sound on afterward. It accepts text, image, video, and audio as inputs, which means you can describe a scene in words, hand it a reference image for style or a starting frame, hand it a reference video for a specific camera move or action you want copied, and hand it an audio clip to set mood or rhythm — combined in a single generation request. It ships as a small family of models — Mini, Fast, and the full Seedance 2.0 — that share the same feature set and trade off quality against cost and speed, which turns out to matter a lot for how you actually work with it day to day.
Mistake one: writing one paragraph instead of three instructions
My early prompts read like a single flowing description: "a wireless earbuds case opening slowly on a marble surface, soft studio lighting, cinematic feel." That produced clips that were fine but generic — competent b-roll, nothing that looked directed.
The fix was separating what I was actually asking for into three distinct things the model treats differently: the camera and subject action, any dialogue or on-screen text, and the sound. Seedance 2.0 wants dialogue written in double quotes so it can voice it with matching lip movement, and it treats audio essentially as a separate sound brief rather than a mood adjective tacked onto the visual description. Once I split my prompts into explicit camera-move-then-action-then-sound structure instead of one adjective-heavy paragraph, the difference was immediate — the model stopped guessing which words were describing the shot versus describing the mood and started following each instruction more literally.
For multi-shot clips, labeling shots explicitly — Shot 1, Shot 2, Shot 3, in order — and describing each with the same camera-then-action-then-position-then-sound structure gave noticeably more consistent results than one long paragraph trying to cover a 10-second clip in a single breath.
Mistake two: not using reference assets to do the describing
The model's multimodal reference support is the actual headline feature, and I ignored it for the first several attempts out of habit — I was used to text-to-image workflows where you describe everything in words. Seedance 2.0 supports up to nine reference images, three video clips, and three audio files per generation, but the practical guidance (which held up in my testing) is to use far fewer than the ceiling allows. Once I started handing it one clean product photo as a style and color reference instead of describing the product's exact appearance in prose, both consistency and detail improved — skin-pore and material-texture level detail is genuinely something the model handles well when it has a real reference to work from rather than an adjective describing it.
One combination worth knowing before you hit it as a confusing failure: audio alone, or text plus audio with no visual reference, won't generate anything. Every job needs a visual anchor — text-to-video, image-to-video, or a reference video — and audio only works layered on top of one of those, not standalone.
The workflow that actually worked: draft cheap, finalize once
Because the model family spans Mini, Fast, and full Seedance 2.0 at different cost and quality points, the workflow that stopped wasting my API budget was drafting on the cheaper, faster tier until the prompt and reference combination was actually behaving the way I wanted, then running the final pass on the full model once. Iterating on the full-quality tier while still figuring out shot structure was the single biggest source of wasted spend early on.
Handling the async job pattern
Video generation isn't a single request-response call the way a chat completion is — you submit a job with your prompt and references, then poll until it's done, then fetch the result. That shape is common across most video generation APIs regardless of provider, so it's worth building the polling logic once instead of writing it inline every time:
// seedance-client.js — submit a generation job and poll until it's ready
async function submitVideoJob(baseURL, apiKey, { prompt, imageRefs = [], model }) {
const response = await fetch(`${baseURL}/video/generations`, {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${apiKey}`,
},
body: JSON.stringify({
model, // e.g. "seedance-2.0-fast" for drafting, "seedance-2.0" for the final pass
prompt,
reference_images: imageRefs,
duration: "auto",
}),
});
if (!response.ok) {
throw new Error(`Job submission failed: ${response.status}`);
}
const { job_id } = await response.json();
return job_id;
}
async function pollUntilDone(baseURL, apiKey, jobId, { intervalMs = 5000, timeoutMs = 300000 } = {}) {
const start = Date.now();
while (Date.now() - start < timeoutMs) {
const res = await fetch(`${baseURL}/video/generations/${jobId}`, {
headers: { Authorization: `Bearer ${apiKey}` },
});
const data = await res.json();
if (data.status === "completed") return data.video_url;
if (data.status === "failed") throw new Error(`Job ${jobId} failed: ${data.error ?? "unknown error"}`);
await new Promise((resolve) => setTimeout(resolve, intervalMs));
}
throw new Error(`Job ${jobId} timed out after ${timeoutMs}ms`);
}
async function generateDraftThenFinal(baseURL, apiKey, prompt, imageRefs) {
// Draft on the cheaper tier first
const draftJobId = await submitVideoJob(baseURL, apiKey, {
prompt,
imageRefs,
model: "seedance-2.0-fast",
});
const draftUrl = await pollUntilDone(baseURL, apiKey, draftJobId);
console.log("Draft ready:", draftUrl);
// Only run the full-quality pass once the prompt is confirmed working
const finalJobId = await submitVideoJob(baseURL, apiKey, {
prompt,
imageRefs,
model: "seedance-2.0",
});
return pollUntilDone(baseURL, apiKey, finalJobId);
}
module.exports = { submitVideoJob, pollUntilDone, generateDraftThenFinal };
This is deliberately generic — exact request and response field names vary by which provider's endpoint you're calling — but the submit-then-poll shape and the draft-then-finalize pattern is what actually made the tool practical to use instead of burning full-price generations while I was still iterating on prompt structure.
Where RouteAI fit in
I ended up calling Seedance 2.0 through RouteAI, which lists it alongside other current models in its catalog, mainly because it meant I didn't need a separate account and billing relationship just to get access to one video model alongside the text models the rest of the internal tool already used. It didn't change anything about the prompting lessons above — those are properties of the model itself, not the gateway in front of it — but consolidating video and text generation behind one key was a smaller convenience worth mentioning rather than a reason to pick a video model in the first place.
What I'd tell someone starting out
Structure your prompt as camera move, then action, then position, then sound — as separate instructions, not one adjective-heavy paragraph. Use a real reference image or video instead of describing appearance in prose whenever you can; the model is genuinely better at matching a reference than interpreting an adjective. Keep reference counts well below the documented limits rather than maxing them out. Draft on the cheaper model tier and only run your final prompt on the full model once it's actually working. And build the polling logic once — you'll be submitting a lot of jobs before the first one looks the way you want.
TL;DR: Splitting Seedance 2.0 prompts into explicit camera/action/sound instructions instead of one descriptive paragraph, leaning on reference images and video instead of prose descriptions, and drafting on a cheaper model tier before the final pass were what actually improved output quality — the API itself is a standard submit-then-poll async pattern worth wrapping once in a reusable client.
Website: https://www.fastrouteai.com
Top comments (0)