DEV Community: Fillies Buffey

[Boost]

Fillies Buffey — Thu, 10 Sep 2026 15:07:33 +0000

Fillies Buffey

Sep 10

Multi-Modal Reference-to-Video: Keeping Characters Consistent with Seedance 3.0

#ai #video #api #seedance

5 min read

Multi-Modal Reference-to-Video: Keeping Characters Consistent with Seedance 3.0

Fillies Buffey — Thu, 10 Sep 2026 15:07:26 +0000

Most AI video demos look great for four seconds. Ask for thirty and the character's jacket changes color, the product label turns into scrambled letters, and the camera forgets where it was pointing.

That's not a prompting problem — it's a reference problem. Long-form generation only holds together when the model has persistent anchors for identity, motion, and pacing instead of re-inventing them every frame. This post walks through the workflow I use for reference-to-video work with Seedance 3.0, including the data structures and the checks that catch drift before you waste a render.

Why single-prompt generation drifts

A text prompt is a lossy compression of the shot you actually want. Every token the model samples is a chance to diverge from your intent, and those divergences compound over hundreds of frames. The usual workarounds have their own costs:

Stitching 5-second clips — continuity breaks at every cut; grading and motion don't survive the seams.
Image-to-video from a single still — locks the first frame, then drifts once the model has to invent what it can't see.
Heavy negative prompting — brittle; it fights the model instead of constraining it.

What actually constrains generation is giving the model the same evidence a human crew would get: what the character looks like, how the camera moves, what the pacing feels like, and where the scene has to land.

What "multi-modal reference" means in practice

Seedance 3.0 accepts a mix of image, video, audio, and text references — up to 50 multi-media inputs in a single generation — and produces clips up to 30 seconds at native 4K. The useful mental model is that each reference has a role:

Reference type	What it anchors	Example
Character / product image	Identity, texture, color	A product photo from three angles
Motion video	Camera path, body movement	A 6-second handheld dolly clip
Audio clip	Pacing, beat structure	A 12-second music bed
Text	Intent, dialogue, world rules	"Neon-lit Tokyo alley, wet asphalt, no text overlays"

The failure mode I see most often is using one reference for two roles. A single image can anchor identity or style — asking it to do both usually gets you neither.

Reference hygiene checklist

Before you spend a render, run the set through this:

One role per reference. Name them (hero_front.png, camera_push.mp4, beat_120bpm.wav) and keep a mapping in your shot list, not in your head.
Match aspect and resolution. Mixing a 4:3 phone photo into a 16:9 4K generation invites cropping artifacts.
Prefer motion references over motion adjectives. "Slow push in" is vague; a 4-second clip of a slow push is not.
Cap the set at what you'll actually verify. 50 references is a ceiling, not a target. Fifteen well-chosen inputs beat forty that overlap.
Freeze identity references between shots. If shot 3 uses a different product photo than shot 2, expect the label to shift.

Structuring a shot list as data

Prompts don't scale; shot lists do. I keep the whole generation as JSON so it can be validated, diffed, and regenerated:

{
  "project": "seedance-3-0-launch",
  "output": { "resolution": "4K", "aspect": "16:9", "max_seconds": 30, "watermark": false },
  "references": {
    "identity": ["refs/hero_front.png", "refs/hero_side.png"],
    "motion": ["refs/camera_push_4s.mp4"],
    "audio": ["refs/bed_120bpm.wav"]
  },
  "shots": [
    {
      "id": "shot-01",
      "setup": "Product sits on a matte black surface, hard key light from camera left",
      "action": "Liquid pours in slow motion, surface tension visible",
      "camera": "Slow push in, 35mm equivalent, no handheld shake",
      "ending_beat": "Camera settles as the last drop lands"
    }
  ]
}

A small validator keeps the set honest before it ever reaches the renderer:

const MAX_REFS = 50;

function validate(plan) {
  const errors = [];
  const refs = Object.values(plan.references).flat();

  if (refs.length > MAX_REFS) {
    errors.push(`too many references: ${refs.length} > ${MAX_REFS}`);
  }
  if (plan.output.max_seconds > 30) {
    errors.push('clip length exceeds the 30s single-pass budget — split the shot list');
  }
  if (!plan.references.identity.length) {
    errors.push('no identity reference: expect character/product drift');
  }
  for (const shot of plan.shots) {
    for (const field of ['setup', 'action', 'camera', 'ending_beat']) {
      if (!shot[field]) errors.push(`${shot.id}: missing ${field}`);
    }
  }
  return errors;
}

Two rules encoded there matter more than they look: never generate without an identity reference, and always declare the ending beat. Generations that have to invent their own landing point are where continuity quietly dies.

Submitting the plan

If you're wiring this into a product rather than clicking through a UI, the workspace exposes an API for teams that want video generation inside their own pipeline. The shape is a single job submission with a callback:

// Replace API_BASE and the auth header with the values from your Seedance 3.0 account.
const API_BASE = process.env.SEEDANCE_API_BASE;

async function submitJob(plan) {
  const errors = validate(plan);
  if (errors.length) throw new Error(errors.join('\n'));

  const res = await fetch(`${API_BASE}/v1/video/jobs`, {
    method: 'POST',
    headers: {
      'content-type': 'application/json',
      authorization: `Bearer ${process.env.SEEDANCE_API_KEY}`,
    },
    body: JSON.stringify(plan),
  });

  if (!res.ok) throw new Error(`submit failed: ${res.status} ${await res.text()}`);
  return res.json(); // { job_id, status, estimated_seconds }
}

Long jobs should be polled or webhooked rather than held open — a 30-second 4K single-pass render is not a request you want blocking a serverless function.

Prompt patterns that survive 30 seconds

Once references are locked, the prompt only has to carry intent. Four fields per shot, in this order:

Setup — the static world. What exists before anything moves.
Action — the single change the shot is about. One verb, not three.
Camera — movement, lens, and whether handheld is allowed.
Ending beat — the frame the shot must resolve on, which is also your cut point.

If a shot needs two actions, it's two shots. Splitting is cheaper than re-rolling a 30-second render.

Quality control before you ship

Check the first three seconds and the last two — drift is almost always visible there first. Then verify:

4K detail holds on the identity reference (logo, label, fabric weave).
No watermark on paid tiers; confirm the plan you're on includes watermark-free output.
Audio keeps sync with the pacing reference across the full clip.
The ending beat matches the plan, so the next shot can cut cleanly.

Cost and throughput notes

One 30-second single-pass generation is dramatically cheaper than six 5-second clips plus edits, and it removes the seam mismatch entirely. Previs passes are where the savings compound: generate a low-stakes version to validate references and pacing, then re-run the same plan for the final. Free tiers with daily credits are enough for reference validation; heavier iteration and commercial licensing live on the paid plans.

Where this fits

If you want to try the workflow above end to end — 50 multi-media references, 30-second single-pass generation, native 4K, watermark-free output, and an API for teams — that's what Seedance 3.0 is built for.

The pipeline matters more than the model, though. Lock your identity references, declare your ending beats, validate before you render, and long-form AI video stops being a slot machine.

If you try the shot-list structure, I'd like to hear which fields you ended up adding — the ending beat is the one most people drop first, and it's the one that hurts most.

How to Structure a 30-Second AI Video Prompt for Seedance 2.5

Fillies Buffey — Sat, 18 Jul 2026 07:46:23 +0000

AI video prompts often fail for a simple reason: they describe a look, but not a sequence.

A phrase such as “make a cinematic product video” may define a mood, but it does not tell the model what should happen in the first shot, how the camera should move, when the main action begins, or how the video should end.

That problem becomes more important when a model can generate a longer clip. The Seedance 2.5 AI Video Generator is designed around clips up to 30 seconds, output paths up to 4K, and a workspace that can accept as many as 50 reference assets. Those capabilities are useful, but they also make prompt organization more important.

This post shows a reusable structure for turning a creative brief into a 30-second AI video prompt.

The five-layer prompt model

I like to separate a prompt into five layers:

Outcome — What is the video supposed to achieve?
Visual constraints — What must remain consistent?
Timeline — What happens during each part of the clip?
Reference mapping — What role does each uploaded asset play?
Acceptance criteria — What should be checked before the result is accepted?

Keeping these layers separate makes the prompt easier to review and revise.

A reusable prompt blueprint

Here is a format that can be stored as YAML, JSON, or a form inside a prompt-building tool:

goal: "Create a 30-second cinematic product reveal"

output:
  duration: "30 seconds"
  aspect_ratio: "16:9"
  resolution: "4K"
  style: "premium, realistic, minimal"

subject:
  product: "matte black wireless speaker"
  must_preserve:
    - product shape
    - logo placement
    - surface material
    - button layout

environment:
  location: "dark studio with a reflective floor"
  lighting: "soft blue rim light and warm key light"
  atmosphere: "subtle haze, no visible crew or equipment"

timeline:
  - time: "0-10s"
    action: "Wide establishing shot. Camera slowly pushes toward the product."
  - time: "11-20s"
    action: "Orbit around the speaker while light reveals the surface texture."
  - time: "21-30s"
    action: "Move into a close-up, show the control buttons, then settle on a clean hero frame."

camera:
  movement: "smooth dolly and controlled orbit"
  lens_feel: "50mm commercial product photography"
  avoid:
    - handheld shake
    - sudden zooms
    - inconsistent direction

references:
  product_images: "control shape, logo, and materials"
  motion_video: "control orbit speed and camera rhythm"
  audio_reference: "guide pacing only"

success_criteria:
  - product identity is stable in every shot
  - camera movement follows one continuous direction
  - text and logo are not distorted
  - final frame is suitable for a landing-page hero section

This structure is intentionally explicit. It gives each instruction a job instead of mixing everything into one paragraph.

Step 1: Start with the business outcome

Before describing lenses or lighting, define the purpose.

Compare these two prompts:

Make a beautiful cinematic video of this product.

Create a 30-second launch video that introduces the product, reveals its materials, and ends on a clean hero frame suitable for a landing page.

The second prompt gives the model a beginning, a middle, and an end. It also gives the creator a way to judge whether the result succeeded.

Step 2: Turn 30 seconds into three beats

A practical 30-second structure is:

0–10 seconds: establish the subject and environment.
11–20 seconds: perform the main action or reveal.
21–30 seconds: resolve the scene and create a usable ending.

This is not a rigid rule. It is a debugging tool.

If the output feels rushed, you can inspect the timeline instead of rewriting the entire prompt. If the ending is weak, revise the final beat. If the subject appears too late, strengthen the opening beat.

Step 3: Assign every reference a role

More reference files do not automatically produce a better result.

A reference set should behave like a small design system:

Product images control shape, texture, color, and branding.
Character images control identity, clothing, and proportions.
Environment images control architecture, palette, and composition.
Motion videos control movement, timing, and camera behavior.
Audio files control rhythm or sound direction.

When two references disagree, the model has to guess. It is usually better to remove a conflicting asset than to add another sentence explaining the conflict.

Step 4: Separate generation from validation

Prompting and reviewing should be two different steps.

After generation, check:

Is the main subject consistent from start to finish?
Does the camera follow the requested path?
Do materials, logos, and product geometry remain stable?
Does the scene have a clear ending?
Is the framing appropriate for the destination?
Would changing one reference solve the problem more directly than adding more prompt text?

This turns iteration into a controlled process instead of random prompt expansion.

Step 5: Make the structure reusable

If you generate AI video regularly, store the blueprint as data.

A small internal tool could provide fields for:

Goal
Duration and aspect ratio
Subject constraints
Three timeline beats
Camera instructions
Reference roles
Success criteria

The tool can then render those fields into a final prompt. This approach also makes it easier to compare versions, reuse successful shot structures, and build prompt templates for product ads, storyboards, character scenes, or social clips.

Final takeaway

Longer AI video generation benefits from production-style thinking.

Instead of writing one dense paragraph, define the outcome, split the scene into timed beats, map every reference to a purpose, and review the output against explicit criteria.

The model still does the generation, but the structure makes the creative intent easier to communicate—and much easier to debug.