Multi-Modal Reference-to-Video: Keeping Characters Consistent with Seedance 3.0

Most AI video demos look great for four seconds. Ask for thirty and the character's jacket changes color, the product label turns into scrambled letters, and the camera forgets where it was pointing.

That's not a prompting problem — it's a reference problem. Long-form generation only holds together when the model has persistent anchors for identity, motion, and pacing instead of re-inventing them every frame. This post walks through the workflow I use for reference-to-video work with Seedance 3.0, including the data structures and the checks that catch drift before you waste a render.

Why single-prompt generation drifts

A text prompt is a lossy compression of the shot you actually want. Every token the model samples is a chance to diverge from your intent, and those divergences compound over hundreds of frames. The usual workarounds have their own costs:

• Stitching 5-second clips — continuity breaks at every cut; grading and motion don't survive the seams.
• Image-to-video from a single still — locks the first frame, then drifts once the model has to invent what it can't see.
• Heavy negative prompting — brittle; it fights the model instead of constraining it.

What actually constrains generation is giving the model the same evidence a human crew would get: what the character looks like, how the camera moves, what the pacing feels like, and where the scene has to land.

What "multi-modal reference" means in practice

Seedance 3.0 accepts a mix of image, video, audio, and text references — up to 50 multi-media inputs in a single generation — and produces clips up to 30 seconds at native 4K. The useful mental model is that each reference has a role:

Reference type	What it anchors	Example
Character / product image	Identity, texture, color	A product photo from three angles
Motion video	Camera path, body movement	A 6-second handheld dolly clip
Audio clip	Pacing, beat structure	A 12-second music bed
Text	Intent, dialogue, world rules	"Neon-lit Tokyo alley, wet asphalt, no text overlays"

The failure mode I see most often is using one reference for two roles. A single image can anchor identity or style — asking it to do both usually gets you neither.

Reference hygiene checklist

Before you spend a render, run the set through this:

1. One role per reference. Name them (hero_front.png, camera_push.mp4, beat_120bpm.wav) and keep a mapping in your shot list, not in your head.
2. Match aspect and resolution. Mixing a 4:3 phone photo into a 16:9 4K generation invites cropping artifacts.
3. Prefer motion references over motion adjectives. "Slow push in" is vague; a 4-second clip of a slow push is not.
4. Cap the set at what you'll actually verify. 50 references is a ceiling, not a target. Fifteen well-chosen inputs beat forty that overlap.
5. Freeze identity references between shots. If shot 3 uses a different product photo than shot 2, expect the label to shift.

Structuring a shot list as data

Prompts don't scale; shot lists do. I keep the whole generation as JSON so it can be validated, diffed, and regenerated:

{
  "project": "seedance-3-0-launch",
  "output": { "resolution": "4K", "aspect": "16:9", "max_seconds": 30, "watermark": false },
  "references": {
    "identity": ["refs/hero_front.png", "refs/hero_side.png"],
    "motion": ["refs/camera_push_4s.mp4"],
    "audio": ["refs/bed_120bpm.wav"]
  },
  "shots": [
    {
      "id": "shot-01",
      "setup": "Product sits on a matte black surface, hard key light from camera left",
      "action": "Liquid pours in slow motion, surface tension visible",
      "camera": "Slow push in, 35mm equivalent, no handheld shake",
      "ending_beat": "Camera settles as the last drop lands"
    }
  ]
}

A small validator keeps the set honest before it ever reaches the renderer:

const MAX_REFS = 50;

function validate(plan) {
  const errors = [];
  const refs = Object.values(plan.references).flat();

  if (refs.length > MAX_REFS) {
    errors.push(`too many references: ${refs.length} > ${MAX_REFS}`);
  }
  if (plan.output.max_seconds > 30) {
    errors.push('clip length exceeds the 30s single-pass budget — split the shot list');
  }
  if (!plan.references.identity.length) {
    errors.push('no identity reference: expect character/product drift');
  }
  for (const shot of plan.shots) {
    for (const field of ['setup', 'action', 'camera', 'ending_beat']) {
      if (!shot[field]) errors.push(`${shot.id}: missing ${field}`);
    }
  }
  return errors;
}

Two rules encoded there matter more than they look: never generate without an identity reference, and always declare the ending beat. Generations that have to invent their own landing point are where continuity quietly dies.

Submitting the plan

If you're wiring this into a product rather than clicking through a UI, the workspace exposes an API for teams that want video generation inside their own pipeline. The shape is a single job submission with a callback:

// Replace API_BASE and the auth header with the values from your Seedance 3.0 account.
const API_BASE = process.env.SEEDANCE_API_BASE;

async function submitJob(plan) {
  const errors = validate(plan);
  if (errors.length) throw new Error(errors.join('\n'));

  const res = await fetch(`${API_BASE}/v1/video/jobs`, {
    method: 'POST',
    headers: {
      'content-type': 'application/json',
      authorization: `Bearer ${process.env.SEEDANCE_API_KEY}`,
    },
    body: JSON.stringify(plan),
  });

  if (!res.ok) throw new Error(`submit failed: ${res.status} ${await res.text()}`);
  return res.json(); // { job_id, status, estimated_seconds }
}

Long jobs should be polled or webhooked rather than held open — a 30-second 4K single-pass render is not a request you want blocking a serverless function.

Prompt patterns that survive 30 seconds

Once references are locked, the prompt only has to carry intent. Four fields per shot, in this order:

• Setup — the static world. What exists before anything moves.
• Action — the single change the shot is about. One verb, not three.
• Camera — movement, lens, and whether handheld is allowed.
• Ending beat — the frame the shot must resolve on, which is also your cut point.

If a shot needs two actions, it's two shots. Splitting is cheaper than re-rolling a 30-second render.

Quality control before you ship

Check the first three seconds and the last two — drift is almost always visible there first. Then verify:

• 4K detail holds on the identity reference (logo, label, fabric weave).
• No watermark on paid tiers; confirm the plan you're on includes watermark-free output.
• Audio keeps sync with the pacing reference across the full clip.
• The ending beat matches the plan, so the next shot can cut cleanly.

Cost and throughput notes

One 30-second single-pass generation is dramatically cheaper than six 5-second clips plus edits, and it removes the seam mismatch entirely. Previs passes are where the savings compound: generate a low-stakes version to validate references and pacing, then re-run the same plan for the final. Free tiers with daily credits are enough for reference validation; heavier iteration and commercial licensing live on the paid plans.

Where this fits

If you want to try the workflow above end to end — 50 multi-media references, 30-second single-pass generation, native 4K, watermark-free output, and an API for teams — that's what Seedance 3.0 is built for.

The pipeline matters more than the model, though. Lock your identity references, declare your ending beats, validate before you render, and long-form AI video stops being a slot machine.

If you try the shot-list structure, I'd like to hear which fields you ended up adding — the ending beat is the one most people drop first, and it's the one that hurts most.