DEV Community: Jerry zhang

Reference Images in AI Video Generation: What They Fix and What They Don't

Jerry zhang — Mon, 19 Oct 2026 14:35:46 +0000

If you've tried to get an AI video model to show your product, your character or your composition from text alone, you've probably hit the wall: the prompt says "matte black bottle with a cork cap" and you get a glossy bottle with a screw top, a different one every run. Reference images are the usual fix. They help a lot. They also come with limits that aren't obvious until you've burned a few generations on them.

This post breaks down how reference-image workflows actually work, what they reliably fix, what they don't, and a few habits that make them more predictable. It's model-agnostic, but I'll use concrete input options from current models so it isn't all hand-waving.

Three different things called "reference"

"Image-to-video" gets used for at least three different input modes. They behave differently, so it's worth separating them.

1. Starting image (first frame)

The image is the opening frame. The model's job is to continue it: add motion, move the camera, let the scene play out. Kling 3.0 Turbo, for example, takes a prompt plus a starting image. Veo 3.1 Quality also accepts a starting image and an end frame.

Good for: product reveals, animating a photo or illustration you've already approved, keeping an exact composition at second zero.

2. Start and end frames

You give the first and the last frame, and the model fills in the motion between them. Kling 3.0 accepts a starting image and an end frame; Seedance 2.0 Mini, Fast and Standard all have a frame mode for first and last frames; Wan 2.7 and Wan 3.0 also expose start/end frames plus references.

Good for: transitions, before/after shots, loops (use the same image for both ends), and any shot where you know exactly where it has to land.

3. Style/subject references (not frames)

One or more images guide what things look like without being used as exact frames. Grok Imagine Video 1.5 accepts up to 7 reference images with a prompt. Seedance 2.0 Mini's reference mode can mix image, video and audio references. Veo 3.1 Fast lists up to 3 reference images; Gemini-Omni takes a prompt plus references.

Good for: keeping a character, outfit, product or art style consistent across several different shots.

The practical point: before you upload anything, check which of these modes the model you selected actually offers. A model being in the video menu doesn't mean it does image-to-video, and a model that does image-to-video may only do one of the three. Also check that the row is still live — some options (including standard Sora 2 and older Seedance V1 variants) are paused in Create even if an older guide still names them.

What reference images reliably fix

Composition and framing. With a starting image, you decide where the subject sits, how big it is, and what's in the background. Text prompts are bad at this; images are good at it.

Subject identity (mostly). Product shape, color, packaging, a character's face and clothing all hold much better with a reference than with a description.

Style. Lighting mood, color grade, illustration style. One good reference often does more than a paragraph of adjectives.

Iteration cost. Once a still is right, you stop re-rolling the whole scene. You're only iterating on motion. Since stills are usually cheaper and faster to generate than video, settling the frame first saves real time and money.

What reference images don't fix

This is where most of the frustration comes from.

They don't add what isn't there. If the reference doesn't contain the object, a motion prompt is not a reliable way to make it appear. "The hand picks up the phone" won't go well if there's no phone in the frame. Make a new still that contains the object, or use a text-led model for that shot.

They don't guarantee a match. A reference is a strong hint, not a lock. Labels, small text, logos and fine patterns are where outputs drift first. Expect to inspect every frame where a logo is visible, and plan to add exact text in your editor instead.

They don't control motion quality. A perfect first frame can still turn into a warped product by second four. The reference controls the start; the model still decides everything after it. That's why the last second of a clip is the first place to look.

They can lock you into the wrong shape. Some image-led modes inherit the aspect ratio of the reference. In Grok Imagine Video 1.5's single-image Stable mode, for instance, the output follows the reference's orientation, so a portrait clip needs a portrait reference. If you upload a landscape photo and select 9:16, check the downloaded file, not the preview box.

They don't solve multi-shot continuity on their own. Using the same reference across shots helps, but a recurring character still drifts between generations. Keep the same reference, the same clothing description and the same setting wording for every shot, and review each result before building the next.

They don't clear rights. If you upload a photo of a real person, a brand's product or someone else's artwork, the model doesn't make that usable. Only upload material you have permission to use.

A workflow that tends to work

Here's the sequence I'd suggest for a single image-led clip:

Write the shot as one sentence. Subject, one action, one camera move. "An unbranded ceramic mug on an oak table; steam rises; slow push-in."
Make or choose the still. Keep it simple: one subject, one surface, one light source. Leave out small text. Generate it at the aspect ratio you'll publish in.
Pick the input mode. Starting image if the opening composition matters, start+end frames if the landing matters, references if consistency across shots matters.
Pick a model that offers that mode and a ratio that matches your still. Confirm the row is still enabled in Create.
Write the prompt for the change, not the picture. The image already shows the mug, the table and the light. Your prompt should describe what moves: the steam, the camera. Repeating the whole scene description often makes the model redraw it.
Generate a short draft at the lowest resolution that lets you judge it. On free studio access, enabled Seedance models are currently limited to 480p — draft there, then re-run a winner at higher quality if your plan allows it.
Watch the whole file. First frame, the main action, last second. Compare edges and labels against the reference.
Change one thing and run it again.

Prompt shape for image-led clips

[What moves] + [how it moves] + [camera] + [what must stay fixed]

Example:
Steam rises slowly from the mug. Camera makes a slow push-in,
one continuous shot. Keep the mug shape, handle and table unchanged.
No text or logos.

Practical tips

Match ratios first. A 9:16 still into a 9:16 setting. Mismatches cause cropping or letterboxing that you'll only notice after download.
Fewer, cleaner references beat more references. If a model accepts several, start with one or two clear ones. Conflicting references (different lighting, different angles) give the model conflicting instructions.
Clean backgrounds. Busy backgrounds in the reference become busy, shifting backgrounds in the clip.
Switching models can drop your references. A different model may not accept the same inputs. Re-check your setup after any switch — many studios ask you to confirm before removing incompatible draft references.
Treat uploads as semi-public. On many services, uploaded references and outputs are stored at shareable URLs. Don't upload anything confidential.
Keep a notes file. Reference used, model, mode, prompt, what drifted. Patterns show up fast.

Trying the workflow

You can run this workflow in most video tools; Runway, Luma, Kling's own app and others all have some form of image-to-video. If you'd like to compare input modes across models without setting up several accounts, the Everygen AI studio at everygenai.io has starting-image, start/end-frame and multi-reference models side by side. Eligible new visitors get welcome credits without signing up; sign in for daily credits after the next 24-hour wait. The studio quotes the credit cost before each run and asks you to confirm before a model switch removes references the new model can't use. Optional Lite membership is $19.90/month if you outgrow the free grants. Disclosure: I work on it. For the submit → review → download steps in more detail, including the reference-orientation gotcha, see the Everygen AI video generator guide.

Takeaway

Reference images are the best tool you have for composition, identity and style, and the weakest tool for motion, fine text and multi-shot continuity. Pick the right reference mode for the job, describe the change rather than the picture, match your aspect ratios, and judge the whole clip, especially the last second.

What's the hardest thing you've tried to keep consistent with a reference: faces, products or style? I'd like to hear what worked.

Writing release notes while Kling 4.0 is still “coming soon”

Jerry zhang — Fri, 25 Sep 2026 06:25:30 +0000

I spent last week rewriting the same three sentences on a landing page.

Not because the product changed. Because I kept mixing what is live, what is reported, and what I wish were true. If you ship AI tools, you’ve probably done the same. Here’s the short version of what burned me — and the Kling 4.0 notes that actually got me excited again.

Release notes pits I fell into (again)

1. Pasting the roadmap into the hero.

I drafted a headline that read like Kling 4.0 was already generating on the site. A teammate caught it. Readers don’t care that “we’re ready.” They care whether the Generate button works. If it doesn’t, say coming soon and stop decorating it.

2. Turning a leak table into a feature checklist.

Someone on the team had a long list: durations, reference caps, Flash vs long video. I almost published it as “Kling 4.0 supports…” with no hedge. That’s how you earn a support ticket on day one. Better: “early reports / what people with access are seeing,” then update when it’s on your own stack.

3. Forgetting the “what you can do today paragraph.

The first soft-launch note I wrote was 100% future tense. A friend replied: “Cool. So… do I close the tab?” Now every note ends with a usable loop for the models that already run — and a clear line that 4.0 isn’t on the site yet.

4. Burying cost honesty.

I once put “free credits in the title and left the quote-before-generate bit in a footnote. People felt baited. Credits up front beats adjectives.

If you’re writing notes for a version that isn’t live: label the buckets. Live / Reported / Not on our site yet. Paste that into the draft before the marketing pass.

What about Kling 4.0 has me most excited

I’m still waiting to wire it up. Kling 4.0 is coming soon on my side — not a pretend launch. But a few people already have clearer notes on what’s in the build, and these are the bits that made me stop doomscrolling and start sketching workflows:

Quick comparison a lot of people ask for first: 4.0 vs Flash vs 3.0. Treat it as a working chart from early notes — especially resolution and long-form rows — and re-check when 4.0 is actually live on your stack. On my site, 4.0 generation is still **coming soon.

Video generation & models

Kling 4.0: about 3–30s across Omni-reference and keyframe workflows; 720p / 1080p; keyframe mode up to ~10 keyframes
Extensions: roughly 3–60s extension steps (source clips often capped around 60s) — this is the dont re-roll the whole spot” piece I’m impatient for
Long video (experimental): reports of 3–120s at 720p / 1080p only (no 2K/4K in those notes), with forward and backward extension
Kling 4.0 Flash: about 3–20s, Omni-ref style, currently called out as 720p-locked
Omni in these notes looks like a workflow inside 4.0, not a separate standalone model name on the box

Multimodal reference limits (the part makers will argue about)

Standard mode: up to about 15 combined references (caps people are citing: ~10 images, ~5 videos, ~7 Elements)
Uploaded reference clips: roughly 230s on 4.0, 2–20s on Flash (input length ≠ output length)
Voice refs: up to ~3 on full 4.0; Flash lists 0 voice inputs in the notes I’ve seen (doesn’t always mean silent output — just no voice inputs yet)
Long video mode: much tighter — hard cap around 5 image/Element refs total; no video or voice inputs in those reports

Image 4.0 / other flags (lighter excitement, still watching)

Default & Series modes, batches of about 1–9 images
Resolution toggles showing 2K / 4K / 8K (native 8K still unconfirmed in what I’ve read)
Early chatter on plugin hooks, experimental audio tools, VIP tiers — I’m not betting the landing page on those yet

API & billing

Full 4.0 sounds like separate API packages and dedicated video-input billing; Flash may ride existing 3.x-style packages. No final pricing in anything Id put in a public changelog without a TBD” stamp.

The five lines I’d put in a “why I’m excited” blurb

If I had to soft-announce without lying:

Up to ~30s generation, with extension paths people are citing toward ~60s
On the order of ~15 reference elements in standard mode
Up to 1080p in the main 4.0 path
Multilingual lip-sync / sound with up to ~3 voice references (where the notes list them)
An experimental long video path people describe as up to ~120s at 1080p, extendable forward and back

That’s the wishlist energy. Not a promise that it’s on my Generate button today.

What I’m doing while I wait (and what you can do too)

Until Kling 4.0 is actually integrated, I keep shipping 3–15s clips with the Kling models that are live — 2.6, 3.0, 3.0 Omni, and the rest of the series you can run now. Same boring loop: short prompt, reference only when identity matters, read the credit quote, then generate.

I keep the waitlist + the “try other Kling models free (eligible users)” path in one place: kling-4.ai. Independent site — not official Kuaishou / Kling. When 4.0 is really there, the release note will say live, not “excited.”

If you’re drafting your own “coming soon” page this week: steal the bucket labels. Future-you will thank present-you when support opens the ticket.

Happy to compare how you’re wording duration vs extension vs long-mode in the comments that’s where most of us accidentally overclaim.

Disclosure: I work on kling-4.ai. Feature numbers above are from early/reported notes others have shared, not a claim that Kling 4.0 generation is live on the site today. No paid placement on DEV.