DEV Community: chen mensen

I Stopped Asking One Prompt to Build the Whole AI Video

chen mensen — Sun, 04 Oct 2026 06:43:58 +0000

TL;DR

For two-person AI video scenes, I now split the work into two passes: approve a still image first, then animate only that composition. It makes failures easier to diagnose and reduces wasted video generations.

🎬 Why the all-in-one prompt kept failing

I used to put everything into one prompt: two people, clothing, background, camera, props, gestures, and motion. When the result failed, I could not tell whether the problem came from the starting composition or the animation.

Two-person scenes multiply the failure points. A model has to keep two faces separate, preserve left/right positions, render hands, and leave enough empty space for movement.

🖼️ Pass one: lock the still frame

I start with a simple scene: two distinct performers, a plain background, one central prop, and soft light. Before moving on, I check:

left/right order
separation between the performers
faces, hands, clothes, and props
room for motion
the final aspect ratio

If the still is wrong, I regenerate the still. Video credits do not need to be part of that debugging loop.

🕺 Pass two: describe only movement

The second prompt does less. I ask for subtle head movement, alternating hand gestures, natural breathing, and a steady camera. Fast turns, crossed arms, and complicated choreography can wait until the stable version works.

I tested this structure with an orange-studio duo workflow in Hotel Lobby AI. The site separates reference-image preparation from video generation, which fits the way I now debug these clips. It does not include an original song or guarantee exact choreography or lip sync, so I treat audio and final editing as separate steps too.

🧪 Keep a tiny generation log

For each attempt, I record the model, aspect ratio, duration, reference-image version, motion prompt, and the main defect. Then I change one variable. This is boring in the best way: after a few runs, successful setups become reusable instead of accidental.

Anyone else using a still-first workflow for multi-person image-to-video? What has helped you keep identities stable?

Treat an AI video prompt as a versioned test case

chen mensen — Sat, 03 Oct 2026 07:36:47 +0000

TL;DR

A generated clip is easier to review when its intent is written down before generation. Keep a small scene record, vary one input at a time, and distinguish a usable take from a visually impressive one. This is a workflow proposal, not a benchmark or a claim of deterministic generation.

For developers preparing short product demos, the difficult part is often the handoff: a clip looks attractive in the generator, then fails when you add the real caption or cut it into a sequence. The prompt is only one piece of that handoff. Crop, subject continuity, timing, and the intended edit matter too.

Start with a scene record

A plain JSON file is enough. The following schema is local bookkeeping, not an API payload for any video service:

{
  "sceneId": "cup-01",
  "revision": 1,
  "subject": "a ceramic cup on a plain table",
  "motion": "a slow camera push-in",
  "composition": "subject below the future caption area",
  "targetAspectRatio": "9:16",
  "reviewChecks": [
    "cup silhouette remains consistent",
    "caption area stays clear",
    "there is a usable continuous section"
  ],
  "attemptLimit": 3
}

The point is to make the acceptance criteria visible. “More cinematic” cannot tell a reviewer whether the result is ready. “The handle stays recognizable throughout the selected section” can.

Do not put credentials or private reference images into a public repository. A scene record can refer to an internal asset identifier instead of embedding the asset itself.

Separate intent, provider settings, and observations

These change for different reasons:

Record	Example	Why keep it separate?
Intent	Leave room for a caption	Comes from the finished edit
Settings	Model, supported duration, aspect ratio	Depends on the selected service/model
Observation	Handle changes halfway through	Describes one actual output

Copying the intent into a provider's prompt is a manual translation step. There is no assumption that every provider supports the same controls. For example, Social Video AI's text-and-image video workspace exposes model-dependent settings and credit costs, then lets you preview and download a clip. The scene record above can accompany that workflow, but it does not imply an API integration, automated posting, or reproducible seeds.

Reference images also do not guarantee exact preservation of a logo, product shape, or person. Treat those as things to inspect, not promises that the input enforces.

Make the review decision explicit

Use a small set of outcomes:

Keep: a usable section meets the scene's requirements.
Trim: the requirements hold only during part of the clip.
Revise: one identifiable problem warrants another attempt.
Stop: the scene needs different source material or ordinary footage.

A useful review note contains a reason and, when applicable, the usable interval. “Trim: opening composition works, but the cup changes shape later” helps the editor more than “version 2 looks best.”

Review the beginning, middle, and end at minimum, then watch the entire selected interval. Put the actual caption over it. A clear region in a still preview may not stay clear after the camera moves.

Version the hypothesis, not just the filename

If a take fails, change one meaningful cause. Keep the reference and framing while simplifying the motion, or keep the motion while changing the crop. Changing everything may produce a better clip, but it gives little evidence about why.

An example naming convention is cup-01-r02-take01. The revision identifies a changed instruction; the take identifies another output from that instruction. This is traceability, not determinism: repeated requests can still differ.

Record the displayed credit cost before committing to another attempt. A simple planning estimate is scenes × attempts per scene × credits per attempt, but costs can vary by model and settings. Recalculate when those settings change; do not treat one estimated number as a universal price.

Finish in the editor

Generation does not finish the social post. Captions, accurate branding, cuts, audio rights, and platform upload need separate checks. Sometimes trimming an existing take is more effective than generating another one.

The useful artifact is not a perfect prompt. It is a scene record plus a review decision that another person can understand. Which field would you add to make that handoff easier?

I stopped making my AI image prompts longer — style references helped more

chen mensen — Mon, 20 Jul 2026 09:13:50 +0000

I used to treat AI image generation like a writing problem. If the output was off, I added more words: more adjectives, more lighting notes, more camera terms, more "high quality" incantations.

That worked sometimes. But for visual work, it also hit a ceiling pretty fast. The prompt got longer, the image got noisier, and I still could not explain exactly why the result felt wrong.

The shift that helped me most was simple: stop relying only on text. Use references to control visual direction.

TL;DR

Long prompts are useful, but they are a weak way to communicate taste.
Reference images help with style, composition, palette, and mood faster than extra adjectives.
The best workflow is not prompt vs reference. It is prompt + reference + small batches + review.
For production work, judge generated images inside the layout where they will actually appear.

Why longer prompts start to fail

Text is great for objects and actions:

a product manager reviewing a dashboard on a laptop

But text is weaker for taste:

premium, modern, clean, cinematic, editorial, soft, minimal

Those words mean different things to different people. They also mean different things to different models. "Clean" might produce a white SaaS landing page look. "Editorial" might push the result toward magazine photography. "Cinematic" might add dramatic shadows you did not want.

So the prompt becomes a pile of corrections:

modern but not too futuristic, clean but not empty, cinematic but not dark,
premium but not luxury, realistic but slightly stylized

At that point, the prompt is not really a creative direction. It is a negotiation.

References are better at carrying visual taste

A reference image can say things that are painful to describe:

How much contrast feels right
Whether the image should feel photographic or illustrated
What kind of color palette is acceptable
How dense the composition should be
What "premium" means in this specific project

Even one reference can reduce ambiguity. Three references, each labeled with a purpose, are even better:

one for lighting
one for composition
one for color or style

The important bit is labeling the references. A random moodboard is just another kind of noise.

The workflow I use now

1. Write the visual job first

Before prompting, I write one sentence:

Create a blog header for a technical article about reference-guided AI image workflows.

That sentence gives the image a job. It is easier to reject a beautiful but useless image when the job is clear.

2. Keep the prompt boring

The prompt should describe subject, format, and constraints. I avoid trying to pack every aesthetic decision into text.

editorial image, designer comparing AI image variations on a review board,
clean workspace, browser-based creative workflow, wide blog header

3. Use references for taste

This is where tools that support reference-guided workflows become useful. A tool like Krea 2 is useful in this context because it lets the image process include prompts, image-to-image direction, references, and aspect ratio choices instead of treating the text prompt as the only control surface.

4. Generate small batches

I avoid changing everything at once. One batch tests composition. Another tests style strength. Another tests crop safety.

If every generation changes prompt, references, ratio, and style at the same time, you learn almost nothing from the result.

5. Review inside the actual layout

Gallery view lies.

An image can look good in isolation and fail as soon as it sits under a headline. It may be too busy. The focal point may be hidden on mobile. The colors may fight the CTA.

So I test the image where it will live: blog header, landing page hero, social card, newsletter, or ad mockup.

The practical rule

Use text for what should be in the image.

Use references for how it should feel.

Use layout review for whether it actually works.

That sounds obvious, but it changed the way I use image generation. I spend less time writing giant prompts and more time building a small visual system that can be reviewed.

Anyone else using references or moodboards in their AI image workflow? Curious how you keep the outputs consistent across a batch.

I tried generating a full song from a text prompt — here's what actually works (and what doesn't)

chen mensen — Sun, 19 Jul 2026 16:14:33 +0000

I needed background music for a side project's demo video. Hiring someone was overkill, stock libraries all sounded the same, and I can't play an instrument. So I went down the "text-to-song" rabbit hole — the class of AI tools where you type a description and get back a finished track with vocals, lyrics, and production. Here's what I learned after a week of messing with it.

TL;DR

Text-to-song has quietly gotten good. A one-line prompt now returns a ~3 min track with coherent structure (intro/verse/chorus).
The prompt that matters isn't the vibe words — it's genre + instrumentation + mood + tempo, in that order.
It's great for demos, placeholders, and ideation. It's not ready to replace a composer for anything where the music is the product.
Watch the licensing fine print. "Royalty-free" means different things on different platforms.

What "text-to-song" actually does

You type something like warm acoustic guitar ballad, male vocals, nostalgic, 80 bpm and the model returns a full song — not a loop, an actual arrangement with a vocal line and lyrics it wrote itself. Under the hood it's the same generative-audio wave that Suno and Udio kicked off; the newer entrants (I bounced between a few, including a free one called Flow Music AI) mostly differ in vocal quality, how much control they give you, and their licensing terms.

The mental model that helped me: it's not a DAW, it's a slot machine you can bias. You don't edit — you describe, roll, and re-roll until something lands, then nudge the prompt.

The workflow that actually produced usable tracks

1. Lead with genre and instrumentation, not adjectives

My first prompts were things like emotional, cinematic, epic. Garbage in, garbage out — I got generic trailer music. What worked was front-loading concrete musical nouns:

lo-fi hip hop, dusty piano, vinyl crackle, mellow, 70 bpm, no vocals

Adjectives (emotional, dreamy) are fine as seasoning, but the genre + instruments do the heavy lifting.

2. Decide vocals vs instrumental up front

Instrumental is dramatically more reliable — no lyrics to go wrong, no uncanny-valley vocals. For anything where the track sits under other audio (demos, videos, podcasts), go instrumental. Only reach for vocals when the song itself is the point.

3. Re-roll before you re-prompt

Same prompt, different seed = different song. Most of my "aha" tracks came from re-rolling an already-decent prompt 3–4 times, not from endlessly rewording. Reword only when the results are consistently off in the same direction.

4. Generate long, trim later

The models output full-length songs. For a 20-second demo bed I'd generate the whole thing and cut the best 20 seconds in any editor. Trying to prompt for a short clip usually gives you a worse full song, not a good short one.

Where it genuinely helped

Demo/placeholder music — the killer use case. Background beds for product videos, app previews, loading screens. "Good enough and free" beats "perfect and expensive" here.
Ideation for actual musicians — a friend who writes uses it to sketch arrangement ideas fast, then rebuilds the good ones properly.
Prototyping vibe — testing whether a scene wants jazz or synthwave before committing budget.

The honest limitations

The stuff the marketing pages skip:

You don't get stems. You get a stereo mix. Want to remix, swap an instrument, or fix one bad bar? You can't — you re-roll the whole thing and hope. This is the single biggest dealbreaker for serious use.
Vocals still tell. Instrumentals pass easily; vocal tracks have a texture that trained ears catch. Fine for a demo, risky for release.
Consistency across tracks is hard. Getting two songs that sound like the same "artist" is luck, not a setting.
Licensing is a minefield. "Royalty-free" on the tin doesn't always mean commercial-safe, and the whole category has open legal questions around training data. If it's going in something commercial, read the actual terms, don't trust the landing page.

So who is this for?

If you need music-shaped audio to fill a gap — a demo, a placeholder, a mood test — text-to-song is genuinely useful today, and the free tiers are enough to try. If the music itself is the deliverable, treat these tools as a sketchpad, not a replacement for someone who knows what they're doing.

I ended up using an instrumental lo-fi track for my demo video. Took maybe ten minutes and zero dollars. Would I ship it as a single? No. Did it solve my actual problem? Completely.

Anyone using AI-generated music in production for real — not demos? Curious how you're handling the licensing side, because that's the part that still makes me nervous.

I stopped writing image prompts and started dropping in reference images instead — here's what I learned

chen mensen — Thu, 09 Jul 2026 04:59:18 +0000

If the slowest part of image generation for you isn't the render time but translating the picture in your head into a prompt, this one's for you. I spent a while digging into the "prompt with images, not words" approach (the thing Google Whisk popularized) and want to share how the workflow actually plays out — and where it falls flat.

TL;DR

Text prompts are hard because you have to verbalize something visual, learn domain vocab (shallow depth of field), and each model interprets words differently.
"Using reference images" actually splits into 3 very different techniques. Only one of them is the low-barrier, web-based kind.
The web-friendly one (caption-mediated) is great for exploration/mockups but cannot preserve identity — that's by design, not a bug.

Why text prompts are the bottleneck

Break it down and there are three structural problems:

Verbalization loss — the image in your head has composition, texture, light and color all at once. Flattening that into a word sequence drops information every time.
Vocabulary dependency — chiaroscuro, isometric view… the words that get you the result are the domain knowledge. You can't write what you don't know.
Model dialects — the same prompt reads differently on different models. Prompts age badly as assets.

So the natural question: why translate something back into words when you can just show it?

"Reference images" is actually 3 different things

This tripped me up at first. There are at least three families, and they behave very differently:

Technique	What it does	Best for	Setup cost
Pixel conditioning (img2img / ControlNet)	Injects contours/depth/pose directly	Locking composition & pose	High (local env)
Feature embedding (IP-Adapter / LoRA)	Turns the image into feature vectors	Reproducing a specific style	High (training)
Caption-mediated (Whisk-style)	A VLM captions the image, feeds text to the generator	Early exploration, style tests	Low (web)

This post is about the third one. A vision-language model (Gemini, etc.) reads your reference images, auto-writes a detailed caption, and hands that text to the image model. You just drag images in — but under the hood it's image → text → image.

The key thing: it extracts the essence of your reference, it does not copy pixels. Remember this — it's the root of the limitation below.

The workflow in practice

Prep references by role — up to 3: subject (what), scene (where), style (what look). You don't need all three. One busy image with too much going on = unwanted features leaking in, so keep it one-role-per-image.
Check the intermediate caption — the single most useful feature to look for in a tool: can you see and edit the caption the AI generated? If yes, you can spot where it misread your reference and fix it at the text level. If it's a black box, you're stuck when it's off.
Review the first result — judge it as "a new image carrying the reference's traits," not "a reproduction." Check (a) how much subject identity survived, (b) whether the style transfer is surface or structural, (c) any unwanted elements crept in.
Iterate via caption edits — the move that works: lock the big picture (composition/style) with images, refine details (color, props, expression) with text. You don't drop text prompts entirely — you go from writing from scratch to writing only the diff.

The honest limitations

Most posts recommending this workflow skip these, so:

Identity is not guaranteed. Since it's caption-mediated, your reference gets "summarized" into text once. A specific face, a brand logo, character details — anything lost in summarization won't come back. Drop in your pet and you get "a similar-looking different dog." That's normal behavior. Need exact identity? That's the pixel-conditioning territory.
The gacha stays. Even with editable captions, final generation is probabilistic. Budget for iteration, not one-shot.
Not a replacement for final production. High-res output still needs human review before it ships. Hands, text, fine patterns — same weak spots as every image model.

Where it actually shines

For me it sped things up in these phases:

Merch/mockups — turning a character into enamel-pin / sticker / plush styles for pre-production previews. "Vibe check" over "exact reproduction," so the weakness doesn't bite.
Character variation exploration — dropping an existing character into different styles/settings, dozens at a time. Great at the concept stage.
Blog/LP visual roughs — a starting point before briefing a designer.

If you want to try the caption-mediated flow without setting up a local environment, Google's Whisk is the original, and web services like Whisk AI layer editable intermediate prompts and model switching on top of the same idea.

Wrap-up

Reference-image workflows fit the phase where you can show what you want but can't easily say it — idea exploration, style experiments, mockup volume. If your requirement is strict composition or faithful reproduction of a specific character, ControlNet/LoRA locally is still the answer.

Text prompts aren't going away. But "verbalize from zero" is quietly being replaced by "reference image + diff edits." The input interface for image generation is inching in that direction.

Anyone else using image-first prompting in production? Curious whether the editable-caption feature actually gets used or if people just re-roll.