The short answer: if the performance already exists, use the video as the performance specification and the image only as the identity reference. A text prompt is useful for inventing a scene; it is a poor substitute for motion, camera and timing that you already want to keep.
“Character swap” is used for several jobs: replacing only a face, transferring an existing performance to a new character, or generating a new scene that merely resembles the source. This article is about the second job: one clear main subject, one replacement character, and a short source clip whose performance should remain recognisable.
Write the input contract before generating
Treat the two inputs as separate controls:
| Input | It should control |
|---|---|
| Reference video | Motion, camera, framing, timing, background and—when selected—original audio |
| Character image | The identity and appearance of the main subject |
You are not asking whether the result is generally attractive. You are asking whether the requested identity changed while the protected parts remained stable.
For a first test, use:
- one MP4 or MOV between 4 and 15 seconds;
- one clear main subject and no hard cuts;
- one sharp, authorised character image;
- similar framing and head angle between the image and the visible subject;
- the original audio when its timing matters.
The 4–15 second range here is the current limit of the browser workflow I built, not a universal limit for every video model.
Test the hardest few seconds first
Do not start with the longest clip you can upload. Find a four- to five-second segment most likely to expose a failure:
- a profile turn;
- a hand crossing the face;
- motion blur;
- a fast camera move;
- small captions or graphics near the subject.
If identity already drifts in that segment, a longer generation will not fix it. Change one variable: use a clearer character image, crop out secondary people, or choose a less obstructed reference. Changing all three at once makes the next result impossible to diagnose.
Keep audio preservation separate from lip sync
“The original soundtrack is present” and “the mouth movement is convincing” are different checks.
When you keep the original sound, compare the same moments in source and output:
- Does speech or music start at the same frame?
- Do visible beats and gestures still land together?
- Do burned-in captions still appear at the intended moment?
- Does the mouth follow the retained speech throughout the clip?
If the words need to change, character replacement is only the first pass. Stabilise the character result before running a separate dubbing or lip-sync pass. Trying to replace identity, rewrite dialogue and repair timing in one generation hides which stage failed.
Review matching frames, then watch in real time
Capture the beginning, middle and end of both files and compare them side by side. Check:
- Identity: does the new character remain recognisable during turns?
- Motion: are gestures and body position still recognisable?
- Scene: did the background, crop or camera drift?
- Occlusion: what happens when an object crosses the subject?
- Audio: is the original track still aligned?
- Text: did captions, prices or interface details change?
Frame checks catch drift, but they can miss unnatural transitions. Finish by watching the entire clip at normal speed with sound.
Know when the task is a bad fit
A single-subject workflow should not be sold as multi-character editing. Split the shot or use a different tool when the source contains:
- two equally prominent people;
- repeated hard cuts;
- a face that is tiny or hidden for most of the clip;
- long occlusions;
- a requirement to remove watermarks or burned-in text during the same pass.
Generative output can change fine detail even when the overall motion and scene remain recognisable. Always review the result before publishing it.
Make cost part of the test plan
The practical unit is output duration. In UseLipSync Recreate, the current rate is 700 credits per output second, and the exact total is shown before submission. A six-second result therefore quotes 4,200 credits. Reserved credits are released automatically if the generation fails technically.
That is another reason to validate the hardest short segment first instead of treating a 15-second generation as the first experiment.
Keep a reproducible test record
For every comparison, record the source clip and duration, character image, sound setting, quoted cost, output, and review notes for identity, motion, scene, occlusion, audio and text.
One successful clip proves only that input pair. A small recorded matrix of easy, profile, occluded and fast-motion cases is much more useful than a highlight reel.
Disclosure: I built UseLipSync. The production workflow accepts an authorised character image plus your own 4–15 second reference clip, shows the exact quote before submission, and can retain the reference audio. The step-by-step guide and real before/after example are here:
Top comments (0)