From Two Photos to a 15-Second Zombie Story: How an AI Photo-to-Video Pipeline Works (Halloween 2026 Creative Coding)

Eason — Thu, 15 Oct 2026 06:32:59 +0000

From Two Photos to a 15-Second Zombie Story: How an AI Photo-to-Video Pipeline Works

By Eason

Every October someone in your group chat asks, "Can you make me into a zombie?" In 2026 the answer is less "open Photoshop" and more "run a pipeline." This autumn a short vertical format called the AI zombie trend (also called the zombie hug or zombie love story) spread on TikTok and Instagram Reels: one person has turned, the other recognizes them, lowers a prop rifle and opens their arms, and the clip hard-cuts to a warm memory of the two of them.

I want to unpack how a product like that works under the hood. This is a concept tutorial, not a vendor-specific API guide, so you can map the stages onto whatever models you like for your own Halloween creative-coding project.

The shape of the problem

Inputs: two still photos (person A, the one who turns; person B, the one who remembers).
Output: a short vertical MP4 with sound, where both people stay recognizable across two very different scenes.

That breaks into five stages:

Input validation and preprocessing
Identity representation
Scene and story conditioning
Image-to-video generation (with a mid-clip cut)
Audio, encoding and delivery

1. Input validation and preprocessing

Garbage in, garbage out applies doubly to generative video. Before anything expensive runs, a good pipeline checks:

One person per photo. A face detector counts faces; more than one is ambiguous ("which one turns?").
Face visibility and lighting. Small, blurry or backlit faces lose identity fast once a model starts adding gore and grime.
Format and size. Normalizing HEIC/WebP/PNG to a common format and capping file size keeps the queue predictable.
Consent and policy. Real products should refuse photos of children and celebrities. That is a product decision, but it belongs at the very front of the pipeline.

Then you crop: a waist-up crop centered on the face gives the video model enough body to animate a run and a hug without losing the face.

2. Identity representation

The hard part of any "make me into X" video is that X must still look like you. Approaches you will see:

Reference-image conditioning. The cropped photo is fed directly to an image-to-video model as the first frame or as a reference.
Identity embeddings. A face-recognition encoder produces a vector that guides generation toward the same person.
Two-subject handling. With two people, you must keep identities from bleeding into each other. Mixed-up faces are the classic failure mode, which is why products that do this often offer a remake.

A useful trick: generate a stylized "zombified" keyframe of person A first (pale skin, torn clothes, clouded eyes) while keeping facial geometry, then animate from that keyframe.

3. Scene and story conditioning

Users should not have to write prompts. Instead, the pipeline maps a few UI choices to a structured prompt template:

relationship: couple | friends | family | dog | cat
scene_a: ruined city street at dusk
scene_b: golden wheat field, warm sunlight
beats: [recognize, lower_rifle, run, leap, cut_to_memory]

Seasonal variants are just new templates. A Halloween pumpkin patch that cuts to an autumn afternoon, or a candle-lit Día de Muertos alley that cuts to a field of marigolds, reuse the same beats with different scene descriptors. Encoding the story as beats, not prose, makes results more consistent.

4. Image-to-video with a hard cut

Most current image-to-video models produce coherent clips of a few seconds. A 15-second story is easier as segments:

Segment 1 (horror): zombie A and survivor B in scene A, recognition and the run.
Segment 2 (memory): both as their normal selves in scene B.

The cut lands mid-leap. That edit is doing real work: it hides the seam between segments and it is the emotional payoff of the format. Generate each segment from its own keyframe, then concatenate. Output is 9:16 because that is where the content lives.

5. Audio, encoding and delivery

Sound matters more than people expect. A low drone for the horror half and a softer track after the cut sells the twist. Mux it in, encode to H.264 MP4 at a mobile-friendly resolution like 720p, and you are done rendering.

Generation takes minutes, not milliseconds, so treat it as an async job: put it on a queue, let the user close the tab, and email a private link when it is ready. Add retention rules (delete source photos after a few days, videos after a few weeks) so you are not hoarding people's faces.

Build it or use it?

If you want to learn, wiring this up yourself is a great Halloween weekend project. Start with a single subject and a single segment, then add the second person and the cut.

If you just want the clip for your feed, a finished tool saves you the GPU bill. For reference, Zombie Hug packages exactly this flow: upload two photos, pick who is in the video and a scene, and get a 15-second 9:16 720p MP4 with sound by email, usually in 5 to 10 minutes. It is paid per video, which is a fair trade against debugging identity bleed at 2 a.m.

Takeaways

Validate and crop inputs aggressively; it is the cheapest quality win.
Preserve identity with reference conditioning, and watch for two-face mixups.
Encode stories as beats, not freeform prompts.
Use the hard cut to hide segment seams and land the emotion.
Run generation as an async job with privacy-minded retention.

Happy haunting, and may your faces never bleed into each other.

DEV Community: Eason