Identity Consistency Is the Hard Part of Two-Person Image Generation

Mia Tanaka — Fri, 24 Jul 2026 19:04:10 +0000

Generating one convincing person from a reference photo is close to solved. Generating two specific people in the same frame, both still recognisable, is a different problem — and it fails in ways single-subject generation does not.

Identity bleed

Put two reference identities into one generation and they contaminate each other. You get a face that is an average of both, or person A wearing person B's jawline. The model has no inherent notion that these are two distinct people who must stay distinct.

This gets worse the more similar the references are — same age range, same hair colour, similar bone structure. Exactly the case that matters most, since people generating a two-person photo are often family or partners.

Why naive approaches fail

Generating separately and compositing keeps identity intact and produces obviously fake results. Lighting direction, colour temperature, film grain, and perspective never match. The human eye is extremely good at detecting this — it reads as a cut-out immediately.

One pass with both references is where bleed happens.

Inpainting the second person into a single-subject generation preserves the first identity well and tends to produce anatomically odd contact points — the classic four-arm hug, or hands that merge.

What helps

Spatial conditioning up front. Deciding where each person goes before generation — via masks or layout conditioning — gives the model a structural reason to keep them separate. Identity constraints work far better when tied to a region than applied globally.

Per-region identity embeddings. Scope each reference to its region rather than injecting both into global conditioning. This is the single biggest lever on bleed.

Explicit contact handling. Poses where subjects touch — arms around shoulders, holding hands — are where anatomy breaks. Worth treating contact-heavy poses as a distinct category with tighter constraints, not as ordinary generations.

Lighting coherence is the tell

Even with identity solved, mismatched lighting kills realism. Both subjects need a shared light model: same key direction, same colour temperature, consistent shadow falloff.

This argues for single-pass generation over compositing — the model gets lighting coherence for free when it renders both subjects together, and getting it right post-hoc is much harder than it sounds.

Evaluation is genuinely awkward

There is no clean automatic metric. Face embedding similarity to each reference is necessary but not sufficient — you can score well on both and still produce an image that looks wrong, because "looks like a real photo of these two people" includes scene plausibility that embeddings do not capture.

Practical compromise: embedding similarity as a gate, human review for the rest.

I work on this at DuoPortrait — two references in, one plausible shared photo out.

Caveat

Everything above is empirical. The field moves fast enough that specific techniques age quickly; the failure modes have been more durable than the fixes.