Why Composited People Look Pasted: Lighting and Color Coherence

Mia Tanaka — Sat, 25 Jul 2026 08:09:51 +0000

You can nail every facial feature and still produce a composite that looks obviously fake. I've watched people burn weeks on identity preservation, get it perfect, then ship an image where two people supposedly standing in the same room are lit by two different suns. The brain catches it instantly, without being able to say why. Lighting coherence is the part nobody writes about, so here's what I've learned making it work.

Why "pasted" is a lighting judgment, not a geometry judgment

Human vision is startlingly bad at absolute measurements and startlingly good at consistency checks. We can't tell you a room's color temperature, but we detect within milliseconds that one face is 5500K daylight and the other 3000K tungsten. Same with shadow direction: nobody can estimate a light's azimuth, but two faces with shadows falling opposite ways read as wrong before you've consciously looked.

Four signals dominate the "pasted" verdict:

Color temperature — the global chromatic cast on skin
Key light direction — where the dominant light comes from, which drives shadow side
Contrast ratio and softness — hard sun vs. overcast vs. bounced indoor light
Edge and atmospheric consistency — how the subject boundary and ambient haze behave

Identity isn't on the list. You can preserve every landmark perfectly and fail all four.

Estimate the lighting before you generate anything

The mistake is treating harmonization as post-processing. By the time you have a composite, half the information is gone. Estimate lighting parameters from each source photo first, then use them to constrain generation or pick which subject gets relit toward the other. Rough but effective estimators, no ML required:

import numpy as np

def estimate_color_temp(skin_rgb):
    """Crude CCT proxy. Higher R/B => warmer light. Relative, not Kelvin."""
    r, g, b = skin_rgb.mean(axis=0)
    return r / max(b, 1e-6)

def estimate_key_direction(gray, face_mask):
    """Shading gradient over the face approximates the light azimuth."""
    gy, gx = np.gradient(gray.astype(float))
    # Light comes FROM the direction of increasing luminance.
    return np.degrees(np.arctan2(-gy[face_mask].mean(),
                                  gx[face_mask].mean()))

def estimate_hardness(gray, face_mask):
    """Contrast ratio proxy; hard light => wide spread."""
    v = gray[face_mask]
    p95, p05 = np.percentile(v, 95), np.percentile(v, 5)
    return (p95 - p05) / max(p95 + p05, 1e-6)

The gradient trick works because a face is roughly convex. Under a single dominant light, luminance falls off with the cosine of the angle between surface normal and light — so the mean image gradient across the face points, near enough, along the projected light direction. It fails on flat overcast lighting (no meaningful gradient, itself a useful answer) and on strong backlighting. Good enough to route decisions.

Now you have a comparison, and it tells you what to do:

delta_azimuth = |dir_A - dir_B|

  < 30 deg  -> compatible; harmonize color only
  30-90 deg -> relight the weaker-key subject toward the stronger one
  > 90 deg  -> conflicting keys; do NOT composite as-is. Pick one as
               authoritative and regenerate the other's shading, or
               choose a scene (overcast) that flattens both.

That last branch matters more than any algorithm. If one photo is backlit sunset and the other front-flash indoors, no harmonization pass saves it. The correct engineering answer is to detect the conflict and change the target scene, not grind harder on blending.

Harmonization: match statistics in a perceptual space, not RGB

The classic move is Reinhard-style statistics transfer: shift and scale each channel so the source's mean and standard deviation match the target's. Done in RGB it wrecks skin tones, because RGB channels are heavily correlated and you rotate hue while trying to fix brightness. Do it in a decorrelated, roughly perceptual space — Lab or Ruderman's lαβ:

def match_stats(src_lab, ref_lab, mask):
    out = src_lab.copy()
    for c in range(3):
        s, r = src_lab[..., c][mask], ref_lab[..., c]
        sd_s = s.std() or 1e-6
        out[..., c] = (src_lab[..., c] - s.mean()) * (r.std() / sd_s) + r.mean()
    return out

Two refinements:

Clamp L harder than a and b. Chroma can move a lot before it looks wrong; luminance cannot. Rescaling L aggressively flattens facial modeling into that waxy, cut-out look. Keep L's scale factor near 1.0 and let a/b do most of the correction.

Match on skin only, apply globally. Compute statistics from the skin mask — hair and clothing have wildly different reflectances and drag your means around — but apply the transform to the whole subject. Skin-only application is a classic bug: a correctly-toned face on a body still lit by the original room.

Shadows are where composites die

Color you can fix. Contact shadows you have to produce. Two subjects on the same ground plane cast shadows in the same direction with consistent softness, and occlude each other at contact points. Without cast shadow, subjects float — the most common tell in generated two-person images.

Minimum viable shadow: project the silhouette along the estimated azimuth, scale by an assumed light elevation, blur proportionally to the hardness estimate, and multiply (never subtract) into the background. Multiply preserves underlying texture; subtraction crushes it to a flat gray patch.

The hard part is consistency of softness. A hard-edged shadow under one person and a soft gradient under the other is as jarring as mismatched color, and it gets skipped because it's invisible in isolation and glaring in comparison.

Getting the full chain right end-to-end — estimate, decide compatibility, relight, then generate contact shadows — is exactly the pipeline behind DuoPortrait, which composites two people from separate photos; running real user uploads through it made clear that the estimator's conflict detection branch earns its keep more than the harmonizer does. Most failures aren't bad blending. They're two photos that should never have been blended in the first place.

So the rule I'd hand anyone starting this: measure lighting compatibility before committing to a composite, and treat incompatibility as a routing decision rather than a problem to blend away. A pipeline that says "these two photos won't work together, here's why" produces better output than one that always returns something.

Identity Consistency Is the Hard Part of Two-Person Image Generation

Mia Tanaka — Fri, 24 Jul 2026 19:04:10 +0000

Generating one convincing person from a reference photo is close to solved. Generating two specific people in the same frame, both still recognisable, is a different problem — and it fails in ways single-subject generation does not.

Identity bleed

Put two reference identities into one generation and they contaminate each other. You get a face that is an average of both, or person A wearing person B's jawline. The model has no inherent notion that these are two distinct people who must stay distinct.

This gets worse the more similar the references are — same age range, same hair colour, similar bone structure. Exactly the case that matters most, since people generating a two-person photo are often family or partners.

Why naive approaches fail

Generating separately and compositing keeps identity intact and produces obviously fake results. Lighting direction, colour temperature, film grain, and perspective never match. The human eye is extremely good at detecting this — it reads as a cut-out immediately.

One pass with both references is where bleed happens.

Inpainting the second person into a single-subject generation preserves the first identity well and tends to produce anatomically odd contact points — the classic four-arm hug, or hands that merge.

What helps

Spatial conditioning up front. Deciding where each person goes before generation — via masks or layout conditioning — gives the model a structural reason to keep them separate. Identity constraints work far better when tied to a region than applied globally.

Per-region identity embeddings. Scope each reference to its region rather than injecting both into global conditioning. This is the single biggest lever on bleed.

Explicit contact handling. Poses where subjects touch — arms around shoulders, holding hands — are where anatomy breaks. Worth treating contact-heavy poses as a distinct category with tighter constraints, not as ordinary generations.

Lighting coherence is the tell

Even with identity solved, mismatched lighting kills realism. Both subjects need a shared light model: same key direction, same colour temperature, consistent shadow falloff.

This argues for single-pass generation over compositing — the model gets lighting coherence for free when it renders both subjects together, and getting it right post-hoc is much harder than it sounds.

Evaluation is genuinely awkward

There is no clean automatic metric. Face embedding similarity to each reference is necessary but not sufficient — you can score well on both and still produce an image that looks wrong, because "looks like a real photo of these two people" includes scene plausibility that embeddings do not capture.

Practical compromise: embedding similarity as a gate, human review for the rest.

I work on this at DuoPortrait — two references in, one plausible shared photo out.

Caveat

Everything above is empirical. The field moves fast enough that specific techniques age quickly; the failure modes have been more durable than the fixes.

DEV Community: Mia Tanaka