DEV Community

AI-KSK
AI-KSK

Posted on Edited on Originally published at huggingface.co

Face Swapping That Survives Upscaling: The Paste-Back Pattern

Face swap workflows usually fail at the edges. The swapped face looks right in isolation and wrong in the image, because the seam does not match the surrounding lighting or the skin texture changes abruptly. The paste-back pattern fixes that by separating the swap from the composite.

The three graphs

Workflow Nodes Base model
Face swap (example) 39 qwen_image_edit_2509_fp8_e4m3fn
Face swap (redraw + paste back) 55 qwen_image_edit_2511_bf16
Klein 9B image upscale 31 flux-2-klein-9b

Two model versions, deliberately

The example graph uses 2509; the paste-back graph uses 2511. Those are different generations of Qwen Image Edit, and the difference matters for face work: later versions handle identity preservation better, which is exactly the failure mode a face swap exposes.

Note that the example graph loads 2509 as fp8_e4m3fn while the paste-back loads 2511 as bf16. Different quantisation, different generation — so the two graphs are not directly comparable.

The BFS head LoRA

Both Qwen graphs load:

LoraLoaderModelOnly → bfs_head_v5_2511_merged_version_rank_16_fp16.safetensors at 1.0

The name is informative: v5, 2511-targeted, rank_16, fp16, merged_version.

Rank 16 is low. A rank-16 LoRA has limited capacity, which for a face-specific LoRA is the right choice — it constrains what the LoRA can change. A high-rank LoRA would let it alter too much and the result stops looking like the target face.

merged_version suggests the weights have been merged rather than kept as a delta. Merged LoRAs sometimes behave differently from deltas at the same nominal weight, because the merge happened against a specific base version.

Why paste-back works

The 55-node graph is 16 nodes larger than the example, and those 16 nodes are the paste-back machinery. The pattern:

  1. Swap the face in a working region, not the whole image
  2. Decode the result
  3. Paste only the swapped region back into the original
  4. Blend the seam

The alternative — swap the face and use the result as the whole image — lets the model alter everything outside the face too. Anything it changed outside the intended region is damage.

Paste-back confines the change. That is why it survives upscaling: an upscaler amplifies whatever the model introduced, and if nothing was introduced outside the face, nothing gets amplified there.

The Klein 9B upscale graph

flux-2-klein-9b with qwen_3_8b (type flux2) and flux2-vae.

31 nodes for upscaling is a considered number: upscaling wants to enhance detail without re-deciding composition, and that means constrained sampling. The graph is not doing face work at all — it is a separate stage the release pairs with the swap.

Where this breaks

  1. Two edit model generations across the graphs, with different quantisation.
  2. Rank 16 limits how much identity can shift. If the target face differs substantially from the source, low rank will not carry enough.
  3. Paste-back needs a good mask. A sloppy region boundary shows as a soft edge after upscale.
  4. The merged_version naming means the LoRA is tied to a specific base version; pairing it with a different base may silently degrade.

Practical order

Run the 39-node example to confirm the LoRA loads and produces a plausible swap. Then move to the 55-node paste-back graph, because that is the one whose result survives an upscale. Upscale last.

Workflows

Three graphs, published on RunningHub with invite code rh-v1111:

  • 2511 face swap (example)
  • 2511 face swap (redraw and paste back)
  • Klein 9B image upscale

Files

https://huggingface.co/AI-KSK/ksk-video-191-comfyui-workflows

Notes

  • Workflow files only. Model weights are not included; each keeps its own licence.
  • Tutorial exports, not a product.
  • Node counts and loader values quoted above are read from the published JSON files.
  • Independent AI-KSK release. Not affiliated with the model authors or RunningHub.

Optional API platform - 88API (affiliate link)

Explore 88API for access to multiple AI model APIs. Available models, routes and prices are listed on the platform; Alipay top-ups are supported.

https://88api.ai/sign-up?aff=usx3

Disclosure: I may earn a commission from eligible purchases after you register through this referral link.

Top comments (0)