Transparent Image Layer Diffusion using Latent Transparency

Controlnetの作者が発表した

This paper enables SD to generate transparent data.
No code?

Dataset
Dataset preparation is conducted in a human-in-the-loop way.

The proposed method categorizes the training into two sets.
In the first stage, base-model training is conducted by only training with transparent images.
In the second stage, the training is conducted by multiple-layer images with foreground LoRA and background Lora.
Parameters in Stable Diffusion are frozen.
This model can be used with ControlNet.

Therefore, the model can create almost all of the properties.

感想
おもろい
広告とかに使えそう