How to Extract LoRA from FLUX Fine Tuning Full Tutorial + Comparison Between Fine Tuning Extraction vs LoRA Training

#tutorial #ai #opensource #news

Details

As you know I have finalized and perfected my FLUX Fine Tuning workflow until something new arrives
It is exactly same as training LoRA just you load config into the DreamBooth tab instead of LoRA tab
Configs and necessary explanation are shared here : https://www.patreon.com/posts/kohya-flux-fine-112099700
Currently we have 16GB, 24GB and 48GB FLUX Fine-Tuning / DreamBooth full check point training configs but all yields same quality and just the training duration changes
Kohya today announced that the lower VRAM configs will get like 30% speed up with Block Swapping technique algorithm improvements, hopefully
It has been commonly asked of me how to extract LoRA from full Fine-Tuned / DreamBooth trained checkpoints of FLUX
So here a tutorial for it with comparison of different settings
In this post, Image 1–5 are links to full images so click them to see / download

We are going to use Kohya GUI
How to install it and use and train full tutorial here : https://youtu.be/nySGu12Y05k
Full tutorial for Cloud services here : https://youtu.be/-uhL2nW7Ddw
The default settings it has is not working good
Thus look at the first image shared in the gallery and set as it is to extract your FLUX LoRAs from Fine Tuned / DreamBooth trained checkpoints
Follow the steps in as in the Image 1

You can change save precision to FP16 or BF16, both will halve the size of the saved LoRA into disk
Are there any quality difference?
You can see comparison in the Image 2 and I didn’t notice any meaningful quality difference
I think FP16 is more close to FP32 saving
Another thing you can change is setting Network Dimension (Rank)
It works as much as up to 640 and above gives error
The more the Rank you save, it is more closer to the original Fine Tuned model, but it will take more space
You can see Network Dimension (Rank) comparison in the Image 3

I find that giving 1.1 strength to extracted LoRA makes it more resembling to the original Fine Tuned / DreamBooth trained full checkpoint when Network Dimension (Rank) is set to 640
You can see full LoRA strengths comparison in Image 4
If you use lower Network Dimension (Rank), you may be need to use higher LoRA strength
I use FLUX in SwarmUI and here full tutorial for SwarmUI
Main tutorial : https://youtu.be/HKX8_F1Er_w
FLUX tutorial : https://youtu.be/bupRePUOA18

With same training dataset (15 images used), same number of steps (all compared trainings are 150 epoch thus 2250 steps), almost same training duration, Fine Tuning / DreamBooth training of FLUX yields the very best results
So yes Fine Tuning is the much better than LoRA training itself
Amazing resemblance, quality with least amount of overfitting issue
Moreover, extracting a LoRA from Fine Tuned full checkpoint, yields way better results from LoRA training itself
Extracting LoRA from full trained checkpoints were yielding way better results in SD 1.5 and SDXL as well
Comparison of these 3 is made in Image 5 (check very top of the images to see)
640 Network Dimension (Rank) FP16 LoRA takes 6.1 GB disk space
You can also try 128 Network Dimension (Rank) FP16 and different LoRA strengths during inference to make it closer to Fine Tuned model
Moreover, you can try Resize LoRA feature of Kohya GUI but hopefully it will be my another research and article later