DEV Community

Cover image for Creating Style Previews with Stable Diffusion
JakovGlavac
JakovGlavac

Posted on

Creating Style Previews with Stable Diffusion

Enhancing Your Shopping Experience with AI and LORA

While wisiting a cloating e-commerce shop, I had a random thought: "Wouldn't it be cool to preview clothes on myself while shopping?" As a developer by trade, and having prior experience with stable diffusion using Dream Booth, I decided to build an MVP to do just that.

Exploring LORA

First, I attempted to use Dream Booth to learn my face. However, I stumbled upon something even cooler: LORA (Low-Rank Adaptation). LORA proved to be faster and easier to train and had the potential for combining different aspects. For instance, you could combine a LORA model of a user's face with a LORA model of the clothes you want that user to wear.

I started by creating a LORA model of my face and had some fun with it, generating cool images like this one:

Image description

Then, I created a LORA model of jackets from images found on an e-commerce website:

Jacket images

Next, I combined these LORA models in a prompt and achieved pretty decent results:

Image description

Image description

Image description

Image description

However, I noticed that the images weren't perfect, especially when it came to small details of the jacket and my face. I ran more prompts and tried to improve the LORA model, but my face was consistently off.

Troubleshooting the LORA Model

I experimented with several solutions:

  1. I attempted to label a person in the jacket dataset as a model and included this as a negative prompt, but it didn't yield significant improvements.

  2. I refined the prompts and used WD4 to annotate images, but the results remained suboptimal.

After trying numerous approaches without groundbreaking success, I decided to explore using a control net to correct the person's pose and apply masks to allow LORA to work on each part of the image independently. Here's what it looked like:

Image description

The results were better, though my face was still not quite right. At first glance, everything seemed to be working okay:

Image description

Image description

Image description

Image description

Encouraged by these results, I attempted to add another piece of clothing to see if it would work with three LORA models. Here are the original images from the shop:

Image description

And the results:

Image description

Image description

Image description

This time, the jacket and jeans looked really good, but the face was worse than in the last batch. Unfortunately, I couldn't pinpoint the exact reason for this.

I'm still working on solving this issue, and when I do, I hope to share it with the community. If anyone has any insights or solutions, I would be incredibly grateful.

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read full post →

Top comments (0)

The Most Contextual AI Development Assistant

Pieces.app image

Our centralized storage agent works on-device, unifying various developer tools to proactively capture and enrich useful materials, streamline collaboration, and solve complex problems through a contextual understanding of your unique workflow.

👥 Ideal for solo developers, teams, and cross-company projects

Learn more