Run Flux.1 on M3 Mac with Diffusers

What is Diffusers?

huggingface / diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.

🤗 Diffusers is the go-to library for state-of-the-art pretrained diffusion models for generating images, audio, and even 3D structures of molecules. Whether you're looking for a simple inference solution or training your own diffusion models, 🤗 Diffusers is a modular toolbox that supports both. Our library is designed with a focus on usability over performance, simple over easy, and customizability over abstractions.

🤗 Diffusers offers three core components:

• State-of-the-art diffusion pipelines that can be run in inference with just a few lines of code.
• Interchangeable noise schedulers for different diffusion speeds and output quality.
• Pretrained models that can be used as building blocks, and combined with schedulers, for creating your own end-to-end diffusion systems.

Installation

We recommend installing 🤗 Diffusers in a virtual environment from PyPI or Conda. For more details about installing PyTorch and Flax, please refer to their official documentation.

PyTorch

With pip (official…

View on GitHub

What is Flux

https://blackforestlabs.ai/announcing-black-forest-labs/

1. Create a virtual env

python3 -m venv fluxtest
source fluxtest/bin/activate

2. Login to Hugging Face via CLI

https://huggingface.co/docs/huggingface_hub/main/en/guides/cli

pip install -U "huggingface_hub[cli]"
huggingface-cli login

3. Install packages

pip install torch==2.3.1
pip install git+https://github.com/huggingface/diffusers.git
pip install transformers==4.43.3 sentencepiece==0.2.0 accelerate==0.33.0 protobuf==5

4. Run a Python script

image.py

import torch
from diffusers import  FluxPipeline
import diffusers

_flux_rope = diffusers.models.transformers.transformer_flux.rope
def new_flux_rope(pos: torch.Tensor, dim: int, theta: int) -> torch.Tensor:
    assert dim % 2 == 0, "The dimension must be even."
    if pos.device.type == "mps":
        return _flux_rope(pos.to("cpu"), dim, theta).to(device=pos.device)
    else:
        return _flux_rope(pos, dim, theta)

diffusers.models.transformers.transformer_flux.rope = new_flux_rope

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", revision='refs/pr/1',  torch_dtype=torch.bfloat16).to("mps")

prompt = "japanese girl, photo-realistic"
out = pipe(
     prompt=prompt,
     guidance_scale=0.,
     height=1024,
     width=1024,
     num_inference_steps=4,
     max_sequence_length=256,
).images[0]
out.save("image.png")

Finally, run a Python script to generate an image.

python image.py

output

Comments

[b alon]

I did exactly your code and I came up with an blurry image

[0xkoji]

what are your Mac specs?

[b alon]

Solve it, downgrade torch to 2.3.1

[舜硯黃]

You can try this blog step by step, it is work for me.

towards-agi.medium.com/how-to-flux...

[Carlos Miguens]

I have exactly this problem too. How did you solve it?