DEV Community

0xkoji
0xkoji

Posted on • Edited on

Run Flux.1 on M3 Mac with Diffusers

What is Diffusers?

GitHub logo huggingface / diffusers

πŸ€— Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.



GitHub GitHub release GitHub release Contributor Covenant X account

πŸ€— Diffusers is the go-to library for state-of-the-art pretrained diffusion models for generating images, audio, and even 3D structures of molecules. Whether you're looking for a simple inference solution or training your own diffusion models, πŸ€— Diffusers is a modular toolbox that supports both. Our library is designed with a focus on usability over performance, simple over easy, and customizability over abstractions.

πŸ€— Diffusers offers three core components:

  • State-of-the-art diffusion pipelines that can be run in inference with just a few lines of code.
  • Interchangeable noise schedulers for different diffusion speeds and output quality.
  • Pretrained models that can be used as building blocks, and combined with schedulers, for creating your own end-to-end diffusion systems.

Installation

We recommend installing πŸ€— Diffusers in a virtual environment from PyPI or Conda. For more details about installing PyTorch and Flax, please refer to their official documentation.

PyTorch

With pip (official…




What is Flux

https://blackforestlabs.ai/announcing-black-forest-labs/

1. Create a virtual env

python3 -m venv fluxtest
source fluxtest/bin/activate
Enter fullscreen mode Exit fullscreen mode

2. Login to Hugging Face via CLI

https://huggingface.co/docs/huggingface_hub/main/en/guides/cli

pip install -U "huggingface_hub[cli]"
huggingface-cli login
Enter fullscreen mode Exit fullscreen mode

3. Install packages

pip install torch==2.3.1
pip install git+https://github.com/huggingface/diffusers.git
pip install transformers==4.43.3 sentencepiece==0.2.0 accelerate==0.33.0 protobuf==5
Enter fullscreen mode Exit fullscreen mode

4. Run a Python script

image.py

import torch
from diffusers import  FluxPipeline
import diffusers

_flux_rope = diffusers.models.transformers.transformer_flux.rope
def new_flux_rope(pos: torch.Tensor, dim: int, theta: int) -> torch.Tensor:
    assert dim % 2 == 0, "The dimension must be even."
    if pos.device.type == "mps":
        return _flux_rope(pos.to("cpu"), dim, theta).to(device=pos.device)
    else:
        return _flux_rope(pos, dim, theta)

diffusers.models.transformers.transformer_flux.rope = new_flux_rope

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", revision='refs/pr/1',  torch_dtype=torch.bfloat16).to("mps")

prompt = "japanese girl, photo-realistic"
out = pipe(
     prompt=prompt,
     guidance_scale=0.,
     height=1024,
     width=1024,
     num_inference_steps=4,
     max_sequence_length=256,
).images[0]
out.save("image.png")
Enter fullscreen mode Exit fullscreen mode

Finally, run a Python script to generate an image.

python image.py
Enter fullscreen mode Exit fullscreen mode

output

output

Top comments (5)

Collapse
 
b_alon_b0ac4986f439386747 profile image
b alon

I did exactly your code and I came up with an blurry image
Image description

Collapse
 
0xkoji profile image
0xkoji

what are your Mac specs?

Collapse
 
b_alon_b0ac4986f439386747 profile image
b alon

Solve it, downgrade torch to 2.3.1

Collapse
 
_7e94daa58788d182faa9d profile image
θˆœη‘―ι»ƒ

You can try this blog step by step, it is work for me.

towards-agi.medium.com/how-to-flux...

Collapse
 
carlos_miguens_606b2b64c9 profile image
Carlos Miguens

I have exactly this problem too. How did you solve it?