DEV Community

0xkoji
0xkoji

Posted on β€’ Edited on

36

Run Flux.1 on M3 Mac with Diffusers

What is Diffusers?

GitHub logo huggingface / diffusers

πŸ€— Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.



GitHub GitHub release GitHub release Contributor Covenant X account

πŸ€— Diffusers is the go-to library for state-of-the-art pretrained diffusion models for generating images, audio, and even 3D structures of molecules. Whether you're looking for a simple inference solution or training your own diffusion models, πŸ€— Diffusers is a modular toolbox that supports both. Our library is designed with a focus on usability over performance, simple over easy, and customizability over abstractions.

πŸ€— Diffusers offers three core components:

  • State-of-the-art diffusion pipelines that can be run in inference with just a few lines of code.
  • Interchangeable noise schedulers for different diffusion speeds and output quality.
  • Pretrained models that can be used as building blocks, and combined with schedulers, for creating your own end-to-end diffusion systems.

Installation

We recommend installing πŸ€— Diffusers in a virtual environment from PyPI or Conda. For more details about installing PyTorch and Flax, please refer to their official documentation.

PyTorch

With pip (official…




What is Flux

https://blackforestlabs.ai/announcing-black-forest-labs/

1. Create a virtual env

python3 -m venv fluxtest
source fluxtest/bin/activate
Enter fullscreen mode Exit fullscreen mode

2. Login to Hugging Face via CLI

https://huggingface.co/docs/huggingface_hub/main/en/guides/cli

pip install -U "huggingface_hub[cli]"
huggingface-cli login
Enter fullscreen mode Exit fullscreen mode

3. Install packages

pip install torch==2.3.1
pip install git+https://github.com/huggingface/diffusers.git
pip install transformers==4.43.3 sentencepiece==0.2.0 accelerate==0.33.0 protobuf==5
Enter fullscreen mode Exit fullscreen mode

4. Run a Python script

image.py

import torch
from diffusers import  FluxPipeline
import diffusers

_flux_rope = diffusers.models.transformers.transformer_flux.rope
def new_flux_rope(pos: torch.Tensor, dim: int, theta: int) -> torch.Tensor:
    assert dim % 2 == 0, "The dimension must be even."
    if pos.device.type == "mps":
        return _flux_rope(pos.to("cpu"), dim, theta).to(device=pos.device)
    else:
        return _flux_rope(pos, dim, theta)

diffusers.models.transformers.transformer_flux.rope = new_flux_rope

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", revision='refs/pr/1',  torch_dtype=torch.bfloat16).to("mps")

prompt = "japanese girl, photo-realistic"
out = pipe(
     prompt=prompt,
     guidance_scale=0.,
     height=1024,
     width=1024,
     num_inference_steps=4,
     max_sequence_length=256,
).images[0]
out.save("image.png")
Enter fullscreen mode Exit fullscreen mode

Finally, run a Python script to generate an image.

python image.py
Enter fullscreen mode Exit fullscreen mode

output

output

Image of AssemblyAI tool

Challenge Submission: SpeechCraft - AI-Powered Speech Analysis for Better Communication

SpeechCraft is an advanced real-time speech analytics platform that transforms spoken words into actionable insights. Using cutting-edge AI technology from AssemblyAI, it provides instant transcription while analyzing multiple dimensions of speech performance.

Read full post

Top comments (5)

Collapse
 
b_alon_b0ac4986f439386747 profile image
b alon β€’

I did exactly your code and I came up with an blurry image
Image description

Collapse
 
0xkoji profile image
0xkoji β€’

what are your Mac specs?

Collapse
 
b_alon_b0ac4986f439386747 profile image
b alon β€’

Solve it, downgrade torch to 2.3.1

Collapse
 
_7e94daa58788d182faa9d profile image
θˆœη‘―ι»ƒ β€’

You can try this blog step by step, it is work for me.

towards-agi.medium.com/how-to-flux...

Collapse
 
carlos_miguens_606b2b64c9 profile image
Carlos Miguens β€’

I have exactly this problem too. How did you solve it?

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

πŸ‘‹ Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay