Craig Morten

Posted on Aug 26, 2022 • Edited on Oct 2, 2022

Setting up Stable Diffusion for MacOS

#ai #python

With the landscape quickly changing, this article is fast becoming outdated!

If you face issues with the tutorial below I recommend you checkout the latest advice here.

Stable Diffusion is a latent text-to-image diffusion model that was recently made open source.

For Linux users with dedicated NVDIA GPUs the instructions for setup and usage are relatively straight forward. However for MacOS users you can't use the project "out of the box". Not to worry! There are some steps to getting it working nevertheless!

Environment Setup

To begin you need Python, Conda, and a few other libraries set up:

# Install Python, Cmake, Git, and Protobuf
brew install python \
  cmake \
  git \
  protobuf

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Install Conda:
## Either use this for older "pre-M1" Macs:
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
bash Miniconda3-latest-MacOSX-x86_64.sh

## Or use this for older M1 Macs:
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh
bash Miniconda3-latest-MacOSX-arm64.sh

You may need to restart your terminal at this point for the changes for the new libraries to be picked up.

Clone this fork of the project and checkout the apple patch branch:

git clone https://github.com/magnusviri/stable-diffusion
cd stable-diffusion
git checkout apple-silicon-mps-support

At this point you will need to make sure you're using Python 3, check out this article for different ways to make Python 3 the default version on your Mac.

Set up the Conda environment:

conda env create -f environment-mac.yaml
conda activate ldm

And finally set the following environment variable:

export PYTORCH_ENABLE_MPS_FALLBACK=1

Code Changes

Our environment is now set up, but we have a few tweaks that we need to allow the code to gracefully fallback to using the CPU (if required!).

Append .contiguous() at ldm/models/diffusion/plms.py#L27 resulting in:

-        attr = attr.to(torch.float32).to(torch.device(self.device_available))
+        attr = attr.to(torch.float32).to(torch.device(self.device_available)).contiguous()

Similarly append a new line x = x.contiguous() after ldm/modules/attention.py#L211 so it looks something like:

def _forward(self, x, context=None):
+       x = x.contiguous()
        x = self.attn1(self.norm1(x)) + x

Download Stable Diffusion Weights

Let's install our diffusion weights

curl "https://www.googleapis.com/storage/v1/b/aai-blog-files/o/sd-v1-4.ckpt?alt=media" > sd-v1-4.ckpt

Create Images 🚀

You should now be ready to generate images on your MacOS device using Stable Diffusion! 🎉 🎉

python scripts/txt2img.py --prompt "a drawing of web developers" --plms --ckpt sd-v1-4.ckpt --skip_grid --n_samples 1

Tricks and hacks gleamed from https://github.com/CompVis/stable-diffusion/issues/25 - credit to all the folks in that thread for figuring out how to get things working!

Oldest comments (41)

Greg Asman • Aug 26 '22 • Edited

Thanks for the detailed instructions! Unfortunately, I'm getting stuck at environment creation where pip is failing with the message below. Any thoughts on how to get over this hurdle? Thanks again!

(Note: It appears to be an issue with onnx. I tried a pip install onnx and it went through every version unsucessfully.)

Pip subprocess error:
  Running command git fetch -q --tags
  Running command git reset --hard -q 24268930bf1dce879235a7fddd0b2355b84d7ea6
  Running command git fetch -q --tags
  Running command git reset --hard -q d50d76daa670286dd6cacf3bcd80b5e4823fc8e1
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [7 lines of output]
      fatal: not a git repository (or any of the parent directories): .git
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/kk/qyr1f5dj017b338m860tt9dm0000gn/T/pip-install-5of30ges/onnx_8af078f05ebe4736b1b65355c7b7406b/setup.py", line 81, in <module>
          assert CMAKE, "Could not find cmake executable!"
      AssertionError: Could not find cmake executable!
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

failed

CondaEnvException: Pip failed

Carlos Merchán • Aug 27 '22

im having the same issue, any clue?

Craig Morten • Aug 28 '22 • Edited

I’ll give my own instructions another go and see if can reproduce…

What version of python/pip are you using? IIRC you need to use python 3

Updates:

If you followed the steps verbatim the conda installation step was incorrect - I have amended!
Just noticed your error is regarding not having cmake, adding some env setup steps for this now! You need to install this brew install cmake

Carlos Merchán • Aug 28 '22

I’ll check again installing cmake
Question: why are you using /Miniconda3-latest-MacOSX-x86_64.sh instead of /Miniconda3-latest-MacOSX-arm64.sh ?

Craig Morten • Aug 29 '22

Miniconda3-latest-MacOSX-x86_64.sh is for "pre-M1" macs, e.g. I use a MacBook Air 2017 which uses an Intel chip.

Miniconda3-latest-MacOSX-arm64.sh is for Apple M1 😄

Carlos Merchán • Aug 29 '22

I was confused because you use apple-silicon-mps-support branch and I thought you had a M1 cpu. Anyway now it's working for me, thanks

Stéphane Vicent • Aug 27 '22 • Edited

Hi Craig,
thanks for this tutorial.

This step doesn't work for me :

curl https://www.googleapis.com/storage/v1/b/aai-blog-files/o/sd-v1-4.ckpt?alt=media > sd-v1-4.ckpt

i get this error :

no matches found: https://www.googleapis.com/storage/v1/b/aai-blog-files/o/sd-v1-4.ckpt?alt=media

Craig Morten • Aug 28 '22

Can you try with quotes around the URL? E.g.

curl "https://www.googleapis.com/storage/v1/b/aai-blog-files/o/sd-v1-4.ckpt?alt=media" > sd-v1-4.ckpt

Constantinez • Aug 29 '22 • Edited

thx for this tutorial
but I got error on my MacBook pro (intel):

(ldm) kos@macbook stable-diffusion % python scripts/txt2img.py --prompt "a drawing of web developers" --plms --ckpt sd-v1-4.ckpt --skip_grid --n_samples 1
zsh: segmentation fault python scripts/txt2img.py --prompt "a drawing of web developers" --plms --ckp

ʞɐoןıu ıɯɐƃo ɟıɹǝ • Aug 30 '22

The only time I got a segfault was when I set KMP_DUPLICATE_LIB_OK ... I posted a couple other solutions above (recommend the "delete the conflicting library" one over the "rebuild with nomkl" but whichever you feel more comfortable with ^_^ )

ʞɐoןıu ıɯɐƃo ɟıɹǝ • Aug 30 '22

FWIW [2022-08-29] —
on an intel mac

1)
ldm/modules/attention.py#L211 already had the patch

2)
ran into issues with:

Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized OMP:

I wound up redoing the environment to include nomkl and it seems to have worked, if ... so ... SO ... slow [4 minutes!] . But I'll take functional over not (thanks!) ... helps me prep for doing this on a real machine, and I can kick things off overnight. ;)

dependencies:
  - nomkl
...

ʞɐoןıu ıɯɐƃo ɟıɹǝ • Aug 30 '22 • Edited

Alternately, with the info that libiomp5.dylib and libomp.dylib were likely conflicts:

 mv /Users/fire/miniconda3/envs/ldm/lib/python3.10/site-packages/torch/lib/libiomp5.dylib /Users/fire/miniconda3/envs/ldm/lib/python3.10/site-packages/torch/lib/libiomp5.bak

seems to have worked, ignoring the nomkl path.

The KMP_DUPLICATE_LIB_OK solution only led to segfaults.

The LD_PRELOAD did nothing, possibly because I was targeting the wrong one, but possibly because the names were actually different?

Sjoerd W. Bijleveld • Aug 30 '22

Hi Craig,
I'm stuck at the point of installing Conda. I get the message: 'zsh: command not found: wget'. It seems wget is a Linux command, but I'm on a iMac (Monterey). Any help is much appreciated!

Sjoerd

ʞɐoןıu ıɯɐƃo ɟıɹǝ • Aug 30 '22

you can install wget with brew, or switch to using curl to get the file also

curl -O URLHERE

Sjoerd W. Bijleveld • Aug 30 '22

Thanks! ATM I'm not able to try it, but also further investigation revealed that probably the path to the zsh shell is not valid. So I will check that also asap.

Sjoerd W. Bijleveld • Aug 31 '22

It turned out I indeed had to install 'wget' with brew. It does now recognise the command. So no problems with the shell path. Thanks so much!

Jeremy Franklin • Aug 30 '22

Thank you a lot! I don't have much experience coding but I managed to followed this true with some added troubleshooting because I couldn't understand quite all of it. But for now it worked!

I only have one question;

you say: Similarly append a new line x = x.contiguous() after ldm/modules/attention.py#L211

but when I went in the code to append this, it was already there without my intervention or anything. Can I just ignore this step or am I missing something?

It does work without the gpu/cpu thing so thanks again.

Anyway I could buy you a coffee?

Jeremy

ʞɐoןıu ıɯɐƃo ɟıɹǝ • Aug 30 '22

Can I just ignore this step or am I missing something?

I expect the codebase just moved forwards and "fixed" this part :)

Jeremy Franklin • Aug 30 '22

Do you have to install the Stable Diffusion Weights every time you use it?

ʞɐoןıu ıɯɐƃo ɟıɹǝ • Aug 30 '22

just each time you install stable diffusion :) it'll wind up downloading a bunch of checkpoints/whatnot the first time you run, but won't need to do that either on subsequent runs.

Nicola Zuliani • Aug 30 '22

Thanks for the instructions, but I found some problems while trying to make this to work.
Many modules looks missing on my MacOS (with M1), that I think I managed to install, but now I'm stucked.
The error I get is:

ImportError: dlopen(/Users/nicol/miniconda3/lib/python3.9/site-packages/_dlib_pybind11.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace '_png_do_expand_palette_rgb8_neon'

Any idea on how to solve it? I tried to install libpng but things don't change.

Thanks again :)

000fff • Aug 31 '22

I am getting a weird error when I try and run the python script.

I have an apple Macbook Pro M1 with 64GB of ram.


(ldm) thomaspetersen@MacBook-Pro stable-diffusion % curl "https://www.googleapis.com/storage/v1/b/aai-blog-files/o/sd-v1-4.ckpt?alt=media" > sd-v1-4.ckpt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 4067M  100 4067M    0     0  63.4M      0  0:01:04  0:01:04 --:--:-- 64.5M
(ldm) thomaspetersen@MacBook-Pro stable-diffusion % python scripts/txt2img.py --prompt "a drawing of web developers" --plms --ckpt sd-v1-4.ckpt --skip_grid --n_samples 1
Intel MKL FATAL ERROR: This system does not meet the minimum requirements for use of the Intel(R) Math Kernel Library.
The processor must support the Intel(R) Supplemental Streaming SIMD Extensions 3 (Intel(R) SSSE3) instructions.
The processor must support the Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) instructions.
The processor must support the Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
(ldm) thomaspetersen@MacBook-Pro stable-diffusion % wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh