Stable Diffusion XL - M1 mac doesn't work with fp16 on tutorial script - LLVM ERROR: Failed to infer result type(s) about generative-models

[mbewley]

Done - sorry - that's all I've tested it on, not sure about whether it impacts more broadly.

[ZelnickB]

Same issue here on MacBook Pro M2 Max in a REPL (using pyenv and pyenv-virtualenv):

>>> from diffusers import DiffusionPipeline
>>> import torch
>>> pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, use_safetensors=True, variant="fp16")
Loading pipeline components...: 100%|█████████████| 7/7 [00:00<00:00,  7.90it/s]
>>> images = pipe(prompt="An astronaut riding a horse").images[0]
loc("varianceEps"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/d9889869-120b-11ee-b796-7a03568b17ac/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":228:0)): error: input types 'tensor<1x77x1xf16>' and 'tensor<1xf32>' are not broadcast compatible
LLVM ERROR: Failed to infer result type(s).
zsh: abort      python
/Users/user/.pyenv/versions/3.11.4/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

After all of this is printed to the console, the REPL exits completely, and I am returned to the shell.

[ZelnickB]

@mbewley, could you please add to the title of this issue that the problem is with Stable Diffusion XL? I believe that this repository is for several generative models and not just SDXL.

[grahamcracker1234]

Can also confirm on MacBook Pro M2 Max running in a conda env. Changing torch_dtype=torch.float16 to torch_dtype=torch.float32 fixed the issue for me.

[WildDanDan]

This is remains a problem on M2 MacBooks with PyTorch@latest on MacOS Sonoma. Using the torch.float32 dtype (or the --no-half CLI arg for AUTOMATIC1111 users) works, albeit at a glacial pace.

[Vargol]

if you're on Sonoma try
pip install -U torch torchvision torchdata torchaudio
Make sure the version of torch it installs is 2.1.

If you not on Sonoma there a load of fp16 fixes that need applying to torch
I've been running with fp16 has ages, and have a git repo showing how to get it working on a 8Gb M1
https://github.com/Vargol/8GB_M1_Diffusers_Scripts/tree/main/sdxl

@WildDanDan I'd look into other SD apps if I was you Auto1111 and Apple Silicon have never mixed that well
I use InvokeAI when not using my own Diffusers scripts but there are others.

[hangerrits]

try the special pipeline:
pipe = StableDiffusionXLPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0")
works for me