Getting an error about diffusers

[willwillems]

I experienced the same issue on A100, tried manually compiling the xformers, also @51dd119 but did not end up working. Switching to Tesla T4 using the precompiled version of xformers "fixed" the issue.

[Dumoque]

I guess I should specify I'm trying to do this locally on a 2080TI with 12GB Vram.

[nerdyrodent]

If Windows can't find /libcudart.so then run:
export LD_LIBRARY_PATH=/usr/lib/wsl/lib:$LD_LIBRARY_PATH

[Dumoque]

I tried the export (both manually running it in ubuntu like you did in the video as well as adding it to the top line of my training file like the pastebin directions say) and am still getting an error. Just to clarify I don't need to edit that line with any unique file path right?

This is the error I'm getting. Looks like its the same error.

/root/anaconda3/envs/diffusers/lib/python3.9/site-packages/bitsandbytes/cuda_setup/paths.py:86: UserWarning: /root/anaconda3/envs/diffusers did not contain libcudart.so as expected! Searching further paths...
warn(
/root/anaconda3/envs/diffusers/lib/python3.9/site-packages/bitsandbytes/cuda_setup/paths.py:20: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('CompVis/stable-diffusion-v1-4')}
warn(
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: WARNING! libcuda.so not found! Do you have a CUDA driver installed? If you are on a cluster, make sure you are on a CUDA machine!
Traceback (most recent call last):
File "/root/github/diffusers/examples/dreambooth/train_dreambooth.py", line 638, in
main()
File "/root/github/diffusers/examples/dreambooth/train_dreambooth.py", line 429, in main
import bitsandbytes as bnb
File "/root/anaconda3/envs/diffusers/lib/python3.9/site-packages/bitsandbytes/init.py", line 6, in
from .autograd._functions import (
File "/root/anaconda3/envs/diffusers/lib/python3.9/site-packages/bitsandbytes/autograd/_functions.py", line 5, in
import bitsandbytes.functional as F
File "/root/anaconda3/envs/diffusers/lib/python3.9/site-packages/bitsandbytes/functional.py", line 13, in
from .cextension import COMPILED_WITH_CUDA, lib
File "/root/anaconda3/envs/diffusers/lib/python3.9/site-packages/bitsandbytes/cextension.py", line 41, in
lib = CUDALibrary_Singleton.get_instance().lib
File "/root/anaconda3/envs/diffusers/lib/python3.9/site-packages/bitsandbytes/cextension.py", line 37, in get_instance
cls._instance.initialize()
File "/root/anaconda3/envs/diffusers/lib/python3.9/site-packages/bitsandbytes/cextension.py", line 15, in initialize
binary_name = evaluate_cuda_setup()
File "/root/anaconda3/envs/diffusers/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py", line 132, in evaluate_cuda_setup
cc = get_compute_capability(cuda)
File "/root/anaconda3/envs/diffusers/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py", line 105, in get_compute_capability
ccs = get_compute_capabilities(cuda)
File "/root/anaconda3/envs/diffusers/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py", line 83, in get_compute_capabilities
check_cuda_result(cuda, cuda.cuDeviceGetCount(ctypes.byref(nGpus)))
AttributeError: 'NoneType' object has no attribute 'cuDeviceGetCount'
Traceback (most recent call last):
File "/root/anaconda3/envs/diffusers/bin/accelerate", line 8, in
sys.exit(main())
File "/root/anaconda3/envs/diffusers/lib/python3.9/site-packages/accelerate/commands/accelerate_cli.py", line 43, in main
args.func(args)
File "/root/anaconda3/envs/diffusers/lib/python3.9/site-packages/accelerate/commands/launch.py", line 837, in launch_command
simple_launcher(args)
File "/root/anaconda3/envs/diffusers/lib/python3.9/site-packages/accelerate/commands/launch.py", line 354, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/root/anaconda3/envs/diffusers/bin/python', 'train_dreambooth.py', '--pretrained_model_name_or_path=CompVis/stable-diffusion-v1-4', '--instance_data_dir=training', '--class_data_dir=classes', '--output_dir=output', '--instance_prompt=Ashtonporter', '--class_prompt=Ashtonporter', '--seed=3434554', '--resolution=512', '--center_crop', '--train_batch_size=1', '--mixed_precision=fp16', '--use_8bit_adam', '--gradient_accumulation_steps=1', '--gradient_checkpointing', '--learning_rate=5e-6', '--lr_scheduler=constant', '--lr_warmup_steps=0', '--num_class_images=100', '--sample_batch_size=4', '--max_train_steps=800']' returned non-zero exit status 1.

[Dumoque]

I just saw the pinned comment on the video and ran "pip install --upgrade bitsandbytes" and did the export LD_LIBRARY_PATH=/usr/lib/wsl/lib:$LD_LIBRARY_PATH again and that seems to have fixed it! Thanks for all the help!