CUDNN_STATUS_NOT_INITIALIZED about diffusers-rs

[ajmwagar]

Dumb question. Do you have a CUDA enabled GPU on your system?

[sak96]

oh yeah i forgot to give details about the machine.

% lspci | grep -i vga
01:00.0 VGA compatible controller: NVIDIA Corporation GA106M [GeForce RTX 3060 Mobile / Max-Q] (rev a1)
05:00.0 VGA compatible controller: Advanced Micro Devices ....

nvidia-smi

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.60.11    Driver Version: 525.60.11    CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   32C    P3    N/A /  N/A |      5MiB /  6144MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A       666      G   /usr/lib/Xorg                       4MiB |
+-----------------------------------------------------------------------------+

6Gb 3060

% nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

using export TORCH_CUDA_VERSION=cu118.
would any other details be required.

i am using f16 model

% sha256sum unet16.bin
5019a4fbb455dd9b75192afc3ecf8a8ec875e83812fd51029d2e19277edddebc  unet16.bin

[rockerBOO]

Could try something like

println!("Cuda available: {}", tch::Cuda::is_available());
println!("Cudnn available: {}", tch::Cuda::cudnn_is_available());

To see if the tch library can see it.

[sak96]

Cuda available: true
Cudnn available: true
Cuda available: true
Cudnn available: true

i am not sure why it printed stuff twice though.

--- a/examples/stable-diffusion/main.rs
+++ b/examples/stable-diffusion/main.rs
@@ -196,6 +196,8 @@ fn run(args: Args) -> anyhow::Result<()> {

 fn main() -> anyhow::Result<()> {
     let args = Args::parse();
+    println!("Cuda available: {}", tch::Cuda::is_available());
+    println!("Cudnn available: {}", tch::Cuda::cudnn_is_available());
     if !args.autocast {
         run(args)
     } else {

EDIT:
i found that the cuda code is also part of the code.

[sak96]

some solution i found was: pytorch/pytorch#16831 (comment)

tch::Cuda::cudnn_set_benchmark(false);

this did not help.

there is another issue with same stuff: tensorflow/tensorflow#6698 (comment) or https://stackoverflow.com/a/52634209
but i dont see any api which can be used to do the same in tch-rs. if you have any idea please let me know.

[eoriont]

Do you have libtorch installed? I had this issue, then fixed it, then forgot exactly what fixed it 😑
The things I tried were: installing the cuda version of pytorch, installing cuda 11.8, installing cudnn 8.9.1, and installing libtorch. After libtorch, it just worked magically.