Memory monitor tracks GPU 0, but the script may be running on another about sygil-webui

[hlky]

multi-GPU scenario

well do you have multiple gpu? can a specific gpu to the load the model to even be set anywhere?

[stevestavropoulos]

CUDA_VISIBLE_DEVICES=1 python .../webui.py will run the script on the second GPU and while nvidia-smi will correctly report 6061MiB / 12196MiB usage on that GPU, the memory monitor from the script will report: Peak memory usage: 19 MiB / 12195 MiB / 0.155% which is from the first GPU.

[hlky]

i can add --gpu option to set the device index, i think you will still need to do the CUDA_VISIBLE_DEVICES=1 part though

[stevestavropoulos]

At the start of the script if you also did an os.environ["CUDA_VISIBLE_DEVICES"] = "{gpu}", then the CUDA_VISIBLE_DEVICES won't be needed.

This is tested and working (I just hardcoded the second GPU for the test):

-- stable-diffusion-webui/webui.py     2022-08-25 23:09:29.752912794 +0300
+++ stable-diffusion-webui/webui-gpu1.py        2022-08-26 19:38:20.803732102 +0300
@@ -1,4 +1,5 @@
 import argparse, os, sys, glob
+os.environ["CUDA_VISIBLE_DEVICES"] = "1"
 import gradio as gr
 import k_diffusion as K
 import math
@@ -177,7 +178,7 @@
     def run(self):
         print(f"[{self.name}] Recording max memory usage...\n")
         pynvml.nvmlInit()
-        handle = pynvml.nvmlDeviceGetHandleByIndex(0)
+        handle = pynvml.nvmlDeviceGetHandleByIndex(1)
         self.total = pynvml.nvmlDeviceGetMemoryInfo(handle).total
         while not self.stop_flag:
             m = pynvml.nvmlDeviceGetMemoryInfo(handle)

[sazabidesu]

Hi, I'm having a similar issue. My first GPU numbered "0" acording to nvidia-smi doesn't get used when I launch webui.py, only the second one numbered "1" gets used which has half the vram of the first one and gives me green images unless I use --full precision and low resolution.
I'm completely noob in all of this, but how can I force it to use the GPU with the most vram?

Thanks in advance.