Running on linux about koboldai-client

[henk717]

I do not have nvidia so my attempt at play-cuda.sh was completely blind and apparently unsuccesful. This will be quite tricky to solve over github since we will have to find out what ends up working one on one.

For that one line don't comment it out and add the following above it and then move these two lines to the bottom of the script:
USER root

This should elevate its permission to root at the last moment and install X11 (this is inside the docker not on your real installation). You need this inside the docker instance so it can draw the file selection window properly.

The missing device is most likely an issue with me plainly copying my amd version hoping it would work. Try removing that from the cuda docker files.

If it still gives issues i recommend joining the kobold discord at https://discord.gg/UCyXV7NssH so we can try and fix this one on one (I am Henky!! there).

Hi on the topic Linux, to get KoboldAI to run on Arch you may need to modify the docker-compose.yml for it to see your nvidia GPU. If you don't it may lock up on large models.

version: "3.2"
services:
koboldai:
build: .
environment:
- DISPLAY=${DISPLAY}
network_mode: "host"
volumes:
- /tmp/.X11-unix:/tmp/.X11-unix
- ../:/content/
- $HOME/.Xauthority:/home/micromamba/.Xauthority:rw
devices:
- /dev/dri
- /dev/nvidia0:/dev/nvidia0
- /dev/nvidiactl:/dev/nvidiactl
- /dev/nvidia-uvm:/dev/nvidia-uvm
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids: ['0']
capabilities: [gpu]
group_add:
- video

[henk717]

Feel free to issue that as a commit here on github, i don't have all the parts for my nvidia gpu yet.

Done, I have made a pull request. Im not sure it's bug free though, but it is a start.

[henk717]

The old one wasn't working on the GPU's in general, leaving this issue open for further testing but i expect this to work well for most people!

Hello, two things. The first an update on nvidia. Sometimes you may get this message like after a reboot,
"Error response from daemon: error gathering device information while adding custom device "/dev/nvidia-uvm": no such file or directory"
In this case you may have to comment out this line
- /dev/nvidia-uvm:/dev/nvidia-uvm
Run ./play-cuda.sh again, have it say GPU NOT FOUND, close the program, uncomment the line again, then rerun the program, don't know why that happens exactly.

Issue number 2 is to do with AMD. I'm working on the docker-rocm/docker-compose.yml file. My GPU was getting marked not found so I made this small change.

devices:
  - /dev/kfd:/dev/kfd
  - /dev/dri:/dev/dri

But now I am getting this error.

"hipErrorNoBinaryForGpu: Unable to find code object for all current devices!"

Does anyone have any ideas how to fix this? Thank you.

[henk717]

Which AMD gpu do you have? Not all of them are supported and you need the built in driver (not the pro driver) + ROCm to get a working conpute stack with them. Only very few xards are supported.