Couldn't start server on WSL, It always get killed about text-generation-webui

[oobabooga]

Could it be that you are running out of RAM while trying to load the model? Can you try python script.py --auto-devices?

[VenkatLohithDasari]

with python server.py --auto-devices

I get same thing...

Loading pygmalion-6b...
Auto-assiging --gpu-memory 7 for your GPU to try to prevent out-of-memory errors.
You can manually set other values.
Killed

btw,
Cudatoolkit Version installed in my WSL2 is 12.0.
Does this thing has to do with this error?

and also one more thing, RAM usage get's spiked to almost 90%, and GPU to around 5%. Once it gets killed, everything drops back, RAM to 50% and GPU to 1% as usual. btw I got 16GB RAM and RTX 3060ti (8GB VRAM only).

Maybe these informations can help a bit so I noted them here.

[oobabooga]

Can you try running git pull inside text-generation-webui to update the repository and then run python server.py --auto-devices again? I added back low_cpu_mem_usage=True. Maybe it will help.

[RememberAlgernon]

Have you set a RAM limit for WSL? It will default to half your system RAM unless you specify otherwise. Details for this can be found here.
https://learn.microsoft.com/en-us/windows/wsl/wsl-config

[VenkatLohithDasari]

Have you set a RAM limit for WSL? It will default to half your system RAM unless you specify otherwise. Details for this can be found here.
https://learn.microsoft.com/en-us/windows/wsl/wsl-config

Thanks, it fixed the problem. But I don't know why but my whole PC get's Frozen when I try --load-in-8bit

Does this thing have to do with WSL2 or due to my GPU?
Edit: Nothing seems to fix this, It's most probably due to my GPU, I guess RTX 3060ti is not enough for 8bit.