Any way to convert larger models to pytorch ? about text-generation-webui

If you have enough RAM you could use the CPU to convert it, without CUDA.

It's useful to create swap so that your system doesn't run out of memory:

$ sudo fallocate -l 32G /swapfile
$ sudo chown 0:0 /swapfile
$ sudo chmod 600 /swapfile
$ sudo mkswap /swapfile
$ sudo swapon /swapfile

Then, remove or comment out .cuda() from this line of the script.

$ python convert-to-torch.py models/GPT-NeoX-20B-Skein
Loading GPT-NeoX-20B-Skein...
Loading checkpoint shards: 100%|██████████| 23/23 [00:39<00:00,  1.70s/it]
Model loaded.
Saving to torch-dumps/GPT-NeoX-20B-Skein.pt
$ ls torch-dumps/
GPT-NeoX-20B-Skein.pt  place-your-pt-models-here.txt

[oobabooga]

I have never managed to do it as well. Maybe @81300 can help.

[Manimap]

Thanks, it worked!
I just created a convert-to-torch-cpu.py with the deleted cuda and it dumped everything in memory.