The current installed version of g++ is greater than the maximum required version by CUDA about gptq-for-llama

[plhosk]

Not sure about Fedora but in Arch you need to install gcc11 which can coexist with the gcc12 that is already installed.

pacman -S gcc11

Then type this in the console you're about to install GPTQ:

export CC=gcc-11 CXX=g++-11

followed by

python setup_cuda.py install

[frandmb]

Update:
Solved by installing g++ through Conda:
conda install -c conda-forge gxx

[frandmb]

Not sure about Fedora but in Arch you need to install gcc11 which can coexist with the gcc12 that is already installed.

pacman -S gcc11

Then type this in the console you're about to install GPTQ:

export CC=gcc-11 CXX=g++-11

followed by

python setup_cuda.py install

Thank you, it seems that it's not possible to install previous gcc versions on Fedora anymore (At least not from the official repos). But the answer should be good for Ubuntu and Arch users

[Bruce-Hopkins]

Update: Solved by installing g++ through Conda: conda install -c conda-forge gxx

I'm using Fedora. I tried this and it still doesn't work.

[frandmb]

Update: Solved by installing g++ through Conda: conda install -c conda-forge gxx

I'm using Fedora. I tried this and it still doesn't work.

I've also installed conda install gcc_linux-64==11.2.0, probably both are needed.
You might need to deactivate and reactivate the Conda environment.
I'm running Nobara which ships with official Nvidia Gpu & Cuda drivers, as the kmod kernel is needed to use Nvidia/Cuda, if you don't have it, you might have to install it through rpmfusion

[mudomau]

Not sure about Fedora but in Arch you need to install gcc11 which can coexist with the gcc12 that is already installed.

pacman -S gcc11

Then type this in the console you're about to install GPTQ:

export CC=gcc-11 CXX=g++-11

followed by

python setup_cuda.py install

Well this got me one step forward, but now the install is erroring out with 'cannot find -lcudart'

"...
/usr/bin/ld: cannot find -lcudart: No such file or directory
collect2: error: ld returned 1 exit status
error: command '/usr/bin/g++-11' failed with exit code 1""

Is this a package I could just copy/paste from somewhere to that directory, or do I need to solve this some other way?

Edit: I was missing CUDA, stupidly. Everything working now.

[eclipseo]

I have Fedora 38 too.

conda install -c conda-forge gcc_linux-64==11.2.0 gxx_linux-64==11.2.0
export PATH=${PATH}:${HOME}/.conda/envs/textgen/bin/
export CC=x86_64-conda-linux-gnu-gcc CXX=x86_64-conda-linux-gnu-g++

replace textgen by your conda env.