oobabooga / gptq-for-llama-wheels

Precompiled Wheels for GPTQ-for-LLaMa

GPTQ-for-LLaMa-Wheels (Windows AMD64)

Precompiled Windows AMD64 Wheels for GPTQ-for-LLaMa CUDA
See the Linux-x64 branch for Linux x86_64 wheels.

---

Wheels in root directory compiled from oobabooga's fork

• Supports Pascal+ (compute 6.0+)

832e220 wheels compiled from latest (as of writing) commit of GPTQ-for-LLaMa

610fdae wheels compiled from equivalent commit of GPTQ-for-LLaMa

0cc4m wheel compiled from 0cc4m's fork for KoboldAI
Deprecated quant_cuda wheel is included for those who want it.

• Supports late-Kepler+ (compute 3.5+)

Wheels are compiled using GitHub Actions.

---

Intended for use with:
text-generation-webui
KoboldAI