triton implementation about autogptq

[PanQiWei]

Hi @qwopqwop200 ! How's your day? 😄

I would like to extend an official invitation for you to join this project and being the maintainer and first-class reviwer on any commits to quantization codes. I believe your expertise will help this project grow more quickly and continue to make important contributions to the community.

Meanwhile, I will continue to extend and optimize this project on the following aspects:

• More faster speed on the execution of model quantization program and more controllable of RAM and VRAM usage.
• Support model offloading so that this project can be used in larger models on consumer level devices.
• An inference engine module that support multi-device and model parallelism inference and compatible with asgi frameworks such as FastAPI.
• An trainer that can fine-tune quantized model on down-stream tasks. (I think this is impossable without your help).
• And more others that community demanded and meaningful.

It would be my great honor to have you join in!

Looking forward to your reply! ❤️

[qwopqwop200]

I think this project is very promising. I am thinking of participating in this project.

[PanQiWei]

Awesome!! What a great job 🎉

I'm also in a progress to learn how to integrate with triton and try to make it compatible with CUDA version in current codes, anyway, I will add your project to README.md and let anyone else to know there is a faster version of AutoGPTQ.

[PanQiWei]

Can't wait to cooperate with you! 🥂