Feature request: Quantize Google Gemma models about aqlm

[Godofnothing]

Hi, @younesbelkada. We have uploaded 2b gemma versions to the huggingface hub:

• gemma-2b-AQLM-2Bit-1x16-hf
• gemma-2b-AQLM-2Bit-2x8-hf

For some reason, 7b model gemma experiences significant decline in performance, making it unusable. In case we manage to resolve the issue, we will upload 7b models as well.

[Godofnothing]

Hi, @younesbelkada. We intend to prepare 2bit gemma models and will come back to you once we have results.

[younesbelkada]

Thanks a lot @Godofnothing ! Looking forward to it !

[younesbelkada]

Very nice thank you @Godofnothing !