print Total number of parameters is incomplete. about aqlm

[BlackSamorez]

Since AQLM employs vector quantization, each codes parameter encodes 8 fp16 weights of the uncompressed model for the 1x16 setup. On top of that, codebooks and scales also contribute a nontrivial number of parameters.
In the end. There really are ~6.5b parameters in ISTA-DASLab/Mixtral-8x7B-Instruct-v0_1-AQLM-2Bit-1x16-hf: some of them are int16 codes encoding larger vectors, some of them serve more complicated roles.
To learn more about the representation, please refer to either the original AQLM paper or this nice blogpost explanation.

[Minami-su]

Mixtral-8x7B-Instruct-v0_1 has approximately 46 billion parameters, obviously the number of parameters being output here is incorrect

[Minami-su]

Since AQLM employs vector quantization, each codes parameter encodes 8 fp16 weights of the uncompressed model for the 1x16 setup. On top of that, codebooks and scales also contribute a nontrivial number of parameters. In the end. There really are ~6.5b parameters in ISTA-DASLab/Mixtral-8x7B-Instruct-v0_1-AQLM-2Bit-1x16-hf: some of them are int16 codes encoding larger vectors, some of them serve more complicated roles. To learn more about the representation, please refer to either the original AQLM paper or this nice blogpost explanation.

huggingface/peft#1594