question ： about autogptq

[fxmarty]

Hi @unicormbu, you can see for example:

AutoGPTQ/auto_gptq/nn_modules/qlinear/qlinear_cuda_old.py

Lines 48 to 55 in 6ba14f1

	self.register_buffer(
	'qweight',
	torch.zeros((infeatures // 32 * self.bits, outfeatures), dtype=torch.int32)
	)
	self.register_buffer(
	'qzeros',
	torch.zeros((math.ceil(infeatures / self.group_size), outfeatures // 32 * self.bits), dtype=torch.int32)
	)

So the grouping is done on the input dimension that is the K dimension in (MK) * (KN).