Question about Maniqa about iqa-pytorch

[chaofengc]

Thanks for your help. After checking the code, I found this is caused by hyper parameter setting. MANIQA with koniq10k used a different scale parameter, fixed as below:

IQA-PyTorch/pyiqa/default_model_configs.py

Lines 273 to 278 in 33a2829

	'maniqa': {
	'metric_opts': {
	'type': 'MANIQA',
	'train_dataset': 'koniq',
	'scale': 0.8,
	},

It should give similar scores as official code.

[wanduoz]

Hi.
After fixing the code, I got iqa score=0.2394, while the official maniqa score of same image was 0.2243. I wonder if this result is reasonable?
Also, I currently used 16 iqa models and I found that the Results Calibration only covered parts of the following models. Due to the mismatching of Maniqa, do I need to check the below models score with the official results one by one?

Calibrated: 'musiq-paq2piq', 'musiq-koniq', 'musiq-spaq', 'musiq-ava', 'paq2piq'

Unknown:  'clipiqa+_rn50_512', 'clipiqa+',  'dbcnn', 'clipiqa+_vitL14_512', 'tres-koniq', 
           'tres-flive', clipiqa', 'cnniqa', ', 'topiq_nr',  'hyperiqa', 'maniqa',

[chaofengc]

For the maniqa if you want to get more close results, you must control the random cropping process. I suggest you to dive deep into the codes and find the problem.

For the models you mentioned without calibration, part of them such as maniqa are taken from official codes with few modifications, therefore does not provide calibration. Others have no official models or weights and are reimplemented and retrained.

It would be very helpful if you could help to double check them one by one.