zhuhancheng / metaiqa Goto Github PK

Hi!Congratulations to your work!
In the process of reproducing the code, I trained the prior knowledge model on TID2013 and Kadid10k, and fine-tuned on the CLIVE data set, but the SROCC was about 0.48, which did not achieve the results in the paper. Do you get similar results? I didn't make any change but the results were not ideal.

can I use this code for training and testing or it is just for testing?
can you please tell me what data it train and what dataset it test?
thanks

What is the best score of sp for model validation?

https://github.com/zhuhancheng/MetaIQA/blob/master/MetaIQA_On_TID2013_KADID.py#L459
best_model = copy.deepcopy(model)
To my understanding, Should here be that test meta_model and save meta_model?

Hi Hancheng,

Thanks your great idea!

I'm trying to reproduce your amazing work but found that the code for the leave-one-distortion-out cross validation on TID2013 and KADID10K (Table 2 of the paper) are not included in this GitHub. I have several questions about the implementation details. How was the fine-tuning done for this experiment? How were the SROCCs reported on the paper selected? Are they the highest SROCCs among the training epochs? Or are they chosen at a fixed epoch?

I would appreciate it greatly if you can release the code. Thank you!

Where is the file image_labeled_by_score.csv? Did i do anything wrong?

Hello! The overall framework of the MetaIQA is composed of two steps: 1. meta-training for quality prior model and fine-tuning for NR-IQA of unknown distortions. When will the first part of the code can be released? Thanks!

when I use the project with torch==1.6.0. it will show me that: torch.nn.modules.module.ModuleAttributeError: 'AvgPool2d' object has no attribute 'divisor_override'
and then it will be right when I change torch version to 0.4.0. So that means MetaIQA use the 0.4.0 version ?

thanks for your work. I want to know how to save the fine-tuned model and then test an image. Could you please tell me?

First of all. thanks for your work. I want known how to convert tid 2013 data range to 0-1. The orignal tid 2013 range is max: 7.21429 min: 0.24242. value / max value ?

Excuse me, I want to train and test new datasets (such as training TID2008, live, csiq, pipal, and so on) with your model. How should I do it?
Thanks so much for your help

Hello! I was trying to open the both trained models with torch.load('MetaIQA/model_IQA/TID2013_IQA_Meta_resnet18.pt') and got this error:

AttributeError Traceback (most recent call last)
in ()
----> 1 model = torch.load('MetaIQA/model_IQA/TID2013_IQA_Meta_resnet18.pt')

/usr/local/lib/python3.6/dist-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
591 return torch.jit.load(f)
592 return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
--> 593 return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
594
595

/usr/local/lib/python3.6/dist-packages/torch/serialization.py in _legacy_load(f, map_location, pickle_module, **pickle_load_args)
771 unpickler = pickle_module.Unpickler(f, **pickle_load_args)
772 unpickler.persistent_load = persistent_load
--> 773 result = unpickler.load()
774
775 deserialized_storage_keys = pickle_module.load(f, **pickle_load_args)

AttributeError: Can't get attribute 'Net' on <module 'main'>

I tried it in my local environment and using colab, so the same error occurs with different python and pycharm versions.
Could you, please, specify the exact versions or maybe check the both models.

Thank you very much for your nice work. It is a quiet great guidance for NR-IQA. Excuse me.
I want to consult where can we get the relevant 5 datasets? Thanks alot.

Hello,
How can I test model for any images?
Can you share Pre-trained Model for testing?
@zhuhancheng

Excuse me, zhuhancheng,
I want to know how you fine-tuning the quality prior model to get the final quality modelmodel_IQA/TID2013_IQA_Meta_resnet18.pt?
Hope to see the reply!

In papers 4.7. Parameters discussion, two key parameters have been discussed.

In MetaIQA_On_TID2013_KADID.py, I'm not find the parameter setting for S.

How do you realize different values for S?

thx

Hello! We kindly invite you to participate in our video quality metrics benchmark. You can submit MetaIQA (or any other your metrics) to the benchmark, following the submission steps, described here. The dataset distortions refer to compression artifacts on professional and user-generated content. The full dataset is used to measure methods overall performance, so we do not share it to avoid overfitting. Nevertheless, we provided the open part of it (around 1,000 videos) within our paper "Video compression dataset and benchmark of learning-based video-quality metrics", accepted to NeurIPS 2022.

是因为pytorch版本不匹配的问题吗？

Hi, zhuhancheng.
Thanks for your excellent work, but I'm a little bit confused about the meta model update location, and the counter location. Why they are in the most inner loop of the train_dataloader? Should not they be in the loop of the task?
https://github.com/zhuhancheng/MetaIQA/blob/30eeeb179bc72cf9408f91e2096726a2713de028/MetaIQA_On_TID2013_KADID.py#L360-366

Thanks for your great work!
I get low srocc on test set ChallengeDB(LIVE_WILD) after training on tid2013 and kadid10k in the first step to train quality prior model.

############# TID 2013 train phase epoch 49 ###############
current loss = 0.006955859042742911
############# Kadid train phase epoch 49 ###############
current loss = 0.012211388528347016
############# test phase epoch 49 ###############
----worker number: 0---- 1
new srocc 0.346739, best srocc 0.403438

I didn't make any change and can you give any suggestions?

Why are the labels not getting normalized in the computeSpearman function when they get normalized during training?

Hi! Thank you for your work. Can you tell me how to get the predicted quality score of a single image? Thank you very much!