PromptIQA: Boosting the Performance and Generalization for No-Reference Image Quality Assessment via Prompts
- The 18th European Conference on Computer Vision ECCV 2024
Zewen Chen, Haina Qin, Jun Wang, Chunfeng Yuan, Bing Li
National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences (NLPR, CASIA).
๐ ๐ ๐ News:
- To be updated...
- โ March, 2023: We created this repository.
- [] Code for PromptIQA
- [] Code for training
- [] Code for testing
- [] Checkpoint
- [] Online Demo on huggingface
This is an official implementation of PromptIQA: Boosting the Performance and Generalization for No-Reference Image Quality Assessment via Prompts by Pytorch.
Due to the diversity of assessment requirements in various application scenarios for the IQA task, existing IQA methods struggle to directly adapt to these varied requirements after training. Thus, when facing new requirements, a typical approach is fine-tuning these models on datasets specifically created for those requirements. However, it is time-consuming to establish IQA datasets. In this work, we propose a Prompt-based IQA (PromptIQA) that can directly adapt to new requirements without fine-tuning after training. On one hand, it utilizes a short sequence of Image-Score Pairs (ISP) as prompts for targeted predictions, which significantly reduces the dependency on the data requirements. On the other hand, PromptIQA is trained on a mixed dataset with two proposed data augmentation strategies to learn diverse requirements, thus enabling it to effectively adapt to new requirements. Experiments indicate that the PromptIQA outperforms SOTA methods with higher performance and better generalization.
Figure1: The framework of the proposed PromptIQA.
The dependencies for this work as follows:
TBU
You can also run the following command to install the environment directly:
TBU
You can download the total 12 datasets from the following download link. (TBU)
Dataset | Task | Ref. Image | Dist. Types | Image Number | Label Type | Range | Download |
---|---|---|---|---|---|---|---|
LIVE | Synthetic Distortion Nature IQA |
29 | 5 | 779 | DMOS | [1, 100] | |
CSIQ | 30 | 6 | 866 | DMOS | [0, 1] | ||
TID2013 | 25 | 24 | 3,000 | MOS | [0, 9] | ||
Kadid-10k | 81 | 25 | 10,125 | MOS | [1, 5] | ||
BID | Authentic Distortion Nature IQA |
- | - | 586 | MOS | [0, 5] | Link |
SPAQ | - | - | 11,125 | MOS | [0, 100] | Link | |
LIVEC | - | - | 1,162 | MOS | [1, 100] | Link | |
KonIQ-10K | - | - | 10,073 | MOS | [0, 100] | Link | |
GFIQA20k | Face IQA | - | - | 19,988 | MOS | [0, 1] | |
AGIQA3k | AI generated IQA | - | - | 2,982 | MOS | [0, 1] | |
AIGCIQA2023 | - | - | 2,400 | MOS | [0, 1] | ||
UWIQA | Underwater IQA | - | - | 890 | MOS | [0, 1] |
- You should replace the dataset path in dataset_info.json to your own dataset path.
- Run the following command to train the PromptIQA (Please review the train.py for more options).
TBU
We achieved state-of-the-art performance on most IQA datasets simultaniously within one single model.
More detailed results can be found in the paper.
Individual Dataset Comparison.
If our work is useful to your research, we will be grateful for you to cite our paper:
We sincerely thank the great work HyperIQA, MANIQA and MoCo. The code structure is partly based on their open repositories.