haishen-ll / the-essence-of-lt-rocognition

This is the code for the paper "Investigate the Essence of Long-Tailed Recognition from a Unified Perspective".

This is the code for the paper "Investigate the Essence of Long-Tailed Recognition from a Unified Perspective".

This repository contains the implementation code for paper:
Investigate the Essence of Long-Tailed Recognition from a Unified Perspective
Lei Liu, and Li Liu
[arXiv]

If you find this code or idea useful, please consider citing our work:

@article{liu2021investigate,
  title={Investigate the Essence of Long-Tailed Recognition from a Unified Perspective},
  author={Liu, Lei and Liu, Li},
  journal={arXiv preprint arXiv:2107.03758},
  year={2021}
}

Overview

In this work, we show theoretically and empirically that, both sample number (imbalanced distribution) and category similarity (similar data) should be responsible for the performance dropping for imbalanced (long-tailed) recognition, and interpret representative and classifier learning from a unified perspective.

Toy Example: Toy example is to show that sample number is not the unique influence factor of long-tailed recognition.

A Unified Perspective for Long-Tailed Learning: In this section, through a simple theoretical model, we first make an intuition on how category similarity influences the classifier estimation via similar samples. With theoretical results, we skillful design verification experiments to confirm the negative effect of similar samples on performance dropping.

Understanding the Learning for Long-Tailed Recognition: Self-supervised pre-training is to help model learn discriminative features for the similar data.

Installation

Prerequisites

• Download CIFAR. Original data will be converted by imbalance_cifar.py.
• Download ImageNet & iNaturalist 2018 dataset, and place them in your data_path. Long-tailed version will be created using train/val splits (.txt files) in corresponding subfolders under imagenet_inat/data/
• Change the data_root in imagenet_inat/main.py accordingly for ImageNet-LT & iNaturalist 2018

Dependencies

• PyTorch (= 1.2)

Code Overview

Main Files

• train_classifier.py: train the classifier with fixed parameters of feature extractor.
• train_feature.py: train the feature extractor with (or without) SSP.
• imagenet_inat/main.py: train model with (or without) SSP, on ImageNet-LT / iNaturalist 2018.
• verification/imbalance_cifar.py: generate imbalanced and similar data for majority classes.
• pretrain_rot.py & pretrain_moco.py: self-supervised pre-training using Rotation prediction or MoCo. This part is refer to [Rethinking the Value of Labels for Improving Class-Imbalanced Learning].(https://github.com/YyzHarry/imbalanced-semi-self).

Main Arguments

• --dataset: name of chosen long-tailed dataset
• --imb_factor: imbalance factor (inverse value of imbalance ratio \rho in the paper)
• --imb_factor_unlabel: imbalance factor for unlabeled data (inverse value of unlabel imbalance ratio \rho_U)
• --pretrained_model: path to self-supervised pre-trained models
• --resume: path to resume checkpoint (also for evaluation)

Getting Started

Toy Example Learning

data sourcing

CIFAR-10-LT: CIFAR-10 unlabeled data is prepared following this repo using the 80M TinyImages.

Setting

To conduct the Toy Example (case 1), first is to fix the sample number for the 10-th class, while the sample number for each top-9 class is manually increased from 100 to 5,000 with the step of 100.

To conduct the Toy Example (case 2), first is to fix the sample number for each top-9 class, while the sample number for the 10-th class is manually decreased from 5,000 to 100 with the step of 100.

Main Results

The results are shown in the following tables.

Simulation Experiment

Setting

To demonstrate this theoretical finding experimentally, we skillfully construct some virtual similar samples with the help of mixup.

Main Results

The results are shown in the following table.

Self-Supervised pretraining

Setting

This part is refer to [Rethinking the Value of Labels for Improving Class-Imbalanced Learning]

Main Results

The results are shown in the following table.

Different Architectures

Main Results

Contact

If you have any questions, feel free to contact us through email ([email protected]) or Github issues.

the-essence-of-lt-rocognition's Issues

Could you please kindly share the parameters used in T-SNE?

Hi @LeiLiu-byte , thanks for your contribution, I tried to reproduce the T-SNE results reported in your paper, but failed, could you please share the parameters used? and which T-SNE library is used?

And when can you release the Appendix?