vibashan / online-da

Official Pytorch Codebase for Towards Online Domain Adaptive Object Detection [WACV 2023]

Towards Online Domain Adaptive Object Detection [WACV 2023]

Vibashan VS, Poojan Oza, Vishal M Patel

[Project Page] [WACV] [pdf] [BibTeX]

Contributions

• To the best of our knowledge, this is the first work to consider both online and offline adaptation settings for detector models.
• We propose a novel unified adaptation framework which makes the detector models robust against online target distribution shifts
• We introduce the MemXformer module, which stores prototypical patterns of the target distribution and provides contrastive pairs to boost contrastive learning on the target domain.

Contents

1. Installation Instructions
2. Dataset Preparation
3. Execution Instructions
   • Training
   • Evaluation
4. Results
5. Citation

Installation Instructions

• We use Python 3.6, PyTorch 1.9.0 (CUDA 10.2 build).
• We codebase is built on Detectron.

conda create -n online_da python=3.6

Conda activate online_da

conda install pytorch==1.9.0 torchvision==0.10.0 torchaudio==0.9.0 cudatoolkit=10.2 -c pytorch

cd online-da
pip install -r requirements.txt

## Make sure you have GCC and G++ version <=8.0
cd ..
python -m pip install -e online-da

Dataset Preparation

• PASCAL_VOC 07+12: Please follow the instructions in py-faster-rcnn to prepare VOC datasets.
• Clipart, WaterColor: Dataset preparation instruction link Cross Domain Detection . Images translated by Cyclegan are available in the website.
• Sim10k: Website Sim10k
• CitysScape, FoggyCityscape: Download website Cityscape, see dataset preparation code in DA-Faster RCNN

Download all the dataset into "./dataset" folder. The codes are written to fit for the format of PASCAL_VOC. For example, the dataset Sim10k is stored as follows.

$ cd ./dataset/Sim10k/VOC2012/
$ ls
Annotations  ImageSets  JPEGImages
$ cat ImageSets/Main/val.txt
3384827.jpg
3384828.jpg
3384829.jpg
.
.

Execution Instructions

Training

• Download the source-trained model weights in source_model folder Link

CUDA_VISIBLE_DEVICES=$GPU_ID python tools/train_onlineda_net.py \ 
--config-file configs/online_da/onda_foggy.yaml --model-dir ./source_model/cityscape_baseline/model_final.pth

Evaluation

• After training, load the teacher model weights and perform evaluation using

CUDA_VISIBLE_DEVICES=$GPU_ID python tools/plain_test_net.py --eval-only \ 
--config-file configs/online_da/foggy_baseline.yaml --model-dir $PATH TO CHECKPOINT

Results

• Pre-trained models can be downloaded from Link.

Citation

If you found Online DA useful in your research, please consider starring ⭐ us on GitHub and citing 📚 us in your research!

@inproceedings{vs2023towards,
  title={Towards Online Domain Adaptive Object Detection},
  author={VS, Vibashan and Oza, Poojan and Patel, Vishal M},
  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
  pages={478--488},
  year={2023}
}

Acknowledgement

We thank the developers and authors of Detectron for releasing their helpful codebases.

online-da's Issues

When can get the code

When can get the code, thank you!

Clarification Regarding the Roles of Key and Query in Cross-Attention Mechanism

First and foremost, I would like to express my appreciation for the outstanding work you have done in this field. Your insights have had a significant impact on my research, and I greatly admire your contributions.

While studying the cross-attention mechanism described in your paper, particularly in Figure 3(a) and the accompanying equations, I encountered some confusion regarding the roles of the Key, Query, and Value components.

In the paper, memory items are treated as the Query, and the teacher RoI features are projected as the Key. However, in the implementation of the code, the memory items are instead treated as the Key. There seems to be a conflict between these two interpretations. Could you please clarify why this is the case?

Thank you very much for your time and assistance. I look forward to your response.

Paper：

Code：

Question about code release

Thanks for your solid work, when will the code be released? I hope it will be soon, thanks!

The details of online setting

Hi,
Nice works! Could you please update more details of online setting?

loss_bbox_reg = 0

Hii,

=I am getting loss_bbox_reg - 0 for all, anything can help me in this regard. Thanks

When will the code be open?

Hi, Vibashan,

Congratulations for this paper being accepted by WACV2023, and it's nice to see that you provided the code link in the paper. To better compare and reference your method, we want to know when the complete code will be made public.

Thanks for your reply.

Training a source model for a new dataset

Hi, I want to train a source model for a new dataset how to implement this?

error: package directory 'projects/PointRend/point_rend' does not exist

Firstly, thanks for your nice work :)

When I try to run python -m pip install -e online-da, I get the error as follows:

Obtaining file:///home/linhb/code/online-da
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/linhb/miniconda3/envs/online_da/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/linhb/code/online-da/setup.py'"'"'; __file__='"'"'/home/linhb/code/online-da/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-zk0oo2k4
       cwd: /home/linhb/code/online-da/
  Complete output (8 lines):
  running egg_info
  creating /tmp/pip-pip-egg-info-zk0oo2k4/detectron2.egg-info
  writing /tmp/pip-pip-egg-info-zk0oo2k4/detectron2.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-pip-egg-info-zk0oo2k4/detectron2.egg-info/dependency_links.txt
  writing requirements to /tmp/pip-pip-egg-info-zk0oo2k4/detectron2.egg-info/requires.txt
  writing top-level names to /tmp/pip-pip-egg-info-zk0oo2k4/detectron2.egg-info/top_level.txt
  writing manifest file '/tmp/pip-pip-egg-info-zk0oo2k4/detectron2.egg-info/SOURCES.txt'
  error: package directory 'projects/PointRend/point_rend' does not exist

Could anyone offer me a hand?

How to change online da to sfda setting?

Hi there!
It seems that the execution instructions from README.md is only for online-da setting.
I'm new to Detectron2 and I wonder how to modify the instructions or code to run sfda part? thx

Issue about the setting of the code

Is the provided code run in the online setting? Does this mean the model is first trained on the entire unlabeled target test set first and then evaluated on the same target test set? So it is a little different from the fully test-time adaption setting proposed by TENT.

Issue about the code

The write operation of the memory bank seems to be inconsistent with the formula in the paper based on code "memory_bank.py" line 69: query_updated_memory[i] = torch.sum(((score[idx,i] / torch.max(score[:,i]))*value[idx].squeeze(1)), dim=0).