h751410234 / datr

A strong DETR-based detector named Domain Adaptive detection TRansformer (DATR) for unsupervised domain adaptation in object detection.

DATR: Unsupervised Domain Adaptive Detection Transformer with Dataset-Level Adaptation and Prototypical Alignment

By Jianhong Han, Liang Chen and Yupei Wang.

This repository contains the implementation accompanying our paper DATR: Unsupervised Domain Adaptive Detection Transformer with Dataset-Level Adaptation and Prototypical Alignment.

If you find it helpful for your research, please consider citing:

@article{han2024datr,
  title={DATR: Unsupervised Domain Adaptive Detection Transformer with Dataset-Level Adaptation and Prototypical Alignment},
  author={Han, Jianhong and Chen, Liang and Wang, Yupei},
  journal={arXiv preprint arXiv:2405.11765},
  year={2024}
}

Acknowledgment

This implementation is bulit upon DINO and RemoteSensingTeacher.

Installation

Please refer to the instructions here. We leave our system information for reference.

• OS: Ubuntu 16.04
• Python: 3.10.9
• CUDA: 11.8
• PyTorch: 2.0.1 (The lower versions of Torch can cause some bugs.)
• torchvision: 0.15.2

Dataset Preparation

Please construct the datasets following these steps:

• Download the datasets from their sources.

• Convert the annotation files into COCO-format annotations.

• Modify the dataset path setting within the script DAcoco.py

  #---源域
    PATHS_Source = {
        "train": ("",  #train image dir
                  ""), #train coco format json file
        "val": ("",    #val image dir
                ""),   #val coco format json file
    }
    #----目标域
    PATHS_Target = {
        "train": ("",  #train image dir
                  ""), #train coco format json file
        "val": ("",    #val image dir
                ""),   #val coco format json file
    }

• Add domain adaptation direction within the script init.py. For example:

    if args.dataset_file == 'city':
        return build_city_DA(image_set, args,strong_aug)

Training / Evaluation / Inference

We provide training script as follows. We divide the training process into two stages. The settings for each stage can be found in the config folder.

(1) For the Burn-In stage:

• Training with single GPU

sh scripts/DINO_train.sh

• Training with Multi-GPU

sh scripts/DINO_train_dist.sh

(2) For the Teacher-Student Mutual Learning stage, it is necessary to use the optimal model obtained from the first stage of training.

• Training with single GPU

sh scripts/DINO_train_self_training.sh

• Training with Multi-GPU

sh scripts/DINO_train_self_training_dist.sh

We provide evaluation script to evaluate pre-trained model.

• Evaluation Model.

sh scripts/DINO_eval.sh

• Evaluation EMA Model.

sh scripts/DINO_eval_for_EMAmodel.sh

We provide inference script to visualize detection results. See inference.py for details

• Inference Model.

python inference.py

• Inference EMA Model.

python inference_ema_model.py 

Pre-trained models

We provide specific experimental configurations and pre-trained models to facilitate the reproduction of our results. You can learn the details of DATR through the paper, and please cite our papers if the code is useful for your papers. Thank you!

Task	mAP50	Config	Model
Cityscapes to Foggy Cityscapes	52.8%	cfg	model
Sim10k to Cityscapes	66.3%	cfg	model
Cityscapes to BDD100K-daytime	41.9%	cfg	model

Reference

https://github.com/IDEA-Research/DINO

https://github.com/h751410234/RemoteSensingTeacher

datr's Issues

遥感数据

您好 我想将xView到Dota任务用这个框架实现 请问除了修改配置中的num_classes之外还要进行什么修改呢 修改num_classes之后仍然有访问报错 看起来是类别数量报错

论文原文及训练问题

您好，请问有论文原文链接吗
以及，这个预训练模型是怎么得到的，只有用这个预训练模型才能复现普通光学图像中的效果吗? 如果我想训练自定义数据集比如光学遥感图像，需要怎么做？
感谢

模型训练

您好 请问这个模型选择单卡训练batchsize为2 在cityscapes到foggycityscapes上训练 效果保持稳健吗

the shape about output_source？in dino.py

Hello, your job is very good! I have some questions in dino.py:
Is the shape of the output_source [bs,N,256] ?
What is the meaning of N ?

#Second.==============objcet query Prototypical Contrast Adaptation======================
#---- For Source Domain

1.--------提取原型特征-----------------

(1)得到最后一层object query 结果

output_source = hs[-1][:,dn_meta['pad_size']:,:] #source_query_feature :[bs,N,256]

显存

您好 我现在是用A6000进行的训练 训练配置和您推荐的一样 但目前面临一个问题 无论是当前DATR的工作 还是您前一篇工作RemoteSensingTeacher 都存在显存占用过高的问题 就是随着训练的不断进行 显存占用在不断增加 最后会因为显存超出而出现core dump 导致训练终止 您有遇到过这两个问题吗 请问针对这一问题 您有什么建议吗 比如第一张图是训练RST时最后一轮显存超了 第二张图DATR刚开始训练 显存占用就达到38000MB左右了

第二阶段训练，加载模型问题

使用单卡训练，第一阶段结束，在第二阶段训练时，报错如下：
Traceback (most recent call last):
File "/media/zh/code/DATR/main_teacher.py", line 565, in
main(args)
File "/media/zh/code/DATR/main_teacher.py", line 234, in main
model_without_ddp.load_state_dict(checkpoint['ema_model'])
KeyError: 'ema_model'

是加载的模型不对么？