hikaritju / ld Goto Github PK
View Code? Open in Web Editor NEWLocalization Distillation for Object Detection (CVPR 2022, TPAMI 2023)
License: Apache License 2.0
Localization Distillation for Object Detection (CVPR 2022, TPAMI 2023)
License: Apache License 2.0
on the 487th line in ld_head.py, as below:
# losses_ld = list(map(lambda x: x / avg_factor, losses_ld))
it wasn't implemented, because losses_ld was samll, wasn't it ?
这样训练的时候losses_ld相对其他loss会很大,代码里为什么会这样实现呢?
我注意到您代码里的实现:
idx_out = torch.ops.torchvision.nms(gibox, giscore, 0.3)[:10]
return idx_out
gi_idx = self.get_gi_region(soft_label, cls_score, anchors,
bbox_pred, soft_targets, stride)
gi_teacher = teacher_x[gi_idx]
gi_student = x[gi_idx]
loss_im = self.loss_im(gi_student, gi_teacher)
并没有像GI 论文里那样使用GIbox ROIAlign 进行FM。
这样的话只选了10个FPN像素进行FM,而且经过NMS 这10个GI box像素附近的点也大概率被抑制掉了,这些像素应该也是信息量比较大的像素。
请问你们这样实现是为什么呢?有试过原文的做法吗,效果怎么样?
想问一下这里是吧多个二分类sigmoid输出转化为softmax下的了吗?这样是否不考虑背景分类的蒸馏
Hello,
Please, is it possible to run the code on cpu?
I would like to use a FAST-RCNN-RN50 as a teacher and SSD-LiteMobileNetv2 as a student. Both models are from MMDetection.
Thank you
Hello
Firstly, thank you for this great work.
Please, is it possible to use the code with whatever teacher/student model from mmdetection?
I am interested to distill knowledge from a large model to a very light student such as MobileNet, and use it for fast inference.
Thank you in advance
LD/mmdet/models/dense_heads/ld_atss.py
Line 467 in 3c5e398
您好,我使用了配置文件:configs/ld/ld_r50_atss_r101_1x.py
其中,bbox_head的type为LDATSSHead
train_cfg 的 assigner 为 ATSSAssigner
但是我发现,在mmdet/models/dense_heads/ld_atss.py第467行的assign_neg函数,并没有定义。
也就是说:LD/mmdet/core/bbox/assigners/atss_assigner.py的class ATSSAssigner 里没有assign_neg函数
it wasn't be installed by clicking the INSTALL.md.
您好,我在最新版的mmdetection中增加了这个仓库中的retinagfl_r101_2x_coco.py
文件和需要的一些head文件
代码的唯一改动是retina_fl_head.py
中的Integral.forward(),因为直接跑会出现数字过大cuda报错,所以将
x = F.linear(x, self.project.type_as(x)).reshape(-1, 4)
改成了
x = F.linear(x.reshape(4, -1, x.shape[-1]), self.project.type_as(x)).reshape(-1, 4)
其他都是按照原配置运行实验,学习率设置为0.01,8卡,具体设置如下
20220308_071908.log
但是实验结果非常糟糕,2022-03-09 20:50:44,029 - mmdet - INFO - Epoch(val) [24][625] bbox_mAP: 0.0010, bbox_mAP_50: 0.0020, bbox_mAP_75: 0.0000, bbox_mAP_s: 0.0000, bbox_mAP_m: 0.0000, bbox_mAP_l: 0.0020, bbox_mAP_copypaste: 0.001 0.002 0.000 0.000 0.000 0.002
想问下是哪里出现了问题?
For something like self.loss_ld = build_loss(dict(type='LocalizationDistillationLoss'...))
I seem not to find the class definition of it in this codebase...
Hello, thanks for such a wonderful work. After reading this paper, I have a question regarding model training.
According to the codes, ground truth annotations are still necessary during knowledge distillation to compute classfication loss, regression loss and DFL. The paper mentions that removing regression loss and DFL only causes a small decrease in mAP.
But if I want to completely get rid of the dependency on ground truth labels, how should I deal with the classfication loss? Does this term affects the final mAP a lot? Would you please share some insights or results about this?
能提供一下gfl_r101_fpn_1x_voc.py文件吗?
LD/mmdet/core/bbox/assigners/atss_assigner.py
Line 250 in 1cb808c
安装您论文给的设置,在源码和mmdetection上都训练不到论文中的效果,甚至连原来gfl都没有,请问这是什么原因?是不是蒸馏后T的值要改变的问题?
I caculated by below formula:
loss = loss_qfl + loss_bbox + loss_dfl + loss_ld
the result was that map was worse.
why?
GFocalV1 + LD R101_R50_1x pan.baidu pw: bvzz, according to which config?
您好,感谢您的很棒的工作。我想将LD用到其他检测器中,关于把regression换成离散化的概率这部分有一个疑惑想要问问作者。我看检测器每一个scale得到的regression都通过相同的integral
将概率转化成lrtb值,那么得到的lrtb都是一样的。所以最终不同feature scale的lrtb范围都是相同的,为什么不把每个scale的lrtb按照比例放缩范围呢?以及如果feature比较大,这样的实现得到的lrtb范围是否有可能小于GT lrtb的范围呢?
LD/mmdet/models/dense_heads/ld_head.py
Line 200 in 2dda5c0
您好!
我注意到LD项目中,config中的gfl文件夹里有复现相关不同网络使用generalized focal loss的代码
我理解的generalized focal loss的改进是在 1.分类分支使用了joint的quality focal loss 2.回归分支采用了general distribution+distribution focal loss,后者的核心是采用概率分布改进了边框拟合方式。
我看到项目中代码,例如gfl_r50_fpn_1x_coco这些,是在loss_cls中使用了quality focal loss,在loss_dfl中使用了distribution focal loss,这很容易理解。
但是,例如retina_gfl_r101_2x,fcos_gfl_r50_center这些配置中,对loss_cls还是使用了focalloss,在loss_bbox使用了GIoUloss(我理解的是quality focal loss中采用IoU label代替one-hot label的意思),不知道distribution focal loss是怎么体现的呢?其中,retinanet(retina_gfl_r101_2x)采用的好像还是deltaxywhbboxcoder?
非常感谢!
please provide teacher model link, thanks.
文章Table4和Table5所述的LD应该是指Main KD + Main LD + VLR LD吧,我看config文件里ld_r18_gflv1_r101_fpn_coco_1x.py只应用了Main LD,但在ld_r50_gflv1_r101_fpn_coco_1x.py中应用了Main KD + Main LD + VLR LD。其他的config需要自己配置编写对吧。谢谢!
Notice
There are several common situations in the reimplementation issues as below
There are several things to do for different cases as below.
Checklist
Describe the issue
A clear and concise description of what the problem you meet and what have you done.
Reproduction
A placeholder for the command.
A placeholder for the config.
Environment
python mmdet/utils/collect_env.py
to collect necessary environment information and paste it here.$PATH
, $LD_LIBRARY_PATH
, $PYTHONPATH
, etc.)Results
If applicable, paste the related results here, e.g., what you expect and what you get.
A placeholder for results comparison
Issue fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!
Hi, in your paper, you use 2 GPUs in all your experiments except for SOTA experiments and 8 GPUs in SOTA experiments, is it? If it is, may I ask why?
Hello again,
I tried to run the training code that is provided in the Readme.md file using 1 gpu.
./tools/dist_train.sh configs/ld/ld_r50_gflv1_r101_fpn_coco_1x.py 1
The only modification I did in the config file, is to precise that I want to run the program for one epoch:
runner = dict( type='EpochBasedRunner', max_epochs= 1 )
I am getting an error while trying to save the checkpoint after the training. This is the complete bug:
2022-02-03 02:45:24,914 - mmdet - INFO - Epoch [1][58550/58633] lr: 2.500e-03, eta: 0:00:46, time: 0.5
58, data_time: 0.004, memory: 4122, loss_cls: 0.7503, loss_bbox: 0.3822, loss_dfl: 0.2494, loss_ld: 0.
2703, loss_ld_vlr: 0.4174, loss_kd: 0.2851, loss_kd_neg: 0.0000, loss_im: 0.3847, loss: 2.7392
2022-02-03 02:45:52,665 - mmdet - INFO - Epoch [1][58600/58633] lr: 2.500e-03, eta: 0:00:18, time: 0.5
55, data_time: 0.004, memory: 4122, loss_cls: 0.7615, loss_bbox: 0.3864, loss_dfl: 0.2457, loss_ld: 0.
2216, loss_ld_vlr: 0.3467, loss_kd: 0.2940, loss_kd_neg: 0.0000, loss_im: 0.3496, loss: 2.6055
2022-02-03 02:46:16,693 - mmdet - INFO - Saving checkpoint at 1 epochs
[ ] 0/5000, elapsed: 0s, ETA:Traceback (most recent c
all last):
File "./tools/train.py", line 187, in <module>
main()
File "./tools/train.py", line 183, in main
meta=meta)
File "/home/edouard/eden/work/codes/LD/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/home/edouard/anaconda3/envs/LD/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py"
, line 125, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/edouard/anaconda3/envs/LD/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py"
, line 54, in train
self.call_hook('after_train_epoch')
File "/home/edouard/anaconda3/envs/LD/lib/python3.7/site-packages/mmcv/runner/base_runner.py", line
308, in call_hook
getattr(hook, fn_name)(self)
File "/home/edouard/eden/work/codes/LD/mmdet/core/evaluation/eval_hooks.py", line 276, in after_trai
n_epoch
gpu_collect=self.gpu_collect)
File "/home/edouard/eden/work/codes/LD/mmdet/apis/test.py", line 97, in multi_gpu_test
result = model(return_loss=False, rescale=True, **data)
File "/home/edouard/anaconda3/envs/LD/lib/python3.7/site-packages/torch/nn/modules/module.py", line
550, in __call__
result = self.forward(*input, **kwargs)
File "/home/edouard/anaconda3/envs/LD/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 458, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/home/edouard/anaconda3/envs/LD/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
result = self.forward(*input, **kwargs)
File "/home/edouard/anaconda3/envs/LD/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 84, in new_func
return old_func(*args, **kwargs)
File "/home/edouard/eden/work/codes/LD/mmdet/models/detectors/base.py", line 183, in forward
return self.forward_test(img, img_metas, **kwargs)
File "/home/edouard/eden/work/codes/LD/mmdet/models/detectors/base.py", line 160, in forward_test
return self.simple_test(imgs[0], img_metas[0], **kwargs)
File "/home/edouard/eden/work/codes/LD/mmdet/models/detectors/single_stage.py", line 120, in simple_test
*outs, img_metas, rescale=rescale)
File "/home/edouard/anaconda3/envs/LD/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 164, in new_func
return old_func(*args, **kwargs)
File "/home/edouard/eden/work/codes/LD/mmdet/models/dense_heads/anchor_head.py", line 583, in get_bboxes
scale_factors, cfg, rescale)
File "/home/edouard/eden/work/codes/LD/mmdet/models/dense_heads/gfl_head.py", line 560, in _get_bboxes
cfg.max_per_img)
File "/home/edouard/eden/work/codes/LD/mmdet/core/post_processing/bbox_nms.py", line 187, in multiclass_nms
return dets, labels[keep]
IndexError: index 8663 is out of bounds for dimension 0 with size 100
Traceback (most recent call last):
File "/home/edouard/anaconda3/envs/LD/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/edouard/anaconda3/envs/LD/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/edouard/anaconda3/envs/LD/lib/python3.7/site-packages/torch/distributed/launch.py", line 263, in <module>
main()
File "/home/edouard/anaconda3/envs/LD/lib/python3.7/site-packages/torch/distributed/launch.py", line 259, in main
cmd=cmd)
subprocess.CalledProcessError: Command '['/home/edouard/anaconda3/envs/LD/bin/python', '-u', './tools/train.py', '--local_rank=0', 'configs/ld/ld_r50_gflv1_r101_fpn_coco_1x.py', '--launcher', 'pytorch']' returned non-zero exit status 1.
PS. all is installed as recommended in the readme file.
Thank you very much for you help
您好,在看您论文和代码的时候,发现您实现了GI(General Instance),但是我好像只找到了特征蒸馏部分,没有找到response-based蒸馏和relation-based蒸馏,请问您有实现这两个部分吗?如果有,方便提供一下吗?非常感谢。
Exciting and excellent work! We are really interested in your AP landscape. Will you released the code or pseudo-code for testing AP landscape?
hi, was it implemented to losses = self(**data)?
on 234th line in base.py in mmdet/models/detectors
我看config里面没有yolox的,如果我想对yolox进行蒸馏是不是需要仿照着写一个配置文件
Hello, may I ask why you used log_softmax
for prediction but softmax
for soft label? Please see the code lines below.
LD/mmdet/models/losses/gfocal_loss.py
Lines 79 to 84 in f94c1d3
And here is the equation in your paper:
Best regards
LD/mmdet/core/bbox/assigners/atss_assigner.py
Lines 289 to 293 in 1cb808c
我用FCOS-GFL-R101的配置预训练了一个教师模型,并用教师训练了学生模型,基本达到了您论文的结果。
我想尝试使用其他的预训练模型训练教师,不知道可不可以,另外代码是不是需要修改某些部分,麻烦帮忙看一下,谢谢。
我对 LD/configs/ld/ld_r50_fcos_r101_1x.py 进行了更改,
teacher_config='configs/fcos/fcos_r101_caffe_fpn_gn-head_mstrain_640-800_2x_coco.py',
teacher_ckpt='configs/ld/fcos_r101_caffe_fpn_gn-head_mstrain_640-800_2x_coco-511424d6.pth'
并下载了mmdetection提供的预训练模型。
请问大佬,mmdetection官方项目中的ld_head.py中loss比LD提供的少了以下几个loss呢?
loss_ld_vlr=losses_ld_vlr,
loss_kd=losses_kd,
loss_kd_neg=losses_kd_neg,
loss_im=losses_im,
请问这是我理解错了还是后续改进了loss的使用?
I download Resnet101 GFocal model from mmdetction(download link:https://download.openmmlab.com/mmdetection/v2.0/gfl/gfl_r101_fpn_mstrain_2x_coco/gfl_r101_fpn_mstrain_2x_coco_20200629_200126-dd12f847.pth), and train by this command line:
bash ./tools/dist_train.sh configs/ld/ld_gflv1_r101_r50_fpn_coco_1x.py 8
but I get the mAP of epoch 12 is 0.3960。this paper give this mAP should be 41.2。
train and eval log:
2021-08-18 10:02:25,727 - mmdet - INFO - Epoch [12][4750/4887] lr: 3.750e-05, eta: 0:02:01, time: 0.882, data_time: 0.015, memory: 6289, loss_cls: 0.3524, loss_bbox: 0.2883, loss_ld: 0.1610, loss_dfl: 0.2044, loss: 1.0061
2021-08-18 10:03:10,220 - mmdet - INFO - Epoch [12][4800/4887] lr: 3.750e-05, eta: 0:01:17, time: 0.890, data_time: 0.014, memory: 6289, loss_cls: 0.3597, loss_bbox: 0.2918, loss_ld: 0.1602, loss_dfl: 0.2045, loss: 1.0162
2021-08-18 10:03:54,650 - mmdet - INFO - Epoch [12][4850/4887] lr: 3.750e-05, eta: 0:00:32, time: 0.888, data_time: 0.015, memory: 6289, loss_cls: 0.3543, loss_bbox: 0.2908, loss_ld: 0.1611, loss_dfl: 0.2060, loss: 1.0122
2021-08-18 10:04:45,038 - mmdet - INFO - Saving checkpoint at 12 epochs
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 5000/5000, 95.6 task/s, elapsed: 52s, ETA: 0s
2021-08-18 10:05:50,605 - mmdet - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=6.00s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type bbox
DONE (t=53.03s).
Accumulating evaluation results...
DONE (t=11.64s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.396
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.572
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.428
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.226
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.435
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.516
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.582
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.582
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.582
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.369
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.630
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.745
2021-08-18 10:07:02,558 - mmdet - INFO - Exp name: ld_gflv1_r101_r50_fpn_coco_1x.py
2021-08-18 10:07:02,559 - mmdet - INFO - Epoch(val) [12][4887] bbox_mAP: 0.3960, bbox_mAP_50: 0.5720, bbox_mAP_75: 0.4280, bbox_mAP_s: 0.2260, bbox_mAP_m: 0.4350, bbox_mAP_l: 0.5160, bbox_mAP_copypaste: 0.396 0.572 0.428 0.226 0.435 0.516
could you help me to slove this problem?
how to set many gpus?
./tools/dist_train.sh configs/ld/ld_gflv1_r101_r50_fpn_coco_1x.py 8
RuntimeError: NCCL error in: /pytorch/torch/lib/c10d/ProcessGroupNCCL.cpp:784, invalid usage, NCCL version 2.7.8
抱歉打扰了,我在复现您的代码时遇到了以下问题。
安装好相应包后我想试试代码能不能跑通,因为只有一个GPU,我将后面参数改为1
/tools/dist_train.sh configs/ld/ld_r50_gflv1_r101_fpn_coco_1x.py 1
报错如下:
Traceback (most recent call last):
File "./tools/train.py", line 15, in
from mmdet.apis import set_random_seed, train_detector
File "/home/cs/LD/mmdet/apis/init.py", line 1, in
from .inference import (async_inference_detector, inference_detector,
File "/home/cs/LD/mmdet/apis/inference.py", line 10, in
from mmdet.core import get_classes
File "/home/cs/LD/mmdet/core/init.py", line 5, in
from .mask import * # noqa: F401, F403
File "/home/cs/LD/mmdet/core/mask/init.py", line 2, in
from .structures import BaseInstanceMasks, BitmapMasks, PolygonMasks
File "/home/cs/LD/mmdet/core/mask/structures.py", line 6, in
import pycocotools.mask as maskUtils
ModuleNotFoundError: No module named 'pycocotools'
Traceback (most recent call last):
File "/home/cs/anaconda3/envs/LD/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/cs/anaconda3/envs/LD/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/cs/anaconda3/envs/LD/lib/python3.7/site-packages/torch/distributed/launch.py", line 263, in
main()
File "/home/cs/anaconda3/envs/LD/lib/python3.7/site-packages/torch/distributed/launch.py", line 259, in main
cmd=cmd)
subprocess.CalledProcessError: Command '['/home/cs/anaconda3/envs/LD/bin/python', '-u', './tools/train.py', '--local_rank=0', 'configs/ld/ld_r50_gflv1_r101_fpn_coco_1x.py', '--launcher', 'pytorch']' returned non-zero exit status 1.
运行环境:ubuntu 20.04
配置:
mmcv-full 1.2.7
torch 1.5.1
cuda 11.4
1.使用teacher-r101-student-r18的config 1x训练结果:
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.323
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.467
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.350
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.179
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.345
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.415
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.538
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.538
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.538
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.317
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.579
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.707
结果差论文很多,这大概是什么原因?
2.config中只加载了teacher网络在coco上的训练好的模型,为啥student却是只加载了imagenet预训练的模型,不应该是基于训练好的coco模型蒸馏吗?感谢回答
作者你好,请问确定Main 和 VLR区域的原理是什么,看代码实在看懵了,不太了解检测
at present, I modify and run through distillation code, using nanodet training framework.
setting parameter reduction='mean', loss_weight=1.0, T=2, alpha=1, beta=1 in LDLoss, did it?
[root][05-08 03:57:02]INFO:warmup|Iter(211/300)| lr:7.06e-02| loss_qfl:0.2829| loss_bbox:0.8766| loss_ld:0.0196| loss_dfl:0.3119|
[root][05-08 03:57:10]INFO:warmup|Iter(221/300)| lr:7.39e-02| loss_qfl:0.2794| loss_bbox:0.8341| loss_ld:0.0250| loss_dfl:0.2975|
[root][05-08 03:57:16]INFO:warmup|Iter(231/300)| lr:7.72e-02| loss_qfl:0.5848| loss_bbox:0.9857| loss_ld:0.0168| loss_dfl:0.3245|
[root][05-08 03:57:23]INFO:warmup|Iter(241/300)| lr:8.05e-02| loss_qfl:0.2754| loss_bbox:0.9521| loss_ld:0.0166| loss_dfl:0.3287|
...
[root][05-08 04:00:07]INFO:train|Epoch1/300|Iter281(280/1543)| lr:1.00e-01| loss_qfl:0.2121| loss_bbox:0.6372| loss_ld:0.0287| loss_dfl:0.2452|
[root][05-08 04:00:11]INFO:train|Epoch1/300|Iter291(290/1543)| lr:1.00e-01| loss_qfl:0.1914| loss_bbox:0.6420| loss_ld:0.0306| loss_dfl:0.2301|
[root][05-08 04:00:15]INFO:train|Epoch1/300|Iter301(300/1543)| lr:1.00e-01| loss_qfl:0.2096| loss_bbox:0.6041| loss_ld:0.0293| loss_dfl:0.2287|
[root][05-08 04:00:20]INFO:train|Epoch1/300|Iter311(310/1543)| lr:1.00e-01| loss_qfl:0.2008| loss_bbox:0.6006| loss_ld:0.0285| loss_dfl:0.2271|
......
Hi,
I found out you use sigmoid for classification (https://github.com/HikariTJU/LD/blob/main/mmdet/models/dense_heads/ld_head.py#L198), but for kd you use softmax, this would be a problem for single class.
Why not use L1?
Any idea?
@HikariTJU 你好, 在论文中有提到使用了Self KD可以涨点。 不过release的代码里,好像没有找到。
因为self KD的做法好像有好几种, 请问可以大概讲一下怎么做的吗?参考的哪篇论文呢?
谢谢!
您好,我按照项目中的要求创建了新的虚拟环境
使用了cuda10.1+pytorch1.5.0
(这里install.md中的mmcv-full安装方式需要更新pip install mmcv-full==1.2.7 -f https://download.openmmlab.com/mmcv/dist/cu101/torch1.5.0/index.html
)
README.md中的configs目录有问题
我只有一个GPU,训练的时候,我修改成了
./tools/dist_train.sh configs/ld/ld_r18_gflv1_r101_fpn_coco_1x.py 1
有一处报错为:
subprocess.CalledProcessError: Command '['/home/a/anaconda3/envs/open-mmlab/bin/python', '-u', './tools/train.py', '--local_rank=0', 'configs/ld/ld_r18_gflv1_r101_fpn_coco_1x.py', '--launcher', 'pytorch']' returned non-zero exit status 1.
我搜索了此种报错的解决方案,有一种是在DistributedDataParallel中添加find_unused_parameters=True
model = torch.nn.parallel.DistributedDataParallel(model,device_ids=[args.local_rank],output_device=args.local_rank, find_unused_parameters=True)
我想知道这个项目的find_unused_parameters应该在哪个文件中设置呢?感谢。
sys.platform: linux
Python: 3.7.11 (default, Jul 27 2021, 14:32:16) [GCC 7.5.0]
CUDA available: True
GPU 0: NVIDIA GeForce RTX 2060 SUPER
CUDA_HOME: /usr/local/cuda-10.2
NVCC: Cuda compilation tools, release 10.2, V10.2.89
GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
PyTorch: 1.5.1
PyTorch compiling details: PyTorch built with:
2021-11-27 22:44:39,967 - mmdet - INFO - Distributed training: True
2021-11-27 22:44:40,128 - mmdet - INFO - Config:
dataset_type = 'CocoDataset'
data_root = 'data/coco/'
img_norm_cfg = dict(
mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
train_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='LoadAnnotations', with_bbox=True),
dict(type='Resize', img_scale=(1333, 800), keep_ratio=True),
dict(type='RandomFlip', flip_ratio=0.5),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='Pad', size_divisor=32),
dict(type='DefaultFormatBundle'),
dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels'])
]
test_pipeline = [
dict(type='LoadImageFromFile'),
dict(
type='MultiScaleFlipAug',
img_scale=(1333, 800),
flip=False,
transforms=[
dict(type='Resize', keep_ratio=True),
dict(type='RandomFlip'),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='Pad', size_divisor=32),
dict(type='ImageToTensor', keys=['img']),
dict(type='Collect', keys=['img'])
])
]
data = dict(
samples_per_gpu=3,
workers_per_gpu=2,
train=dict(
type='CocoDataset',
ann_file='data/coco/annotations/instances_train2017.json',
img_prefix='data/coco/images/',
pipeline=[
dict(type='LoadImageFromFile'),
dict(type='LoadAnnotations', with_bbox=True),
dict(type='Resize', img_scale=(1333, 800), keep_ratio=True),
dict(type='RandomFlip', flip_ratio=0.5),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='Pad', size_divisor=32),
dict(type='DefaultFormatBundle'),
dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels'])
]),
val=dict(
type='CocoDataset',
ann_file='data/coco/annotations/instances_val2017.json',
img_prefix='data/coco/images/',
pipeline=[
dict(type='LoadImageFromFile'),
dict(
type='MultiScaleFlipAug',
img_scale=(1333, 800),
flip=False,
transforms=[
dict(type='Resize', keep_ratio=True),
dict(type='RandomFlip'),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='Pad', size_divisor=32),
dict(type='ImageToTensor', keys=['img']),
dict(type='Collect', keys=['img'])
])
]),
test=dict(
type='CocoDataset',
ann_file='data/coco/annotations/instances_val2017.json',
img_prefix='data/coco/images/',
pipeline=[
dict(type='LoadImageFromFile'),
dict(
type='MultiScaleFlipAug',
img_scale=(1333, 800),
flip=False,
transforms=[
dict(type='Resize', keep_ratio=True),
dict(type='RandomFlip'),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='Pad', size_divisor=32),
dict(type='ImageToTensor', keys=['img']),
dict(type='Collect', keys=['img'])
])
]))
evaluation = dict(interval=1, metric='bbox')
optimizer = dict(type='SGD', lr=0.00375, momentum=0.9, weight_decay=0.0001)
optimizer_config = dict(grad_clip=None)
lr_config = dict(
policy='step',
warmup='linear',
warmup_iters=500,
warmup_ratio=0.001,
step=[8, 11])
runner = dict(type='EpochBasedRunner', max_epochs=12)
checkpoint_config = dict(interval=1)
log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')])
custom_hooks = [dict(type='NumClassCheckHook')]
dist_params = dict(backend='nccl')
log_level = 'INFO'
load_from = None
resume_from = None
workflow = [('train', 1)]
teacher_ckpt = 'https://download.openmmlab.com/mmdetection/v2.0/gfl/gfl_r101_fpn_mstrain_2x_coco/gfl_r101_fpn_mstrain_2x_coco_20200629_200126-dd12f847.pth'
model = dict(
type='KnowledgeDistillationSingleStageDetector',
pretrained='torchvision://resnet18',
teacher_config='configs/gfl/gfl_r101_fpn_mstrain_2x_coco.py',
teacher_ckpt=
'https://download.openmmlab.com/mmdetection/v2.0/gfl/gfl_r101_fpn_mstrain_2x_coco/gfl_r101_fpn_mstrain_2x_coco_20200629_200126-dd12f847.pth',
output_feature=True,
backbone=dict(
type='ResNet',
depth=18,
num_stages=4,
out_indices=(0, 1, 2, 3),
frozen_stages=1,
norm_cfg=dict(type='BN', requires_grad=True),
norm_eval=True,
style='pytorch'),
neck=dict(
type='FPN',
in_channels=[64, 128, 256, 512],
out_channels=256,
start_level=1,
add_extra_convs='on_output',
num_outs=5),
bbox_head=dict(
type='LDHead',
num_classes=80,
in_channels=256,
stacked_convs=4,
feat_channels=256,
anchor_generator=dict(
type='AnchorGenerator',
ratios=[1.0],
octave_base_scale=8,
scales_per_octave=1,
strides=[8, 16, 32, 64, 128]),
loss_cls=dict(
type='QualityFocalLoss',
use_sigmoid=True,
beta=2.0,
loss_weight=1.0),
loss_dfl=dict(type='DistributionFocalLoss', loss_weight=0.25),
loss_ld=dict(
type='KnowledgeDistillationKLDivLoss', loss_weight=0.25, T=10),
reg_max=16,
loss_bbox=dict(type='GIoULoss', loss_weight=2.0)),
train_cfg=dict(
assigner=dict(type='ATSSAssigner', topk=9),
allowed_border=-1,
pos_weight=-1,
debug=False),
test_cfg=dict(
nms_pre=1000,
min_bbox_size=0,
score_thr=0.05,
nms=dict(type='nms', iou_threshold=0.6),
max_per_img=100))
work_dir = './work_dirs/ld_r18_gflv1_r101_fpn_coco_1x'
gpu_ids = range(0, 1)
Traceback (most recent call last):
File "/home/a/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/utils/registry.py", line 179, in build_from_cfg
return obj_cls(**args)
TypeError: init() missing 1 required positional argument: 'loss_im'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/a/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/utils/registry.py", line 179, in build_from_cfg
return obj_cls(**args)
File "/home/a/LD/mmdet/models/detectors/kd_one_stage.py", line 35, in init
pretrained)
File "/home/a/LD/mmdet/models/detectors/single_stage.py", line 30, in init
self.bbox_head = build_head(bbox_head)
File "/home/a/LD/mmdet/models/builder.py", line 59, in build_head
return build(cfg, HEADS)
File "/home/a/LD/mmdet/models/builder.py", line 34, in build
return build_from_cfg(cfg, registry, default_args)
File "/home/a/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/utils/registry.py", line 182, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
TypeError: LDHead: init() missing 1 required positional argument: 'loss_im'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./tools/train.py", line 187, in
main()
File "./tools/train.py", line 161, in main
test_cfg=cfg.get('test_cfg'))
File "/home/a/LD/mmdet/models/builder.py", line 77, in build_detector
return build(cfg, DETECTORS, dict(train_cfg=train_cfg, test_cfg=test_cfg))
File "/home/a/LD/mmdet/models/builder.py", line 34, in build
return build_from_cfg(cfg, registry, default_args)
File "/home/a/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/utils/registry.py", line 182, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
TypeError: KnowledgeDistillationSingleStageDetector: LDHead: init() missing 1 required positional argument: 'loss_im'
Traceback (most recent call last):
File "/home/a/anaconda3/envs/open-mmlab/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/a/anaconda3/envs/open-mmlab/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/a/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/distributed/launch.py", line 263, in
main()
File "/home/a/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/distributed/launch.py", line 259, in main
cmd=cmd)
subprocess.CalledProcessError: Command '['/home/a/anaconda3/envs/open-mmlab/bin/python', '-u', './tools/train.py', '--local_rank=0', 'configs/ld/ld_r18_gflv1_r101_fpn_coco_1x.py', '--launcher', 'pytorch']' returned non-zero exit status 1.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.