krumo / domain-adaptive-faster-rcnn-pytorch Goto Github PK

my env is
torch 1.4.0
torchvision 0.5.0
cuda 10.1

Traceback (most recent call last):
  File "D:/Domain-Adaptive-Faster-RCNN-PyTorch/train_net.py", line 18, in <module>
    from maskrcnn_benchmark.engine.inference import inference
  File "D:\Domain-Adaptive-Faster-RCNN-PyTorch\maskrcnn_benchmark\engine\inference.py", line 10, in <module>
    from maskrcnn_benchmark.data.datasets.evaluation import evaluate
  File "D:\Domain-Adaptive-Faster-RCNN-PyTorch\maskrcnn_benchmark\data\datasets\evaluation\__init__.py", line 3, in <module>
    from .coco import coco_evaluation
  File "D:\Domain-Adaptive-Faster-RCNN-PyTorch\maskrcnn_benchmark\data\datasets\evaluation\coco\__init__.py", line 1, in <module>
    from .coco_eval import do_coco_evaluation
  File "D:\Domain-Adaptive-Faster-RCNN-PyTorch\maskrcnn_benchmark\data\datasets\evaluation\coco\coco_eval.py", line 10, in <module>
    from maskrcnn_benchmark.structures.boxlist_ops import boxlist_iou
  File "D:\Domain-Adaptive-Faster-RCNN-PyTorch\maskrcnn_benchmark\structures\boxlist_ops.py", line 6, in <module>
    from maskrcnn_benchmark.layers import nms as _box_nms
  File "D:\Domain-Adaptive-Faster-RCNN-PyTorch\maskrcnn_benchmark\layers\__init__.py", line 8, in <module>
    from .nms import nms
  File "D:\Domain-Adaptive-Faster-RCNN-PyTorch\maskrcnn_benchmark\layers\nms.py", line 4, in <module>
    from ._utils import _C
  File "D:\Domain-Adaptive-Faster-RCNN-PyTorch\maskrcnn_benchmark\layers\_utils.py", line 39, in <module>
    _C = _load_C_extensions()
  File "D:\Domain-Adaptive-Faster-RCNN-PyTorch\maskrcnn_benchmark\layers\_utils.py", line 35, in _load_C_extensions
    extra_include_paths=extra_include_paths,
  File "C:\Users\wzr\Miniconda3\envs\da\lib\site-packages\torch\utils\cpp_extension.py", line 684, in load
    is_python_module)
  File "C:\Users\wzr\Miniconda3\envs\da\lib\site-packages\torch\utils\cpp_extension.py", line 881, in _jit_compile
    return _import_module_from_library(name, build_directory, is_python_module)
  File "C:\Users\wzr\Miniconda3\envs\da\lib\site-packages\torch\utils\cpp_extension.py", line 1089, in _import_module_from_library
    file, path, description = imp.find_module(module_name, [path])
  File "C:\Users\wzr\Miniconda3\envs\da\lib\imp.py", line 297, in find_module
    raise ImportError(_ERR_MSG.format(name), name=name)
ImportError: No module named 'torchvision'

Hi, thanks for sharing the code.
But when I try to convert cityscapes to coco format using convert_cityscapes_to_coco.py script, I found that the csHelpers and cityscapesscripts are missing. Could you please update it, thanks in advance.

First of all, thanks for your excellent work！

May I ask you whether the result map 41.0 is with FPN or not since it is too much better than paper's.
I see the configuration in log.txt is without the FPN.

Thanks again

File "D:\000_zhuang\DA_F_RCNN_torch_c\maskrcnn_benchmark\data\transforms\transforms.py", line 60, in call
target = target.resize(image.size)
AttributeError: 'list' object has no attribute 'resize'

When calculating the loss of the rpn part, how do you deal with the label of the target domain? I found you treat source and target domain data equally？I only notice you process the target domain label when calculate the loss of roi head.
Looking forward to your reply

Hi,

Thanks for sharing your code. I have a question:

I would like to compute and log the losses for the validation set during training. For doing this I followed a more recent version of mask_rcnn lines(128-174). However, the problem is that for evaluating on validation set batch size is 1 and for computing, the consistency loss batch size must be 2. I would like to know why is it the case?
Is it ok to use the batch size of 2 for computing this loss on the validation set?
Or do you have any other suggestions as a workaround regarding this?
Thank you

For example ('AP',2.2241e-05)

Hi, thanks for your work. Can you share the processed versions of cityscape and foggy cityscape datasets to reproduce the results?

Thank you.

Have you tried res50 for sim10--cityscape experiment? Could you show the results of the experiment? I did the experiment but I only observed that img_head has good performance, but ins_head does not seem to have much effect.

File "D:\000_zhuang\DA_F_RCNN_torch_c\maskrcnn_benchmark\structures\boxlist_ops.py", line 27, in boxlist_nms
keep = _box_nms(boxes, score, nms_thresh)
RuntimeError: Not compiled with GPU support

i can't from maskrcnn_benchmark import _C and there isn't _C

Hi,
Thanks for your awesome codes! The websit of Sim10k dataset is down. Could you share the Sim10k data by google drive or baidu drive?

Hi~Thanks for your share at first.And I have some problem with your work.How can I eval the AP for each classes for the experiment of cityscapes -> foggy_cityscapes like your paper.

The original paper and other papers used VGG backbone instead of ResNet-50-fpn, so is the improvement of mAP reported on README come from a much stronger backbone? Or other tricks?

ImportError: cannot import name '_C' from 'maskrcnn_benchmark' (/home/crx/zhanzan/Domain-Adaptive-Faster-RCNN-PyTorch_5/maskrcnn_benchmark/init.py)

I can run train_net.py in terminal successfully.
But I can't run this in pycharm.

Hi, I noticed that in rpn/loss.py, prepare_targets you have an output named "masks", but in call, you only do "torch.cat" to it and nothing else. So I am wondering what is the use of it? Thanks!

In the paper "Domain Adaptive Faster R-CNN for Object Detection in the Wild" https://arxiv.org/pdf/1803.03243.pdf

The Loss is described as follows:
The final training loss of the proposed network is a summation of each individual part, which can be written as:
L = Ldet + λ(Limg + Lins + Lcst)
where λ is a trade-off parameter to balance the Faster R-CNN loss and our newly added domain adaptation components

In the code the loss is defined as:
losses = sum(loss for loss in loss_dict.values())
Where there is no reference to a trade-off parameter.

Is this handled somewhere else in the code or is the trade-off parameter left out intentionally?

I installed ninja on the server, but there was an error when running pycharm. Strangely, I didn't have this error when running on the server

I have tried rebuilding the build folder. It does not help. Could you suggest a fix?

Hello, I run the test_net.py with the pre-trained model you provided (https://polybox.ethz.ch/index.php/s/OgkNFJHVkEscTO0) , but my AP50 is only 0.21. I can't find the reason, could you help me?

Now I got model_0042500.pth and dont't know how to resume my training from here

Hello,

I was wondering what the difference between proposals and da_proposals in the forward method of box_head.py

When debugging the code-base they appear to be identical

Is there any difference between the two?

In the box_head.py forward method, proposals are used in the feature extractor passed to the predictor to generate the class_logits and box_regrssion needed to get the loss_classifier and loss_box_reg from the loss_evaluator() ... then da_proposals are used in the same way as above to run the loss_evaluator once again to obtain da_ins_labels.

        if self.training:
            # Faster R-CNN subsamples during training the proposals with a fixed
            # positive / negative ratio
            with torch.no_grad():
                proposals = self.loss_evaluator.subsample(proposals, targets)

        # extract features that will be fed to the final classifier. The
        # feature_extractor generally corresponds to the pooler + heads
        x = self.feature_extractor(features, proposals)
        # final classifier that converts the features into predictions
        class_logits, box_regression = self.predictor(x)

        if not self.training:
            result = self.post_processor((class_logits, box_regression), proposals)
            return x, result, {}, x, None

        loss_classifier, loss_box_reg, _ = self.loss_evaluator(
            [class_logits], [box_regression]
        )

        if self.training:
            with torch.no_grad():
                da_proposals = self.loss_evaluator.subsample_for_da(proposals, targets)

        da_ins_feas = self.feature_extractor(features, da_proposals)
        class_logits, box_regression = self.predictor(da_ins_feas)
        lc2, lbr2, da_ins_labels = self.loss_evaluator(
            [class_logits], [box_regression]
        )

Unless I'm missing something could you not call the loss evaluator once and then return the da_ins_labels like this if they are identical anyway?

        loss_classifier, loss_box_reg, da_ins_labels = self.loss_evaluator(
            [class_logits], [box_regression]
        )

I feel I could be missing something here, it would be great if you could advise. Thanks! 😄

Hi. Thanks for your great work.
I want more DA configs for train. such as faster_rcnn_x_101, r_101.
Do you have plan to make these things?

Hi, 这篇论文 https://arxiv.org/abs/1812.04798 里面认为 da-faster-rcnn的ins level会降低map
我几个月前做的时候加入ins level也发现了这个情况
这个你怎么看呢？

Hello, thanks for your excellent work！
Could you tell me how to train DA from VOC to Clipart or Watercolor. The result I tried by myself is just 0.02. Maybe the config file requires some changes? Or something else?
Thanks a lot~

the link of pretrained model is invalid, could you upload it again

Hello,
thank you for your incredible work. Since this is one of the most important paper for unsupervised domain adaptation for object detection I have reimplemented this architecture using Detectron2. This allow many people to easily use this architecture because i created a Google Colab Notebook. Could you please add this repo to your section "Other implementation"?
link to the repo: https://github.com/GiovanniPasq/DA-Faster-RCNN
Thank you

能提供以一下kitti转为coco style的程序吗？

Hi, I wonder how to set up the images and annotations in the dataset?
for example, how to place the and folder?
Very need your help and answer, thank you!

Thank you very much for providing such excellent open source code for learning, and compiling according to the official guidance of mask-rcnn, the environment cannot be compiled successfully. Can you provide the environment configuration that can successfully compile and run this project? I have tried to use pytorch1.0, torchvision=0.2.2 and pytorch1.2, torchvision=0.4.0, but the project cannot be run correctly.

比如（'AP',2.2241e-05）

Original domain adaptive faster rcnn is unsupervised for target domain, which means that there isn't any label of target domain dataset. But in paths_catalog.py: it provides the ann_file for foggy_cityscapes_...._train_cocostyle, why? Doesn't it mean that the model already see the foggy_cityscapes image in the training process?How can I train the model with unlabeled target domain dataset?
Thanks!

Hello @krumo , thanks for the great work! I am trying to use your project with a ResNet backbone with a feature pyramid network on top, I would like to ask you if you have any clue about how to apply the domain discriminator at the image level in this case, as the backbone returns feature maps of different sizes, and it is not possible to simply concatenate them. I've noticed that it is stated that DomainAdaptationModule works for both FPN and non-FPN and that a boolean USE_FPN parameter is mentioned but not used, maybe you already thought of an expected behavior?

This implementation is based on maskrcnn-benchmark, from which the faster rcnn is not the same in the original paper, for example
1 origin backbone: vgg16, here: resnet
2 origin roi pooler: roi pool, here: roi align
3 origin roi feature: 2 mlp, here ResNet50Conv5
and so on
These difference is default in the maskrcnn-benchmark.
People who refers to this implementation so really pay attention to these difference.

Hi, when i run the script to test trained model, KeyError occurs.
the used script: python tools/test_net.py --config_file "configs/da_faster_rcnn/e2e_da_faster_rcnn_R_50_C4_cityscapes_to_foggy_cityscapes.yaml" MODEL.WEIGHT trainedmodels/model_final.pth
detailed error:
Traceback (most recent call last):
File "tools/test_net.py", line 98, in
main()
File "tools/test_net.py", line 50, in main
cfg.merge_from_file(args.config_file)
File "/home/casia_robot_lwy/anaconda3/envs/py3_env/lib/python3.7/site-packages/yacs/config.py", line 213, in merge_from_file
self.merge_from_other_cfg(cfg)
File "/home/casia_robot_lwy/anaconda3/envs/py3_env/lib/python3.7/site-packages/yacs/config.py", line 217, in merge_from_other_cfg
_merge_a_into_b(cfg_other, self, self, [])
File "/home/casia_robot_lwy/anaconda3/envs/py3_env/lib/python3.7/site-packages/yacs/config.py", line 460, in _merge_a_into_b
_merge_a_into_b(v, b[k], root, key_list + [k])
File "/home/casia_robot_lwy/anaconda3/envs/py3_env/lib/python3.7/site-packages/yacs/config.py", line 473, in _merge_a_into_b
raise KeyError("Non-existent config key: {}".format(full_key))
KeyError: 'Non-existent config key: MODEL.DOMAIN_ADAPTATION_ON'

in trainer.py line 141-146
images = (source_images+target_images).to(device)
targets = [target.to(device) for target in list(source_targets+target_targets)]

loss_dict = model(images, targets)

losses = sum(loss for loss in loss_dict.values())

Does it seem that the target label is also used for calculating the detector_losses and proposal_losses?

Hello, I used cityscapes and foggy_cityscapes dataset to train the model according to the example usage,after 70000 iter ,the AP50 index obtained is only 0.12. I would like to ask where there may be a problem.

Hello. I checked your new git.
Thanks for your great work.

My question is the meaning of many losses.

• loss /
• loss_classifier /
• loss_box_reg /
• loss_objectness /
• loss_rpn_box_reg /
• loss_da_image /
• loss_da_instance /
• loss_da_consistency

where can I found detailed explanation of these loss?

Hello! Can you upload the pretrained model again? Thank you！

Hi,
Thanks for sharing your code.
But why are the the results are different from the numbers in the paper.

hello,I meet this problem, who knows how to solve it?

Hi, thanks for your excellent work. I have never accessed cityscapes dataset before, and the official download page really confuses me since there're so many individual packages (sub-dataset?) like gtFine, gtCoarse, leftImg8bit etc. I'm wondering which package should I download for this project? Are the data structures of these packages actually the same?

Same question comes with foggy_dataset, many thanks :)

In the call of modelling.rpn.loss.py a masks variable is generated

    def __call__(self, anchors, objectness, box_regression, targets):
        """
        Arguments:
            anchors (list[BoxList])
            objectness (list[Tensor])
            box_regression (list[Tensor])
            targets (list[BoxList])

        Returns:
            objectness_loss (Tensor)
            box_loss (Tensor
        """
        anchors = [cat_boxlist(anchors_per_image) for anchors_per_image in anchors]
        labels, regression_targets, masks = self.prepare_targets(anchors, targets)
        
        masks = torch.cat(masks, dim=0)
        
        sampled_pos_inds, sampled_neg_inds = self.fg_bg_sampler(labels)
        sampled_pos_inds = torch.nonzero(torch.cat(sampled_pos_inds, dim=0)).squeeze(1)
        sampled_neg_inds = torch.nonzero(torch.cat(sampled_neg_inds, dim=0)).squeeze(1)

        sampled_inds = torch.cat([sampled_pos_inds, sampled_neg_inds], dim=0)

        objectness, box_regression = \
                concat_box_prediction_layers(objectness, box_regression)

        objectness = objectness.squeeze()

        labels = torch.cat(labels, dim=0)
        regression_targets = torch.cat(regression_targets, dim=0)

        box_loss = smooth_l1_loss(
            box_regression[sampled_pos_inds],
            regression_targets[sampled_pos_inds],
            beta=1.0 / 9,
            size_average=False,
        ) / (sampled_inds.numel())

        objectness_loss = F.binary_cross_entropy_with_logits(
            objectness[sampled_inds], labels[sampled_inds]
        )

        return objectness_loss, box_loss

It is returned from prepare_targets in line 2 and then the cat() function is called to on itself to generate a new masks var in line 3.

However, it is not used in the rest of the call function.

Should these masks have been used somewhere or are they a redundant return from prepare_targets() ?

Thanks

when I run the test_net.py via using the following cmd, "IndexError: list index out of range" occurs.
used cmd:
python tools/test_net.py --config_file "configs/da_faster_rcnn/e2e_da_faster_rcnn_R_50_C4_cityscapes_to_foggy_cityscapes.yaml" MODEL.WEIGHT trainedmodels/model_cst.pth
File "/home/casia_robot_lwy/Documents/code_cv/da_object_detetion/Domain-Adaptive-Faster-RCNN-PyTorch-master/maskrcnn-benchmark/maskrcnn_benchmark/structures/segmentation_mask.py", line 464, in getitem
selected_polygons.append(self.polygons[i])

occured error:
IndexError: list index out of range

errored image is 462/500 in val set of cityscapes, specifically, it is lindau_000020_000019_leftImg8bit.png

FileNotFoundError: [Errno 2] No such file or directory: './datasets/caronly_filtered_gtFine_val.json'