lkeab / gsnet Goto Github PK

Official code of ECCV 2020 paper "GSNet: Joint Vehicle Pose and Shape Reconstruction with Geometrical and Scene-aware Supervision". GSNet performs joint vehicle pose estimation and vehicle shape reconstruction with single RGB image as input.

Home Page: https://arxiv.org/abs/2007.13124

Python 90.17% C++ 3.50% Cuda 6.31% Shell 0.02%

eccv vehicle-pose-shape-reconstruction 3d-shape-reconstruction eccv2020 car-models car-shape 6dof-pose 3d-model key-points-detection pca-models

gsnet's Introduction

GSNet: Joint Vehicle Pose and Shape Reconstruction with Geometrical and Scene-aware Supervision [ECCV'20]

Code and 3D car mesh models for the ECCV 2020 paper "GSNet: Joint Vehicle Pose and Shape Reconstruction with Geometrical and Scene-aware Supervision". GSNet performs joint vehicle pose estimation and vehicle shape reconstruction with single RGB image as input.

[arXiv]|[Paper]|[Project Page]

Abstract

We present a novel end-to-end framework named as GSNet (Geometric and Scene-aware Network), which jointly estimates 6DoF poses and reconstructs detailed 3D car shapes from single urban street view. GSNet utilizes a unique four-way feature extraction and fusion scheme and directly regresses 6DoF vehicle poses and shapes in a single forward pass. Extensive experiments show that our diverse feature extraction and fusion scheme can greatly improve model performance. Based on a divide-and-conquer 3D shape representation strategy, GSNet reconstructs 3D vehicle shape with great detail (1352 vertices and 2700 faces). This dense mesh representation further leads us to consider geometrical consistency and scene context, and inspires a new multi-objective loss function to regularize network training, which in turn improves the accuracy of 6D pose estimation and validates the merit of jointly performing both tasks.

Results on ApolloCar3D benchmark

(Check Table 3 of the paper for full results)

Method	A3DP-Rel-mean	A3DP-Abs-mean
DeepMANTA (CVPR'17)	16.04	20.1
3D-RCNN (CVPR'18)	10.79	16.44
Kpt-based (CVPR'19)	16.53	20.4
Direct-based (CVPR'19)	11.49	15.15
GSNet (ECCV'20)	20.21	18.91

Installation

We build GSNet based on the Detectron2 developed by FAIR. Please first follow its readme file. We recommend the Pre-Built Detectron2 (Linux only) version with pytorch 1.5 by the following command:

python -m pip install detectron2 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.5/index.html

Dataset Preparation

The ApolloCar3D dataset is detailed in paper ApolloCar3D and the corresponding images can be obtained from link. We provide our converted car meshes (same topology), kpts, bounding box, 3d pose annotations etc. in coco format under the car_deform_result and datasets/apollo/annotations folders.

Environment

Python 3.6
Numpy 1.16
PyTorch >= 1.0.1
CUDA 9/10
Softras
Pyrender

Using Our Car mesh models

car_deform_result: We provide 79 types of ground truth car meshes with the same topology (1352 vertices and 2700 faces) converted using SoftRas (https://github.com/ShichenLiu/SoftRas)

The file car_models.py has a detailed description on the car id and car type correspondance.

merge_mean_car_shape: The mean car shape of the four shape basis used by four independent PCA models.

pca_components: The learned weights of the four PCA models.

How to use our car mesh models? Please refer to the class StandardROIHeads in roi_heads.py, which contains the core inference code for ROI head of GSNet. It relies on the SoftRas to load and manipulate the car meshes.

Run GSNet

Please follow the readme page (including the pretrained model).

Citation

Please star this repository and cite the following paper in your publications if it helps your research:

@InProceedings{gsnet2020ke,
    author = {Ke, Lei and Li, Shichao and Sun, Yanan and Tai, Yu-Wing and Tang, Chi-Keung},
    title = {GSNet: Joint Vehicle Pose and Shape Reconstruction with Geometrical and Scene-aware Supervision},
    booktitle = {European Conference on Computer Vision (ECCV)},
    year = {2020}
}

Related repo based on detectron2: BCNet, CVPR'21- a bilayer instance segmentation method

License

A MIT license is used for this repository. However, certain third-party datasets, such as (ApolloCar3D), are subject to their respective licenses and may not grant commercial use.

gsnet's People

Contributors

Stargazers

Watchers

gsnet's Issues

Training codes

Hi @lkeab ,

Thanks for the great work! I wonder if you will be able to share the training codes and some detailed training procedures so we are able to train the model ourselves. Please let us know if that's possible, thank you!

pytorch version for detectron2 under gsnet/reference_code/GSNet-release

Hello,

Firstly, thanks for the codes of your impressive research.

I have an issue when running

CUDA_VISIBLE_DEVICES=0 python3 demo/demo.py --config-file configs/COCO-Keypoints/keypoint_rcnn_R_101_FPN_3x_apollo.yaml \
  --input 'datasets/coco/val_apollo/' \
  --output 'medium_result_val/' \
  --opts MODEL.WEIGHTS ./output/model_final.pth

The error information:

Traceback (most recent call last):
  File "demo/demo.py", line 11, in <module>
    from detectron2.data.detection_utils import read_image
  File "/home/Test/code/gsnet/reference_code/GSNet-release/detectron2/data/__init__.py", line 4, in <module>
    from .build import (
  File "/home/Test/code/gsnet/reference_code/GSNet-release/detectron2/data/build.py", line 14, in <module>
    from detectron2.structures import BoxMode
  File "/home/Test/code/gsnet/reference_code/GSNet-release/detectron2/structures/__init__.py", line 2, in <module>
    from .boxes import Boxes, BoxMode, pairwise_iou
  File "/home/Test/code/gsnet/reference_code/GSNet-release/detectron2/structures/boxes.py", line 8, in <module>
    from detectron2.layers import cat
  File "/home/Test/code/gsnet/reference_code/GSNet-release/detectron2/layers/__init__.py", line 3, in <module>
    from .deform_conv import DeformConv, ModulatedDeformConv
  File "/home/Test/code/gsnet/reference_code/GSNet-release/detectron2/layers/deform_conv.py", line 10, in <module>
    from detectron2 import _C
ImportError: /home/Test/code/gsnet/reference_code/GSNet-release/detectron2/_C.cpython-36m-x86_64-linux-gnu.so: undefined symbol: THPVariableClass

Env information

python: 3.6.2
CUDA: 10.1
pytorch: 1.5.0
ubuntu: 18.04

The existing solutions about this issue give me that the version of pytorch is incompatible with detectron2.

What I tried

Remove the detectron2 and detectron2.egg-info you provided and reinstall detectron2 via

python -m pip install detectron2 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.5/index.html

By doing this I got:

Traceback (most recent call last):
  File "demo/demo.py", line 73, in <module>
    cfg = setup_cfg(args)
  File "demo/demo.py", line 28, in setup_cfg
    cfg.MODEL.FCOS.INFERENCE_TH = args.confidence_threshold
  File "/home/Test/anaconda3/envs/gsnet/lib/python3.6/site-packages/yacs/config.py", line 141, in __getattr__
    raise AttributeError(name)
AttributeError: FCOS

It looks like that the the version of detectron2 provided in GSNet-release is different from that from https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.5/index.html

So could you please give some hints about the version of pytorch used for the detectron2 provided in GSNet-release?

Many thanks in advance.

Missing attributes

I am trying to run bash.run, unsuccessfully.

I get an attribute Error:
AttributeError: Cannot find field 'predict_rotation' in the given Instances!

There are other fields missing as well...
Any ideas?

Thanks!

predict_trans error

I meet this error when i use run.sh, i have successfully install the detectron2

Traceback (most recent call last):
File "demo2/demo.py", line 90, in
predictions, visualized_output = demo.run_on_image(img, save_name)
File "D:\ApolloCar3D\detectron2-master\demo2\predictor.py", line 107, in run_on_image
for index in range(instances.predict_trans.shape[0]):
File "d:\apollocar3d\detectron2-master\detectron2\structures\instances.py", line 65, in getattr
raise AttributeError("Cannot find field '{}' in the given Instances!".format(name))
AttributeError: Cannot find field 'predict_trans' in the given Instances!

Geometric consistency loss

Hi @lkeab, First of all, thank you for such a nice work! and making it public. I am curious how you are doing the geoemtric consistency loss (because it looks like there are no 3D semantic coordinates as mentioned in the paper). Could you please point me where this is implemented in your code?

Thank you,
Srikanth

About kpts_mapping.npy

Hi, I met a problem when reading the code.
As mentioned in other issues, the kpts_mapping.npy is used to get 66 3d coordinates of 3D semmantic points in the mesh model.
But I can not understand how to use this file to get the results specifily. Would you answer this problem ?
Thanks

66 3D vertices

Great research! Could you please tell me where to find the "66 3D vertices " mentioned below eq.(4) ? Are they included in this project?

About 3D semantic keypoints on the predicted mesh

Hi, I have a question about 3D semantic keypoints on the predicted mesh as you mentioned eq(2) above in your paper? How to get them? Are they generated along with model reconstruction?

CUDA11.1+Torch 1.8

I have successfully built Detectron2 under this environment. But when i build SoftRas，i meet this problem. Can you give me some advice? Thanks so much.

1 error detected in the compilation of "D:/ApolloCar3D/各种pip包/SoftRas-master/soft_renderer/cuda/soft_rasterize_cuda_kernel.cu".
soft_rasterize_cuda_kernel.cu
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\torch\utils\cpp_extension.py", line 1673, in _run_ninja_build
env=env)
File "C:\Users\One\anaconda3\envs\detectron2\lib\subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "setup.py", line 38, in
cmdclass={'build_ext': BuildExtension}
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\setuptools_init_.py", line 153, in setup
return distutils.core.setup(**attrs)
File "C:\Users\One\anaconda3\envs\detectron2\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "C:\Users\One\anaconda3\envs\detectron2\lib\distutils\dist.py", line 955, in run_commands
self.run_command(cmd)
File "C:\Users\One\anaconda3\envs\detectron2\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\setuptools\command\install.py", line 67, in run
self.do_egg_install()
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\setuptools\command\install.py", line 109, in do_egg_install
self.run_command('bdist_egg')
File "C:\Users\One\anaconda3\envs\detectron2\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "C:\Users\One\anaconda3\envs\detectron2\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\setuptools\command\bdist_egg.py", line 164, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\setuptools\command\bdist_egg.py", line 150, in call_command
self.run_command(cmdname)
File "C:\Users\One\anaconda3\envs\detectron2\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "C:\Users\One\anaconda3\envs\detectron2\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\setuptools\command\install_lib.py", line 11, in run
self.build()
File "C:\Users\One\anaconda3\envs\detectron2\lib\distutils\command\install_lib.py", line 107, in build
self.run_command('build_ext')
File "C:\Users\One\anaconda3\envs\detectron2\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "C:\Users\One\anaconda3\envs\detectron2\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\setuptools\command\build_ext.py", line 79, in run
_build_ext.run(self)
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\Cython\Distutils\old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "C:\Users\One\anaconda3\envs\detectron2\lib\distutils\command\build_ext.py", line 339, in run
self.build_extensions()
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\torch\utils\cpp_extension.py", line 708, in build_extensions
build_ext.build_extensions(self)
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\Cython\Distutils\old_build_ext.py", line 195, in build_extensions
_build_ext.build_ext.build_extensions(self)
File "C:\Users\One\anaconda3\envs\detectron2\lib\distutils\command\build_ext.py", line 448, in build_extensions
self._build_extensions_serial()
File "C:\Users\One\anaconda3\envs\detectron2\lib\distutils\command\build_ext.py", line 473, in _build_extensions_serial
self.build_extension(ext)
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\setuptools\command\build_ext.py", line 196, in build_extension
_build_ext.build_extension(self, ext)
File "C:\Users\One\anaconda3\envs\detectron2\lib\distutils\command\build_ext.py", line 533, in build_extension
depends=ext.depends)
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\torch\utils\cpp_extension.py", line 690, in win_wrap_ninja_compile
with_cuda=with_cuda)
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\torch\utils\cpp_extension.py", line 1359, in _write_ninja_file_and_compile_objects
error_prefix='Error compiling objects for extension')
File "C:\Users\One\anaconda3\envs\detectron2\lib\site-packages\torch\utils\cpp_extension.py", line 1683, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension

Using SoftRas to generate deformed car models

Hi,

Thank you for open sourcing your code. I have one question: how did you use SoftRas to generate the deformed car models that have the same topology? You only briefly mentioned it in the paper, but I have not found code in the repo to do this. Thanks!

Many warnings while loading the pretrained model, and no predictions

Hi there. When I run the demo, I get the warnings below. Like this one:

WARNING [09/23 05:51:06 d2.checkpoint.c2_model_loading]: Shape of roi_heads.box_predictor.bbox_pred.bias in checkpoint is torch.Size([4]), while shape of roi_heads.box_predictor.bbox_pred.bias in model is torch.Size([316]).

Can you tell me what is going on here? The predictions are empty also:

Predicitons:  {'instances': Instances(num_instances=0, image_height=563, image_width=750, fields=[pred_boxes: Boxes(tensor([], device='cuda:0', size=(0, 4))), scores: tensor([], device='cuda:0'), pred_classes: tensor([], device='cuda:0', dtype=torch.int64), pred_keypoints: tensor([], device='cuda:0', size=(0, 17, 3)), pred_keypoint_heatmaps: tensor([], device='cuda:0', size=(0, 17, 56, 56))])}
[09/23 05:51:08 detectron2]: /content/example_images//licenseplates-20220825-0034-768.jpg: detected 0 instances in 1.25s




/content/gsnet/reference_code/GSNet-release
[09/23 05:51:03 detectron2]: Arguments: Namespace(confidence_threshold=0.5, config_file='configs/COCO-Keypoints/keypoint_rcnn_R_101_FPN_3x_apollo.yaml', input=['/content/example_images/'], opts=['MODEL.WEIGHTS', './output/model_final.pth'], output='medium_results_val/', video_input=None, webcam=False)
WARNING [09/23 05:51:06 d2.checkpoint.c2_model_loading]: Shape of roi_heads.box_predictor.bbox_pred.bias in checkpoint is torch.Size([4]), while shape of roi_heads.box_predictor.bbox_pred.bias in model is torch.Size([316]).
WARNING [09/23 05:51:06 d2.checkpoint.c2_model_loading]: roi_heads.box_predictor.bbox_pred.bias will not be loaded. Please double check and see if this is desired.
WARNING [09/23 05:51:06 d2.checkpoint.c2_model_loading]: Shape of roi_heads.box_predictor.bbox_pred.weight in checkpoint is torch.Size([4, 1024]), while shape of roi_heads.box_predictor.bbox_pred.weight in model is torch.Size([316, 1024]).
WARNING [09/23 05:51:06 d2.checkpoint.c2_model_loading]: roi_heads.box_predictor.bbox_pred.weight will not be loaded. Please double check and see if this is desired.
WARNING [09/23 05:51:06 d2.checkpoint.c2_model_loading]: Shape of roi_heads.keypoint_head.score_lowres.bias in checkpoint is torch.Size([66]), while shape of roi_heads.keypoint_head.score_lowres.bias in model is torch.Size([17]).
WARNING [09/23 05:51:06 d2.checkpoint.c2_model_loading]: roi_heads.keypoint_head.score_lowres.bias will not be loaded. Please double check and see if this is desired.
WARNING [09/23 05:51:06 d2.checkpoint.c2_model_loading]: Shape of roi_heads.keypoint_head.score_lowres.weight in checkpoint is torch.Size([512, 66, 4, 4]), while shape of roi_heads.keypoint_head.score_lowres.weight in model is torch.Size([512, 17, 4, 4]).
WARNING [09/23 05:51:06 d2.checkpoint.c2_model_loading]: roi_heads.keypoint_head.score_lowres.weight will not be loaded. Please double check and see if this is desired.
[09/23 05:51:06 d2.checkpoint.c2_model_loading]: Following weights matched with model:
| Names in Model                                  | Names in Checkpoint                                                                                  | Shapes                                          |
|:------------------------------------------------|:-----------------------------------------------------------------------------------------------------|:------------------------------------------------|
| backbone.bottom_up.res2.0.conv1.*               | backbone.bottom_up.res2.0.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (64,) (64,) (64,) (64,) (64,64,1,1)             |
| backbone.bottom_up.res2.0.conv2.*               | backbone.bottom_up.res2.0.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (64,) (64,) (64,) (64,) (64,64,3,3)             |
| backbone.bottom_up.res2.0.conv3.*               | backbone.bottom_up.res2.0.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,64,1,1)        |
| backbone.bottom_up.res2.0.shortcut.*            | backbone.bottom_up.res2.0.shortcut.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight} | (256,) (256,) (256,) (256,) (256,64,1,1)        |
| backbone.bottom_up.res2.1.conv1.*               | backbone.bottom_up.res2.1.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (64,) (64,) (64,) (64,) (64,256,1,1)            |
| backbone.bottom_up.res2.1.conv2.*               | backbone.bottom_up.res2.1.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (64,) (64,) (64,) (64,) (64,64,3,3)             |
| backbone.bottom_up.res2.1.conv3.*               | backbone.bottom_up.res2.1.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,64,1,1)        |
| backbone.bottom_up.res2.2.conv1.*               | backbone.bottom_up.res2.2.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (64,) (64,) (64,) (64,) (64,256,1,1)            |
| backbone.bottom_up.res2.2.conv2.*               | backbone.bottom_up.res2.2.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (64,) (64,) (64,) (64,) (64,64,3,3)             |
| backbone.bottom_up.res2.2.conv3.*               | backbone.bottom_up.res2.2.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,64,1,1)        |
| backbone.bottom_up.res3.0.conv1.*               | backbone.bottom_up.res3.0.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (128,) (128,) (128,) (128,) (128,256,1,1)       |
| backbone.bottom_up.res3.0.conv2.*               | backbone.bottom_up.res3.0.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (128,) (128,) (128,) (128,) (128,128,3,3)       |
| backbone.bottom_up.res3.0.conv3.*               | backbone.bottom_up.res3.0.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (512,) (512,) (512,) (512,) (512,128,1,1)       |
| backbone.bottom_up.res3.0.shortcut.*            | backbone.bottom_up.res3.0.shortcut.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight} | (512,) (512,) (512,) (512,) (512,256,1,1)       |
| backbone.bottom_up.res3.1.conv1.*               | backbone.bottom_up.res3.1.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (128,) (128,) (128,) (128,) (128,512,1,1)       |
| backbone.bottom_up.res3.1.conv2.*               | backbone.bottom_up.res3.1.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (128,) (128,) (128,) (128,) (128,128,3,3)       |
| backbone.bottom_up.res3.1.conv3.*               | backbone.bottom_up.res3.1.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (512,) (512,) (512,) (512,) (512,128,1,1)       |
| backbone.bottom_up.res3.2.conv1.*               | backbone.bottom_up.res3.2.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (128,) (128,) (128,) (128,) (128,512,1,1)       |
| backbone.bottom_up.res3.2.conv2.*               | backbone.bottom_up.res3.2.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (128,) (128,) (128,) (128,) (128,128,3,3)       |
| backbone.bottom_up.res3.2.conv3.*               | backbone.bottom_up.res3.2.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (512,) (512,) (512,) (512,) (512,128,1,1)       |
| backbone.bottom_up.res3.3.conv1.*               | backbone.bottom_up.res3.3.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (128,) (128,) (128,) (128,) (128,512,1,1)       |
| backbone.bottom_up.res3.3.conv2.*               | backbone.bottom_up.res3.3.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (128,) (128,) (128,) (128,) (128,128,3,3)       |
| backbone.bottom_up.res3.3.conv3.*               | backbone.bottom_up.res3.3.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (512,) (512,) (512,) (512,) (512,128,1,1)       |
| backbone.bottom_up.res4.0.conv1.*               | backbone.bottom_up.res4.0.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,512,1,1)       |
| backbone.bottom_up.res4.0.conv2.*               | backbone.bottom_up.res4.0.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.0.conv3.*               | backbone.bottom_up.res4.0.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.0.shortcut.*            | backbone.bottom_up.res4.0.shortcut.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight} | (1024,) (1024,) (1024,) (1024,) (1024,512,1,1)  |
| backbone.bottom_up.res4.1.conv1.*               | backbone.bottom_up.res4.1.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.1.conv2.*               | backbone.bottom_up.res4.1.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.1.conv3.*               | backbone.bottom_up.res4.1.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.10.conv1.*              | backbone.bottom_up.res4.10.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.10.conv2.*              | backbone.bottom_up.res4.10.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.10.conv3.*              | backbone.bottom_up.res4.10.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.11.conv1.*              | backbone.bottom_up.res4.11.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.11.conv2.*              | backbone.bottom_up.res4.11.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.11.conv3.*              | backbone.bottom_up.res4.11.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.12.conv1.*              | backbone.bottom_up.res4.12.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.12.conv2.*              | backbone.bottom_up.res4.12.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.12.conv3.*              | backbone.bottom_up.res4.12.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.13.conv1.*              | backbone.bottom_up.res4.13.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.13.conv2.*              | backbone.bottom_up.res4.13.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.13.conv3.*              | backbone.bottom_up.res4.13.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.14.conv1.*              | backbone.bottom_up.res4.14.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.14.conv2.*              | backbone.bottom_up.res4.14.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.14.conv3.*              | backbone.bottom_up.res4.14.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.15.conv1.*              | backbone.bottom_up.res4.15.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.15.conv2.*              | backbone.bottom_up.res4.15.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.15.conv3.*              | backbone.bottom_up.res4.15.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.16.conv1.*              | backbone.bottom_up.res4.16.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.16.conv2.*              | backbone.bottom_up.res4.16.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.16.conv3.*              | backbone.bottom_up.res4.16.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.17.conv1.*              | backbone.bottom_up.res4.17.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.17.conv2.*              | backbone.bottom_up.res4.17.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.17.conv3.*              | backbone.bottom_up.res4.17.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.18.conv1.*              | backbone.bottom_up.res4.18.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.18.conv2.*              | backbone.bottom_up.res4.18.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.18.conv3.*              | backbone.bottom_up.res4.18.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.19.conv1.*              | backbone.bottom_up.res4.19.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.19.conv2.*              | backbone.bottom_up.res4.19.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.19.conv3.*              | backbone.bottom_up.res4.19.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.2.conv1.*               | backbone.bottom_up.res4.2.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.2.conv2.*               | backbone.bottom_up.res4.2.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.2.conv3.*               | backbone.bottom_up.res4.2.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.20.conv1.*              | backbone.bottom_up.res4.20.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.20.conv2.*              | backbone.bottom_up.res4.20.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.20.conv3.*              | backbone.bottom_up.res4.20.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.21.conv1.*              | backbone.bottom_up.res4.21.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.21.conv2.*              | backbone.bottom_up.res4.21.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.21.conv3.*              | backbone.bottom_up.res4.21.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.22.conv1.*              | backbone.bottom_up.res4.22.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.22.conv2.*              | backbone.bottom_up.res4.22.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.22.conv3.*              | backbone.bottom_up.res4.22.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}   | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.3.conv1.*               | backbone.bottom_up.res4.3.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.3.conv2.*               | backbone.bottom_up.res4.3.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.3.conv3.*               | backbone.bottom_up.res4.3.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.4.conv1.*               | backbone.bottom_up.res4.4.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.4.conv2.*               | backbone.bottom_up.res4.4.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.4.conv3.*               | backbone.bottom_up.res4.4.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.5.conv1.*               | backbone.bottom_up.res4.5.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.5.conv2.*               | backbone.bottom_up.res4.5.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.5.conv3.*               | backbone.bottom_up.res4.5.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.6.conv1.*               | backbone.bottom_up.res4.6.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.6.conv2.*               | backbone.bottom_up.res4.6.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.6.conv3.*               | backbone.bottom_up.res4.6.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.7.conv1.*               | backbone.bottom_up.res4.7.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.7.conv2.*               | backbone.bottom_up.res4.7.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.7.conv3.*               | backbone.bottom_up.res4.7.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.8.conv1.*               | backbone.bottom_up.res4.8.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.8.conv2.*               | backbone.bottom_up.res4.8.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.8.conv3.*               | backbone.bottom_up.res4.8.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res4.9.conv1.*               | backbone.bottom_up.res4.9.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| backbone.bottom_up.res4.9.conv2.*               | backbone.bottom_up.res4.9.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| backbone.bottom_up.res4.9.conv3.*               | backbone.bottom_up.res4.9.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| backbone.bottom_up.res5.0.conv1.*               | backbone.bottom_up.res5.0.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (512,) (512,) (512,) (512,) (512,1024,1,1)      |
| backbone.bottom_up.res5.0.conv2.*               | backbone.bottom_up.res5.0.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (512,) (512,) (512,) (512,) (512,512,3,3)       |
| backbone.bottom_up.res5.0.conv3.*               | backbone.bottom_up.res5.0.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (2048,) (2048,) (2048,) (2048,) (2048,512,1,1)  |
| backbone.bottom_up.res5.0.shortcut.*            | backbone.bottom_up.res5.0.shortcut.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight} | (2048,) (2048,) (2048,) (2048,) (2048,1024,1,1) |
| backbone.bottom_up.res5.1.conv1.*               | backbone.bottom_up.res5.1.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (512,) (512,) (512,) (512,) (512,2048,1,1)      |
| backbone.bottom_up.res5.1.conv2.*               | backbone.bottom_up.res5.1.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (512,) (512,) (512,) (512,) (512,512,3,3)       |
| backbone.bottom_up.res5.1.conv3.*               | backbone.bottom_up.res5.1.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (2048,) (2048,) (2048,) (2048,) (2048,512,1,1)  |
| backbone.bottom_up.res5.2.conv1.*               | backbone.bottom_up.res5.2.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (512,) (512,) (512,) (512,) (512,2048,1,1)      |
| backbone.bottom_up.res5.2.conv2.*               | backbone.bottom_up.res5.2.conv2.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (512,) (512,) (512,) (512,) (512,512,3,3)       |
| backbone.bottom_up.res5.2.conv3.*               | backbone.bottom_up.res5.2.conv3.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}    | (2048,) (2048,) (2048,) (2048,) (2048,512,1,1)  |
| backbone.bottom_up.stem.conv1.*                 | backbone.bottom_up.stem.conv1.{norm.bias,norm.running_mean,norm.running_var,norm.weight,weight}      | (64,) (64,) (64,) (64,) (64,3,7,7)              |
| backbone.fpn_lateral2.*                         | backbone.fpn_lateral2.{bias,weight}                                                                  | (256,) (256,256,1,1)                            |
| backbone.fpn_lateral3.*                         | backbone.fpn_lateral3.{bias,weight}                                                                  | (256,) (256,512,1,1)                            |
| backbone.fpn_lateral4.*                         | backbone.fpn_lateral4.{bias,weight}                                                                  | (256,) (256,1024,1,1)                           |
| backbone.fpn_lateral5.*                         | backbone.fpn_lateral5.{bias,weight}                                                                  | (256,) (256,2048,1,1)                           |
| backbone.fpn_output2.*                          | backbone.fpn_output2.{bias,weight}                                                                   | (256,) (256,256,3,3)                            |
| backbone.fpn_output3.*                          | backbone.fpn_output3.{bias,weight}                                                                   | (256,) (256,256,3,3)                            |
| backbone.fpn_output4.*                          | backbone.fpn_output4.{bias,weight}                                                                   | (256,) (256,256,3,3)                            |
| backbone.fpn_output5.*                          | backbone.fpn_output5.{bias,weight}                                                                   | (256,) (256,256,3,3)                            |
| proposal_generator.rpn_head.anchor_deltas.*     | proposal_generator.rpn_head.anchor_deltas.{bias,weight}                                              | (12,) (12,256,1,1)                              |
| proposal_generator.rpn_head.conv.*              | proposal_generator.rpn_head.conv.{bias,weight}                                                       | (256,) (256,256,3,3)                            |
| proposal_generator.rpn_head.objectness_logits.* | proposal_generator.rpn_head.objectness_logits.{bias,weight}                                          | (3,) (3,256,1,1)                                |
| roi_heads.box_head.fc1.*                        | roi_heads.box_head.fc1.{bias,weight}                                                                 | (1024,) (1024,12544)                            |
| roi_heads.box_head.fc2.*                        | roi_heads.box_head.fc2.{bias,weight}                                                                 | (1024,) (1024,1024)                             |
| roi_heads.box_predictor.cls_score.*             | roi_heads.box_predictor.cls_score.{bias,weight}                                                      | (80,) (80,1024)                                 |
| roi_heads.keypoint_head.conv_fcn1.*             | roi_heads.keypoint_head.conv_fcn1.{bias,weight}                                                      | (512,) (512,256,3,3)                            |
| roi_heads.keypoint_head.conv_fcn2.*             | roi_heads.keypoint_head.conv_fcn2.{bias,weight}                                                      | (512,) (512,512,3,3)                            |
| roi_heads.keypoint_head.conv_fcn3.*             | roi_heads.keypoint_head.conv_fcn3.{bias,weight}                                                      | (512,) (512,512,3,3)                            |
| roi_heads.keypoint_head.conv_fcn4.*             | roi_heads.keypoint_head.conv_fcn4.{bias,weight}                                                      | (512,) (512,512,3,3)                            |
| roi_heads.keypoint_head.conv_fcn5.*             | roi_heads.keypoint_head.conv_fcn5.{bias,weight}                                                      | (512,) (512,512,3,3)                            |
| roi_heads.keypoint_head.conv_fcn6.*             | roi_heads.keypoint_head.conv_fcn6.{bias,weight}                                                      | (512,) (512,512,3,3)                            |
| roi_heads.keypoint_head.conv_fcn7.*             | roi_heads.keypoint_head.conv_fcn7.{bias,weight}                                                      | (512,) (512,512,3,3)                            |
| roi_heads.keypoint_head.conv_fcn8.*             | roi_heads.keypoint_head.conv_fcn8.{bias,weight}                                                      | (512,) (512,512,3,3)                            |
Skip loading parameter 'roi_heads.box_predictor.bbox_pred.bias' to the model due to incompatible shapes: (4,) in the checkpoint but (316,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.bbox_pred.weight' to the model due to incompatible shapes: (4, 1024) in the checkpoint but (316, 1024) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.keypoint_head.score_lowres.bias' to the model due to incompatible shapes: (66,) in the checkpoint but (17,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.keypoint_head.score_lowres.weight' to the model due to incompatible shapes: (512, 66, 4, 4) in the checkpoint but (512, 17, 4, 4) in the model! You might want to double check if this is expected.
Some model parameters or buffers are not found in the checkpoint:
roi_heads.box_predictor.bbox_pred.{bias, weight}
roi_heads.keypoint_head.score_lowres.{bias, weight}
The checkpoint state_dict contains keys that are not used by the model:
  roi_heads.fuse_box_pos.{bias, weight}
  roi_heads.fuse_box_pos_1.{bias, weight}
  roi_heads.fuse_heatmap_conv.{bias, weight}
  roi_heads.fuse_heatmap_conv_1.{bias, weight}
  roi_heads.fuse_heatmap_conv_2.{bias, weight}
  roi_heads.fuse_heatmap_conv_3.{bias, weight}
  roi_heads.fuse_kpts_pos.{bias, weight}
  roi_heads.fuse_kpts_pos_1.{bias, weight}
  roi_heads.fuse_roi_feature_conv.{bias, weight}
  roi_heads.fuse_roi_feature_conv_1.{bias, weight}
  roi_heads.fuse_roi_feature_conv_2.{bias, weight}
  roi_heads.regress_car_cls.{bias, weight}
  roi_heads.regress_car_cluster_type.{bias, weight}
  roi_heads.regress_car_params_0.{bias, weight}
  roi_heads.regress_car_params_1.{bias, weight}
  roi_heads.regress_car_params_2.{bias, weight}
  roi_heads.regress_car_params_3.{bias, weight}
  roi_heads.regress_rotation.{bias, weight}
  roi_heads.regress_translation.{bias, weight}

the pretrained model has been removed

Could you please give us a new link to the pretrained model. The old link doesn't contain anything anymore.

how to use this project only as keypoint detector

Hello, i just need a strong key point detector for traffic task, i don't need to create a 3d scene.
What part of this project runs the inference?

Demo failed because of FCOS

First of all, I'd like to thank you for your code.

I followed instructions in README, but I have a problem with the demo caused by an error below:

[10/29 03:45:35 detectron2]: Arguments: Namespace(confidence_threshold=0.5, config_file='configs/COCO-Keypoints/keypoint_rcnn_R_101_FPN_3x_apollo.yaml', input=['datasets/coco/val_apollo/'], opts=['MODEL.WEIGHTS', './output/model_final.pth'], output='medium_result_val/', video_input=None, webcam=False)
Traceback (most recent call last):
  File "demo/demo.py", line 73, in <module>
    cfg = setup_cfg(args)
  File "demo/demo.py", line 28, in setup_cfg
    cfg.MODEL.FCOS.INFERENCE_TH = args.confidence_threshold
  File "/home/azureuser/.virtualenvs/py36/lib/python3.6/site-packages/yacs/config.py", line 141, in __getattr__
    raise AttributeError(name)
AttributeError: FCOS

Could you help point out what I should do to make it run successfully? Thanks in advance.

No code about loss function

Hello,
I want to retrain the model using my own datasets, but I find there is no code about training, such as loss function. Could you please add this part?

AttributeError: Cannot find field 'predict_trans' in the given Instances!

I'm trying to run the run.sh script in order to give a try to the provided demo, but I got the following error

File "/usr/local/lib/python3.7/dist-packages/detectron2/structures/instances.py", line 65, in getattr
raise AttributeError("Cannot find field '{}' in the given Instances!".format(name))
AttributeError: Cannot find field 'predict_trans' in the given Instances!

My environment :
torchvision=0.6.0
CUDA = cu10.1
torch=1.5.0

kpts_mapping

Hi, firstly thanks for your work.
I have a question about the relationship between 1352 vertices and 66 keypoints. There is a file, named kpts_mapping.npy. It's not really used in your code as I understand. What means the content of this file? And how could I understand the relationship between mesh vertices and detected keypoints?
Thanks a lot.

Abrut kill happening when trying to run the demo.py

Hello,

Firstly, thanks for the codes of your impressive research.

when i tired to the below command to to execute the demo.py ( i don't see the datasets folder )
CUDA_VISIBLE_DEVICES=0 python3 demo/demo.py --config-file configs/COCO-Keypoints/keypoint_rcnn_R_101_FPN_3x_apollo.yaml
--input 'datasets/coco/val_apollo/'
--output 'medium_result_val/'
--opts MODEL.WEIGHTS ./output/model_final.pth

by reading the parser arguments i learnt --input should be list of input images with this i changed the command as follows

command : CUDA_VISIBLE_DEVICES=0 python3 demo/demo.py --config-file configs/COCO-Keypoints/keypoint_rcnn_R_101_FPN_3x_apollo.yaml --input 'datasets/apollo/annotations/test/images/171206_034559609_Camera_5.jpg' --output 'medium_result_val/' --opts MODEL.WEIGHTS ./output/model_final.pth

Output
name: coco_2014_train
name: coco_2014_val
name: coco_2014_minival
name: coco_2014_minival_100
name: coco_2014_valminusminival
name: coco_2017_train
name: coco_2017_val
name: coco_2017_occ
name: coco_2017_test
name: coco_2017_test-dev
name: coco_2017_val_100
name: apollo_train
name: apollo_val
name: keypoints_coco_2014_train
name: keypoints_coco_2014_val
name: keypoints_coco_2014_minival
name: keypoints_coco_2014_valminusminival
name: keypoints_coco_2014_minival_100
name: keypoints_coco_2017_train
name: keypoints_coco_2017_val
name: keypoints_coco_2017_val_100
[10/13 00:22:46 detectron2]: Arguments: Namespace(config_file='configs/COCO-Keypoints/keypoint_rcnn_R_101_FPN_3x_apollo.yaml', webcam=False, video_input=None, input=['datasets/apollo/annotations/test/images/171206_034559609_Camera_5.jpg'], output='medium_result_val/', confidence_threshold=0.5, opts=['MODEL.WEIGHTS', './output/model_final.pth'])

Killed

can you please help me where I'm going wrong

Is there any trained model for evaluation or any training script available?

Hi,

I wanted to try your method and evaluate the results for a better knowledge, can you provide the scripts and files regarding the same?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.