GithubHelp home page GithubHelp logo

s95huang / monoflex Goto Github PK

View Code? Open in Web Editor NEW

This project forked from neolixcn/monoflex

0.0 0.0 0.0 908 KB

Released code for Objects are Different: Flexible Monocular 3D Object Detection, CVPR21

License: MIT License

Shell 0.01% C++ 11.92% Python 72.95% C 1.30% Cuda 13.83%

monoflex's Introduction

MonoFlex

Released code for Objects are Different: Flexible Monocular 3D Object Detection, CVPR21.

Work in progress.

Installation

This repo is tested with Ubuntu 20.04, python==3.7, pytorch==1.4.0 and cuda==10.1

conda create -n monoflex python=3.7

conda activate monoflex

Install PyTorch and other dependencies:

conda install pytorch==1.4.0 torchvision==0.5.0 cudatoolkit=10.1 -c pytorch

pip install -r requirements.txt

Build DCNv2 and the project

cd models/backbone/DCNv2

. make.sh

cd ../../..

python setup develop

Data Preparation

Please download KITTI dataset and organize the data as follows:

#ROOT		
  |training/
    |calib/
    |image_2/
    |label/
    |ImageSets/
  |testing/
    |calib/
    |image_2/
    |ImageSets/

Then modify the paths in config/paths_catalog.py according to your data path.

Training & Evaluation

Training with one GPU. (TODO: The multi-GPU training will be further tested.)

CUDA_VISIBLE_DEVICES=0 python tools/plain_train_net.py --batch_size 8 --config runs/monoflex.yaml --output output/exp

The model will be evaluated periodically (can be adjusted in the CONFIG) during training and you can also evaluate a checkpoint with

CUDA_VISIBLE_DEVICES=0 python tools/plain_train_net.py --config runs/monoflex.yaml --ckpt YOUR_CKPT  --eval

You can also specify --vis when evaluation to visualize the predicted heatmap and 3D bounding boxes. The pretrained model for train/val split and logs are here.

Note: we observe an obvious variation of the performance for different runs and we are still investigating possible solutions to stablize the results, though it may inevitably due to the utilized uncertainties.

inference

CUDA_VISIBLE_DEVICES=0 python tools/plain_train_net.py --config runs/monoflex.yaml --ckpt ./model_moderate_best_soft.pth --eval --vis

ConvertOnnx

  1. replace some folders and files with those named with suffix of "_onnx":
  • model/backbone/DCNv2_onnx -> model/backbone/DCNv2
  • model/head/detector_head_onnx.py -> model/head/detector_head.py
  • model/head/detector_predictor_onnx.py -> model/head/detector_predictor.py
  • model/detector_onnx.py -> model/detector.py
  1. change some files:
  • change import way of DCNv2 in model/backbone/dla_dcn.py:
# from model.backbone.DCNv2.dcn_v2 import DCN
from model.backbone.DCNv2.modules.deform_conv import ModulatedDeformConvPack as DCN
  • change dcn extension name in setup.py
 ext_modules = [
       extension(
           "deform_conv_cuda",
           sources,
           include_dirs=include_dirs,
           define_macros=define_macros
  1. rebuild dcn module and whole project, then run the following command to convert onnx.
CUDA_VISIBLE_DEVICES=2 python tools/convert_onnx.py --config runs/monoflex.yaml --ckpt ./model_moderate_best_soft_noInabn.pth --export-name "monoflex_new.onnx"

Note that if "new-ckpt" parameter and "ckpt" parameter are both given, the ckpt will be converted to new state dict without InplaceABN opterator and save to "new-ckpt". And if you want to compare the results of model with "new-ckpt" and "ckpt", try change the code around line 109.

Citation

If you find our work useful in your research, please consider citing:

@InProceedings{MonoFlex,
    author    = {Zhang, Yunpeng and Lu, Jiwen and Zhou, Jie},
    title     = {Objects Are Different: Flexible Monocular 3D Object Detection},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {3289-3298}
}

Acknowlegment

The code is heavily borrowed from SMOKE and thanks for their contribution.

monoflex's People

Contributors

zhangyp15 avatar pyten avatar penpenpen-thu avatar utc1205 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.