qiqihaer / 3dssd-pytorch-openpcdet

The successful implementation of 3DSSD in Pytorch

3DSSD-pytorch-openPCDet

The successful implementation of 3DSSD in Pytorch

Thanks for the [OpenPCDet]!!! This implementation of the 3DSSD is mainly based on the pcdet v0.3.

Preparation

1. Clone this repository

2. Install the Python dependencies.

pip install -r requirements.txt

3. Option: Install the spconv. Please follow the instructions in the OpenPCDet. This 3DSSD does not use spconv. However, for better use the OpenPCDet, I recommend to install it.

4. Install the pcdet library.

python setup.py develop

5. Install the pointnet2_3DSSD libarary.

cd pcdet/ops/pointnet2/pointnet2_3DSSD/
python setup.py develop

Train a Model

I have set the default config file to the 3DSSD model. So, you can just run:

cd tools
python train.py

The trainning log and tensorboard log are saved into output dir

Eval the model

You can follow the instructions in the openPCDet. Please set the --cfg_file to 'cfgs/kitti_models/3DSSD_openPCDet.yaml'

Pretrained Weights

I have tested this code on ubuntu16.04+cuda9.0+pytorch1.1+python3.6+spconv1.0

The pretrained weights are in the output/kitti_models/3DSSD_openPCDet/3DSSD/ckpt/. The eval performance on the Car class is as follows:

Car [email protected], 0.70, 0.70:
bbox AP:96.5468, 90.0235, 89.4066
bev  AP:90.3444, 88.0784, 86.0698
3d   AP:89.2219, 78.8593, 77.5890
aos  AP:96.52, 89.95, 89.25
Car [email protected], 0.70, 0.70:
bbox AP:98.2011, 95.0305, 92.6650
bev  AP:93.2919, 89.1952, 88.1910
3d   AP:91.4331, 82.2283, 77.8059
aos  AP:98.18, 94.93, 92.49
Car [email protected], 0.50, 0.50:
bbox AP:96.5468, 90.0235, 89.4066
bev  AP:96.6237, 90.1257, 89.6772
3d   AP:96.5594, 90.0998, 89.6259
aos  AP:96.52, 89.95, 89.25
Car [email protected], 0.50, 0.50:
bbox AP:98.2011, 95.0305, 92.6650
bev  AP:98.3041, 95.4983, 95.0182
3d   AP:98.2703, 95.3970, 94.8667
aos  AP:98.18, 94.93, 92.49

3dssd-pytorch-openpcdet's Issues

No module named 'viz_tools'

Your job hels me a lot. but when I run
python train.py
it reminds me that
Traceback (most recent call last): File "train.py", line 6, in <module> from test import repeat_eval_ckpt File "/home/hby/hdd/chenyanbin/3DSSD-pytorch-openPCDet/tools/test.py", line 13, in <module> from eval_utils import eval_utils File "/home/hby/hdd/chenyanbin/3DSSD-pytorch-openPCDet/tools/eval_utils/eval_utils.py", line 8, in <module> from pcdet.models import load_data_to_gpu File "/home/hby/hdd/chenyanbin/3DSSD-pytorch-openPCDet/pcdet/models/__init__.py", line 6, in <module> from .detectors import build_detector File "/home/hby/hdd/chenyanbin/3DSSD-pytorch-openPCDet/pcdet/models/detectors/__init__.py", line 1, in <module> from .detector3d_template import Detector3DTemplate File "/home/hby/hdd/chenyanbin/3DSSD-pytorch-openPCDet/pcdet/models/detectors/detector3d_template.py", line 7, in <module> from .. import backbones_2d, backbones_3d, dense_heads, roi_heads File "/home/hby/hdd/chenyanbin/3DSSD-pytorch-openPCDet/pcdet/models/backbones_3d/__init__.py", line 1, in <module> from .pointnet2_backbone import PointNet2Backbone, PointNet2MSG File "/home/hby/hdd/chenyanbin/3DSSD-pytorch-openPCDet/pcdet/models/backbones_3d/pointnet2_backbone.py", line 8, in <module> from viz_tools.viz_utils import point_viz ModuleNotFoundError: No module named 'viz_tools'
and the github don't have the folder viz_tools. Could you help me ?
Thank you very much.

The error when I run train.py

when I run train.py under tools, I encounter this error. I wonder how to solve this.

Traceback (most recent call last):
File "train.py", line 198, in
main()
File "train.py", line 112, in main
total_epochs=args.epochs
File "/home/yifan/Code/OpenPCDet/pcdet/datasets/init.py", line 51, in build_dataloader
logger=logger,
File "/home/yifan/Code/OpenPCDet/pcdet/datasets/kitti/kitti_dataset.py", line 23, in init
dataset_cfg=dataset_cfg, class_names=class_names, training=training, root_path=root_path, logger=logger
File "/home/yifan/Code/OpenPCDet/pcdet/datasets/dataset.py", line 32, in init
) if self.training else None
File "/home/yifan/Code/OpenPCDet/pcdet/datasets/augmentor/data_augmentor.py", line 23, in init
cur_augmentor = getattr(self, cur_cfg.NAME)(config=cur_cfg)
File "/home/yifan/Code/OpenPCDet/pcdet/datasets/augmentor/data_augmentor.py", line 31, in gt_sampling
logger=self.logger
File "/home/yifan/Code/OpenPCDet/pcdet/datasets/augmentor/database_sampler.py", line 21, in init
with open(str(db_info_path), 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/yifan/Code/SESS/3DSSD-pytorch-openPCDet-main/data/kitti/kitti_dbinfos_train.pkl'

Does anyone have any idea?

我在训练的时候遇到如下问题，请问怎么解决呢？

File "train.py", line 198, in
main()
File "train.py", line 170, in main
merge_all_iters_to_one_epoch=args.merge_all_iters_to_one_epoch
File "/data-input/3DSSD-pytorch-openPCDet/tools/train_utils/train_utils.py", line 93, in train_model
dataloader_iter=dataloader_iter
File "/data-input/3DSSD-pytorch-openPCDet/tools/train_utils/train_utils.py", line 40, in train_one_epoch
loss.backward()
File "/opt/conda/lib/python3.7/site-packages/torch/tensor.py", line 166, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/opt/conda/lib/python3.7/site-packages/torch/autograd/init.py", line 99, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.)

python = 3.7
pytorch = 1.3.1
cuda = 10.0.130
cudnn = 7.6.3
spconv = 1.2.1

error when I train the model

2021-02-23 22:06:27,598 INFO Start training
kitti_models/3DSSD_openPCDet(3DSSD)
epochs: 0it [00:00, ?it/s]
2021-02-23 22:06:28,232 INFO End training kitti_models/3DSSD_openPCDet(3DSSD)
When I run the command python train.py ,it doesnot work ,the problem is in line 3.

Getting error when i run python setup.py develop

Getting following error when i run python setup.py develop in pcdet/ops/pointnet2/pointnet2_3DSSD folder

running develop
/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running egg_info
writing pointnet2_3DSSD.egg-info/PKG-INFO
writing dependency_links to pointnet2_3DSSD.egg-info/dependency_links.txt
writing top-level names to pointnet2_3DSSD.egg-info/top_level.txt
Traceback (most recent call last):
  File "setup.py", line 4, in <module>
    setup(
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command
    super().run_command(command)
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/command/develop.py", line 34, in run
    self.install_for_development()
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/command/develop.py", line 110, in install_for_development
    self.run_command('egg_info')
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command
    super().run_command(command)
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 308, in run
    self.find_sources()
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 316, in find_sources
    mm.run()
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 560, in run
    self.add_defaults()
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 597, in add_defaults
    sdist.add_defaults(self)
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/command/sdist.py", line 106, in add_defaults
    super().add_defaults()
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/_distutils/command/sdist.py", line 252, in add_defaults
    self._add_defaults_ext()
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
    build_ext = self.get_finalized_command('build_ext')
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 304, in get_finalized_command
    cmd_obj = self.distribution.get_command_obj(command, create)
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 861, in get_command_obj
    cmd_obj = self.command_obj[command] = klass(self)
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 351, in __init__
    if not is_ninja_available():
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1310, in is_ninja_available
    subprocess.check_call('ninja --version'.split(), stdout=devnull)
  File "/home/mcw/anaconda3/envs/3dssd/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ninja', '--version']' returned non-zero exit status 245.

About nuscenes dataset

Dear author: have you trained the 3DSSD on nuscenes dataset? The 3DSSD's author have not publish the detail training parameter, and i tried to train it but got very low mAP, only around 20%

How to export model to onnx?

Hi, thanks for your work on 3DSSD in pytorch!
Have you ever exported trained model to onnx?

ninja: build stopped: subcommand failed.

I compiled 3dssd on the installed pcdet 0.5.2
An error occurred after run setup.py:

ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1673, in _run_ninja_build
env=env)
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "setup.py", line 22, in
cmdclass={'build_ext': BuildExtension}
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/site-packages/setuptools/command/develop.py", line 34, in run
self.install_for_development()
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/site-packages/setuptools/command/develop.py", line 114, in install_for_development
self.run_command('build_ext')
File "/home/xxx/anaconda3/envs/xxj/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 708, in build_extensions
build_ext.build_extensions(self)
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
_build_ext.build_extension(self, ext)
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
depends=ext.depends)
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 538, in unix_wrap_ninja_compile
with_cuda=with_cuda)
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1359, in _write_ninja_file_and_compile_objects
error_prefix='Error compiling objects for extension')
File "/home/xxx/anaconda3/envs/xx/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1683, in _run_ninja_build
raise RuntimeError(message) from e

appreciate it if you could give me some advice

Question about vote_loss

In tensorflow implementation and your 3DSSD-pytorch implementation, I can see there is a vote_loss.
https://github.com/qiqihaer/3DSSD-pytorch/blob/d0c856e4c1b6cb69810fc0b0562ebc9a42e5bace/lib/builder/loss_builder.py#L190

Is it still available in this repo? If not, should I add vote_loss to improve the performance?

And I am not sure how to get end_points[maps_dict.PL_LABEL_BOXES_3D] from forward_ret_dict. Could you give me some hints please?

vote_offset = self.forward_ret_dict['ctr_offsets']
vote_base = self.forward_ret_dict['centers_origin']
bs, pts_num, _ = vote_offset.shape
gt_boxes_3d = end_points[maps_dict.PL_LABEL_BOXES_3D] 

Appreciate your efforts and thank you for your time

Why is the accuracy so low on pedestrian and cyclist?

Thank you very much for providing this version of the code !

But I don't understand why the accuracy of pedestrian and cyclist is so low !
Is the parameter setting of a part of the code wrong?

What should I modify to achieve satisfactory results on these two categories?

If you have any helpful ideas, please let me know! Thank you very much！！！

I can't understand some of your code

Hi! I'm learning 3DSSD recently, your job is great and it is very helpful to me.
In the pointnet2_modules.py
class PointnetSAModuleMSG_SSD(_PointnetSAModuleBase):
def init(self, *, npoint: int,

:param npoint: int
But in the 3DSSD_openPCDet.yaml,
NPOINTS: [[4096], [512], [256, 256], [256, 0], [-1], [256]]
[256, 256] is a list, why it works?