song-jingyu / pointpainting Goto Github PK

when i run python train.py --cfg_file cfgs/kitti_models/pointpillar_painted.yaml

it says that :

Traceback (most recent call last):
File "train.py", line 198, in
main()
File "train.py", line 112, in main
total_epochs=args.epochs
File "/media/user/date/ZXH/PointPainting-main/detector/tools/pcdet/datasets/init.py", line 46, in build_dataloader
dataset = all[dataset_cfg.DATASET](
KeyError: 'PaintedKittiDataset'

How can i fix it?? @Song-Jingyu 谢谢！！！

RuntimeError: Error compiling objects for extension

Thank you for providing the code and your efforts. I have tried to replicate the results but when I run painting.py, I get a point cloud that does not seem correct. As you can see in the image below, the whole left part should not be there. Do you have any idea about it?
Thanks in advance :

)

I try to run the whole framework in ROS. The FPS of my node is about 1.6 and the most time consuming is PointPainting. The part of pointpainting cost about 0.5 seconds(deeplabv3plus).
But I'm not sure the performance is the normal result or just something wrong with my code.
Could you tell me what's the FPS of the whole framework？

Hi,
Thank you for your excellent work! I tried to reproduce the results in the source paper with your code, but failed（with default configture）. Have you reproduce the similarity performance on kitti val dataset with the paper's, please?

Thank you for your work, but I want to know that how can I get the last two pictures in "README"?
I mean "Accuracy for each object category and task difficulty/Average accuracy across different object category"
thanks

请问在运行demo的命令中，--ext .npy是什么？可以提供一下你成功的命令的例子吗？
I want to know what is "--ext .npy" when I run the demo.py ?Can you offer a successfull command example?
并且，你的readme里只写了如何开始训练，没有写如何测试。测试的命令是和openpcdet相同的吗？
Moreover, eadme only writes how to start training, not how to test. Is the test command the same as openpcdet?

Hello, thanks for your excellent work.I recently tried to run your work in the environment of ubuntu18.04 and python3.7.The previous work has been completed, but I encountered difficulties in the final visualization step.
After I installed 'mayavi' according to your prompt, I executed the visualization command 'python demo.py --cfg_file cfgs/kitti_models/pointpillar_painted.yaml --ckpt /home/ypx/ws/PointPainting_ws/detector/output/kitti_models/pointpillar_painted After /default/ckpt/checkpoint_epoch_80.pth --data_path /home/ypx/ws/PointPainting_ws/detector/data/kitti/training/painted_lidar/000000.npy --ext .npy', an error 'ImportError: Could not import appeared backend for traitsui. Make sure you have a suitable UI toolkit like PyQt/PySide or wxPython installed.’.
I tried to solve it, replaced various versions of 'mayavi, pyqt5, vtk, PySide', but still had the same error.So I would like to know what version of the visualization software you use? And what are its dependencies?Can you give me some guidance?Looking forward to your reply.

Thanks to share this repository. Inspried by this project, I'm devising a new 3D detection model.

train, val, and test.txt provided by OpenPCDet, kiti_infos_train.pkl and kiti_infos_val.pkl each consist of about 3800 datasets, but did the PointPainting paper use them as they are?

I know that what is actually applied to train and evaluation is kiti_infos_train.pkl and kiti_infos_val.pkl, and there are no overlapping samples. So I wonder if it is okay to use it as it is given and settle for a good result.

many many problems about version.
Im serious.

Hi,
I wonder to know that how to calculate the mAP which describes in the pointpainting paper. It seems that didn't explain what it is in the paper. Can u provide the mAP code? or tell us how to calculate it.

Hello,
while running this part of the code

$ cd detector
$ python -m pcdet.datasets.kitti.painted_kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/painted_kitti_dataset.yaml
$ cd tools
$ python train.py --cfg_file cfgs/kitti_models/pointpillar_painted.yaml

I ran to this error

/content/PointPainting/detector
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 183, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/usr/lib/python3.7/runpy.py", line 109, in _get_module_details
import(pkg_name)
File "/content/PointPainting/detector/pcdet/init.py", line 4, in
from .version import version
ModuleNotFoundError: No module named 'pcdet.version'
/content/PointPainting/detector/tools
2022-08-11 10:36:39,752 INFO Start logging
2022-08-11 10:36:39,753 INFO CUDA_VISIBLE_DEVICES=ALL
2022-08-11 10:36:39,753 INFO cfg_file cfgs/kitti_models/pointpillar_painted.yaml
2022-08-11 10:36:39,753 INFO batch_size 4
2022-08-11 10:36:39,753 INFO epochs 80
2022-08-11 10:36:39,753 INFO workers 8
2022-08-11 10:36:39,753 INFO extra_tag default
2022-08-11 10:36:39,753 INFO ckpt None
2022-08-11 10:36:39,753 INFO pretrained_model None
2022-08-11 10:36:39,753 INFO launcher none
2022-08-11 10:36:39,753 INFO tcp_port 18888
2022-08-11 10:36:39,753 INFO sync_bn False
2022-08-11 10:36:39,753 INFO fix_random_seed False
2022-08-11 10:36:39,753 INFO ckpt_save_interval 1
2022-08-11 10:36:39,753 INFO local_rank 0
2022-08-11 10:36:39,753 INFO max_ckpt_save_num 30
2022-08-11 10:36:39,754 INFO merge_all_iters_to_one_epoch False
2022-08-11 10:36:39,754 INFO set_cfgs None
2022-08-11 10:36:39,754 INFO max_waiting_mins 0
2022-08-11 10:36:39,754 INFO start_epoch 0
2022-08-11 10:36:39,754 INFO save_to_file False
2022-08-11 10:36:39,754 INFO cfg.ROOT_DIR: /content/OpenPCDet
2022-08-11 10:36:39,754 INFO cfg.LOCAL_RANK: 0
2022-08-11 10:36:39,754 INFO cfg.CLASS_NAMES: ['Car', 'Pedestrian', 'Cyclist']
2022-08-11 10:36:39,754 INFO
cfg.DATA_CONFIG = edict()
2022-08-11 10:36:39,754 INFO cfg.DATA_CONFIG.DATASET: PaintedKittiDataset
2022-08-11 10:36:39,754 INFO cfg.DATA_CONFIG.DATA_PATH: ../data/kitti
2022-08-11 10:36:39,754 INFO cfg.DATA_CONFIG.POINT_CLOUD_RANGE: [0, -39.68, -3, 69.12, 39.68, 1]
2022-08-11 10:36:39,754 INFO
cfg.DATA_CONFIG.DATA_SPLIT = edict()
2022-08-11 10:36:39,754 INFO cfg.DATA_CONFIG.DATA_SPLIT.train: train
2022-08-11 10:36:39,754 INFO cfg.DATA_CONFIG.DATA_SPLIT.test: val
2022-08-11 10:36:39,754 INFO
cfg.DATA_CONFIG.INFO_PATH = edict()
2022-08-11 10:36:39,755 INFO cfg.DATA_CONFIG.INFO_PATH.train: ['kitti_infos_train.pkl']
2022-08-11 10:36:39,755 INFO cfg.DATA_CONFIG.INFO_PATH.test: ['kitti_infos_val.pkl']
2022-08-11 10:36:39,755 INFO cfg.DATA_CONFIG.FOV_POINTS_ONLY: True
2022-08-11 10:36:39,755 INFO
cfg.DATA_CONFIG.DATA_AUGMENTOR = edict()
2022-08-11 10:36:39,755 INFO cfg.DATA_CONFIG.DATA_AUGMENTOR.DISABLE_AUG_LIST: ['placeholder']
2022-08-11 10:36:39,755 INFO cfg.DATA_CONFIG.DATA_AUGMENTOR.AUG_CONFIG_LIST: [{'NAME': 'gt_sampling', 'USE_ROAD_PLANE': True, 'DB_INFO_PATH': ['kitti_dbinfos_train.pkl'], 'PREPARE': {'filter_by_min_points': ['Car:5', 'Pedestrian:5', 'Cyclist:5'], 'filter_by_difficulty': [-1]}, 'SAMPLE_GROUPS': ['Car:15', 'Pedestrian:15', 'Cyclist:15'], 'NUM_POINT_FEATURES': 8, 'DATABASE_WITH_FAKELIDAR': False, 'REMOVE_EXTRA_WIDTH': [0.0, 0.0, 0.0], 'LIMIT_WHOLE_SCENE': False}, {'NAME': 'random_world_flip', 'ALONG_AXIS_LIST': ['x']}, {'NAME': 'random_world_rotation', 'WORLD_ROT_ANGLE': [-0.78539816, 0.78539816]}, {'NAME': 'random_world_scaling', 'WORLD_SCALE_RANGE': [0.95, 1.05]}]
2022-08-11 10:36:39,755 INFO
cfg.DATA_CONFIG.POINT_FEATURE_ENCODING = edict()
2022-08-11 10:36:39,755 INFO cfg.DATA_CONFIG.POINT_FEATURE_ENCODING.encoding_type: absolute_coordinates_encoding
2022-08-11 10:36:39,755 INFO cfg.DATA_CONFIG.POINT_FEATURE_ENCODING.used_feature_list: ['x', 'y', 'z', 'intensity', 's0', 's1', 's2', 's3']
2022-08-11 10:36:39,755 INFO cfg.DATA_CONFIG.POINT_FEATURE_ENCODING.src_feature_list: ['x', 'y', 'z', 'intensity', 's0', 's1', 's2', 's3']
2022-08-11 10:36:39,755 INFO cfg.DATA_CONFIG.DATA_PROCESSOR: [{'NAME': 'mask_points_and_boxes_outside_range', 'REMOVE_OUTSIDE_BOXES': True}, {'NAME': 'shuffle_points', 'SHUFFLE_ENABLED': {'train': True, 'test': False}}, {'NAME': 'transform_points_to_voxels', 'VOXEL_SIZE': [0.16, 0.16, 4], 'MAX_POINTS_PER_VOXEL': 32, 'MAX_NUMBER_OF_VOXELS': {'train': 16000, 'test': 40000}}]
2022-08-11 10:36:39,755 INFO cfg.DATA_CONFIG.BASE_CONFIG: cfgs/dataset_configs/painted_kitti_dataset.yaml
2022-08-11 10:36:39,755 INFO
cfg.MODEL = edict()
2022-08-11 10:36:39,755 INFO cfg.MODEL.NAME: PointPillar
2022-08-11 10:36:39,755 INFO
cfg.MODEL.VFE = edict()
2022-08-11 10:36:39,755 INFO cfg.MODEL.VFE.NAME: PillarVFE
2022-08-11 10:36:39,755 INFO cfg.MODEL.VFE.WITH_DISTANCE: False
2022-08-11 10:36:39,756 INFO cfg.MODEL.VFE.USE_ABSLOTE_XYZ: True
2022-08-11 10:36:39,756 INFO cfg.MODEL.VFE.USE_NORM: True
2022-08-11 10:36:39,756 INFO cfg.MODEL.VFE.NUM_FILTERS: [64]
2022-08-11 10:36:39,756 INFO
cfg.MODEL.MAP_TO_BEV = edict()
2022-08-11 10:36:39,756 INFO cfg.MODEL.MAP_TO_BEV.NAME: PointPillarScatter
2022-08-11 10:36:39,756 INFO cfg.MODEL.MAP_TO_BEV.NUM_BEV_FEATURES: 64
2022-08-11 10:36:39,756 INFO
cfg.MODEL.BACKBONE_2D = edict()
2022-08-11 10:36:39,756 INFO cfg.MODEL.BACKBONE_2D.NAME: BaseBEVBackbone
2022-08-11 10:36:39,756 INFO cfg.MODEL.BACKBONE_2D.LAYER_NUMS: [3, 5, 5]
2022-08-11 10:36:39,756 INFO cfg.MODEL.BACKBONE_2D.LAYER_STRIDES: [2, 2, 2]
2022-08-11 10:36:39,756 INFO cfg.MODEL.BACKBONE_2D.NUM_FILTERS: [64, 128, 256]
2022-08-11 10:36:39,756 INFO cfg.MODEL.BACKBONE_2D.UPSAMPLE_STRIDES: [1, 2, 4]
2022-08-11 10:36:39,756 INFO cfg.MODEL.BACKBONE_2D.NUM_UPSAMPLE_FILTERS: [128, 128, 128]
2022-08-11 10:36:39,756 INFO
cfg.MODEL.DENSE_HEAD = edict()
2022-08-11 10:36:39,756 INFO cfg.MODEL.DENSE_HEAD.NAME: AnchorHeadSingle
2022-08-11 10:36:39,756 INFO cfg.MODEL.DENSE_HEAD.CLASS_AGNOSTIC: False
2022-08-11 10:36:39,756 INFO cfg.MODEL.DENSE_HEAD.USE_DIRECTION_CLASSIFIER: True
2022-08-11 10:36:39,756 INFO cfg.MODEL.DENSE_HEAD.DIR_OFFSET: 0.78539
2022-08-11 10:36:39,757 INFO cfg.MODEL.DENSE_HEAD.DIR_LIMIT_OFFSET: 0.0
2022-08-11 10:36:39,757 INFO cfg.MODEL.DENSE_HEAD.NUM_DIR_BINS: 2
2022-08-11 10:36:39,757 INFO cfg.MODEL.DENSE_HEAD.ANCHOR_GENERATOR_CONFIG: [{'class_name': 'Car', 'anchor_sizes': [[3.9, 1.6, 1.56]], 'anchor_rotations': [0, 1.57], 'anchor_bottom_heights': [-1.78], 'align_center': False, 'feature_map_stride': 2, 'matched_threshold': 0.6, 'unmatched_threshold': 0.45}, {'class_name': 'Pedestrian', 'anchor_sizes': [[0.8, 0.6, 1.73]], 'anchor_rotations': [0, 1.57], 'anchor_bottom_heights': [-0.6], 'align_center': False, 'feature_map_stride': 2, 'matched_threshold': 0.5, 'unmatched_threshold': 0.35}, {'class_name': 'Cyclist', 'anchor_sizes': [[1.76, 0.6, 1.73]], 'anchor_rotations': [0, 1.57], 'anchor_bottom_heights': [-0.6], 'align_center': False, 'feature_map_stride': 2, 'matched_threshold': 0.5, 'unmatched_threshold': 0.35}]
2022-08-11 10:36:39,757 INFO
cfg.MODEL.DENSE_HEAD.TARGET_ASSIGNER_CONFIG = edict()
2022-08-11 10:36:39,757 INFO cfg.MODEL.DENSE_HEAD.TARGET_ASSIGNER_CONFIG.NAME: AxisAlignedTargetAssigner
2022-08-11 10:36:39,757 INFO cfg.MODEL.DENSE_HEAD.TARGET_ASSIGNER_CONFIG.POS_FRACTION: -1.0
2022-08-11 10:36:39,757 INFO cfg.MODEL.DENSE_HEAD.TARGET_ASSIGNER_CONFIG.SAMPLE_SIZE: 512
2022-08-11 10:36:39,757 INFO cfg.MODEL.DENSE_HEAD.TARGET_ASSIGNER_CONFIG.NORM_BY_NUM_EXAMPLES: False
2022-08-11 10:36:39,757 INFO cfg.MODEL.DENSE_HEAD.TARGET_ASSIGNER_CONFIG.MATCH_HEIGHT: False
2022-08-11 10:36:39,757 INFO cfg.MODEL.DENSE_HEAD.TARGET_ASSIGNER_CONFIG.BOX_CODER: ResidualCoder
2022-08-11 10:36:39,757 INFO
cfg.MODEL.DENSE_HEAD.LOSS_CONFIG = edict()
2022-08-11 10:36:39,757 INFO
cfg.MODEL.DENSE_HEAD.LOSS_CONFIG.LOSS_WEIGHTS = edict()
2022-08-11 10:36:39,757 INFO cfg.MODEL.DENSE_HEAD.LOSS_CONFIG.LOSS_WEIGHTS.cls_weight: 1.0
2022-08-11 10:36:39,757 INFO cfg.MODEL.DENSE_HEAD.LOSS_CONFIG.LOSS_WEIGHTS.loc_weight: 2.0
2022-08-11 10:36:39,757 INFO cfg.MODEL.DENSE_HEAD.LOSS_CONFIG.LOSS_WEIGHTS.dir_weight: 0.2
2022-08-11 10:36:39,757 INFO cfg.MODEL.DENSE_HEAD.LOSS_CONFIG.LOSS_WEIGHTS.code_weights: [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
2022-08-11 10:36:39,758 INFO
cfg.MODEL.POST_PROCESSING = edict()
2022-08-11 10:36:39,758 INFO cfg.MODEL.POST_PROCESSING.RECALL_THRESH_LIST: [0.3, 0.5, 0.7]
2022-08-11 10:36:39,758 INFO cfg.MODEL.POST_PROCESSING.SCORE_THRESH: 0.1
2022-08-11 10:36:39,758 INFO cfg.MODEL.POST_PROCESSING.OUTPUT_RAW_SCORE: False
2022-08-11 10:36:39,758 INFO cfg.MODEL.POST_PROCESSING.EVAL_METRIC: kitti
2022-08-11 10:36:39,758 INFO
cfg.MODEL.POST_PROCESSING.NMS_CONFIG = edict()
2022-08-11 10:36:39,758 INFO cfg.MODEL.POST_PROCESSING.NMS_CONFIG.MULTI_CLASSES_NMS: False
2022-08-11 10:36:39,758 INFO cfg.MODEL.POST_PROCESSING.NMS_CONFIG.NMS_TYPE: nms_gpu
2022-08-11 10:36:39,758 INFO cfg.MODEL.POST_PROCESSING.NMS_CONFIG.NMS_THRESH: 0.01
2022-08-11 10:36:39,758 INFO cfg.MODEL.POST_PROCESSING.NMS_CONFIG.NMS_PRE_MAXSIZE: 4096
2022-08-11 10:36:39,758 INFO cfg.MODEL.POST_PROCESSING.NMS_CONFIG.NMS_POST_MAXSIZE: 500
2022-08-11 10:36:39,758 INFO
cfg.OPTIMIZATION = edict()
2022-08-11 10:36:39,758 INFO cfg.OPTIMIZATION.BATCH_SIZE_PER_GPU: 4
2022-08-11 10:36:39,758 INFO cfg.OPTIMIZATION.NUM_EPOCHS: 80
2022-08-11 10:36:39,759 INFO cfg.OPTIMIZATION.OPTIMIZER: adam_onecycle
2022-08-11 10:36:39,759 INFO cfg.OPTIMIZATION.LR: 0.003
2022-08-11 10:36:39,759 INFO cfg.OPTIMIZATION.WEIGHT_DECAY: 0.01
2022-08-11 10:36:39,759 INFO cfg.OPTIMIZATION.MOMENTUM: 0.9
2022-08-11 10:36:39,759 INFO cfg.OPTIMIZATION.MOMS: [0.95, 0.85]
2022-08-11 10:36:39,759 INFO cfg.OPTIMIZATION.PCT_START: 0.4
2022-08-11 10:36:39,759 INFO cfg.OPTIMIZATION.DIV_FACTOR: 10
2022-08-11 10:36:39,759 INFO cfg.OPTIMIZATION.DECAY_STEP_LIST: [35, 45]
2022-08-11 10:36:39,759 INFO cfg.OPTIMIZATION.LR_DECAY: 0.1
2022-08-11 10:36:39,759 INFO cfg.OPTIMIZATION.LR_CLIP: 1e-07
2022-08-11 10:36:39,759 INFO cfg.OPTIMIZATION.LR_WARMUP: False
2022-08-11 10:36:39,759 INFO cfg.OPTIMIZATION.WARMUP_EPOCH: 1
2022-08-11 10:36:39,759 INFO cfg.OPTIMIZATION.GRAD_NORM_CLIP: 10
2022-08-11 10:36:39,759 INFO cfg.TAG: pointpillar_painted
2022-08-11 10:36:39,759 INFO cfg.EXP_GROUP_PATH: kitti_models
Traceback (most recent call last):
File "train.py", line 198, in
main()
File "train.py", line 112, in main
total_epochs=args.epochs
File "/content/OpenPCDet/pcdet/datasets/init.py", line 51, in build_dataloader
dataset = all[dataset_cfg.DATASET](
KeyError: 'PaintedKittiDataset'

I searched, but I couldn't find something usefull!
Do you have any idea?

How did you visualize the painted point cloud, especially this part how did you get it. I followed all the steps and used the visualization like you suggested but i am not getting the painted point cloud like the image given below:

what's the difference between image_2 and image_3?

RuntimeError: Error compiling objects for extension

I run codes in painting.py. I found it needs at least two cameras to complete the painting process. According to the paper, there seem no limits on the number of cameras.
Now, I got a dataset that only uses one camera. (it mimics the Kitti dataset)
I try to copy image_2 to image_3.
But it does not work.
When I run the test.py, I got a result that all metrics are 0.

Could you give me some advice on how to revise the code in painting.py to generate a normal result?

How to set up the environment without GPU?

hello ! thank you so much! when i run python.py ,there are some error,so i want to ask you for help.
INFO - 2021-12-17 16:03:24,257 - database_sampler - Database filter by difficulty Pedestrian: 2168 => 2075
2021-12-17 16:03:24,258 INFO Database filter by difficulty Cyclist: 704 => 581
INFO - 2021-12-17 16:03:24,258 - database_sampler - Database filter by difficulty Cyclist: 704 => 581
2021-12-17 16:03:24,264 INFO Loading KITTI dataset
INFO - 2021-12-17 16:03:24,264 - painted_kitti_dataset - Loading KITTI dataset
2021-12-17 16:03:24,399 INFO Total samples for KITTI dataset: 3712
INFO - 2021-12-17 16:03:24,399 - painted_kitti_dataset - Total samples for KITTI dataset: 3712
段错误 (核心已转储)
this is the problem.
good luck!

When running demo.py, the point cloud in the visualization is not colored. Is this a normal horse? My visualization-related files use the source code in the Openpcet project and have not been modified. Do I need to modify the following visualization-related code? I hope you can give me some advice. Thank you very much, sorry for your inconvenience.

我在自己的机器上复现了Pointpainting，但是最终的结果无法达到原文的效果，尤其是行人类别，Mod。仅有46.0145，请问这是什么情况。
I reproduced Pointpainting on my own machine, but the final result cannot reach the original, especially for the pedestrian category, the Mod is only 46.0145.

Hello, thanks for your excellent work,
Could you tell me which which KItte dataset do I need to download?
there are :
Download left color images of object data set (12 GB)
Download right color images, if you want to use stereo information (12 GB)
Download the 3 temporally preceding frames (left color) (36 GB)
Download the 3 temporally preceding frames (right color) (36 GB)
Download Velodyne point clouds, if you want to use laser information (29 GB)
Download camera calibration matrices of object data set (16 MB)
Download training labels of object data set (5 MB)
......
and so on.
Please tell me, I am looking for your reply!

Thanks for your good job, but I have a dataset question. The error I met as follows:

2022-11-23 19:53:31,224 INFO **********************Start training home/seu/data_backup/new_disk/bbdata/pro_detect/PointPainting-main/detector/tools/cfgs/kitti_models/pointpillar_painted(default)********************** epochs: 0%| | 0/10 [00:00<?, ?it/s] epochs: 0%| | 0/10 [00:09<?, ?it/s] Traceback (most recent call last): File "/home/seu/data_backup/new_disk/bbdata/pro_detect/PointPainting-main/detector/tools/train_utils/train_utils.py", line 93, in train_model dataloader_iter=dataloader_iter File "/home/seu/data_backup/new_disk/bbdata/pro_detect/PointPainting-main/detector/tools/train_utils/train_utils.py", line 19, in train_one_epoch batch = next(dataloader_iter) File "/home/seu/anaconda3/envs/DRLearning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 517, in __next__ data = self._next_data() File "/home/seu/anaconda3/envs/DRLearning/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 557, in _next_data data = self._dataset_fetcher.fetch(index) # may raise StopIteration File "/home/seu/anaconda3/envs/DRLearning/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/seu/anaconda3/envs/DRLearning/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp> data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/seu/data_backup/new_disk/bbdata/pro_detect/PointPainting-main/detector/pcdet/datasets/kitti/painted_kitti_dataset.py", line 409, in __getitem__ data_dict = self.prepare_data(data_dict=input_dict) File "/home/seu/data_backup/new_disk/bbdata/pro_detect/PointPainting-main/detector/pcdet/datasets/dataset.py", line 124, in prepare_data 'gt_boxes_mask': gt_boxes_mask File "/home/seu/data_backup/new_disk/bbdata/pro_detect/PointPainting-main/detector/pcdet/datasets/augmentor/data_augmentor.py", line 93, in forward data_dict = cur_augmentor(data_dict=data_dict) File "/home/seu/data_backup/new_disk/bbdata/pro_detect/PointPainting-main/detector/pcdet/datasets/augmentor/database_sampler.py", line 213, in __call__ data_dict = self.add_sampled_boxes_to_scene(data_dict, sampled_gt_boxes, total_valid_sampled_dict) File "/home/seu/data_backup/new_disk/bbdata/pro_detect/PointPainting-main/detector/pcdet/datasets/augmentor/database_sampler.py", line 134, in add_sampled_boxes_to_scene [-1, self.sampler_cfg.NUM_POINT_FEATURES]) ValueError: cannot reshape array of size 308 into shape (8) python-BaseException

Towards the debug, I find the problem is the number of points, I.e the points of "file_path" ( '.bin' ) can not be devided by 8, so, how I solve the problem? I use the kitti datasets。
"
obj_points = np.fromfile(str(file_path), dtype=np.float32).reshape(
[-1, self.sampler_cfg.NUM_POINT_FEATURES])
"
please help me ....

Hi. Thanks for sharing your work. I am struggling to visualise the painted point cloud. Can you share the code you used to get the projected point cloud images for the Framework Overview?

Thank you in advance.

I trained my model and when i run the demo I get this result :

If it is to go by this approach, I should have got the painted point cloud in this image, but I dont think the point cloud are painted here. I used one of the .npy files from training/painted_lidar folder. Is there anything wrong I am doing here. Your suggestion would be greatly appreciated.

Nice work! 👏

@Song-Jingyu
How much effort do you estimate to integrate your code into the official OpenPCDet repository?
It would be great if you could create a pull-request that merges your modifications/additions into the official OpenPCDet repository!

Hi, since we want to check if we obtain reasonable results, so we wonder if you would like to share the AP values for every class and every difficulty?

Thanks in advance :)

Hello, I read your paper. I think your idea is really interesting. I just want to know if the BEV mAP values in below table in your paper are based on recall 40 or recall11. Thank you.

I want to make compare to non-painted bin file. I just change the cfg file when runing training command. Should I change any other settings?

LINK : fatal error LNK1181: 无法打开输入文件“D:\A_github\PointPainting-main\detector\build\temp.win-amd64-3.8\Release\pcdet\ops\iou3d_nms\src\iou3d_cpu.obj”
error: command 'D:\A-Visual Studio-c++\Professional\VC\Tools\MSVC\14.34.31933\bin\HostX86\x64\link.exe' failed with exit status 1181

I tried demo.py to seem how my painted detector model works and task was killed by system at about 180th instance.

I checked GPU status and GPU memory keeps going up which eventually caused OOM and process was killed by system. Any ideas that might work this out will be grateful.

Can you explain why you project the points on the segmentation map after they were already projected to camera coordinates? Start of the code I am confused about is below.

https://github.com/HuangCongQing/PointPainting-notes/blob/hcq/20221203/painting/painting.py#L179

i tried to add 3 more rgb color attributions, but it turns out
Warning: NaN or Inf found in input tensor
what can i do about this?
hoping for your reply

Hi,

may I ask how do you get the semantic segmentation images that are shown in the 'Results & Discussion' subsection?
Thanks in advance.

A problem was encountered while running the command"python painting.py". Does anyone have a solution？
“ImportError:libtorch_cpu.so:cannot open shared object file:No such file or directory.”
After checking the tracback , It seems to be because of mmcv, but mmcv has been installed with the documentation.

Could you please tell me how to run the test set?

I run : "python test.py --cfg_file cfgs/kitti_models/pointpillar_painted.yaml --batch_size 1 --ckpt /home/wenhao/桌面/PointPainting-main/detector/output/kitti_models/pointpillar_painted/default/ckpt/checkpoint_epoch_80.pth".

But the result of "pointpillar+DeepLab V3+" seemed have no different with original "pointpillar"

Hello, I have completed some training and testing of pointpillars_painted through train.py and evel.py, but the following problems cannot be solved when running demo.py, I hope to get your reply, thank you.

python -m pcdet.datasets.kitti.painted_kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/painted_kitti_dataset.yaml
gives me the following error:

in transform_points_to_voxels
from spconv.utils import VoxelGenerator
ImportError: cannot import name 'VoxelGenerator' from 'spconv.utils' (/home/UNT/sd0570/anaconda3/envs/pointpainting/lib/python3.8/site-packages/spconv/utils/init.py)

I am getting this import error. Is it because of the mismatch in spconv version? I just wanted to know which version of spconv you are using because openpcdet has special requirements. Could you please specify the version of following:

1. CUDA
2. Pytorch
3. spconv
4. cudatoolkit

Hello

While trying to execute the code
!python -m pcdet.datasets.kitti.painted_kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/painted_kitti_dataset.yaml

the following error had occurred
/usr/local/lib/python3.7/runpy.py:125: RuntimeWarning: 'pcdet.datasets.kitti.painted_kitti_dataset' found in sys.modules after import of package 'pcdet.datasets.kitti', but prior to execution of 'pcdet.datasets.kitti.painted_kitti_dataset'; this may result in unpredictable behaviour
warn(RuntimeWarning(msg))
/content/drive/MyDrive/Pytorch/PointPainting-main/detector/pcdet/datasets/kitti/painted_kitti_dataset.py:442: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
dataset_cfg = EasyDict(yaml.load(open(sys.argv[2])))
Traceback (most recent call last):
File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/content/drive/MyDrive/Pytorch/PointPainting-main/detector/pcdet/datasets/kitti/painted_kitti_dataset.py", line 448, in
save_path=ROOT_DIR / 'data' / 'kitti'
File "/content/drive/MyDrive/Pytorch/PointPainting-main/detector/pcdet/datasets/kitti/painted_kitti_dataset.py", line 424, in create_kitti_infos
dataset = PaintedKittiDataset(dataset_cfg=dataset_cfg, class_names=class_names, root_path=data_path, training=False)
File "/content/drive/MyDrive/Pytorch/PointPainting-main/detector/pcdet/datasets/kitti/painted_kitti_dataset.py", line 23, in init
dataset_cfg=dataset_cfg, class_names=class_names, training=training, root_path=root_path, logger=logger
File "/content/drive/MyDrive/Pytorch/PointPainting-main/detector/pcdet/datasets/dataset.py", line 34, in init
self.dataset_cfg.DATA_PROCESSOR, point_cloud_range=self.point_cloud_range, training=self.training
File "/content/drive/MyDrive/Pytorch/PointPainting-main/detector/pcdet/datasets/processor/data_processor.py", line 16, in init
cur_processor = getattr(self, cur_cfg.NAME)(config=cur_cfg)
File "/content/drive/MyDrive/Pytorch/PointPainting-main/detector/pcdet/datasets/processor/data_processor.py", line 48, in transform_points_to_voxels
from spconv.utils import VoxelGenerator
File "/usr/local/lib/python3.7/site-packages/spconv-2.1.21-py3.7-linux-x86_64.egg/spconv/utils/init.py", line 27, in
from spconv.core_cc.csrc.sparse.all.ops1d import Point2Voxel as Point2VoxelGPU1d
ModuleNotFoundError: No module named 'spconv.core_cc.csrc.sparse.all.ops1d'; 'spconv.core_cc.csrc.sparse.all' is not a package

hello， @Song-Jingyu ，Your work is great. I am now thinking about how to use your Point Painting framework on other lidar detection frameworks, such as Voxel-RCNN on OpenPCDet or some frameworks that are not integrated in OpenPCDet.
Looking forward to your answer.
Thank you very much.

Hi Song-Jingyu,

Thanks for your work with point painting. Just got a question about the model architecture in regards to adding more lidar input features.

Did you have to modify the model architecture to be able to accommodate more input features?

I have looked through the code and configs but didn't seem to find any modifications. However, I'm not too clear how the model can be fed with additional input channels (s0,s1,s2,s3) and learn to take advantage of those features without modifying the layer shapes, loss function, or some sort.

Could you help me clarify this? Also, what are s0,s1,s2,s3? Do these form a one-hot vector of class labels of the classes "0: background, 1: bicycle, 2: car, 3: person"?