wangyueft / pillar-od

Pillar-based Object Detection for Autonomous Driving

Prerequisite

TensorFlow (https://www.tensorflow.org/install)

TensorFlow Addons (https://www.tensorflow.org/addons/overview)

Waymo Open Dataset (https://github.com/waymo-research/waymo-open-dataset)

Lingvo (https://github.com/tensorflow/lingvo)

Data

1. Download the data from https://waymo.com/open/

2. Pre-process the data using the script "data/generate_waymo_dataset.sh"

Train and eval

Check "train.py", "eval.py", and "config.py"

Evaluation using pretrained models

1. Download the weights from https://drive.google.com/file/d/16cFbbKfEXc5uH7V6xDw6fy3lVBCgHLNd/view?usp=sharing

2. For car, python eval.py --class_id=1 --nms_iou_threshold=0.7 --pillar_map_size=256 --ckpt_path=/path/to/checkpoints --data_path=/path/to/data --model_dir=/path/to/results

   For pedestrian, python eval.py --class_id=2 --nms_iou_threshold=0.2 --pillar_map_size=512 --ckpt_path=/path/to/checkpoints --data_path=/path/to/data --model_dir=/path/to/results

If you find this repo useful for your research, please consider citing the paper

@inproceedings{
   wang2020,
   title={Pillar-based Object Detection for Autonomous Driving},
   author={Wang, Yue and Fathi, Alireza and Kundu, Abhijit and Ross, David A. and Pantofaru, Caroline and Funkhouser, Thomas A. and Solomon, Justin M.},
   booktitle={The European Conference on Computer Vision ({ECCV})},
   year={2020}
}

pillar-od's Issues

miss tensorflow-datasets in requirements.txt

• OK

pip install tensorflow-datasets=2.0.0

• Error(this will install tensorflow-datasets=3.2.1, and it will has version conflict with waymo.open_dataset )

pip install tensorflow-datasets

Question about ema_model

pillar-od/train.py

Lines 115 to 130 in 3eac6a9

	model = builder.PillarModel(
	class_id=FLAGS.class_id,
	norm_type=FLAGS.norm_type,
	act_type=FLAGS.act_type,
	nms_iou_threshold=FLAGS.nms_iou_threshold,
	nms_score_threshold=FLAGS.nms_score_threshold,
	max_nms_boxes=FLAGS.max_nms_boxes,
	use_oriented_per_class_nms=FLAGS.use_oriented_per_class_nms)
	ema_model = builder.PillarModel(
	class_id=FLAGS.class_id,
	norm_type=FLAGS.norm_type,
	act_type=FLAGS.act_type,
	nms_iou_threshold=FLAGS.nms_iou_threshold,
	nms_score_threshold=FLAGS.nms_score_threshold,
	max_nms_boxes=FLAGS.max_nms_boxes,
	use_oriented_per_class_nms=FLAGS.use_oriented_per_class_nms)

pillar-od/train.py

Lines 208 to 226 in 3eac6a9

	with tf.GradientTape() as tape:
	preds = model(inputs, training=True)
	if optimizer.iterations < 1:
	ema_model(inputs, training=False)
	cls_loss, loc_loss = model.compute_loss(inputs, preds)
	mean_cls_loss = tf.reduce_mean(cls_loss)
	mean_loc_loss = tf.reduce_mean(loc_loss)
	weight_loss = tf.reduce_sum(model.losses)
	loss = (mean_cls_loss + mean_loc_loss + weight_loss)
	training_vars = model.trainable_variables
	grads = tape.gradient(loss, training_vars)
	optimizer.apply_gradients(zip(grads, training_vars))
	train_loss_metric.update_state(loss)
	train_loc_loss_metric.update_state(mean_loc_loss)
	train_cls_loss_metric.update_state(mean_cls_loss)
	weight_loss_metric.update_state(weight_loss)
	_momentum_update(model, ema_model)

Thanks for your great work. But I still confuse about the effect of ema_model, since the pointpillars does not apply the same setting. Could you explain the reason why you apply ema_model here, and add an ablation study on ema_model?

Could you add a license like MIT oder Apache?

Question about cylindrical view

pillar-od/network.py

Lines 280 to 284 in 8a26d57

	self.cylinder_view_grid_size = (2560, 100, 1)
	self.cylinder_view_grid_range_x = (-np.pi, np.pi)
	self.cylinder_view_grid_range_y = (-3.0, 7.0)
	self.cylinder_view_grid_range_z = (0.0, 107.0)

The order of the coordinates in the cylindrical view is different from Table 4 in the paper (arxiv version). What's more, the grid size of the SPV is missing. Could you give the grid size of the SPV?