yanx27 / urban3d-2021-2nd

The 2nd solution (68.1 mIoU) for 1st Urban3D challenge.

2nd Solution for Urban3D@ICCV2021

This repository is for the runner up solution for the challenge Urban3D@ICCV2021.

News

• We achieve the 3rd place in Urban3D@ECCV2022！The 1st solution also uses this codebase!

Getting Started

Requirements

• Ubuntu 16.04
• Anaconda with python=3.7
• tensorFlow=1.14
• cuda=10.1
• cudnn=7.6.5
• others: pip install termcolor opencv-python toposort h5py easydict

Compile custom operators

sh init.sh

Data preparation

(1) Download the files named "data_release.zip" here. Uncompress it.

(2) Preparing the dataset

python datasets/prepare_data.py --dataset_path $YOURPATH
cd $YOURPATH; 
cd ../; mkdir original_block_ply; mv data_release/train/* original_block_ply; mv data_release/test/* original_block_ply;
mv data_release/grid* ./

(3) The data should organized in the following format:

/Dataset/SensatUrban/
          └── original_block_ply/
                  ├── birmingham_block_0.ply
                  ├── birmingham_block_1.ply 
		  ...
	    	  └── cambridge_block_34.ply 
          └── grid_0.200/
	     	  ├── birmingham_block_0_KDTree.pkl
                  ├── birmingham_block_0.ply
		  ├── birmingham_block_0_proj.pkl 
		  ...
	    	  └── cambridge_block_34.ply 

Training

We train our model with two V100 GPUs. If you want to use other type of GPU with smaller batch size or other model architecture, you can modify the configuration in cfgs/sensaturban/baseline_version1.yaml.

python function/train_evaluate_sensaturban.py --cfg cfgs/sensaturban/baseline_version1.yaml  --gpus 0 1

• The results will be saved in ./log/.
• You can use --trainval to train on both trianing and validation sets.

Evaluation

python function/evaluate_sensaturban.py  --load_path [YOUR_MODEL_PATH]  --cfg cfgs/sensaturban/baseline_version1.yaml

• The results will be saved in ./log_eval/.

Submit to benchmark

python function/test_sensaturban.py  --load_path [YOUR_MODEL_PATH]  --cfg cfgs/sensaturban/baseline_version1.yaml

• The results will be saved in ./log_test/.
• Submit the results to the evaluation server.

Pre-trained model

We currently release a checkpoints with about 66% mIoU, and more powerful checkpoints will be released in the future.

Input	mIoU	Accuracy	Checkpoints
xyz+rgb	65.8	94.0	BaiduNetdisk:1wt2

You can specific the checkpoint in evaluation or testing by using

--load_path [CHECKPOINTS_PATH]/model.ckpt-600 

Citation

If you find our work useful in your research, please consider citing:

@inproceedings{yan2021urban3d,
  title={Urban3D-2021-2rd},
  author={Xu Yan},
  booktitle={https://github.com/yanx27/Urban3D-2021-2nd},
  year={2021}
}

Acknowledgement

This project is not possible without multiple great opensourced codebases.

• SensatUrban
• CloserLook3D

urban3d-2021-2nd's Issues

Incompatible shapes

solved

solved

solved

Do you know how to calculate the parameter volume suitable for specific GPU size?

Hi @yanx27 @BIRlz

Do you know how to calculate the parameter volume suitable for specific GPU size? When I revise the model with batch size 12, using 4 32G GPU cards, there is an OOM error. Before revise the model, 1 32G GPU card could run the model with batch size 12. Thanks.

ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[70395,26,128,1] and type float on /job:localhost/replica:0/task:0/device:GPU:3 by allocator GPU_3_bfc
[[node gpu_3/SceneSegModel/resnext_backbone/res1_bottleneck_resnext0/split_layer/transform_layer_1/conv2/local_aggregation_card_1/Mul (defined at /export/home//SensatUrban_sol_tf_GPU_90_batch_12/models/local_aggregation_operators.py:453) = Mul[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:3"](gpu_3/SceneSegModel/resnext_backbone/res1_bottleneck_resnext0/split_layer/transform_layer_1/conv2/local_aggregation_card_1/ExpandDims_1, gpu_3/SceneSegModel/resnext_backbone/res1_bottleneck_resnext0/split_layer/transform_layer_1/conv2/local_aggregation_card_1/Reshape)]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

     [[{{node Mean_156/_8673}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_37211_Mean_156", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

Best wishes.

How to upgrade this code from tensorflow-gpu==1.14 with cuda90 into tensorflow-gpu==2.8.0 with cuda11.1?

Hi @yanx27

How to upgrade this code from tensorflow-gpu==1.14 with cuda90 into tensorflow-gpu==2.8.0 with cuda11.1?

The following errors occur.

Successfully opened dynamic library libcudart.so.11.0
/usr/bin/ld: cannot find -ltensorflow_framework
collect2: error: ld returned 1 exit status
/usr/bin/ld: cannot find -ltensorflow_framework
collect2: error: ld returned 1 exit status
/usr/bin/ld: cannot find -ltensorflow_framework

test code error after submitting

Hi @BIRlz @yanx27 ,

I have run this code, and submit the test results to the Codalab evalution system. But the error occurs like this. Have you ever met this error before? Or, how to set the GPU when testing?

Error:
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
Traceback (most recent call last):
File "/tmp/codalab/tmpD6ePHX/run/program/evaluate.py", line 118, in
m_iou,iou,m_acc = evaluate_iou(pred_files,target_files)
File "/tmp/codalab/tmpD6ePHX/run/program/evaluate.py", line 93, in evaluate_iou
assert (target_files[i] in pred_files)
AssertionError

Extraction code

Hey , what is the extraction code to retrieve the weights?