dw1hh / deeplabv3plus-pytorch

Here is a pytorch implementation of deeplabv3+. The project supports COCO object detection dataset pretrain and PASCAL VOC 2012 train/val.

Most recurrent update:

2018.11.26 - Update including support Xception network, multi-scale test, network output stride modification, pure train set finetuning, and more dataset interface (PASCAL Context, Cityscapes, ADE20K)

2018.09.28 - Add python evaluation func in ./lib/datasets/VOCDataset.py

2018.09.21 - Fix some bugs in ./lib/datasets

DeepLabv3plus Semantic Segmentation in Pytorch

Here is a pytorch implementation of deeplabv3+. The project support variants of dataset including MS COCO object detection dataset, PASCAL VOC, PASCAL Context, Cityscapes, ADE20K.

Configuration Environment

Ubuntu16.04 + python3.6 + pytorch0.4 + CUDA8.0

You should run the code on more than 2 GPU because only multi-GPU version is supported at present.

Anaconda virtual environment is suggested.

Please install tensorboardX for loss curves and segmentation visualization.

SynchronizedBatchNorm(from vacancy) has been used in our model, which will not be installed manually.

Dataset

1. Download VOC2012 dataset and VOCdevkit, keep the VOC2012 folder in VOCdevkit as $root/data/VOCdevkit/VOC2012/...., or you can modified path in $root/lib/datasets/VOCDataset.py

2. Download VOC augmented segmentation dataset from DrSleep. Put the annotations in $root/data/VOCdevkit/VOC2012/SegmentationClass/ and name list file at $root/data/VOCdevkit/VOC2012/ImageSets/Segmentation/trainaug.txt

3. Download COCO2017 object detection dataset here for pretrain, keep the path as $root/data/MSCOCO/...., or change it in$root/lib/datasets/COCODataset.py

4. (optional) Install Matlab for VOC2012 evaluation (VOC official evaluation code in VOCdevkit is written by Matlab, and we have reimplement a python version)

5. (optional) You can also download ADE20K dataset, Cityscapes, PASCAL Context as same as VOC and COCO, check the dataset path setting in $root/lib/datasets/xxxDataset.py.

Network

Now the project support modified ResNet and Xception network as backbone.

For ResNet backbone, the structure is modified according to [2] which repeats layer4 four times with atrous rate as 1,2,1. We load part of ResNet parameters from pytorch pretrained model and random initialize others.

As for Xception, we keep the same structure with offical tensorflow code and transform the pretrained parameter file from .ckpt to .pth manually!

Now the best model achieve 81.937% on VOC2012 validation set with Xception, output stride=16, multi-scale and flip test, and pretrained on COCO, finetuning on VOC trainaug set and VOC train set. Discussion about finetuing tricks is welcomed by email([email protected]) or issues!!!

Here are some pretrained model for download(deeplabv3+ models are trained on multi GPUs, it may cause error when loading by cpu version):

Name	mIoU on PASCAL VOC val set
xception_pytorch_imagnet
xception_pytorch_coco
deeplabv3plus_xception_VOC2012_COCO	81.937%

Model checkpoint with higher performance will be updata once achieve.

Train & Test

Please check the configuration file $root/experiment/project_name/config.py first to meet your requires.

Please check the pretrained checkpoint path in $root/lib/net/xception.py and $root/lib/net/resnet_atrous.py

Please set visible gpu devices before training.

export CUDA_VISIBLE_DEVICES=0,1

Then you can train the network as you wish.

cd $root/experiment/project_name
python train.py

You can also test the performance of trained network by:

cd $root/experiment/project_name
python test.py

Implementation Detail

• deeper ResNet101 backbone
• atrous convolution
• multi-gpu support
• synchronized batch normalization
• decoder
• model pretrained on COCO
• flip test
• xception as backbone
• multiscale test
• achieve the performance mentioned in paper.

Discussion about learning tricks is welcomed ([email protected]).

References

1. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, Hartwig Adam. In ECCV, 2018.

2. Rethinking Atrous Convolution for Semantic Image Segmentation Liang-Chieh Chen, George Papandreou, Florian Schroff, Hartwig Adam. arXiv: 1706.05587, 2017.