lxtgh / gald-dgcnet

Source code and model GALD net (BMVC-2019) and Dual-Seg Net (BMVC-2019)

GALD-Net-v2 (TIP-2021)

Note that our GALD-v2 (improved version of GALD-v1) has been accept by TIP-2021! It achieves 83.5 mIoU using ResNet101 backbone!.

GALD-Net & Dual-Seg Net （BMVC-2019）

This is PyTorch re-implementation of GALD-net and Dual-Seg. Both papers were accepted by BMVC-2019 and achieve state-of-the-art results on the Cityscapes and Pascal Context datasets.

High Performance Road Scene Semantic Segmentaion 🎉

There is also a co-current repo for Fast Road Scene Semantic Segmentation:Fast_Seg ⚡ and thanks for your attention 😃

GALDNet

DualGCNSegNet

Training & Validation

Requirements

pytorch >= 1.1.0 apex opencv-python

Pretrained Model

Baidu Pan Link: https://pan.baidu.com/s/1MWzpkI3PwtnEl1LSOyLrLw passwd: 4lwf Google Drive Link: https://drive.google.com/file/d/1JlERBWT8fHvf-uD36k5-LRZ5taqUbraj/view?usp=sharing, https://drive.google.com/file/d/1gGzz_6ZHUSC4A3SO0yg8-uLE0iiPdO4H/view?usp=sharing

Training

Note that we use apex to speed up training process. At least 8 gpus with 12GB are needed since we need batch size at least 8 and crop size at least 800 on Cityscapes dataset. Please see train_distribute.py for the details.

sh ./exp/train_dual_seg_r50_city_finetrain.sh

You will get the model with 79.6~79.8 mIoU.

sh ./exp/train_dual_seg_r101_city_finetrain.sh

You will get the model with 80.3~80.4 mIoU.

Validation

sh ./exp/tes_dualseg_r50_city_finetrain.sh

Trained Model

Model trained with the Cityscapes fine dataset:

Dual-Seg-net: ResNet 50, ResNet 101

Some Advice on Training

Please see the Common.md for the details for using the coarse data training. Or you can refer to our GLAD paper(last part) for reference.

GALD-Net (BMVC 2019,arxiv)

We propose Global Aggregation then Local Distribution (GALD) scheme to distribute global information to each position adaptively according to the local information around the position. GALD net achieves top performance on Cityscapes dataset. Both source code and models will be available soon. The work was done at DeepMotion AI Research

DGCNet (BMVC 2019,arxiv)

We propose Dual Graph Convolutional Network (DGCNet) to model the global context of the input feature by modelling two orthogonal graphs in a single framework. (Joint work: University of Oxford, Peking University and DeepMotion AI Research)

Comparisons with state-of-the-art models on Cityscapes dataset

Method	Conference	Backbone	mIoU(%)
RefineNet	CVPR2017	ResNet-101	73.6
SAC	ICCV2017	ResNet-101	78.1
PSPNet	CVPR2017	ResNet-101	78.4
DUC-HDC	WACV2018	ResNet-101	77.6
AAF	ECCV2018	ResNet-101	77.1
BiSeNet	ECCV2018	ResNet-101	78.9
PSANet	ECCV2018	ResNet-101	80.1
DFN	CVPR2018	ResNet-101	79.3
DSSPN	CVPR2018	ResNet-101	77.8
DenseASPP	CVPR2018	DenseNet-161	80.6
OCNet	-	ResNet-101	81.7
CCNet	ICCV2019	ResNet-101	81.4
GALD-Net	BMVC2019	ResNet50	80.8
GALD-Net	BMVC2019	ResNet101	81.8
DGCN-Net	BMVC2019	ResNet101	82.0
GALD-Net(use coarse data)	BMVC2019	ResNet101	82.9
GALD-NetV2(use coarse data)	TIP2021	ResNet101	83.5
GALD-Net(use Mapillary)	BMVC2019	ResNet101	83.3

Detailed Results are shown

GALD-Net: here
GFF-Net:here
Both are (Single Model Result)

Citation

Please refer our paper for more detail. If you find the codebase useful, please consider citing our paper.

@inproceedings{xiangtl_gald
title={Global Aggregation then Local Distribution in Fully Convolutional Networks},
author={Li, Xiangtai and Zhang, Li and You, Ansheng and Yang, Maoke and Yang, Kuiyuan and Tong, Yunhai},
booktitle={BMVC2019},
}

@inproceedings{zhangli_dgcn
title={Dual Graph Convolutional Network for Semantic Segmentation},
author={Zhang, Li(*) and Li, Xiangtai(*) and Arnab, Anurag and Yang, Kuiyuan and Tong, Yunhai and Torr, Philip HS},
booktitle={BMVC2019},
}

License

MIT License

Acknowledgement

Thanks to previous open-sourced repo:
Encoding
CCNet
TorchSeg
pytorchseg

gald-dgcnet's Issues

how to plot the error map?

the availability of the code

Hi,

I am creating a new benchmark with fine-grained annotation in aerial imagery. In the paper, I am doing a heavy benchmark on state-of-the-art methods such as PSPNet, Deeplabv+, BiSeNet, DenseASSP and so on for semantic segmentation. I am wondering whether it is possible to train your algorithm on this data for the sake of benchmark with proper citation.

Are there plans to release the code for GALD-v2

stated performance of DGCNet cannot meet

Dear author,
Thank you for publish this repo. I tried with original training/validation settings , as stated in readme, the resnet50 DualGCN for cityscape shall reach ~79%, However mine only reaches 67%
The details are attached as below:

{"meanIU": 0.6755697115491832, "IU_array": [0.9699699360909015, 0.7804632423588875, 0.909098167476979, 0.47001360372795786, 0.53806631978281, 0.5845193952059451, 0.5983268985770329, 0.7145573340347664, 0.9099576676527088, 0.5301010366577027, 0.9328700787203849, 0.7606536183056566, 0.524505490879149, 0.9179202810460247, 0.45918543000315803, 0.6450625060757722, 0.39362567755342187, 0.49597250072955573, 0.7009553345556664]}

While I havenot change any settings in the original repo, my pytorch version is 1.6.0.
Could you explain a little bit why this performance gap is so large?

The use of 16bit images and stereo

On the cityscapes score, it is shown that you use both 16bit images and stereo data, but I cannot find any explanation in the paper. Could you clarify why you use these data and their effect?

The availability of the code about "Dual Graph Convolutional Network for Semantic Segmentation" .

I am very expect that you can release the code about "Dual Graph Convolutional Network for Semantic Segmentation". Thanks.

Not able to download the pre-trained model from baidu pan

Hi,

Thanks for the repo.

I am trying to download the pre-trained weights of the model using Baidu Pan Link (https://pan.baidu.com/s/1MWzpkI3PwtnEl1LSOyLrLw). But there is no option to download the files. Is it possible to share models on google drive?

Thanks for your help

Regards,
Achuth

The availability ofpaper?

The paper?

file not found error

Where can I find this file ?
FileNotFoundError: [Errno 2] No such file or directory: '/home/lxt/pretrained/GALD_res101_map_831.pth'
In addition do I have to use this file "'./resnet101-deep.pth'" in place of deeplab_resnet.pth

File missing!!

python file demo.py missing in the repository

The code for GALD-v2

When will it be open sourced?

About the performance of GALD

Thanks for your work in semantic segmentation. I'm confused about the reason of worse performance with your galdnet in https://github.com/speedinghzl/pytorch-segmentation-toolbox.

The Paper?

Pretrained weights missing

Link given https://github.com/lxtGH/GALD-DGCNet#pretrained-model is dead.

GFF-Net

你好，在GFF这篇论文的表1的对比实验中，SUM和Cat是先对Backbone的所有特征图进行1x1降维到256，然后sum/cat，最后再加两次3x3卷积。而FPN从公式来看，只用了最后一个FPN层，也即1/4的特征图，没有将1/32、1/16、1/8、1/4所有特征图进行融合。而GFF是将FPN的所有层进行Cat后进行预测，这个对比是否还合理呢。还是我理解错了，FPN如果只用1/4特征图，和sum也没多少区别了？
能否解答下疑惑呢，谢谢。

Why do you downsample feature first and then input GA module? ( x = self.down(x) x = self.long_relation(x))

class GALDBlock(nn.Module):
def forward(self, x):
size = x.size()[2:]
x = self.down(x)
x = self.long_relation(x)
# local attention
x = F.interpolate(x,size=size, mode="bilinear", align_corners=True)
res = x
x = self.local_attention(x)
return x + res

why you first downsample feature, and then use GA module, which is is inconsistent with that in the paper.

class GALDBlock(nn.Module):
def forward(self, x):
size = x.size()[2:]
x = self.down(x)
x = self.long_relation(x)
# local attention
x = F.interpolate(x,size=size, mode="bilinear", align_corners=True)
res = x
x = self.local_attention(x)
return x + res

only two gpus

Sorry to interrupt you, I want to ask, if I only have two GPUs, each with 12GB, can I run your semantic segmentation code? If I run, what needs to be modified, thank you

The paper for GALD-Net-v2

Can you release the paper for GALD-Net-v2?

I have encounter the problem KeyError: 'CascadeRelatioNet_res50' and I find DeeplabV3.py is void

Hello，Thank you for sharing the code。
I run python train_distribute.py ,encounter porblem as follows
Traceback (most recent call last): File "train_distribute.py", line 262, in <module> main() File "train_distribute.py", line 157, in main deeplab = models.__dict__[args.arch](num_classes=args.num_classes, data_set=args.data_set) KeyError: 'CascadeRelatioNet_res50'

another question is
DANet.py DeeplabV3.py GloreNet.py .Why are they empty

something wrong with trainning DualSeg

File missing

Hello, I encountered a lot of problems when using the code, can you provide a detailed documentation, thank you! ! Where can resnet50-deep.pth be downloaded?

请问一下什么时候公开源码呢 期待呀

About the GFF module

Sorry to bother...Could you please point out where the code for the GFF mode is? Thx a lot.

怎么样设置才能把miou复现的和论文一样高呢