xrachitx / sdl-skeleton

A toolbox for object skeleton detection, can also be used for edge detection, building extraction and road extraction. TIP (2021)

SDL-Skeleton

Our paper has been accepted by TIP(2021), the NAS part of code and the pretrained model will come soon !

SDL-Skeleton is a FREE toolbox for object skeleton detection, which also has strong adaptability to general pixel-wise binary classification tasks, such as edge detection, saliency detection, line detetection, building extraction and road extraction. This code is based on the implementation of HED and SRN.

SDL-Skeleton includes popular skeleton detection methods, including HED¹, SRN², HiFi³, DeepFlux⁴ and our newly proposed Ada-LSN⁵. Ada-LSN achieved the state-of-the-art results across all skeleton dataset, for example, we achieved 0.786 performace on sklarge dataset.

Figure 1: Skeleton detection examples.

Figure 2: Edge detection examples.

Figure 3: Building extraction examples.

Figure 4: Road extraction examples.

Requirements

• python 3
• pytorch >= 0.4
• torchvision

Pretrained models

Datasets

Skeleton Detection

Five commonly used skeleton datasets are used, including sklarge、sk506、sympascal、symmax and whsymmax. You also can download all these datasets at here, password:x9bd and revaluation code at here, password:zyqn.

The preliminary data augmentation code can be downloaded at sklarge, including resizing images to 3 scales (0.8x, 1.0x, and 1.2x), rotating for 4 directions (0◦, 90◦, 180◦,and 270◦), flipping in 2 orientations (left-to-right and up-to-down). After that, you can use resolution normalization technology (dataRN.py), which helps for skeleton detection because of their different image size.

Other tasks

We also test our methods on edge detection, building extraction and road extraction.

Usages

Skeleton Detection

Test HED and SRN by run:

python train.py --network 'hed'            # HED
python train.py --network 'srn'            # SRN
python train.py --network 'deep_flux'      # DeepFlux

At the same time, modify the saved path of the network model in engines/trainer.py. If you want to test DeepFlux, you also need to modify the data loader to use datasets/sklarge_flux.py. As for HiFi, we only implemented the network structure, with lacking the multi-scale annotation datasets.

Test Ada-LSN by run:

python train_AdaLSN.py

Our Ada-LSN supports different backbones, including VGG, ResNet, Res2Net and Inception. Simply modify the Ada_LSN/model.py to switch between different backbones. The performance of these different backbones on the sklarge dataset is as follows：

backbones	VGG	ResNet50	Res2Net	InceptionV3
F-score	0.763	0.764	0.768	0.786

Other tasks

Our Ada-LSN also can be used for other pixel-wise binary classification tasks. We archieved state-of-the-art performace in edge detection and road extraction. We think Ada-LSN is also suitable for other tasks, for example, in subsequent experiments, we found Ada-LSN also works well on building extraction. You can use our method to simply modify the data path and run:

python train_AdaLSN.py

Please refer to ODN⁶ for saliency detection and earthquake detection

Reference

• [1] S. Xie and Z. Tu, “Holistically-nested edge detection,” in IEEE ICCV, 2015

• [2] W. Ke, J. Chen, J. Jiao, G. Zhao, and Q. Ye, “SRN: side-output residual network for object symmetry detection in the wild,” in IEEE CVPR,2017.

• [3] K. Zhao, W. Shen, S. Gao, D. Li, and M. Cheng, “Hi-fi: Hierarchical feature integration for skeleton detection,” in IJCAI, 2018.

• [4] Y. Wang, Y. Xu, S. Tsogkas, X. Bai, S. J. Dickinson, and K. Siddiqi, “Deepflux for skeletons in the wild,” in IEEE CVPR, 2019.

• [5] Chang Liu*, Yunjie Tian*, Jianbin Jiao and Qixiang Ye, "Adaptive Linear Span Network for Object Skeleton Detection", https://arxiv.org/abs/2011.03972.

• [6] Chang Liu, Fang Wan, Wei Ke, Zhuowei Xiao, Yuan Yao, Xiaosong Zhang and Qixiang Ye, "Orthogonal Decomposition Network for Pixel-Wise Binary Classification", in IEEE CVPR, 2019.

• [7] Chang Liu, Wei Ke, Fei Qin and Qixiang Ye, "Linear Span Network for Object Skeleton Detection", in IEEE ECCV, 2018.