Deep Learning Pipeline for Camelyon 2016 dataset

Weizhe Li, Weijie Chen

0 - Preparation

0.1 Set up deep learning environment.

0.2 ASAP installation and image display

Compare the ASAP with OpenSlide: ASAP doesn’t have detailed manual to describe its commands; OpenSlide has a much better document for its commands. ASAP has a GUI; OpenSlide doesn’t

0.3 Mask file generation

Mask file is the ground truth for model training. Mask file has the exact same dimensions as its corresponding WSI image. Mask file is a binary file with normal tissue coded as ‘0’ and tumor tissue coded as ‘1’ for each corresponding pixel of WSI image.
Mask file generation
However, the code provided by the organizer is misleading. The mask file generated by the code from their paper is all ‘0’; then another piece of code suggested generated mask file that can be open only by ASAP GUI, not by its command line and OpenSlide.
Time consuming

WSI and Mask file: tumor_026

WSI and Mask file: tumor_005

1 - WSI Visulization with Annotation

Annotation Visulization

Annotation Visulization Over Image Base on xml file

Annotation Visulization Over Image Base on mask file

2 - Image Preprocess

2.1 Image Segmentation

To reduce computation, the blank regions (no tissue) on slide will be excluded.

Color space switch to HSV
Tissue region segmentation (Otsu’s method of foreground segmentation)

2.2 Patch Extraction

Step 1 : Randomly extract patches (256 x 256) on the tissue region at the level of 40x

	Tumor slide : 1K positive and 1K negative from each slide

	Normal slide: 1K negative from each slide

Step 2 : Crop 224x224 patches and conduct image augmentation

stain normalization (Method II)

The color variety among patches

The patches before and after stain normalization

flip
adding color noise (Method II)

Step 3 : Image Generator

Patches:

Ground Truth:

3 - Training Neural Network

3.1 FCN

Lambda, Normalize input (x / 255.0 - 0.5), outputs 256x256x3

Convolution1, 5 x 5 kernel, stride 2, outputs 128x128x100
Maxpooling1, 2 x 2 window, stride 2, outputs 64x64x100
Convolution2, 5 x 5 kernel, stride 2, outputs 32x32x200
Maxpooling2, 2 x 2 window, stride 2, outputs 16x16x200
Convolution3, 3 x 3 kernel, stride 1, outputs 16x16x300
Convolution4, 3 x 3 kernel, stride 1, outputs 16x16x300
Dropout, 0.1 rate
Convolution5, 1x1 kernel, stride 1, outputs 16x16x2
Deconvolution, 31 x 31 kernel, stride 16, outputs 256x256x2

-FCN training

FCN prediction

3.2 U-net

U-net training

Learning Curve

Prediction by trained U-net:

3.3 GoogleNet

-- step 1: Model Training

Training GoogleNet

Optimization method: Stochastic gradient descent
Weight initialization: Random sampling from a Gaussian distribution
Batch size: 32
Batch normalization: No
Regularization: L2-regularization (0.0005) and 50% dropout
Learning rate: 0.01, multiplied by 0.5 every 50,000 iterations (0.01, multiplied by 0.1 per epoch)
Activation function: ReLu
Loss function: Cross-entropy
Number of training epochs/iterations: 300,000 iterations

-- step 2: Negative Mining

Extract additional training patches from false positive regions

4 - Prediction and Evaluation

4.1 Make predictions and construct heatmaps

Test images were divided into non-overlapping small patches; each patch will get a predicted image for each pixel assigned by probability. Heatmap is a way to display the probability

Put all the patches together and get prediction for the whole slide (code for heatmap generation based on predicted values).

Results

Heatmap for tumor_026:

The overview of heatmap for tumor_026:

Comparison of predicted with ground truth for tumor_005:

4.2a Slide-based Classification

Extracting Features for whole-slide image classification task

Global Features Extraction

The ratio between the area of metastatic regions and the tissue area.
The sum of all cancer metastases probailities detected in the metastasis identification task, divided by the tissue area. caculate them at 5 different thresholds (0.5, 0.6, 0.7, 0.8, 0.9), so the total 10 global features

Local Features Extraction

Based on 2 largest metastatic candidate regions (select them based on a threshold of 0.5).

9 features were extracted from the 2 largest regions:

Area: the area of connected region
Eccentricity: The eccentricity of the ellipse that has the same second-moments as the region
Extend: The ratio of region area over the total bounding box area
Bounding box area
Major axis length: the length of the major axis of the ellipse that has the same normalized second central moments as the region
Max/mean/min intensity: The max/mean/minimum probability value in the region
Aspect ratio of the bounding box
Solidity: Ratio of region area over the surrounding convex area

code for Global and Local Features Extraction

4.2b Lesion-based Detection

Extract Patches near Tumor Regions
Training model again (model-2)
Combine Model-1 and Model-2, do prediction

4.3 ROC and FROC Generation

Teams using GoogleNet

HMS&MIT, HMS&MGH(model I), Smart Imaging(model II), Osaka University, CAMP-TUM(model II), Minsk Team, DeepCare

References

Wang, D., et al.: Deep learning for identifying metastatic breast cancer https://arxiv.org/abs/1606.05718
Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5820737/?report=reader#!po=59.4340

flamingofugang / deeplearningcamelyon Goto Github PK

deeplearningcamelyon's Introduction