GithubHelp home page GithubHelp logo

deadskull7 / one-stop-for-covid-19-infection-and-lung-segmentation-plus-classification Goto Github PK

View Code? Open in Web Editor NEW
69.0 1.0 26.0 35.37 MB

βœ‹πŸΌπŸ›‘ This one stop project is a complete COVID-19 detection package comprising of 3 tasks: β€’ Task 1 --> COVID-19 Classification β€’ Task 2 --> COVID-19 Infection Segmentation β€’ Task 3 --> Lung Segmentation

License: MIT License

Jupyter Notebook 99.51% Python 0.49%
covid-19 machine-learning deep-neural-networks segmentation classification image-processing computer-vision convolutional-neural-networks unet-image-segmentation threshold

one-stop-for-covid-19-infection-and-lung-segmentation-plus-classification's Introduction

βœ‹πŸΌπŸ›‘ One-Stop-for-COVID-19-Infection-and-Lung-Segmentation-plus-Classification

The project is a complete COVID-19 detection package comprising of 3 tasks:

➀ Task 1: COVID-19 Infection Segmentation
➀ Task 2: COVID-19 Classification
➀ Task 3: Lung Segmentation

Sample data

-----------------------------------------------------------------------------------------------------------------------------------------

βž₯ Some Instructions and Guidelines for Code Execution

  1. All scripts are exactly the same with the notebooks having same titles.
  2. Any necessary package to be installed is placed on top of each of the six scripts/notebooks.
  3. Scripts/Notebooks are using kaggle and Colab collectively, so will contain some exclusive operations as per the platform like importing data from kaggle to colab needs a unique API key, mounting drive and authentication, etc

-----------------------------------------------------------------------------------------------------------------------------------------

βž₯ Following are the details about the scripts/notebooks

β˜… task1_preprocessing_plus_unet_with_comments.py ---> Contains the maximum number of comments and explanation. Any doubt, if persists probably could be rectified here. It contains the UNet training for infection mask prediction (Task- 1)

β˜… task1_crossval_3folds_unet.py --> Contains the cross-validation (3-folds) for TASK-1.

β˜… task1_crossval_4folds_unet.py --> Contains the cross-validation (4-folds) for TASK-1.

β˜… task1_unet_plus_plus.py --> Contains the unet++ training for TASK-1.

β˜… task2_covid19_classifcation.py --> Contains the covid-19 classification (TASK-2).

β˜… task3_lung_segmentation_unet.py --> Contains Unet training for lung segmentation (TASK-3).

β˜… app.py --> Contains a runner code for running any file listed above.

# app.py

from task1_crossval_3folds_unet import *
from task1_crossval_4folds_unet import *
from task1_preprocessing_plus_unet_with_comments import *
from task1_unet_plus_plus import *
from task2_covid19_classifcation import *
from task3_lung_segmentation_unet import *


print("\n\n\n\n")
print("--------------------------------------------------------------------------------------")
print(" 'one' --> Task1: 3-fold cross-validation UNet (Infection Segmentation)")
print(" 'two' --> Task1: 4-fold cross-validation UNet (Infection Segmentation)")
print(" 'three' --> Task1: UNet original holdout method (Infection Segmentation)")
print(" 'four' --> Task1: UNet++ holdout method (Infection Segmentation)")
print(" 'five' --> Task2: COVID-19 Classification")
print(" 'six' --> Task3: Lung Segmentation")
print("--------------------------------------------------------------------------------------")
print("\n\n\n\n\n")


print("Enter from one of the {'one', 'two', 'three', 'four', 'five', 'six', 'seven'}")
num = input()


if num == 'one':
	three_fold_runner_unet_infection_segmentation()
    

if num == 'two':
	four_fold_runner_unet_infection_segmentation()


if num == 'three':
	holdout_runner_unet_infection_segmentation()


if num == 'four':
	holdout_runner_unetplusplus_infection_segmentation()


if num == 'five':
	runner_classification()


if num == 'six':
	runner_lung_segmentation()

-----------------------------------------------------------------------------------------------------------------------------------------

Note: Not all are displayed here, rest could be fetched after running notebooks

RESULT TABLE
+----------------------------------------+----------------------------+-----------------------+---------------------------+------------------------+--------+--------------+----------+
|                                        |    Dice (Mean of folds)    |  IOU (Mean of folds)  | Precision (Mean of folds) | Recall (Mean of folds) | AUCROC |   F1 Score   | Accuracy |
+----------------------------------------+----------------------------+-----------------------+---------------------------+------------------------+--------+--------------+----------+
| Task1: Infection Segmentation (3-fold) |            0.948           |         0.903         |           0.947           |         0.950          |    -   | same as dice |     -    |
+----------------------------------------+----------------------------+-----------------------+---------------------------+------------------------+--------+--------------+----------+
| Task1: Infection Segmentation (4-fold) |            0.956           |         0.917         |           0.955           |         0.958          |    -   | same as dice |     -    |
+----------------------------------------+----------------------------+-----------------------+---------------------------+------------------------+--------+--------------+----------+
|          Task2: Classification         |              -             |           -           |           0.987           |          0.989         |  0.998 |     0.988    |   0.982  |
+----------------------------------------+----------------------------+-----------------------+---------------------------+------------------------+--------+--------------+----------+
|        Task3: Lung Segmentation        |            0.984           |         0.969         |             -             |            -           |    -   | same as dice |     -    |
+----------------------------------------+----------------------------+-----------------------+---------------------------+------------------------+--------+--------------+----------+
|                                        | Note: precision and recall | values are as per the | best threshold for dice   |                        |        |              |          |
+----------------------------------------+----------------------------+-----------------------+---------------------------+------------------------+--------+--------------+----------+

-----------------------------------------------------------------------------------------------------------------------------------------

Preprocessing Stage

1.) Removing incomplete and fauty images
2.) Separate model for empty mask prediction
3.) Use of Contrast Limited Adaptive Histogram Equalization ( ) for image enhancement
4.) Cropping the Region of Interst (ROI) using Otsu's binarization and other approaches
5.) Data Augmentation

Cropping out contour with the largest area
CLAHE only Output Before-After Image (after all preprocessing steps)
Augmentated CT Scans (Task: 1)
Augmented Infections Masks (Task: 1)
Augmentated CT Scans (Task: 3)
Augmented Lung Masks (Task: 3)
DICE v/s IOU and some relationship
β€’ Dice (S) = 2|𝐴∩𝐡|)/(|𝐴| + |𝐡|) = 2𝑇𝑃/(2𝑇𝑃+𝐹𝑃+𝐹𝑁)
β€’ IOU (J) = (|𝐴∩𝐡|)/(|𝐴βˆͺ𝐡|) = 𝑇𝑃/(𝑇𝑃+𝐹𝑃+𝐹𝑁)
β€’ J = S/(S-2)
β€’ πœ•π½/πœ•π‘† = 2/[(2 βˆ’π‘ )]^2

β€’ Interpretations:

β€’ We can say that IOU(J) is always more punishing or has less value than the corresponding dice(S) at the same threshold.

β€’ Another takeaway is that πœ•π½/πœ•π‘† = 2/[(2 βˆ’π‘ )]^2 which is basically the slope and is a continuous increasing function for S ∈ [0,1].

β€’ For (S = 0.586, J = 0.414), the value of slope equals to 1.

β€’ Above two points combinedly establishes a relationship that for all S > 0.586, rate of increase of IOU is greater than the dice whereas for all S < 0.586, the rate of increase of dice is greater than the IOU.

Training Stage

(Remains same for all tasks)

Exponential decaying LR (Step and continuous) Exponential decaying LR (Step and continuous) with Variation
Cosine Annealing Cyclical LR Blend of Cosine Annealing and Exponential Decay

Results with UNet (Task: 1)

Trianing curve for Dice Coefficient Training curve for BCE + Dice Loss
Optimizing threshold with small step size
Precision and recall curves v/s thresholds
Some Actual Vs Predicted Masks Some Actual Vs Predicted Masks

4-Fold Cross-validation Results on Task: 1 (UNet)

4-fold threshold vs split number dataframe for DICE Brief report acquired from dataframe
4-fold threshold vs split number dataframe for IOU Brief report acquired from dataframe
4-fold threshold vs split number dataframe for PRECISION Brief report acquired from dataframe
4-fold threshold vs split number dataframe for RECALL Brief report acquired from dataframe
Some Actual v/s Predicted Masks by 4 Unet models of 4-fold Cross-Validation

Results with UNet++ (Task: 1)

Training curve for Dice Coefficient Training curve for BCE + Dice Loss
Optimizing threshold with small step size

Results with CNN (Task: 2)

Classification loss curve
Distribution of TN, TP, FN, FP with Threshold 0.50 Distribution of TN, TP, FN, FP with Best Threshold 0.81
ROC Curve with Threshold 0.50 ROC Curve with Threshold 0.81
Confusion matrix with Threshold 0.50 Confusion matrix with Best Threshold 0.81

Results with UNet (Task: 3)

Training curve for Dice Coefficient Training curve for BCE + Dice Loss
Optimizing threshold with small step size
Actual v/s Predicted Lung Masks Actual v/s Predicted Lung Masks

License

MIT License

Copyright (c) 2020 Rohit Verma

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

one-stop-for-covid-19-infection-and-lung-segmentation-plus-classification's People

Contributors

deadskull7 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

one-stop-for-covid-19-infection-and-lung-segmentation-plus-classification's Issues

unetpp dataset not find?

Hi~ I have tried your notebook in this repo. When downloading the datasets, I got a http 404 error in kaggle datasets download -d deadskull7/unetpp.

It seems the dataset have been removed or made private, how can I find it?

BinaryClassification Function

Hi Rohit,

Thank you for your great work, I'm a beginner to python, in "task2_covid19_classifcation.py", the BinaryClassification was used but not defined before. Can you please explain this?

Best Regards

bc1 = BinaryClassification(y_valid, predictions, labels=["Class 0", "Class 1"], threshold = 0.50)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.