pascalcpp / cmtgf

a cascaded multi-task generative framwork for dectecting aortic dissection using non-contrast enhanced computed tomography

Introduction

Contrast-enhanced computed tomography (CE-CT) is the gold standard for diagnosing aortic dissection (AD). However, contrast agents can cause allergic reactions or renal failure in some patients. Moreover, AD diagnosis by radiologists using non-contrast-enhanced CT (NCE-CT) images has poor sensitivity. To address this issue, we propose a novel cascaded multi-task generative framework for AD detection using NCE-CT volumes.

The framework includes a 3D nnU-Net and a 3D multi-task generative architecture (3D MTGA). Specifically, the 3D nnU-Net was employed to segment aortas from NCE-CT volumes. The 3D MTGA was then employed to simultaneously synthesize CE-CT volumes, segment true & false lumen, and classify the patient as AD or non-AD. It has great potential to reduce the misdiagnosis of AD using NCE-CT in clinical practice.

#--------------------Normal--------------------##---------------------- AD----------------------#

[Input NCE-CT] | [Real CE-CT] | [Fake CE-CT] & [Input NCE-CT] | [Real CE-CT] | [Fake CE-CT]

Multi-task Generative Architecture in PyTorch

We provide PyTorch implementations for Multi-task Generative Architecture (MTGA).

The code was inspired by Pix2pix and modified by Xiang-Yu Xiong.

Note: The current software works well with PyTorch 1.4.0+. Check out the older branch that supports PyTorch 0.4.

You may find useful information in training/test tips and frequently asked questions.

A Cascaded multi-task generative framework (CMTGF) for Aortic Dissection Detection: Paper

The 3D U-Net was a older nnU-Net created by FabianIsensee, it was used to segment aortic mask from NCE-CT.

The following 3D MTGA was created by Xiang-Yu Xiong, it contains a generator, a discriminator and a classifier.

Prerequisites

• Linux or Windows (Pycharm + Anaconda)
• Python 2 or 3
• CPU or NVIDIA GPU + CUDA CuDNN

Getting Started

Installation

• Install PyTorch 1.4.0+ and torchvision from http://pytorch.org and other dependencies (e.g., visdom and dominate). You can install all the dependencies by

pip install -r requirements.txt

• Clone this repo:

git clone https://github.com/yXiangXiong/CMTGF
cd CMTGF

• For Anaconda users, you can use pip to install PyTorch and other libraries.

MTGA train/test

• Train the MTGA:

python train.py --dataroot D:\...\Random_AorticData --name aortas_nce2ce --model paired_pix2pix3d --netC CNN_3D --dataset_mode aligned --input_nc 1 --output_nc 2 --gpu_ids 0 --batch_size 1 --netG unet_256 --no_html --pool_size 0 --norm batch --no_flip --print_freq 10 --lambda_C 1 --lambda_L1 200 --display_id -1

• Test the MTGA:

python test.py --dataroot D:\...\Random_AorticData --name aortas_nce2ce --netC CNN_3D --model paired_pix2pix3d --netG unet_256 --dataset_mode aligned --input_nc 1 --output_nc 2 --norm batch --gpu_ids 0 --how_many 32 --eval --which_epoch 200

[Datasets]

create a directory below and add your own datasets.

all 3D images have same size of 256x256x512.

Random_AorticData：
|─trainA
│      001_plain.nii.gz # non-contrast enhanced images of aorta
│      002_plain.nii.gz
│      003_plain.nii.gz
│      ...
│      
├─trainB
│      001_enhance.nii.gz # contrast enhanced images of aorta
│      002_enhance.nii.gz
│      003_enhance.nii.gz
│      ...
│      
├─trainC
│      001_aorta.nii.gz # annotated true and false lumen masks by radiologists
│      002_aorta.nii.gz
│      003_aorta.nii.gz
│      ...
│      
├─testA
│      004_plain.nii.gz # non-contrast enhanced images of aorta
│      005_plain.nii.gz
│      ...
│      
├─testB
│      004_enhance.nii.gz # contrast enhanced images of aorta
│      005_enhance.nii.gz
│      ...
│      
├─testC
│      004_aorta.nii.gz # annotated true and false lumen masks by radiologists
│      005_aorta.nii.gz
│      ...
│      
└─ground_truth_classification.csv
# csv file has two columes: first is xxx.plain.nii.gz and second is 0/1. 0 mean normal and 1 mean aortic dissection.

Training/Test Tips

Best practice for training and testing your models.

Citation

If you are interested in this project and use this code for your research, please cite our papers.

@inproceedings{xiong2021CDLF,
  author    = {Xiangyu Xiong, Xiuhong Guan, Chuanqi Sun, Tianjing Zhang, Hao Chen, Yan Ding, Zhangbo Cheng, Lei Zhao, Xiaohai Ma, Guoxi Xie},
  title     = {A Cascaded Deep Learning Framework for Detecting Aortic Dissection
               Using Non-contrast Enhanced Computed Tomography},
  booktitle = {43rd Annual International Conference of the {IEEE} Engineering in
               Medicine {\&} Biology Society, {EMBC} 2021, Mexico, November 1-5,
               2021},
  year      = {2021},
}

@ARTICLE{xiong2022CMTGF,
  author={Xiong, Xiangyu and Ding, Yan and Sun, Chuanqi and Zhang, Zhuoneng and Guan, Xiuhong and Zhang, Tianjing and Chen, Hao and Liu, Hongyan and Cheng, Zhangbo and Zhao, Lei and Ma, Xiaohai and Xie, Guoxi},
  journal={IEEE Journal of Biomedical and Health Informatics}, 
  title={A Cascaded Multi-Task Generative Framework for Detecting Aortic Dissection on 3D Non-contrast-enhanced Computed Tomography},
  year={2022}
}

Related Projects

CycleGAN-Torch | pix2pixHD | EC-GAN | BicycleGAN

Aortic Dissection Detection (Non-contrast Enhanced CT) Papers Collection

2022 Frontiers in Cardiovascular Medicine. Advanced Warning of Aortic Dissection on Non-Contrast CT: The Combination of Deep Learning and Morphological Characteristics.

2021 IEEE EMBC. A Cascaded Deep Learning Framework for Detecting Aortic Dissection Using Non-contrast Enhanced Computed Tomography.

2020 Medical Image Analysis. Detection, segmentation, simulation and visualization of aortic dissections: A review.

2020 European Radiology. Deep learning algorithm for detection of aortic dissection on non-contrast-enhanced CT.

Acknowledgments

Our code is inspired by pytorch-pix2pix.