m3dv / ribseg

[MICCAI'21 & TMI'23] RibSeg Dataset and Point Cloud Baselines for Rib Segmentation from CT Scans

RibSeg

Please see the RibSeg v2 paper on IEEE TMI 2023, for code of RibSeg v1, please refer to the branch ribsegv1.

Released! Welcome to use and leave comments! Please cite RibSeg v2!

RibSeg v2 dataset, description document, and annotations as mesh here.

To load the data:

seg: import nibabel as nib seg = nib.load(file name).get_fdata() # seg is a np array / volume of (512,512,N) with rib labels

cl: import numpy as np cl = np.load(file name)['cl'] # cl is a np array of (24,500,3), each rib contains 500 points

Paper (MICCAI'21) | Dataset (Zenodo)

Jiancheng Yang, Shixuan Gu, Donglai Wei, Hanspeter Pfister, Bingbing Ni

Manual rib inspections in computed tomography (CT) scans are clinically critical but labor-intensive, as 24 ribs are typically elongated and oblique in 3D volumes. Automatic rib segmentation methods can speed up the process through rib measurement and visualization. However, prior arts mostly use in-house labeled datasets that are publicly unavailable and work on dense 3D volumes that are computationally inefficient. To address these issues, we develop a labeled rib segmentation benchmark, named RibSeg, including 490 CT scans (11,719 individual ribs) from a public dataset. For ground truth generation, we used existing morphology-based algorithms and manually refined its results. Then, considering the sparsity of ribs in 3D volumes, we thresholded and sampled sparse voxels from the input and designed a point cloud-based baseline method for rib segmentation. The proposed method achieves state-of-the-art segmentation performance (Dice≈95%) with significant efficiency (10∼40× faster than prior arts).

Dataset

The RibSeg Dataset contains annotations for both rib segmentation and centerline.

Rib Segmentation	Rib Centerline

Overview of RibSeg dataset:

Subset	No. of CT Scans	No. of Individual Ribs
Training	320	7,670
Development	50	1,187
Test	120	2,862

Model Training (RibSeg v1)

For training data, please download the source CT scans from RibFrac Dataset to ./data/ribfrac directory:

Download

For the source CT scans, please refer to the MICCAI 2020 RibFrac Challenge on grand-challenge.org (click Join first).

For annotations, please download RibSeg dataset to ./data/ribseg directory on Zenodo.

Data Preparation

run data_prepare.py to create data for training.

Based on RibFrac dataset and RibSeg dataset, we binarized the CT scans and the annotations for rib segmentation to ./data/pn for the convenience of training PointNet++.

Model

You can train your model through the following command line:

python train_ribseg.py --model pointnet2_part_seg_msg --log_dir <model_directory>

You can test your model through the following command line:

python test_ribseg.py --log_dir <model_directory>

You can conduct inference through the following command line:

python inference.py --log_dir <model_directory>

You can run our model through the following command line:

python inference.py --log_dir c2_a

You can get the volume-wise test result through the following command line:

python post_proc.py

Citation

If you find this project useful, please cite our paper as:

Liang Jin, Shixuan Gu, Donglai Wei, Jason Ken Adhinarta, Kaiming Kuang, Yongjie Jessica Zhang, Hanspeter Pfister, Bingbing Ni, Jiancheng Yang, Ming Li. "RibSeg v2: A Large-scale Benchmark for Rib Labeling and Anatomical Centerline Extraction". IEEE Transactions on Medical Imaging (TMI), 2023.

Jiancheng Yang, Shixuan Gu, Donglai Wei, Hanspeter Pfister, Bingbing Ni. "RibSeg Dataset and Strong Point Cloud Baselines for Rib Segmentation from CT Scans". International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2021.

or using bibtex:

@article{ribsegv2,
  title={RibSeg v2: A Large-scale Benchmark for Rib Labeling and Anatomical Centerline Extraction},
  author={Jin, Liang and Gu, Shixuan and Wei, Donglai and Adhinarta, Jason Ken and Kuang, Kaiming and Zhang, Yongjie Jessica and Pfister, Hanspeter and Ni, Bingbing and Yang, Jiancheng and Li, Ming},
  journal={IEEE Transactions on Medical Imaging (TMI)},
  year={2023},
  publisher={IEEE}
}

@inproceedings{ribsegv1,
  title={RibSeg Dataset and Strong Point Cloud Baselines for Rib Segmentation from CT Scans},
  author={Yang, Jiancheng and Gu, Shixuan and Wei, Donglai and Pfister, Hanspeter and Ni, Bingbing},
  booktitle={International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)},
  pages={611--621},
  year={2021},
  organization={Springer}
}

ribseg's Issues

train_rigseg.py labels wierds

Hi,
I found these labels in train_rigseg.py file:

seg_classes = {'rib':[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24],'Earphone': [44, 45], 'Motorbike': [30, 31, 32, 33, 34, 35], 'Rocket': [41, 42, 43],
'Car': [29], 'Laptop': [28 ], 'Cap': [26], 'Skateboard': [46],
'Mug': [36], 'Guitar': [39, 40], 'Bag': [27], 'Lamp': [25],
'Table': [47], 'Airplane': [48], 'Pistol': [38],
'Chair': [37], 'Knife': [49]}

In each Epoch a have these result:

eval mIoU of Airplane nan
eval mIoU of Bag nan
eval mIoU of Cap nan
eval mIoU of Car nan
eval mIoU of Chair nan
eval mIoU of Earphone nan
eval mIoU of Guitar nan
eval mIoU of Knife nan
eval mIoU of Lamp nan
eval mIoU of Laptop nan
eval mIoU of Motorbike nan
eval mIoU of Mug nan
eval mIoU of Pistol nan
eval mIoU of Rocket nan
eval mIoU of Skateboard nan
eval mIoU of Table nan
eval mIoU of rib nan

How to use this code on ribfrac dataset for five kinds of rib segmentation

Hello, I want to change the ribseg code for the segmentation of five rib fracture types in the ribfrac data set. What are the main changes? thank you

How to number the ribs

Please tell me how to number the ribs after they are divided. For example, the first root and the second root on the left, can you give me some suggestions

Why is there a scapula in the result of my segmentation?

problem with data_prepare.py

Hi,
I found this error: NameError: name 'points' is not defined

I found that you put in comment :

choice = np.random.choice(temp.shape[0], 30000, replace=False)

downsample

points = temp[choice, :]

Why ?

issue in inference.py code branch ribsegv1, np.load both data and seg are same?

raise ValueError("Cannot load file containing pickled data "
ValueError: Cannot load file containing pickled data when allow_pickle=False

Labels missing for several training images

The ribseg labels appear to be missing for many of the ribfrac images.

This is causing data_prepare.py to fail with errors such as:

$ python data_prepare.py 
Traceback (most recent call last):
  File "/home/david/miniconda3/envs/RibSeg/lib/python3.7/site-packages/nibabel/loadsave.py", line 42, in load
    stat_result = os.stat(filename)
FileNotFoundError: [Errno 2] No such file or directory: './data/RibSeg/nii/RibFrac43-rib-seg.nii.gz'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "data_prepare.py", line 102, in <module>
    main()
  File "data_prepare.py", line 20, in main
    label = nib.load('./data/RibSeg/nii/'+data[:-12]+'rib-seg.nii.gz')
  File "/home/david/miniconda3/envs/RibSeg/lib/python3.7/site-packages/nibabel/loadsave.py", line 44, in load
    raise FileNotFoundError("No such file or no access: '%s'" % filename)
FileNotFoundError: No such file or no access: './data/RibSeg/nii/RibFrac43-rib-seg.nii.gz'

Label Accuracy is counted by dice threshold

Hello!

RibSeg/segmentation/metrics.py

Line 53 in 5a0d496

if dice_temp > 0.7:

- this line means that you are using dice threshold to calculate Label Accuracy. But in the article (https://arxiv.org/abs/2210.09309) says that the Label Accuracy is calculated by the Recall threshold (Section V ). Which version is correct?

Some error when I use post_proc.py

Hi,
Thanks for your contribution, I'm trying to reproduce your results and get a nibabel file to see the 3D segmentation.
However, when I try this code:
dilated = sitk.BinaryDilate(lmage_array, 3, sitk.sitkBall)
It raises a TypeError: in method 'BinaryDilate', argument 2 of type 'std::vector< unsigned int,std::allocator< unsigned int > >', do you know why?
The version of my SimpleITK is 2.0.2, hope to get your reply soon.

An error occurred while training the model“ModuleNotFoundError: No module named 'models.CLNet'”

Traceback (most recent call last):
File "e:/code/RibSeg/train_ribseg.py", line 204, in
main(args)
File "e:/code/RibSeg/train_ribseg.py", line 71, in main
MODEL = importlib.import_module('models.'+args.model)
File "D:\Anaconda\envs\pytorch\lib\importlib_init_.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'models.CLNet'

Dataset problem

Excuse me, is there a label file for the test set. I only downloaded the test set image file from ribfrac challenge, but there is no test set label file. Can I provide relevant files, thank you!!

data_prepare.py seems to be the binary version seg_classes = {'rib':[0,1]}

Hi,

I didn't find where you use the centerlines files to prepare the data. In data_prepare.py file you only read *-rib-seg.nii, in this way your problem is a binary segmentation, the label is rib = 1.

I didn't find where you go from a segmented binary volume (*-rib-seg.nii) to 24 rib labels, without using the information from *-rib-cl.nii files.

during training and inference

ValueError: Cannot load file containing pickled data when allow_pickle=False

Please tell me how do you achieve the indicators of center line evaluation？

First of all, thank you very much for your excellent work. I would like to ask how do you achieve the indicators of center line evaluation? The only metrics I found in the code were for splitting.

About the ribseg dataset

Hello, I want to locate the location of my rib fracture (for example, the first one on the left or the fifth one on the right). Therefore, I need a CT file with 24 rib segmentation masks, calculate its bbox coordinates according to the mask marks of each rib, and then match it with the bbox coordinates of the rib fracture I detected, so as to locate. Can the CT of the ribseg dataset achieve my goal?

forward的时候报错，尚未找到原因，大家碰到过吗

1.运行forward报错
‘RibSeg-main/models/pointnet2_part_seg_msg.py", line 45, in forward
cls_label_one_hot = cls_label.view(B,16,1).repeat(1,1,N)
RuntimeError: shape '[8, 16, 1]' is invalid for input of size 8
cls_label的shape是[8,1,1]

改成 cls_label_one_hot = cls_label.view(B,1,1).repeat(1,16,N)
可运行，但不知是否正确

2.data_prepare中采样固定采样数目为30000
choice = np.random.choice(temp.shape[0], 30000, replace=False)
没不是全部输出所有样本，这样data_prepare生成的样本只有30000个点，dataloader中的重采样也没有意义了

Visualization of prediction results

Hello, I want to convert the post-processing results into NII files. Here is my conversion program and visualization, but the visualization result is not good. Can you give me some advice? I'm looking forward to your reply