lfz / dsb2017

The solution of team 'grt123' in DSB2017

Paper link

https://arxiv.org/abs/1711.08324 Please cite this paper if you find this project helpful for your research.

Dependencies

Ubuntu 14.04, python 2.7, CUDA 8.0, cudnn 5.1, h5py (2.6.0), SimpleITK (0.10.0), numpy (1.11.3), nvidia-ml-py (7.352.0), matplotlib (2.0.0), scikit-image (0.12.3), scipy (0.18.1), pyparsing (2.1.4), pytorch (0.1.10+ac9245a) (anaconda is recommended)

This is my configuration, I am not sure about the compatability of other versions

Instructions for runing

Testing

1. unzip the stage 2 data
2. go to root folder
3. open config_submit.py, filling in datapath with the stage 2 data path
4. python main.py
5. get the results from prediction.csv

if you have bug about short of memory, set the 'n_worker_preprocessing' in config_submit.py to a int that is smaller than your core number.

Training

1. Install all dependencies
2. Prepare stage1 data, LUNA data, and LUNA segment results (https://luna16.grand-challenge.org/download/), unzip them to separate folders
3. Go to ./training and open config_training.py
4. Filling in stage1_data_path, luna_raw, luna_segment with the path mentioned above
5. Filling in luna_data, preprocess_result_path, with tmp folders
6. bash run_training.sh and wait for the finishing of training (it may take several days)

If you do not have 8 GPUs or your the memory of your GPUs is less than 12 GB, decrease the number of -b and -b2 in run_training.sh, and modify the 'CUDA_VISIBLE_DEVICES=0,1,..,n_your_gpu'. The time of training is very long (3~4 days with 8 TITANX).

Brief Introduction to algorithm

Extra Data and labels: we use LUNA16 as extra data, and we manually labeled the locations of nodules in the stage1 training dataset. We also manually washed the label of LUNA16, deleting those that we think irrelavent to cancer. The labels are stored in ./training./detector./labels.

The training involves four steps

1. prepare data

   All data are resized to 1x1x1 mm, the luminance is clipped between -1200 and 600, scaled to 0-255 and converted to uint8. A mask that include the lungs is calculated, luminance of every pixel outside the mask is set to 170. The results will be stored in 'preprocess_result_path' defined in config_training.py along with their corresponding detection labels.

2. training a nodule detector

   in this part, a 3d faster-rcnn is used as the detector. The input size is 128 x 128 x 128, an online hard negative sample mining method is used. The network structure is based on U-net.

3. get all proposals

   The model trained in part 2 was tested on all data, giving all suspicious nodule locations and confidences (proposals)

4. training a cancer classifier

   For each case, 5 proposals are samples according to its confidence, and for each proposal a 96 x 96 x 96 cubes centered at the proposal center is cropped.

   These proposals are fed to the detector and the feature in the last convolutional layer is extracted for each proposal. These features are fed to a fully-connected network and a cancer probability $P_i$ is calculated for each proposal. The cancer probability for this case is calculated as:

   $P = 1-(1-P_d)\Pi(1-P_i)$,

   where the $P_d$ stand for the probability of cancer of a dummy nodule, which is a trainable constant. It account for any possibility that the nodule is missed by the detector or this patient do not have a nodule now. Then the classification loss is calculated as the cross entropy between this $P$ and the label.

   The second loss term is defined as: $-\log(P)\boldsymbol{1}(y_{nod}=1 & P<0.03)$, which means that if this proposal is manually labeled as nodule and its probability is lower than 3%, this nodule would be forced to have higher cancer probability. Yet the effect of this term has not been carefully studied.

   To prevent overfitting, the network is alternatively trained on detection task and classification task.

The network archetecture is shown below

dsb2017's Issues

我不清楚数据预处理这段理解对不对？

你好，lfz。我不知道你们对数据预处理具体怎么做的，下面是我的理解，不知道和你们理解的一不一样。如果不一样望指出区别！
原文：
All data are resized to 1x1x1 mm, the luminance is clipped between -1200 and 600, scaled to 0-255 and converted to uint8. A mask that include the lungs is calculated, luminance of every pixel outside the mask is set to 170.
（1）将体素转变为1x1x1
（2）保留原图像上-1200到600区间内的亮度值，将小于-1200的值全都变成-1200，大于600的值全都变成600？
（3）然后将图像的体素值同比放缩到0-255，然后使用uint8将体素中的小数转成整数
（4）找出肺部区域，最后对肺部之外的体素值全都设为170

A typo or an explicit design in detect net's 5-dimension output?（大神求回复）

I found there seems a typo there

        self.output = nn.Sequential(nn.Conv3d(self.featureNum_back[0], 64, kernel_size = 1),
                                    nn.ReLU(),
                                    #nn.Dropout3d(p = 0.3),
                                   nn.Conv3d(64, 5 * len(config['anchors']), kernel_size = 1))

To my limited knownledge, 1 out of the 5-dimension output should be the probability of a region proposal, which should be in [0,1], ususally we use sigmoid. Other 4 should be the bbox regressor, the linear activation is suitable.

So I'm wondering whether it's just typo, or your explicit design?

Hardware for prediction?

Is it also necessary to have a GPU with at least 12 GB of memory in order to predict the tumor or is it possible to use a normal desktop GPU?
Do you know how long it would take to predict the cancer probability for one 3D scan (with which GPU/CPU)?
I can't install CUDA so I'm not able to figure that out myself...
Thanks a lot!

Initialize weights?

It seems this project uses the default weight initialization method. But as I know, sometimes weight initialization plays a very important role, so how to initialize the model parameters?
I have learned some methods to initialize simple models, but it's not easy to initialize this model since it contains many PostRes models.

what is the minimal memory requirement ?

when I run CUDA_VISIBLE_DEVICES=0,1,2 python main.py --model res18 -b 4 --resume results/res18/$eps.ckpt --test 1 in 'run_training.sh',the program crashed. My computer has 32g memory,is it enough?

About the role of lung_segment file?

Hello, I'd like to replace the data for training, but I don't know "seg - lungs - LUNA16" files in the image the effect of preparation process.thank you

What does the loss layer mean?

Could you please tell me some details?

what does LabelMapping and select_samples do?

I guess that LabelMapping is trying to transfer the ground truth label "bboexs" to a "label map", the size of which is 32 * 32 * 32 * 3 * 5. But I can't understand the details in the LabelMapping function. Could you please explain it? And what does "anchors" do ?

class LabelMapping(object):
    def __init__(self, config, phase):
        self.stride = np.array(config['stride'])
        self.num_neg = int(config['num_neg'])
        self.th_neg = config['th_neg']
        self.anchors = np.asarray(config['anchors'])
        self.phase = phase
        if phase == 'train':
            self.th_pos = config['th_pos_train']
        elif phase == 'val':
            self.th_pos = config['th_pos_val']

            
    def __call__(self, input_size, target, bboxes):
        stride = self.stride
        num_neg = self.num_neg
        th_neg = self.th_neg
        anchors = self.anchors
        th_pos = self.th_pos
        
        output_size = []
        for i in range(3):
            assert(input_size[i] % stride == 0)
            output_size.append(input_size[i] / stride)
        
        label = -1 * np.ones(output_size + [len(anchors), 5], np.float32)
        offset = ((stride.astype('float')) - 1) / 2
        oz = np.arange(offset, offset + stride * (output_size[0] - 1) + 1, stride)
        oh = np.arange(offset, offset + stride * (output_size[1] - 1) + 1, stride)
        ow = np.arange(offset, offset + stride * (output_size[2] - 1) + 1, stride)

        for bbox in bboxes:
            for i, anchor in enumerate(anchors):
                iz, ih, iw = select_samples(bbox, anchor, th_neg, oz, oh, ow)
                label[iz, ih, iw, i, 0] = 0

        if self.phase == 'train' and self.num_neg > 0:
            neg_z, neg_h, neg_w, neg_a = np.where(label[:, :, :, :, 0] == -1)
            neg_idcs = random.sample(range(len(neg_z)), min(num_neg, len(neg_z)))
            neg_z, neg_h, neg_w, neg_a = neg_z[neg_idcs], neg_h[neg_idcs], neg_w[neg_idcs], neg_a[neg_idcs]
            label[:, :, :, :, 0] = 0
            label[neg_z, neg_h, neg_w, neg_a, 0] = -1

        if np.isnan(target[0]):
            return label
        iz, ih, iw, ia = [], [], [], []
        for i, anchor in enumerate(anchors):
            iiz, iih, iiw = select_samples(target, anchor, th_pos, oz, oh, ow)
            iz.append(iiz)
            ih.append(iih)
            iw.append(iiw)
            ia.append(i * np.ones((len(iiz),), np.int64))
        iz = np.concatenate(iz, 0)
        ih = np.concatenate(ih, 0)
        iw = np.concatenate(iw, 0)
        ia = np.concatenate(ia, 0)
        flag = True 
        if len(iz) == 0:
            pos = []
            for i in range(3):
                pos.append(max(0, int(np.round((target[i] - offset) / stride))))
            idx = np.argmin(np.abs(np.log(target[3] / anchors)))
            pos.append(idx)
            flag = False
        else:
            idx = random.sample(range(len(iz)), 1)[0]
            pos = [iz[idx], ih[idx], iw[idx], ia[idx]]
        dz = (target[0] - oz[pos[0]]) / anchors[pos[3]]
        dh = (target[1] - oh[pos[1]]) / anchors[pos[3]]
        dw = (target[2] - ow[pos[2]]) / anchors[pos[3]]
        dd = np.log(target[3] / anchors[pos[3]])
        label[pos[0], pos[1], pos[2], pos[3], :] = [1, dz, dh, dw, dd]
        return label        

def select_samples(bbox, anchor, th, oz, oh, ow):
    z, h, w, d = bbox
    max_overlap = min(d, anchor)
    min_overlap = np.power(max(d, anchor), 3) * th / max_overlap / max_overlap
    if min_overlap > max_overlap:
        return np.zeros((0,), np.int64), np.zeros((0,), np.int64), np.zeros((0,), np.int64)
    else:
        s = z - 0.5 * np.abs(d - anchor) - (max_overlap - min_overlap)
        e = z + 0.5 * np.abs(d - anchor) + (max_overlap - min_overlap)
        mz = np.logical_and(oz >= s, oz <= e)
        iz = np.where(mz)[0]
        
        s = h - 0.5 * np.abs(d - anchor) - (max_overlap - min_overlap)
        e = h + 0.5 * np.abs(d - anchor) + (max_overlap - min_overlap)
        mh = np.logical_and(oh >= s, oh <= e)
        ih = np.where(mh)[0]
            
        s = w - 0.5 * np.abs(d - anchor) - (max_overlap - min_overlap)
        e = w + 0.5 * np.abs(d - anchor) + (max_overlap - min_overlap)
        mw = np.logical_and(ow >= s, ow <= e)
        iw = np.where(mw)[0]

        if len(iz) == 0 or len(ih) == 0 or len(iw) == 0:
            return np.zeros((0,), np.int64), np.zeros((0,), np.int64), np.zeros((0,), np.int64)
        
        lz, lh, lw = len(iz), len(ih), len(iw)
        iz = iz.reshape((-1, 1, 1))
        ih = ih.reshape((1, -1, 1))
        iw = iw.reshape((1, 1, -1))
        iz = np.tile(iz, (1, lh, lw)).reshape((-1))
        ih = np.tile(ih, (lz, 1, lw)).reshape((-1))
        iw = np.tile(iw, (lz, lh, 1)).reshape((-1))
        centers = np.concatenate([
            oz[iz].reshape((-1, 1)),
            oh[ih].reshape((-1, 1)),
            ow[iw].reshape((-1, 1))], axis = 1)
        
        r0 = anchor / 2
        s0 = centers - r0
        e0 = centers + r0
        
        r1 = d / 2
        s1 = bbox[:3] - r1
        s1 = s1.reshape((1, -1))
        e1 = bbox[:3] + r1
        e1 = e1.reshape((1, -1))
        
        overlap = np.maximum(0, np.minimum(e0, e1) - np.maximum(s0, s1))
        
        intersection = overlap[:, 0] * overlap[:, 1] * overlap[:, 2]
        union = anchor * anchor * anchor + d * d * d - intersection

        iou = intersection / union

        mask = iou >= th
        #if th > 0.4:
         #   if np.sum(mask) == 0:
          #      print(['iou not large', iou.max()])
           # else:
            #    print(['iou large', iou[mask]])
        iz = iz[mask]
        ih = ih[mask]
        iw = iw[mask]
        return iz, ih, iw

Access to trained models

Hi. Would you be able to provide access to your trained models? Thanks!

Can I use the reposity to process the LIDC-IDRI dataset?

Hi LFZ,

It seems the LIDC-IDRI has the same format as the kaggle dataset. Do you think the usage has some problems such as annotation difference, data format difference according to your expertise on the medical imaging processing?

Thanks a lot!
Wentao

Mistake

mistake...

the nodule confidence score is pbb[:,0] ?

I have run your code successfully ! In the test_detect.py:

pbb,mask = get_pbb(output,thresh,ismask=True)
The pbb shape is [*,5] , so the nodule cofidence sorce is pbb[:,0] ?

the two models, classifier.ckpt and detector.ckpt, are the final models?

if not, could you upload the two models?
Thanks. :)

I can't find luna segment results ?

Hi, lfz ：
This is a great work . Read your code is very enjoyable . But I can't find luna segment mhd results :
files = [f for f in os.listdir(luna_segment) if f.endswith('mhd')]

     So,the lung seg part is from your own tool which not appear in this github code ?

Confusion about saving label as zeros in function 'savenpy' of full_prep.py.

I don't understand why you just save the labels as zeros.

Run main.py for training error

I had finishing prepare datas.When I train with prepared DSB3 data and LUNA16 data,it is something wrong with DataLoader.
using gpu 0,1
Traceback (most recent call last):
File "/home/wisdom/soft/pycharm-community-2017.1/helpers/pydev/pydevd.py", line 1578, in
globals = debugger.run(setup['file'], None, None, is_module)
File "/home/wisdom/soft/pycharm-community-2017.1/helpers/pydev/pydevd.py", line 1015, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/home/wisdom/PycharmProjects/DSB2017-master/training/detector/main.py", line 348, in
main()
File "/home/wisdom/PycharmProjects/DSB2017-master/training/detector/main.py", line 167, in main
train(train_loader, net, loss, epoch, optimizer, get_lr, args.save_freq, save_dir)
File "/home/wisdom/PycharmProjects/DSB2017-master/training/detector/main.py", line 179, in train
for i, (data, target, coord) in enumerate(data_loader):
File "/home/wisdom/anaconda3/envs/DSB2017/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 198, in next
return self._process_next_batch(batch)
File "/home/wisdom/anaconda3/envs/DSB2017/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 239, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
TypeError: Traceback (most recent call last):
File "/home/wisdom/anaconda3/envs/DSB2017/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 41, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/home/wisdom/anaconda3/envs/DSB2017/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 109, in default_collate
transposed = zip(*batch)
TypeError: zip argument #3 must support iteration

I had modified the GPU=1,2 and workers=10
Thank you!

ERROR In Compiling ANY Code (Process finished with exit code 0)

I downloaded the code and whenever I compile any code, the error below keeps showing up. No matter what code I change, the same error still shows up. Is it possible if anyone could please tell me how to fix this problem?

The error is "process finished with exit code 0"

prepare issue

when I run python prepare.py in my pc(ubuntu,16G),it occurs memory error to me.how can I solve the problem?thx.

Please for help about the file 'lung_segment' !!!

我想问一下，lung mask 是如何生成的？step1文件可以生成，是要自己补充代码来生成.mhd文件吗，如果这样生成完文件名字也要变成如015.mhd文件还是说名字不变。
还有prepare.py 的366行的'.zraw’是就应该这样，还是说应该‘.raw’

希望得到你的帮助！！

Run main.py for testing, but got out of memory

Hi,

I got this error message when I run python main.py for testing:

THCudaCheck FAIL file=/home/xxx/pytorch-0.1.12/torch/lib/THC/generic/THCStorage.cu line=66 error=2 : out of memory
Traceback (most recent call last):
File "main.py", line 58, in
test_detect(test_loader, nod_net, get_pbb, bbox_result_path,config1,n_gpu=config_submit['n_gpu'])

my computer have 2 GTX 1080p GPU with memory 8G, which is less than Titan X with 12 G memory, is that the problem?

Any help would be appreciate.

What does the coord mean here？

Could you please tell me the meaning of coord in comb2 = self.back2(torch.cat((rev2, out2, coord), 1))?

how to dump detected bounding boxes and visualize the nodule detection?

Hi, is there a way to dump the detected bounding boxes for each scan?
For testing, I saw .pbb and .lbb files generated, but couldn't understand what was written out, because for example .pbb is a 2D array with the 2nd dimension of 5 for most cases, as for .lbb, they're 0s

Could you please explain a bit:

1. how to dump bboxes?
2. how to interpret the .pbb and .lbb results?

Many thanks!

Why not flip the sliceim when we filp the mask?

in https://github.com/lfz/DSB2017/blob/master/training/prepare.py#L196, if is_flip==True, we flip the mask. But I don't find the code to flip the sliceim. Is this a bug?

UnicodeDecodeError: 'ascii' codec can't decode byte 0x8c in position 16: ordinal not in range(128)

When I run the mian.py at DSB2017-master directory, I got this error. It happened when I load the classifier.ckpt. The classifier.ckpt is downloaded from this github.
Traceback (most recent call last):
File "main.py", line 68, in
checkpoint = torch.load(config_submit['classifier_param'])
File "/usr/local/lib/python3.5/dist-packages/torch/serialization.py", line 229, in load
return _load(f, map_location, pickle_module)
File "/usr/local/lib/python3.5/dist-packages/torch/serialization.py", line 377, in _load
result = unpickler.load()
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8c in position 16: ordinal not in range(128)
checkpoint = torch.load(config_submit['classifier_param'])
How can resolve it ?

One question with implementing and extending your method (potential publication request)

Hi, sir,

Thanks for your repository. I am a student that have recently implemented your method with tensorflow and have extended it to be a cascaded system for lung nodule detection problem. I have validated it on LUNA dataset. From your part of code, I have used the "preprocessing" step (prepare.py) and also the "detector" code as my initialized nodule proposals. Then I added two cascaded parts for position refinement and false positive reduction.

I am writing to you to ask if I could use your code for my project and submit an abstract to a conference. My work has been inspired by your idea a lot so I want to give you a co-authorship. Meanwhile, if there is any existing publication from you on this, could you please let me know so I put them into my reference list?

Please let me know what you think. I respect your opinion and would like to discuss more.

Thanks,

how much score in the leaderboard?

I have get the result and submitted to kaggle leaderboard. But the private score is 0.43, much larger than 0.39.

preprocessing luna gets wrong

starting preprocessing luna
252
278
556
523
746
435
231
MetaImage: Read: Cannot open data file
166
608
409
722
179
276
788
547
807
549
659
383
575
182
706
Traceback (most recent call last):
File "prepare.py", line 378, in
preprocess_luna()
File "prepare.py", line 287, in preprocess_luna
_=pool.map(partial_savenpy_luna,range(N))
File "/usr/lib64/python2.7/multiprocessing/pool.py", line 250, in map
return self.map_async(func, iterable, chunksize).get()
File "/usr/lib64/python2.7/multiprocessing/pool.py", line 554, in get
raise self._value
RuntimeError: Exception thrown in SimpleITK ReadImage: /tmp/SimpleITK-build/ITK/Modules/IO/Meta/src/itkMetaImageIO.cxx:483:
itk::ERROR: MetaImageIO(0x6f83b30): File cannot be read:
/data/tmp/luna_data/15.mhd for reading.
Reason: Success
^Z[4] 退出 1 nohup python -u prepare.py 2>&1

error while running for testing

Hi, fangzhou,
When I used your code for the testing data, it gave me the following error:

starting preprocessing
bug in stage2
Traceback (most recent call last):
  File "main.py", line 29, in <module>
    use_existing=config_submit['use_exsiting_preprocessing'])
  File "/workspace/hshu/lungcancer/winners/1st/DSB2017/preprocessing/full_prep.py", line 124, in full_prep
    _=pool.map(partial_savenpy,range(N))
  File "/workspace/hshu/anaconda2/lib/python2.7/multiprocessing/pool.py", line 251, in map
    return self.map_async(func, iterable, chunksize).get()
  File "/workspace/hshu/anaconda2/lib/python2.7/multiprocessing/pool.py", line 567, in get
    raise self._value
IndexError: list index out of range
bug in acd91ceec65820f4d55d71efe870093b.dcm

I revised your config_submit.py to be the following:

config = {'datapath':'/projects-priv10/hshu/data_science_bowl_2017/stage2_image/',
          'preprocess_result_path':'./prep_result/',
          'outputfile':'prediction.csv',

          'detector_model':'net_detector',
         'detector_param':'./model/detector.ckpt',
         'classifier_model':'net_classifier',
         'classifier_param':'./model/classifier.ckpt',
         'n_gpu':2,
         'n_worker_preprocessing':None,
         'use_exsiting_preprocessing':False,
         'skip_preprocessing':False,
         'skip_detect':False}

I used 7za e stage2.7z to unzip the stage2 data.

I am not sure why the error came out. Could you help provide some suggestion?
Thanks.

Something wrong with the features exporting

end preprocessing
(18L, 1L, 208L, 208L, 208L)
Traceback (most recent call last):
File "main.py", line 62, in
test_detect(test_loader, nod_net, get_pbb, bbox_result_path,config1,n_gpu=config_submit['n_gpu'])
File "/home/xx/py/DSB2017-master/test_detect.py", line 55, in test_detect
output = split_comber.combine(output,nzhw=nzhw)
File "/home/xx/py/DSB2017-master/split_combine.py", line 83, in combine
splits[0].shape[4]), np.float32)
IndexError: tuple index out of range

I'm wondering if the codes:
if 'output_feature' in config:
if config['output_feature']:
isfeat = True
are used for exporting the features. If not, how can I review the features or proposals detected by the model?

Potential improvement in nodule detection

Hi lfz,

I have observed one thing during playing with your detector. Looks like your detector performs much better on CTs with smaller slice thickness (i.e., higher resolution on z axis). This is significant as I observed ~500 proposals (before iou step) for higher resolution CTs and around ~2000 ~ 3000 proposals for lower resolution CTs. Do you think it could be beneficial to train two detectors for these two types of CTs?

RuntimeError: CUDNN_STATUS_INTERNAL_ERROR

I run the program as testing phase instruction said.'RuntimeError: CUDNN_STATUS_INTERNAL_ERROR' occured.What make this happen??

One potential bug in line 45, step1.py

LFZ,

Please look at the line 45 in step1.py. Should it be np.float(slope) *......? Or is it my DICOM issue?

I am using python 2.7.

Thanks,

how to convert detected nodule coordinates back to original scanning coordinates?

Hi @lfz

Can you please advise how to convert the nodule detection results (coordinates in .pbb) files back to the original coordinate system in the scan? For example, all luna16 data has been pre-processed and normalized, so if we want to test on luna16, how do we get the corresponding coordinates in the original system? Any ideas? I assume this reverse transform would be scan-dependent.

Thanks!

error in ‘checkpoint = torch.load(config_submit['classifier_param'])’（求大神回复渣交小妹）

when I run the main.py, there is something wrong happening when I run this code'checkpoint = torch.load(config_submit['classifier_param'])’. The error is 'UnicodeDecodeError: 'ascii' codec can't decode byte 0x8c in position 16: ordinal not in range(128)' ,however the same code that 'checkpoint = torch.load(config_submit['detector_param'])' is ok.
dear 大神（look forward to your responses ）

'illegal instruction' if 'use_exsiting_preprocessing':True?

Hi,

Thanks for posting the code for us to reproduce the results!
One issue I ran into is when trying to use existing pre-processing results, I got illegal instructions right after pre-processing. Seemed to be coming from the following:

self.preBlock = nn.Sequential(                                                                                                                                                                               
            nn.Conv3d(1, 24, kernel_size = 3, padding = 1),                                                                                                                                                          
            nn.BatchNorm3d(24),                                                                                                                                                                                      
            nn.ReLU(inplace = True),                                                                                                                                                                                 
            nn.Conv3d(24, 24, kernel_size = 3, padding = 1),                                                                                                                                                         
            nn.BatchNorm3d(24),                                                                                                                                                                                      
            nn.ReLU(inplace = True))

This is for testing using an existing model and existing pre-processed data.
Do you know what happened here? And how to fix this?
Thanks!

training a nodule detector error

I only use .mhd to train without DSB3 data,but in step 2,when I run ./run_training.sh,it is output some error.

Traceback (most recent call last):
File "main.py", line 350, in
main()
File "main.py", line 168, in main
train(train_loader, net, loss, epoch, optimizer, get_lr, args.save_freq, save_dir)
File "main.py", line 180, in train
for i, (data, target, coord) in enumerate(data_loader):
File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 212, in next
return self._process_next_batch(batch)
File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 239, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
AssertionError: Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 41, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "DSB2017-master/training/detector/data.py", line 103, in getitem
label = self.label_mapping(sample.shape[1:], target, bboxes)
File "DSB2017-master/training/detector/data.py", line 281, in call
assert(input_size[i] % stride == 0)
AssertionError

In README.md, I see that the input size is 128x128x128,but the input_size(in DSB2017-master/training/detector/data.py", line 281 ) of my data is not all 128,there are some values is 144,127,75 and so on. Is the input_size must be 128? And I did not find the code to precess the data to 128128128,where is it?Thank you very much...

What hardware spec you have used ?

Why dsb's data have coord?

in detector/labels/label_qualified.csv
I see 008464bb8521d09a42985dd8add3d0d2 coordx1 coordx2 coordx3 diameter
Isn't the label of 008464bb8521d09a42985dd8add3d0d2 just 1 or 0 ?

Are both benign and malign nodules annotated with label 1 in nodule detection?

I was wondering: in the nodule detection network, do you annotate both benign and malign nodules with label 1?
Then in the lung classification network, you aim to distinguish the benign and malign nodules?

train issue

I am confused when i saw run_training.sh codes..

especially the read lines, what are their specific features?

program hangs when using "use_exsiting_preprocessing: True"

Hi,

I ran into this situation when specifying "use_exsiting_preprocessing: True", below is what I had from the console:

starting preprocessing
09aec51e845d17f2ec5459b5e292abb8 had been done
417e3c40213fe0b8474b1ff74318f14c had been done
6f76555e8e65be4886e8868d7ba3f510 had been done
ed43e376ff1b9dbf971e6a1d6d9f863b had been done
9e041750951de329e7c7a20acdb9c88f had been done
c0f0e7479900bd05e349af2df690be73 had been done
a77b5a7654310f555e04a046711b81b6 had been done
aa070ed19af0fad480a807b75e0a816b had been done
cd4f000784503503feb427133a49ca3a had been done
789503ce9fceedc4a7b21f939b40ed82 had been done
end preprocessing
(12L, 1L, 208L, 208L, 208L)

and it hung without moving any further: no use is shown on CPU and GPU.
Any ideas why so?

Other configs that might be related:

          'detector_model':'net_detector',
          'detector_param':'./model/detector.ckpt',
          'classifier_model':'net_classifier',
          'classifier_param':'./model/classifier.ckpt',
          'n_gpu':1,
          'n_worker_preprocessing':2,
          'use_exsiting_preprocessing':True,
          'skip_preprocessing':False,
          'skip_detect':False}

I'm using a single GPU card for testing only 10 scans from kaggle stage2 data.
Please advise.
Thanks!

Jin

training/detector/bbox出现的问题

在运行CUDA_VISIBLE_DEVICES=0 python main.py --model res18 -b 8 --resume results/res18/$eps.ckpt --test 1时，
print出[0,'dsb2017']时，是不是要等很久才能print下一个？我现在print出了[1, 'DSB2017']后就已经没反应了这是正常的吗？

The input dimension in detector.

In detector step, why the input dimension are different in training (1L,128L, 128L, 128L) )and testing (1L, 208L, 208L, 208L))? It's mean the input of the network is variable?

Something about the luna data used for training

Hi lfz,
I have one question about the luna data used during training.
There're 1187 and 684 nodules with diameter greater than 3mm and 6mm respectively, but it seems only 478 nodules are labeled as positive during training. What about the remaining 684-478=106 nodules? Is there any other screening criteria than diameter?

what does the value of b mean?

I mean I have two TITANXs, so what value should I change b into？

sitk.ReadImage(filename) is broken.

in prepare.py:
if name=='main':
full_prep(step1=True,step2=True) is ok.
prepare_luna() is ok.

When it ran preprocess_luna(),
line 199: Mask,origin,spacing,isflip = load_itk_image(os.path.join(luna_segment,name+'.mhd'))

in load_itk_image(),
line 61: itkimage = sitk.ReadImage(filename)

I get:
starting preprocessing luna
MetaImage: Read: Cannot open data file
Traceback (most recent call last):
File "prepare.py", line 377, in
preprocess_luna()
File "prepare.py", line 286, in preprocess_luna
_=pool.map(partial_savenpy_luna,range(1))
File "/usr/lib64/python2.7/multiprocessing/pool.py", line 250, in map
return self.map_async(func, iterable, chunksize).get()
File "/usr/lib64/python2.7/multiprocessing/pool.py", line 554, in get
raise self._value
RuntimeError: Exception thrown in SimpleITK ReadImage: /tmp/SimpleITK-build/ITK/Modules/IO/Meta/src/itkMetaImageIO.cxx:483:
itk::ERROR: MetaImageIO(0x6df4d60): File cannot be read:
/luna16/data/original_lung_masks/734.mhd for reading.
Reason: Success

why do you concat coord with rev2 and out2 in detect network

eg. comb2 = self.back2(torch.cat((rev2, out2,coord), 1))

Can you upload some essential files.

Thanks for your code, good work. While, I'm a little confused about the config files.

config = {'stage1_data_path':'/work/DataBowl3/stage1/stage1/', # Kaggle第一阶段原始数据
'luna_raw':'/work/DataBowl3/luna/raw/', #LUNA16原始数据
'luna_segment':'/work/DataBowl3/luna/seg-lungs-LUNA16/', #LUNA16肺分割mask
'luna_data':'/work/DataBowl3/luna/allset', #LUNA16预处理之后的数据
'preprocess_result_path':'/work/DataBowl3/stage1/preprocess/', # Kaggele第一阶段预处理之后的数据
'luna_abbr':'./detector/labels/shorter.csv', # LUNA16的文件名对数据之间的映射（原始文件名为1.3.6.1.4.1.14519.5.2.1.6279.6001.100225287222365663678666836860，之后用0表示）
'luna_label':'./detector/labels/lunaqualified.csv', # 筛选之后的LUNA16的结节信息（排除官方给出的结节中直径较小的那些）
'stage1_annos_path':['./detector/labels/label_job5.csv',
'./detector/labels/label_job4_2.csv',
'./detector/labels/label_job4_1.csv',
'./detector/labels/label_job0.csv',
'./detector/labels/label_qualified.csv'],
'bbox_path':'../detector/results/res18/bbox/',
'preprocessing_backend':'python'
}

can you give me the luna_abbr, luna_label files. If have time, can you explain each items in this config files.

how to measure the result good or not?

Epoch 130 (lr 0.00010)
Train: tpr 0.00, tnr 100.00, total pos 1317, total neg 3762, time 715.10
loss 0.7080, classify loss 0.6632, regress loss 0.0048, 0.0045, 0.0050, 0.0305

Validation: tpr 0.00, tnr 100.00000000, total pos 216, total neg 19458320, time 18.70
loss 0.7367, classify loss 0.6942, regress loss 0.0033, 0.0035, 0.0031, 0.0327

Train, epoch 130, loss2 20.3725, miss loss 0.5777, acc 0.2627, tpn 419, fpn 1176, fnn 0, time 1497.73, lr 0.00010
Valid, epoch 130, loss2 20.4470, miss loss 0.6395, acc 0.2600, tpn 91, fpn 259, fnn 0, time 95.32
Valid, epoch 130, loss2 20.3725, miss loss 0.5829, acc 0.2627, tpn 419, fpn 1176, fnn 0, time 423.61

is this training result normal?

extract mask

@lfz a lot of really cool image processing tricks that I think would be useful to keep in your back pocket
for any similar domain. Some confused, why Mindist smaller than 62mm, volume between 0.68L and 7.5L , and Area.Thank you!