ekagra-ranjan / ae-cnn

ICVGIP' 18 Oral Paper - Classification of thoracic diseases on ChestX-Ray14 dataset

Hi there 👋

Checkout my Professional Portfolio to know more about my work.

ae-cnn's Issues

Trouble reproducing baseline results

Hi there! First of all, I think the paper is really interesting. Second, I appreciate that this is one of the only open-sourced repos I can find that provides training code for the NIH dataset and uses the official train-test split.

I am trying to independently reproduce the DenseNet121 baseline you provide in the paper (over 0.82 AUROC with augmentation), and not really coming close. I am using the exact same data splits as you, but have written my own training and model definition scripts; I am still using pre-trained ImageNet weights provided by torchvision and, as far as I can tell, all the same hyperparameters and preprocessing steps as you have.

I've tried unweighted cross-entropy, class-weighted cross-entropy, heavy augmentation (including elastic deformations, cutout, etc.), light augmentation (e.g., just random crop to 224 and horizontal flip), using 14 classes, using 15 classes (considering "No Finding" to be another output class). No matter what I've tried, I see rather quick convergence and overfitting (best validation loss achieved no later than epoch 7), and the highest validation AUROC I've seen is 0.814. This is considerably lower than the 0.839 validation AUROC you report in Table 2 for BL5-DenseNet121. The absolute best test set results I've achieved with a DenseNet121 architecture is 0.807 AUROC with 8x test-time augmentation.

I'm pretty puzzled by this because I don't think random variation in training or minor implementation differences should cause a >0.015-point drop in AUROC... Of course there are a million potential sources for this difference, but maybe you can help me pinpoint it.

For your final models, did you use 14 or 15 classes? Also, would you be able to provide any sort of training/learning curve showing loss or AUROC vs. # epochs? I am suspicious of how quickly my baseline models are converging, and am wondering how my training trajectories compare to yours.

.pth file of the model used in paper

Sir, can you provide model weight file?

What is the purpose of applying TenCrop to the test dataset and increasing the Y labels?

Hi,
It would be great if you clarify the following operation in test dataset:

AE-CNN/ae-cnn/densenet121/ChexnetTrainer.py

Line 136 in de16e25

transformList.append(transforms.TenCrop(transCrop))

Best

License file is missing

Hello,

It would be great if you could open your code.

Cheers,

Trained weights

Hi,
I couldn't find the trained weights. Can you share it please?