This repository contains code to train a self-supervised learning model on chest X-ray images that lack explicit annotations and evaluate this model's performance on pathology-classification tasks.
License: MIT License
Hi,
we're from the University of Wuerzburg and tried to replicate your project for German report data.
For now, we simply tried to get your code to run and train on MIMIC with the default settings provided as well as the settings provided in your paper. Of course, we made sure to have the same package versions as in the project.
However, we quickly get NaN loss after some iterations. So first, we tried to create a subsample of the dataset. For a very small dataset (~300 images), the training does converge. However, even for 1000 images the loss does not get smaller. We also tried several different learning rates and hyperparameters, but nothing helped so far.
I was hoping that you might be familiar with our problems and give us advice here.
Thanks in advance!
Getting the error in title when training.
Full command: python3 run_train.py --cxr_filepath {path to cxr.h5} --txt_filepath {path to mimic_impressions.csv}
Full traceback:
Traceback (most recent call last):
File "/home/ec2-user/MedZero/CheXzero/run_train.py", line 143, in <module>
model = model_pipeline(args)
File "/home/ec2-user/MedZero/CheXzero/run_train.py", line 44, in model_pipeline
train(model, data_loader, device, criterion, optimizer, config)
File "/home/ec2-user/MedZero/CheXzero/run_train.py", line 85, in train
for data in tqdm(loader):
File "/opt/conda/envs/medzero/lib/python3.9/site-packages/tqdm/std.py", line 1195, in __iter__
for obj in iterable:
File "/opt/conda/envs/medzero/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
data = self._next_data()
File "/opt/conda/envs/medzero/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "/opt/conda/envs/medzero/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
data.reraise()
File "/opt/conda/envs/medzero/lib/python3.9/site-packages/torch/_utils.py", line 434, in reraise
raise exception
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/opt/conda/envs/medzero/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "/opt/conda/envs/medzero/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
return self.collate_fn(data)
File "/opt/conda/envs/medzero/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 74, in default_collate
return {key: default_collate([d[key] for d in batch]) for key in elem}
File "/opt/conda/envs/medzero/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 74, in <dictcomp>
return {key: default_collate([d[key] for d in batch]) for key in elem}
File "/opt/conda/envs/medzero/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 68, in default_collate
return torch.tensor(batch, dtype=torch.float64)
TypeError: must be real number, not str
I'm getting memory errors when running run_preprocess.py even with 512GB of RAM. Is this unusual? What type of RAM requirements would you suggest for doing the preprocessing?
Thank you for you awesome code and work.
I have a question:
In your train_batch function in the run_train.py you are negating the images with -images in the forward pass:
logits_per_image, logits_per_text = model(-images, texts)
Why do you do so?
Also in the train function you wanted to get the "highest_val_auc" which you never used. How would you define the AUC in such a scenario?
`def train(model, loader, device, criterion, optimizer, config):
model_save_dir = os.path.join(config.save_dir, config.model_name)
if not os.path.exists(model_save_dir):
# Create a new folder if not exists
os.makedirs(model_save_dir)
# Run training
total_batches = len(loader) * config.epochs
example_ct = 0 # number of examples seen
batch_ct = 0
report_freq = config.log_interval
highest_val_auc = 0 # save highest mean auc
for epoch in range(config.epochs):
running_loss = 0.0 # running loss over batch
for data in tqdm(loader):
# get the images
images = data['img']
texts = data['txt']
texts = preprocess_text(texts, model)
# perform step for a single batch
loss = train_batch(images, texts, model, device, criterion, optimizer)
example_ct += len(images)
batch_ct += 1
running_loss += loss.item()
# Report metrics every `report_freq` batch
if (batch_ct % report_freq) == 0:
train_log(running_loss / report_freq, example_ct, epoch)
running_loss = 0.0
if (batch_ct % config.save_interval) == 0:
model_path = os.path.join(model_save_dir, "checkpoint_{batch_ct}.pt".format(
batch_ct=str(batch_ct),
))
print("Saved checkpoint to: ", model_path)
save(model, model_path)`
I hope you can help me with my questions!
Thanks again for making your code publicly avaiable and greetings from Germany
Hello, thank you for sharing the code.
There seems to be one problem loading the h5 file into the CXRdataset class.
Wouldn't the key value be cxr rather than cxr_unprocessed in L39 (also, L42)?
Lines 24 to 43 in c303e5c
Hi, thank you for sharing your excellent work.
I found that the preprocessing in img_to_h5 function fails when I passed resolution argument. Probably the cause is in the following line.
CheXzero/preprocess_padchest.py
Line 186 in 809d6f9
Should be:
img = preprocess(img_pil, desired_size=resolution)
Tested on some pneumonia image, getting these predictions:
[array([[0.49470618, 0.49632818, 0.501808 , 0.49816415, 0.49443188,
0.4973687 , 0.5035202 , 0.50338894, 0.49944744, 0.4980956 ,
0.5006629 , 0.5109319 , 0.5024933 , 0.48189908],
[0.48987147, 0.4886013 , 0.505182 , 0.50687265, 0.49118397,
0.50182277, 0.5023532 , 0.50404346, 0.49911276, 0.49427748,
0.49890172, 0.51351357, 0.5014591 , 0.48301208]], dtype=float32)]
What does it mean? I think predictions should give 1.0 in total, here it seems to suspect every pathology with 50% probability, how so?
The same happens for both best_64_5e-05_original_22000_0.864.pt and best_128_5e-05_original_22000_0.855.pt weights.
ground_truth_3.csv
I was able to train a model with a small dataset and wanted to test the model with the original JPGs and giving a grount_truth.csv file encoded as 0s for all labels except "No Finding". I am getting back the prediction averages, but bootstrap_results[1] returns a table full of NaNs. What could be the issue?
Attached is a screenshot of the issue.
Hope you can help us.
When I ran the zero shot.ipynb file, I'm not sure why the average AUC of "No Finding" from the result is only 0.0700.
Hi, Thanks for the code you've provided.
I was trying to run the train on default specs of the repository. However I observed that the training loss gets suck at a value after a few minibatch cycles for both pretrained and non-pretrained cases. I tried this on the complete MIMIC dataset as well as a smaller batch I created for testing, but the loss was getting stuck at a value in all cases rather than moving towards overfitting.
Is this expected behaviour? Has anyone else also come across this?
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.