ganslate-team / ganslate Goto Github PK
View Code? Open in Web Editor NEWSimple and extensible GAN image-to-image translation framework. Supports natural and medical images.
Home Page: https://ganslate.readthedocs.io
License: Other
Simple and extensible GAN image-to-image translation framework. Supports natural and medical images.
Home Page: https://ganslate.readthedocs.io
License: Other
validate_after
__init__.py
s should contain no code except for necessary imports to be able to access some modules more easily
While running on a new machine, weights and biases throws a login error if results are chosen to not be visualized. This needs to be fixed. Maybe upgrade wandb version
see if it can be improved so that is general and that it is a single call, preferably in BaseGAN, and takes care if it's a cycle-consistency model or not and so on.
rn the problem is that the calculation of metrics is located in different places in GANs (see CycleGAN, RevGAN), which makes it not very readable and a pain to remember and place properly in new GAN implementations.
Maybe a storage class that can be used for wandb and tensorboard trackers? Ideas are welcome.
http://utw10503.utweb.utexas.edu/publications/2009/cl_spie09.pdf
3 component SSIM is shown to correspond better with human perception of Mean Opinion Score ( a rating based on human quality scoring). It is shown to give better SSIM values compared to blurrier images.
The investigation here would be to see if changing weights to promote texture over edges and smoothness would be more valuable.
Replace with this: https://github.com/facebookresearch/fastMRI/blob/master/fastmri/losses.py
Not necessary to divide the D loss by 2 since this repo, unlike the original CycleGAN repo, has separate LR param for G and D. Reference: junyanz/pytorch-CycleGAN-and-pix2pix#720
Should it be done?
Check for conf[conf.mode].batch_size * communication.get_world_size() > len(dataloader.dataset)
The issue discussed was what can be done to consider the effect of negative covariance between the image pixels.
Further reading led to an implementation of SSIM as a valid distance measure which leads it to be modified from being just a ( 1 - SSIM index) call.
Paper can be found below:
https://ece.uwaterloo.ca/~z70wang/publications/TIP_SSIM_MathProperties.pdf
Will read through, document and implement soon.
Proposed solution:
The loss functions must be modified to have the loss reduction to none so that per voxel loss is obtained. Once this is obtained,
This loss masking needs to go into L1, SSIM and all the GAN losses.
This is important to handle the case when terminal slices of a CBCT scan have artifacts or incomplete anatomical representation. This can potentially corrupt training and should ideally be removed in the dataloader pre-registration.
Possible methods:
enforce that each dataset class is a child of a base dataset class
Implement an inference script along with necessary config that will be dataloader and model agnostic.
Checkpoint to be saved when training is stopped. Implement as Jonas did.
evaluators have wandb, inferer doesnt. neither have tensorboard. make sure they work in DDP
When logging very often, the fact that the images have to be logged can make the iteration longer than what the calculated computation time and data-loading time state.
Add a warning message about that when training is starting if the log_freq is low.
Double check and verify if metrics are accurately computed before running final experiments along with @ibro45
Reference Paper: https://openaccess.thecvf.com/content_ICCV_2019/papers/Shocher_InGAN_Capturing_and_Retargeting_the_DNA_of_a_Natural_Image_ICCV_2019_paper.pdf
Could be implemented in a different way but the general idea is to have discrimination on different scales.
Hi ~ friends ~ I write some code:
1.body_mask.py:if apply_bound is False,output the original size of image
2.cut_lung_from_background.py:It can cut lung from the background stable.
But I can not push to origin/zhixiang branch.Maybe its permissions issue?
Code are in here now:
https://github.com/ZhixiangWang-CN/Radiomics/blob/master/ImageSegmentation/body_mask.py
https://github.com/ZhixiangWang-CN/Radiomics/blob/master/ImageSegmentation/cut_lung_from_background.py
@surajpaib once we're almost ready for running final experiments
Case: Due to the masking of patient body while computing metrics, a lot of values are zero skewing the true metric values. Decide if while taking the mean of the metric, the mask should be applied over the elements.
When using multi-channel input, PatchGAN produces multi-channel output. Shouldn't th output be just single channel?
in our projects we have completely separate datasets for this, the question here is if paired and unpaired ImageDatasets should have their valtest equivalents or just the same dataset class that handles different modes using conf.mode
With the current implementation if the batch size is 2 and data points are 5:
The final averaging is done in this way,
(average of first batch metrics with 2 data points + average of second batch metrics with 2 data points + third batch metric with only one data point ) / 3
For batch size 1, the implementation works this way,
( first data point metric + .... + 5th data point metric)/5
These values are not synonymous and need to be fixed so that the same values are given no matter what the implementation detail.
Separate the current z-score function so that :
Currently, when loading a checkpoint, the optimizers are loaded as well. Make it optional, explain why might be useful to have that option.
Usecases:
The Inferer can be called as Validator during training
A separate inference config might not be necessary as most of the parameters are defined in train_config.yaml. Needed parameters for inference should be made mandatory as CLI arguments or fed through config. However, a mandatory separate config should not be needed. A design decision on what parameters need to provided need to be made.
if cli.config:
inference_conf = OmegaConf.load(cli.pop("config"))
inference_conf = OmegaConf.merge(inference_conf, cli)
else:
inference_conf = cli
# Fetch the config that was used during training of this specific run
train_conf = Path(inference_conf.logging.checkpoint_dir) / "training_config.yaml"
Maybe conf here points to train_conf itself and separate inference_conf and train_conf are not needed
If the defined 3D patch size is big enough in x and y dimension to encompass the whole body, center cropping it in x and y would most likely be ideal, while z position could be selected as usual.
Add an option to StochasticFocalPatchSampler for center cropping in x and y.
Does this need to inherit from dataset in the TrainConfig?
There could be cases where you want to run evaluation with different parameters in the dataset -> For example, with bound to avoid checking out of patient body areas although the training still includes it. For now, these are two separate configs and care should be taken to define it as you need it.
Is it necessary to enforce that the dataset uses image_size (instead of load_size) or patch_size to indicate what's the size of the inputs? What would benefit from it?
project_dir: "./projects/nki_cervix_cbct_to_ct"
use_cuda: True
n_iters: 200000
n_iters_decay: 0
batch_size: 1
logging:
checkpoint_dir: ./checkpoints/cbct_ex3/
wandb:
project: "NKI_CBCT_CT"
log_freq: 50
checkpoint_freq: 10000
dataset:
name: "CBCTtoCTDataset"
root: /home/rt/workspace_suraj/cervix_resampled
hounsfield_units_range: [-1024, 2048]
num_workers: 4
patch_size: [32, 32, 32]
gan:
name: "PiCycleGAN"
generator:
name: Vnet3D
in_channels: 1
use_memory_saving: False
use_inverse: True
is_separable: False
down_blocks: [2, 2, 3]
up_blocks: [3, 3, 3]
discriminator:
name: "PatchGAN3D"
n_layers: 2
in_channels: 1
optimizer:
lambda_A: 25.0
lambda_B: 25.0
lambda_identity: 0.0
lambda_inverse: 0.0
proportion_ssim: 0.84
lr_D: 0.0001
lr_G: 0.0002
Reference Post can be found here: https://discourse.itk.org/t/ct-patient-bed-and-clothing-removal-from-dicom-files/3278/2
Add timestamp to it so that it doesn't overwrite. Option to not do it? Make sure it works in distributed training.
@surajpaib the ValTest metric calculation isn't compatible with bigger than 1 batch size. Problem is that the batch dimension is squeezed, which works fine for BS=1, but not otherwise. the scikit metrics don't operate on batches, so it'll be necessary to loop over the batch and average the metrics
https://github.com/Maastro-CDS-Imaging-Group/midaGAN/blob/06d6c419fca81327008d2a2cdbf92f126cff1c19/midaGAN/utils/metrics/val_test_metrics.py#L14
https://github.com/Maastro-CDS-Imaging-Group/midaGAN/blob/06d6c419fca81327008d2a2cdbf92f126cff1c19/midaGAN/utils/metrics/val_test_metrics.py#L64
Also, is this slice num alright for regular images or? @surajpaib
@JulianPosch and I are installing the repo and it would be nice to have an automated CI build using github actions to run a small snippet of the code and show if tests are passing.
I've done something similar for the seg pipeline but it is broken now, so maybe we can sit together and fix/ add it @ibro45
E.g.
val:
freq: 1000
multi_dataset:
lungs:
name: "CBCTtoCTValTestDataset"
root: "/workspace/train_val/val"
phantoms:
name: "CBCTtoCTValTestDataset"
root: "/workspace/train_val/val_phantom"
test:
dataset: "${val.multi_dataset}"
sliding_window: "${val.sliding_window}"
build_loader
selects a single dataset from multi_dataset
, assigns it to dataset
and sets multi_dataset
to None
. This breaks interpolation when it's referring to multi_dataset
.
Error:
omegaconf.errors.ConfigKeyError: str interpolation key 'val.multi_dataset' not found
full_key: test.dataset
reference_type=Optional[Dict[Union[str, Enum], Any]]
object_type=dict
Chinmay Rao 8:14 PM
I have some suggestions. Let me know what you think. First, regarding the naming conventions of the networks. I found them a bit confusing.
From the current code, G_A is the generator for A->B, for example. Maybe, we can change it to G_AB.
Also currently, D_A seems to be the discriminator that takes real_B and fake_B as inputs (i.e. domain B). Maybe, we can instead call it D_B, since it works in domain B. This would be similar as in the paper - domain Y and D_Y
If we can change the names, then we can have naming conventions for loggable (metrics, losses, predictions) based on them (edited)
Ibrahim Hadzic 8:16 PM
i agree. This D_A think must have been a mistake in naming i guess, which just propagated to everything
8:16
G
8:16
G_AB makes perfect sense
8:16
also, do you think these names as such are good
8:17
or would you prefer generator_ab instead of G_AB
8:17
i think that the latter, while not being great regarding the python conventions, is still more easily readable
8:17
in this case
๐
1
Chinmay Rao 8:17 PM
I agree. IMO, G_AB is good
New
8:21
For the loggable stuff -------------------------
Maybe have a naming format like (ignore the spaces) ==> mode - type - specific_component
For example, I'd suggest the following changes -----------------------------------
Loggable losses:
loss_G_A --> train-loss-adv_G_AB
loss_D_A --> train-loss-adv_D_B
loss_cycle_A --> train-loss-cycle_ABA
2. Logable predictions:
Train D_A_fake --> train-pred-D_A_fake
3. Loggable metrics:
mse --> val-mse-A (i.e. MSE between fake A and reference A)
Ibrahim Hadzic 8:28 PM
i would actually leave out "train" from train-loss-adv_G_AB but otherwise yes
Ibrahim Hadzic 8:30 PM
for the point 3, i agree with "val" but I'm wondering about "A". We're calculating only A->B and a lot of stuff is done that way in the framework only for A->B, like testing let's say, because B->A is gonna be the case only for cycle-consistency models
๐
1
1 reply
Today at 8:32 PMView thread
Ibrahim Hadzic 8:31 PM
so you have a prob with generalizability
๐
1
8:31
but if you really need to do B->A, you could manually take care of it lets say, but that's not elegant
not sure why only for 3D training
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.