thomasneff / generative-adversarial-network-based-synthesis-for-supervised-medical-image-segmentation Goto Github PK

Code for the paper 'Generative Adversarial Network based Synthesis for Supervised Medical Image Segmentation'

License: MIT License

Python 98.04% TeX 1.96%

generative-adversarial-network-based-synthesis-for-supervised-medical-image-segmentation's Introduction

Generative-Adversarial-Network-based-Synthesis-for-Supervised-Medical-Image-Segmentation

Code for the paper 'Generative Adversarial Network based Synthesis for Supervised Medical Image Segmentation'

This modification adds the ability to generate pixel-wise segmentations to the GAN. Currently, it assumes that the images are grayscale, therefore the GAN model only handles 2 image channels (1 for the image, 1 for the segmentation)

For more details, check out our paper.

For citations (bibtex):

Neff, Thomas and Payer, Christian and Stern, Darko and Urschler, Martin (2017). 
Generative Adversarial Network based Synthesis for Supervised Medical Image Segmentation.
In Proceedings of the OAGM&ARW Joint Workshop, pp. 140-145.

Credits

Credits to sugyan for their tf-dcgan implementation, which this code is based on.

How to use/modify

In train.py, the 'idlist_*' variables defined at the start of 'train' need to point to text files containing a list of image/segmentation filenames, one for each line.

The '*_folder_path* variables need to point to the folder containing the files defined in the idlists.

(You can easily change the data loading, the dcgan model just takes image batches as a tensor as input.)

generative-adversarial-network-based-synthesis-for-supervised-medical-image-segmentation's People

Contributors

Stargazers

Watchers

Forkers

geekyspartan chenailian yf817 binlv ece7048 medical-images-process gsunfeng chuanzhidong forlife-socialism ai-ml-labs kspkoyyada

generative-adversarial-network-based-synthesis-for-supervised-medical-image-segmentation's Issues

RandomShuffleQueue error while running train.py

before d gen
2
4
2048
before d real
2
4
2048
2018-09-04 14:58:47.443976: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-09-04 14:58:47.499029: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-09-04 14:58:47.499312: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1344] Found device 0 with properties:
name: GeForce GTX 980 Ti major: 5 minor: 2 memoryClockRate(GHz): 1.076
pciBusID: 0000:01:00.0
totalMemory: 5.93GiB freeMemory: 5.45GiB
2018-09-04 14:58:47.499326: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0
2018-09-04 14:58:47.680835: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-09-04 14:58:47.680862: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917] 0
2018-09-04 14:58:47.680868: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0: N
2018-09-04 14:58:47.681035: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5221 MB memory) -> physical GPU (device: 0, name: GeForce GTX 980 Ti, pci bus id: 0000:01:00.0, compute capability: 5.2)
Traceback (most recent call last):
File "train.py", line 159, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 126, in run
_sys.exit(main(argv))
File "train.py", line 155, in main
train()
File "train.py", line 90, in train
_, g_loss_value, d_loss_value = sess.run([train_op, dcgan.losses['g'], dcgan.losses['d']])
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 905, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1140, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1321, in _do_run
run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1340, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_3_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 128, current size 0)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

Caused by op u'shuffle_batch', defined at:
File "train.py", line 159, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 126, in run
_sys.exit(main(argv))
File "train.py", line 155, in main
train()
File "train.py", line 54, in train
images, segmentation_images = load_image_and_segmentation_from_idlist(idlist_tensor_img, idlist_tensor_seg, batch_size, 16, 2560, shift_params = [-128, -0.5], rescale_params = [128, 0.5], shuffle = True)
File "/home/aditi/Documents/PhD/Steel_Seg/Data/Ground-Truth-Generation/GAN_trained_to_get_GT/Generative-Adversarial-Network-based-Synthesis-for-Supervised-Medical-Image-Segmentation-master/load_folder_images.py", line 92, in load_image_and_segmentation_from_idlist
min_after_dequeue=min_queue_examples)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/input.py", line 1301, in shuffle_batch
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/input.py", line 847, in _shuffle_batch
dequeued = queue.dequeue_many(batch_size, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/data_flow_ops.py", line 483, in dequeue_many
self._queue_ref, n=n, component_types=self._dtypes, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 3476, in queue_dequeue_many_v2
component_types=component_types, timeout_ms=timeout_ms, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 3290, in create_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1654, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

OutOfRangeError (see above for traceback): RandomShuffleQueue '_3_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 128, current size 0)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

Using the net after training

(requested 128, current size 0) what is the 128 represent ?

OutOfRangeError (see above for traceback): RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 128, current size 0)
[[node shuffle_batch (defined at :84) ]]

i tried the suggested solutions and delete the path for .txt file and just put the name, this error rise
idlist_img_name = 'imagetxt.txt'
idlist_seg_name = 'labeltext.txt'
IOError: [Errno 2] No such file or directory: 'imagetxt.txt'
what did you do to make it work

tuple shape mismatch error while training

before d gen
2
4
2048
before d real
2
4
2048
2018-07-03 11:11:01.074867: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-07-03 11:11:01.117290: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties:
name: Quadro K420 major: 3 minor: 0 memoryClockRate(GHz): 0.8755
pciBusID: 0000:03:00.0
totalMemory: 1.94GiB freeMemory: 1.35GiB
2018-07-03 11:11:01.117324: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Ignoring visible gpu device (device: 0, name: Quadro K420, pci bus id: 0000:03:00.0, compute capability: 3.0) with Cuda compute capability 3.0. The minimum required Cuda capability is 3.5.
2018-07-03 11:11:01.117336: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-07-03 11:11:01.117346: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929] 0
2018-07-03 11:11:01.117353: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0: N
2018-07-03 11:11:02.343335: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
2018-07-03 11:11:02.412697: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
2018-07-03 11:11:02.428441: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
2018-07-03 11:11:02.428441: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
2018-07-03 11:11:02.429115: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
2018-07-03 11:11:02.437389: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
2018-07-03 11:11:02.437405: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
2018-07-03 11:11:02.441379: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
2018-07-03 11:11:02.449401: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
2018-07-03 11:11:02.451722: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
2018-07-03 11:11:02.457215: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
2018-07-03 11:11:02.457665: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
2018-07-03 11:11:02.458851: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
2018-07-03 11:11:02.459306: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
2018-07-03 11:11:03.544010: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
2018-07-03 11:11:03.636651: W tensorflow/core/framework/op_kernel.cc:1318] OP_REQUIRES failed at queue_ops.cc:105 : Invalid argument: Shape mismatch in tuple component 1. Expected [128,128,1], got [128,128,3]
Traceback (most recent call last):
File "train.py", line 158, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 126, in run
_sys.exit(main(argv))
File "train.py", line 154, in main
train()
File "train.py", line 89, in train
_, g_loss_value, d_loss_value = sess.run([train_op, dcgan.losses['g'], dcgan.losses['d']])
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 900, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1135, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1316, in _do_run
run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1335, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 64, current size 0)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

Caused by op u'shuffle_batch', defined at:
File "train.py", line 158, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 126, in run
_sys.exit(main(argv))
File "train.py", line 154, in main
train()
File "train.py", line 54, in train
images, segmentation_images = load_image_and_segmentation_from_idlist(idlist_tensor_img, idlist_tensor_seg, batch_size, 16, 2560, shift_params = [-128, -0.5], rescale_params = [128, 0.5], shuffle = True)
File "/home/nikhitha/Srijita_work/Image_Preprocess/Generative-Adversarial-Network-based-Synthesis-for-Supervised-Medical-Image-Segmentation-master/load_folder_images.py", line 84, in load_image_and_segmentation_from_idlist
min_after_dequeue=min_queue_examples)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/input.py", line 1300, in shuffle_batch
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/input.py", line 846, in _shuffle_batch
dequeued = queue.dequeue_many(batch_size, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/data_flow_ops.py", line 483, in dequeue_many
self._queue_ref, n=n, component_types=self._dtypes, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 3480, in queue_dequeue_many_v2
component_types=component_types, timeout_ms=timeout_ms, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 3392, in create_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1718, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

OutOfRangeError (see above for traceback): RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 64, current size 0)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

Hi , I am getting this tuple shape mismatch error , can you please provide me with the solution to this ?
It is urgent .

Parsing the training data

Hi,
First off, thank you for sharing this code. Your paper is really interesting!
I am trying to run the code with the JSRT dataset, before I try it on my own. I am using 1024x1024 image-mask pair of JSRT data in png format. I used the whole 247 images as training data and specified the paths as specified in your readme file:

idlist_img_name = os.path.normpath("/home/gpux/JSRT_data/lung_id.txt")
idlist_seg_name = os.path.normpath("/home/gpux/JSRT_data/mask_id.txt")
img_folder_path = os.path.normpath('/home/gpux/JSRT_data/lung')
seg_folder_path = os.path.normpath('/home/gpux/JSRT_data/mask')

But when I train the model, I keep getting the following error:
OutOfRangeError (see above for traceback): RandomShuffleQueue '_3_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 16, current size 0)

I am not sure what I am doing wrong here. Do you have any suggestions?
(For complete output of the training command please see the attached file: error_msg.txt)

I am using tensorflow 1.4.0 and python 2.7.12+.

The generator loss is so high

The code is running for my data now but the generator loss is up to 10 while the d loss is 0.0004
What should I change to make it work?

event logs and pytorch

tf.app.flags.DEFINE_string('train_dir', '/tmp/dcgan_train',
"""Directory where to write event logs """
"""and checkpoint.""")
tf.app.flags.DEFINE_integer('max_steps', 100000,
"""Number of batches to run.""")
tf.app.flags.DEFINE_boolean('log_device_placement', False,
"""Whether to log device placement.""")

tf.app.flags.DEFINE_string("working_directory", "working_dir", "")
tf.app.flags.DEFINE_string("checkpoint_dir", "checkpoints", "")

can you please tell me what kind of files i should put here, i am not familiar with tf
and what is the possibility to converst the model to pytorch ?

if the image dim changes to 512 does the number of layers change or stays the same?

Using the Net after Training

Hey ,

Can you please explan what does this sess.run() actually stores in this model and how exactly do you use the model for segmentation?

Error during training

2048
before d real
2
4
2048
2018-07-03 10:57:54.138909: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-07-03 10:57:54.193468: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties:
name: Quadro K420 major: 3 minor: 0 memoryClockRate(GHz): 0.8755
pciBusID: 0000:03:00.0
totalMemory: 1.94GiB freeMemory: 1.36GiB
2018-07-03 10:57:54.193503: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Ignoring visible gpu device (device: 0, name: Quadro K420, pci bus id: 0000:03:00.0, compute capability: 3.0) with Cuda compute capability 3.0. The minimum required Cuda capability is 3.5.
2018-07-03 10:57:54.193516: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-07-03 10:57:54.193524: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929] 0
2018-07-03 10:57:54.193532: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0: N
Traceback (most recent call last):
File "train.py", line 158, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 126, in run
_sys.exit(main(argv))
File "train.py", line 154, in main
train()
File "train.py", line 89, in train
_, g_loss_value, d_loss_value = sess.run([train_op, dcgan.losses['g'], dcgan.losses['d']])
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 900, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1135, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1316, in _do_run
run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1335, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 64, current size 0)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

I have put the image text list and segmentation mask list in appropriate folder structure . Still I am getting this error . Can you please help me resolve this ?

Problem while parsing the training data

 [[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

I have tried changing the paths respectively

idlist_img_name = "list_of_img_ids.txt"
idlist_seg_name = "list_of_seg_ids.txt"
img_folder_path = "/home/nikhitha/Srijita_work/GAN/image_folder/images"
seg_folder_path = "/home/nikhitha/Srijita_work/GAN/segmentation_folder/masks"

the text files are kept in the folder GAN where the train.py file is also kept.

list_of_img_ids.txt is written in the following way

1.png
2.png
3.png
4.png
5.png
6.png
7.png
8.png

...
....

Can you please provide me with the solution to this problem , @jeya-maria-jose it will of great help if you also provide me some insights on this .

Thankyou in Advance .