shinseung428 / globallocalimagecompletion_tf

Globally and Locally Consistent Image Completion

Globally and Locally Consistent Image Completion

Tensorflow implementation of Globally and Locally Consistent Image Completion on celebA dataset.

What's different from the paper

• smaller image input size (128x128)
• smaller patch sizes
• less number of training iteration (500,000 iterations in the paper)
• Adam optimizer used instead of Adadelta

Requirements

• Opencv 2.4
• Tensorflow 1.4

Folder Setting

-data
  -img_align_celeba
    -img1.jpg
    -img2.jpg
    -...

Train

$ python train.py 

To continue training

$ python train.py --continue_training=True

Test

$ python test.py --img_path=./data/test/test_img.jpg

Use your mouse to erase pixels in the image.
When you're done, press ENTER.
Result will be shown in few seconds.

Results

globallocalimagecompletion_tf's Issues

Code wrong in network.py?

conv3 = batch_norm(conv1, name="bn3") should be conv3 = batch_norm(conv3, name="bn3") in global_discriminator?

License

Could you add a license to this project?

Same masks each epoch?

I could be wrong, but because load_train_data of ops.py is called only once in init of network.py, block_patch of ops.py, which makes the random masks, is called once.
So the same masks are used each epoch? It would be better if new masks are generated each epoch.

How to set the Tc, Td and learning rate

Thank you for your share, however i still have some problems when i train the model on my own dataset. I notice that the c_loss and G_loss decrese regularly, but the D_loss decreses to 0.000 only after several steps (within one epoch). I wonder if it's because the Discreminator is too strong. The initial Td = 1, much smaller than Tc. I heard that Td have to set larger than Tc, which is totally different from your work. So, can you help me solve the problem about how to set the Tc and Td.

What's more, I also have some problem about the learning rate, you set it to be 1, and when i try to train the model, i set it to be 0.0001. It seems to make no sense when i see the training result. The tendncy of the C_loss and G_loss is same as yours.

Hi, how to modify the code if I want to use bigger input images?

Training duration

Hi,

Thanks for sharing your code! Could you help me with the number of iterations the model was trained on and how long did it take to train and what hardware (CPU, GPU etc.) resources did you use?

post-processing

hi,i read your paper,there is a post-processing method after output,but i don't find it in your code.could you give it to me or help me find it?

Is this code wrong?In network.py

In network.py Line 76-77,
self.fake_d_logits = tf.concat([self.local_fake_d_logits, self.global_fake_d_logits], axis=1)
self.real_d_logits = tf.concat([self.local_fake_d_logits, self.global_fake_d_logits], axis=1)
,I think it should be the
self.fake_d_logits = tf.concat([self.local_fake_d_logits, self.global_fake_d_logits], axis=1)
self.real_d_logits = tf.concat([self.local_reald_logits, self.globalreal_d_logits], axis=1)
?Aha?

test dataset

is the test dataset necessary?. because your load_test_data consist of many errors and you have't called it.

no result

when i use test.py i can erase pixels in the image ,however,when i press ENTER , there no result and any reaction, what's the problem?

Your implement is different from the algorithm in the paper?

In the Paper ， the training 3 stages are not Corresponding with your implement codes：
your codes:
1)training C
2)traing D
3)traing C
but in the paper:
1)training C
2)traing D
3)traing D && C

Can't load save_path when it is None.

Hello, I'd like to run the test.py. But there is something wrong .Can you help me ?

Traceback (most recent call last):
File "test.py", line 108, in
main(args)
File "test.py", line 106, in main
test(args, sess, model)
File "test.py", line 66, in test
saver.restore(sess, last_ckpt)
File "/home/guanyun/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1796, in restore
raise ValueError("Can't load save_path when it is None.")
ValueError: Can't load save_path when it is None.

Test script doesn't work

Hey!
Thank you very much for the implementation of the network. I tried to run the test script with an image from the celeb dataset but it seems that the checkpoints (that were downloaded with download.py) don't fit the current architecture. I get the following error:

InvalidArgumentError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

Assign requires shapes of both tensors to match. lhs shape= [256] rhs shape= [64]
[[node save/Assign_93 (defined at test.py:68) ]]
[[node save/RestoreV2 (defined at test.py:68) ]]

Can anyone tell me how to adapt the network such that it fits the pre-trained weights?
Thanks!

ValueError: Negative dimension size caused by

Thank you for sharing you codes, however, when i run 'train.py', i met the problem as follows: ValueError: Negative dimension size caused by subtracting 5 from 1 for 'global_discriminator/conv5/Conv2D' (op: 'Conv2D') with input shapes: [64,1,1,512], [5,5,512,512].

Pre-training model

Can someone have a pre-training model, can you please send me a copy?

Could you please share the download link of the trained model? I cannot use your download code

Could you please share the download link of the trained model? I cannot use your download code

How to save the test image

@shinseung428 , Thank you for your work! I notice that you try to save the test image in line 73 to 76 in the train.py. However, i don't know how to write the codes which can save the test image based on your codes in the train.py 'line 73 to 76'.

I captured the part of you codes as follows,

--train.py

import tensorflow as tf
from config import *
from network import *


def train(args, sess, model):
.
.
.
.        
69        # Check Test image results every time epoch is finished
70        if step*args.batch_size >= model.data_count:
71            saver.save(sess, args.checkpoints_path + "/model", global_step=epoch)
72
73            #res_img = sess.run(model.test_res_imgs)
74           
75           # save test img result
76            #img_tile(epoch, args, res_img)

When i uncomment the line-73 to line-76, i met the problem "InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype float and shape [64,64,64,3]"

Can you provide the complete codes for saving the test image results?

Train and test image format are different?

Hi~ I found that the train and test images reading format are different, will it be influence the result of test?

No test results

Hello, I got such an image when the test was completed.Everything that has content is black. May I ask why?

Thanks a lot for the code is working. Could you share the trained model?

Hey, could you share the trained model? The train takes too much time, almost a day for each epoch using celebA. I can't get the correct fixed image as result.
Thanks a lot, for the code is working.
It would be my pleasure to get the trained model.

Wrong Implement?

Your implement can run but get bad result,because in network.py global and local discriminator are wrong implemented,would you please right it as soon as possible?

hi, I noticed that in parper , the author mentioned he used a constant color in input image x, but in your code ,you choose 1, maybe the author's idea is the mean pixel value of the training dataset?

Can you help me solve the following problem ？TypeError: Input 'input_sizes' of 'Conv2DBackpropInput' Op has type float32 that does not match expected type of int32.

Can you help me solve the following problem when running the program?

2018-04-23 16:05:10.450605: I C:\tf_jenkins\home\workspace\rel-win\M\windows\PY\35\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
Traceback (most recent call last):
File "E:\Anaconda3\envs\tensorflow1.4\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 510, in _apply_op_helper
preferred_dtype=default_dtype)
File "E:\Anaconda3\envs\tensorflow1.4\lib\site-packages\tensorflow\python\framework\ops.py", line 926, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "E:\Anaconda3\envs\tensorflow1.4\lib\site-packages\tensorflow\python\framework\ops.py", line 774, in _TensorTensorConversionFunction
(dtype.name, t.dtype.name, str(t)))
ValueError: Tensor conversion requested dtype int32 for Tensor with dtype float32: 'Tensor("completion_net/deconv1/conv2d_transpose/output_shape:0", shape=(4,), dtype=float32)'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "test.py", line 108, in
main(args)
File "test.py", line 103, in main
model = network(args)
File "C:\Users\hasee\Desktop\GlobalLocalImageCompletion_TF-master\GlobalLocalImageCompletion_TF-master\network.py", line 27, in init
self.build_model()
File "C:\Users\hasee\Desktop\GlobalLocalImageCompletion_TF-master\GlobalLocalImageCompletion_TF-master\network.py", line 50, in build_model
self.recon_img, self.g_nets = self.completion_net(self.perturbed_img, name="completion_net")
File "C:\Users\hasee\Desktop\GlobalLocalImageCompletion_TF-master\GlobalLocalImageCompletion_TF-master\network.py", line 212, in completion_net
deconv1 = deconv2d(conv8, [self.batch_size, input_shape[1]/2, input_shape[2]/2, 128], name="deconv1")
File "C:\Users\hasee\Desktop\GlobalLocalImageCompletion_TF-master\GlobalLocalImageCompletion_TF-master\architecture.py", line 51, in deconv2d
padding="SAME")
File "E:\Anaconda3\envs\tensorflow1.4\lib\site-packages\tensorflow\python\ops\nn_ops.py", line 1211, in conv2d_transpose
name=name)
File "E:\Anaconda3\envs\tensorflow1.4\lib\site-packages\tensorflow\python\ops\gen_nn_ops.py", line 778, in conv2d_backprop_input
use_cudnn_on_gpu=use_cudnn_on_gpu, data_format=data_format, name=name)
File "E:\Anaconda3\envs\tensorflow1.4\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 533, in _apply_op_helper
(prefix, dtypes.as_dtype(input_arg.type).name))
TypeError: Input 'input_sizes' of 'Conv2DBackpropInput' Op has type float32 that does not match expected type of int32.

Why is the loss so large in my experiment?

I try to train my model by using Celeba dataset.
However, when I start to train my model, the value of the loss has been so large all the time,which seems not to be smaller.
In fact, the loss is almost 120000.
How can I solve this problem?
Thanks!

A negative dimension during conv4d in network.py

Hi, I wonder why is there would be a negative dimension during the training, the result is shown in the snapshot

@shinseung428 Hi, I also find an error. In network.py Line 273.

@shinseung428 Hi, I also find an error. In network.py Line 273.
conv3 = batch_norm(conv1, name="bn3").
I think conv1 should be replaced by conv3.

Originally posted by @Arbiter0 in #4 (comment)

In test.py why we need "cv2.setMouseCallback('mask',erase_rec)''

In test.py why we need "cv2.setMouseCallback''? Because of cv2.setMouseCallback('image',erase_rec), we get the image with hole, why we need the mask?

Model effect

I iterated 400 times to train the model, but the ability to repair the image is very poor, I did not modify the code, I would like to ask you this is also the case?

Hi，how to improve GPU utilization

I use your code to train,but the GPU utilization only is 3% on my computer,why?Training completely spend for a week,now,I test a image.Result not be shown as fast as you said.I have waited for 10 minutes.Could you please help me?Thanks.

Hi Can you share your training model

Thank you!

training error

hi~ When I began to train the model. An problem occurred that
OutOfRangeError (see above for traceback): RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 16, current size 0)

I wonder why