zengxianyu / co-mod-gan-pytorch

pytorch implementation of the paper ``Large Scale Image Completion via Co-Modulated Generative Adversarial Networks"

Hi there 👋

zengxianyu/zengxianyu is a ✨ special ✨ repository because its README.md (this file) appears on your GitHub profile.

Here are some ideas to get you started:

• 🔭 I’m currently working on phd
• 🌱 I’m currently learning machine learning
• 👯 I’m looking to collaborate on computer vision

co-mod-gan-pytorch's Issues

The output is still incomplete

python test.py -i imgs/ffhq_in.png -m imgs/ffhq_m.png -o ./imgs/example_output.jpg -c checkpoints/co-mod-gan-ffhq-9-025000.pth
doesn't work

inference for NAN

why the model forward generate the nan?

multiple GPU training issue

Hi, thanks for open sourcing the code. I have tried using multiple GPU training below

python train.py \
	--batchSize 8 \
	--nThreads 8 \
	--name "$exp_name" \
	--load_pretrained_g_ema "$pretrain_weight" \
	--train_image_dir "$dataset_root"/"img_512" \
	--train_image_list "$dataset_root"/"train_img_list.txt" \
	--train_image_postfix ".png" \
	--val_image_dir "$dataset_root""/img_512" \
	--val_image_list "$dataset_root"/"val_mask_list.txt" \
	--val_mask_dir "$dataset_root"/"mask_512" \
	--val_image_postfix ".png" \
	--load_size 512 \
	--crop_size 512 \
	--z_dim 512 \
	--validation_freq 10000 \
	--niter 50 \
	--dataset_mode trainimage \
	--trainer stylegan2 \
	--dataset_mode_train trainimage \
	--dataset_mode_val valimage \
	--model comod \
	--netG comodgan \
	--netD comodgan \
	--no_l1_loss \
	--no_vgg_loss \
	--preprocess_mode scale_shortside_and_crop \
	--save_epoch_freq 10 \
	--gpu_id 0,1,2,3
	$EXTRA

and received the error: (This problem didn't have in the single gpu training)

(epoch: 1, iters: 9904, time: 0.171) GAN: 1.7399 path: 0.0003 D_real: 0.4633 D_Fake: 0.6500 r1: 0.2954
(epoch: 1, iters: 10000, time: 0.215) GAN: 1.4925 path: 0.0003 D_real: 0.3935 D_Fake: 0.9652 r1: 0.2954
saving the latest model (epoch 1, total_steps 10000)
Saved current iteration count at ./checkpoints/comod-ffhq-512-4gpus/iter.txt.
doing validation
warnings.warn('Was asked to gather along dimension 0, but all '
Traceback (most recent call last):
File "train.py", line 138, in
generated,_ = model(data_ii, mode='inference')
File "/binaries/anaconda3/envs/torch_py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/binaries/anaconda3/envs/torch_py36/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 168, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/binaries/anaconda3/envs/torch_py36/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 178, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/binaries/anaconda3/envs/torch_py36/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 86, in parallel_apply
output.reraise()
File "/binaries/anaconda3/envs/torch_py36/lib/python3.6/site-packages/torch/_utils.py", line 434, in reraise
raise exception
TypeError: Caught TypeError in replica 3 on device 3.
Original Traceback (most recent call last):
File "/binaries/anaconda3/envs/torch_py36/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
output = module(*input, **kwargs)
File "/binaries/anaconda3/envs/torch_py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
TypeError: forward() missing 1 required positional argument: 'data'

Do you know what could be the problem?

Run Test.sh problem

Hello! I run test.sh but got issue

 --which_epoch co-mod-gan-places2-050000 \
                                     ^
SyntaxError: leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers

Could you please add the training code for the co-mod-gan?

Can you upload the training codes? Appreciated.

Is there a torch version of the co-mod-gan-coco-stuff-025000.pth pretrained model ?

Is there a torch version of the co-mod-gan-coco-stuff-025000.pth pretrained model ?

input size

Hello, can the input of this network be an image of any size?

Diversity of generated images

Hi @zengxianyu !!
Thank you for sharing your nice work!
I’d like to ask you about how much generated images have diversity.

I think your architecture depends on StyleGAN so that the generated images have several structures or appearance.
However, without truncation trick, the generated images look similar among all batchs.

Is it correct output?
Input image

Output images

from pytorch_fid.fid_model import FIDModel

ModuleNotFoundError : No module named 'pytorch_fid.fid_model' and I used 'pip install pytorch_fid 'still cannot be solved. Could anyone else tell me to download this dependency library?

about co-mod-gan-coco-stuff-025000.pkl and co-mod-gan-places2-050000.pkl

thanks. nice work. have you convert other models to torch format ?

Conversion from TF Version to PyTorch

Hi,
Do you mind elaborating on how you converted the TF Version to PT ? Did you convert using onnx or some other model converter and then wrote the PT code or did you train the PT code from scratch and created your own weights ? On the first glance it seems you did the former ?

Training Code for Co-Mod-GAN

Hi , is there any plan to add code for training ?
The code for the discriminator and other parts of the model also doesn't exist. Is there any plan to add them ? Thanks .

Code for TF weight to convert Torch weight

hello
Would you mind adding the code to convert the TF weight to Torch weight ?

No module named 'data.coco_dataset'

I would like to know how this error is resolved
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'data.coco_dataset'

How to convert tf model to pytorch model?

noise injection

Thanks for your contributions.
I have a question, is there no noise injection?