yunjey / stargan

StarGAN - Official PyTorch Implementation (CVPR 2018)

Hi there 👋

stargan's Issues

I wonder in the test phase, if i can control or modify the target label as i want? and how to do it?

rafd_crop_size?

can anyone please let me know what is irafd_crop_size? when you tarin star gan on your dataset

Bad test results

HI
thank you for share your work
I trained the celeba model,now I test the model. The test result is the mode has very good performance on training data, poor performance on my own production of data.
e.g. my test data:
/home/dms/eclipse/workspace/stargan_pytorch04/tangni/results/1-images.jpg
The generate picture has two mouthes, It is looks terrible!
Can you help me If I want to has a good performance on my data,how can I do it?

Why it generate one image only at training and testing time?

Please help me why it generate only one image 8 times during training samples

Test time also generate only image multiple times

Training info:
Training data: RaFD
Arranged according to your instructions
Used these configurations for train following your instructions
$ python main.py --mode train --dataset RaFD --image_size 128 --c_dim 8
--sample_dir stargan_rafd/samples --log_dir stargan_rafd/logs
--model_save_dir stargan_rafd/models --result_dir stargan_rafd/results

--Batch_szie=1 is not work?

Is anyone tried --Batch_szie=1?

celeba dataset can not be downloaded

"--sample_dir" may not work as expected

So far as I know, I see "--sample_dir" arg does not work as expected. It is not used in the code.

Out of memory on GPU Titan X

Hi,

I set batch_size=3, and tried the code on GPU Titan X. The memory cost keeps on increasing, and soon causes out of memory. Must I use a GPU with 24 GB memory? Any suggestions are appreciated.

Thanks!

i wonder if i can do multi-attribute transfer cross two dataset

issues about code

hello, I saw your code of papar "StarGAN". I have some issues.

1 In data_loder.py line 43 (random.shuffle(lines)), each time you get_loader, the lines will be newly shuffled, so these train and test samples may be duplicate ?

2 in paper, equation (6), I saw the first item of loss G is L_{adv}, but in the code, I only saw loss of D(G(x,c)). I have understood that the first item of the equation 6 has no need to put into the loss function, but how about the last item.

Thank you.

Custom evaluation dataset

Hello, thank you for your code.
I performed the training stage and got the model - works perfectly on the default datasets. Is it possible to cast the model on my custom imageset? I created a proper folder structure, created valid metafile, filled all the paths (as i think) and got error

Traceback (most recent call last):
  File "main.py", line 106, in <module>
    main(config)
  File "main.py", line 46, in main
    solver.test()
  File "/home/username/Documents/StarGAN/solver.py", line 679, in test
    target_c_list = self.make_celeb_labels(org_c)
  File "/home/username/Documents/StarGAN/solver.py", line 181, in make_celeb_labels
    c[i] = 0 if c[i] == 1 else 1   # opposite value
IndexError: index 4 is out of range for dimension 0 (of size 4)

Is it possible in general? Can you provide a kind of instruction, so people can use it without tough reverse engineering? Thanks

No data_loader defined for 'test' mode, and small documentation issue

At this point self.data_loader is not defined:

https://github.com/yunjey/StarGAN/blob/master/solver.py#L671

I solved it on my local machine doing:

        # Set dataloader
        if self.dataset == 'CelebA':
            self.data_loader = self.celebA_loader
        else:
            self.data_loader = self.rafd_loader

at the start of test, but not sure how you want to handle 'Both' there so I didn't submit a PR.

Finally, the application states that:

Translated test images and saved into ./test/results..!                                                                                                                                                                                                                                                                       
Translated test images and saved into ./test/results..!                                                                                                                                                                                                                                                                       
Translated test images and saved into ./test/results..!                                                                                                                                                                                                                                                                       
Translated test images and saved into ./test/results..!                                                                                                                                                                                                                                                                       
Translated test images and saved into ./test/results..!  

but it appears to actually be putting them into ./test/samples

Thanks for releasing this.

Attribute list

Hi, I want to re-implement your work. Can you please let me know how you have created the attribute list? Is it manually or you used any code for this?

Please help me

I want to use the pre-trained model for less than 5 attributes by using :

python main.py --mode test --dataset CelebA --image_size 256 --c_dim 2
--selected_attrs Blond_Hair Brown_Hair
--model_save_dir='stargan_celeba_256/models'
--result_dir='stargan_celeba_256/results'

but when I try it I get this:
Loading the trained models from step 200000...
Traceback (most recent call last):
File "main.py", line 110, in
main(config)
File "main.py", line 49, in main
solver.test()
File "C:\Users\gigi\PycharmProjects\StarGANPyTorch\venv\solver.py", line 526, in test
self.restore_model(self.test_iters)
File "C:\Users\gigi\PycharmProjects\StarGANPyTorch\venv\solver.py", line 103, in restore_model
self.G.load_state_dict(torch.load(G_path, map_location=lambda storage, loc: storage))
File "C:\Users\gigi\PycharmProjects\StarGANPyTorch\venv\lib\site-packages\torch\nn\modules\module.py", line 721, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Generator:
While copying the parameter named "main.0.weight", whose dimensions in the model are torch.Size([64, 5, 7, 7]) and whose dimensions in the checkpoint are torch.Size([64, 8, 7, 7]).

When I run the command with the 5 attributes (as you suggest in the README.md ) I do not get this error... What I could do to solve it??

RuntimeError: dimension out of range

Hi,

I am trying to train the model on RaFD using python3 and torch v3. I've already prepared the dataset but after one epoch, this is what I am getting. Can someone please help me solve this?

I printed the size of out_cls, real_label.

I also tried training on CelebA+RaFD but I got cuda memory error. My gpu has only 8Gb of memory space ( I have two of them). Any workaround?

Thanks!

D
Discriminator(
(main): Sequential(
(0): Conv2d (3, 64, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(1): LeakyReLU(0.01, inplace)
(2): Conv2d (64, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(3): LeakyReLU(0.01, inplace)
(4): Conv2d (128, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(5): LeakyReLU(0.01, inplace)
(6): Conv2d (256, 512, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(7): LeakyReLU(0.01, inplace)
(8): Conv2d (512, 1024, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(9): LeakyReLU(0.01, inplace)
(10): Conv2d (1024, 2048, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(11): LeakyReLU(0.01, inplace)
)
(conv1): Conv2d (2048, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(conv2): Conv2d (2048, 8, kernel_size=(2, 2), stride=(1, 1), bias=False)
)
The number of parameters: 44786624
Classification Acc (8 emotional expressions): ['12.50']

Elapsed

Classification Acc (8 emotional expressions): ['12.50']
Elapsed [0:00:04.683473], Epoch [1/200], Iter [10/31], G/loss_cls: 2.1469, G/loss_rec: 0.6333, D/loss_fake: 2.5479, D/loss_cls: 6.0393, D/loss_real: -26.6812, G/loss_fake: -1.5794, D/loss_gp: 0.3869
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
Classification Acc (8 emotional expressions): ['18.75']
Elapsed [0:00:07.881092], Epoch [1/200], Iter [20/31], G/loss_cls: 3.8383, G/loss_rec: 0.5828, D/loss_fake: -5.0278, D/loss_cls: 9.1143, D/loss_real: -30.6130, G/loss_fake: 4.7769, D/loss_gp: 0.7379
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
Classification Acc (8 emotional expressions): ['31.25']
Elapsed [0:00:11.084384], Epoch [1/200], Iter [30/31], G/loss_cls: 3.3109, G/loss_rec: 0.5486, D/loss_fake: 3.4921, D/loss_cls: 4.8124, D/loss_real: -38.8579, G/loss_fake: -0.7082, D/loss_gp: 0.6353
torch.Size([8]) torch.Size([1])
Traceback (most recent call last):
File "main.py", line 106, in
main(config)
File "main.py", line 41, in main
solver.train()
File "/home/ramin/codes/StarGAN/solver.py", line 274, in train
d_loss_cls = F.cross_entropy(out_cls, real_label)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py", line 1140, in cross_entropy
return nll_loss(log_softmax(input, 1), target, weight, size_average, ignore_index, reduce)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py", line 786, in log_softmax
return torch._C._nn.log_softmax(input, dim)
RuntimeError: dimension out of range (expected to be in range of [-1, 0], but got 1)

The word "Tranlsation" is misspelled

interpolation and extrapolation

Is there a way to interpolate/extrapolate the latent space? In order to see the distribution near the inferenced image y given input image x and condition label c.

Will interpolating on c work? The input is currently (h, w, 3 + nc). With the extra nc channels of all zeros or all ones, I'm thinking if there's a way to use them for interpolation.

tensorboard

How can I use the pre-trained models on my own images?

How can I get the model used in this paper ?

How could I get RaFD dataset?

Hello,

CelebA dataset is available in download.sh, but I couldn't find RaFD dataset anywhere.

Could you offer a method to get RaFD dataset?

Thanks a lot

Dimension does not match

python:3.6
pytorch:0.4.0
tensorflow:1.4.0

When running the following code , there is a problem:
python main.py --mode test --dataset CelebA --image_size 128 --c_dim 5 \ --sample_dir stargan_celeba/samples --log_dir stargan_celeba/logs \ --model_save_dir stargan_celeba/models --result_dir stargan_celeba/results \ --selected_attrs Black_Hair Blond_Hair Brown_Hair Male Young

RuntimeError: Error(s) in loading state_dict for Discriminator:
While copying the parameter named "conv2.weight", whose dimensions in the model are torch.Size([5, 2048, 2, 2]) and whose dimensions in the checkpoint are torch.Size([5, 2048, 4, 4]).

Question About the RaFD dataset.

Could you sharing training/testing splitting on RaFD dataset of your StarGAN paper?
Moreover, I download RaFD which has 8,040 images, while in your paper you reported it only has 4,824 images, may you also provide the images you used in your paper?

Request access from RaFD

Greetings,
i sent request for dataset in RaFD, but steel wating approval. Can you please approve it?
P.S. My login is bengalua.
Best regards,
Vladimir.

Please help me

I am confused about the version of StarGAN, because when I running it on GPU, it said:
File "/home/willamyeoh/StarGAN/solver.py", line 53, in init
self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
AttributeError: module 'torch' has no attribute 'device'
in the version I download to GPU, the line 53 of solver.py is: self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
and I found there is another version which I cloned on my own Mac is :self.dtype = torch.cuda.FloatTensor if torch.cuda.is_available() else torch.FloatTensor
Does this has anything to do with the running problem? and could you give me a hint how to solve this problem?

I failed to train with all attributes of celebA

When I use all attributes of celebA to train stargan, It dose not work.

when will update to pytorch0.4+？

Pre-trained models

Dear @yunjey,
Thank you for your nice work. Would you please release your pre-trained models?

Training on my dataset

I wanted to know if is it possible to train my dataset, I want to convert eyes from opened to closed and also the mouth

Please help me to solve this

Preprocess RaFD dataset

Hi, @yunjey
I am confused about the method of preprocessing RaFD, where the download images have initial size 618 x 1024. If I crop the images to 256 x 256, some images became partial absent, especially when the camera angle is 45 degree or 135 degree.

So I crop the initial images as follow code:

elif dataset == 'RaFD':
    box = (100, 150, 600, 700)
    if mode == 'train':
        transform = transforms.Compose([
            transforms.Lambda(lambda x: x.crop(box)),
            transforms.Resize(image_size),
            transforms.RandomHorizontalFlip(),
            transforms.ToTensor(),
            transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
    else:
        transform = transforms.Compose([
            transforms.Lambda(lambda x: x.crop(box)),
            transforms.Resize(image_size),
            transforms.ToTensor(),
            transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
    dataset = ImageFolder(image_path, transform)

And I get some samples like

But I am not sure whether the method is right, could you help me? Thanks in advance.

Inference issue with custom test method

Hi,

I try to write a method that allows me to do inference on one single input image and looked into the solver.test() method for that. My goal is to have a method fakeimage = process(inputimg,attributes).
From what I understand is, that I have to:

1. load the input image
2. use the torchvision transformation pipeline similar to the data_loader on the image
3. load the weights (G.load_state_dict)
4. pass image + attribute array to the network: fakeimg = self.G(image, target_c)

I do not quite understand on how to setup target_c. Also I do not use a batch for inference like in the original test method but only want to pass one single image.

In step 4 I am stuck with this error:

/data/StarGAN/model.py in forward(self, x, c)
     56         # replicate spatially and concatenate domain information
     57         c = c.unsqueeze(2).unsqueeze(3)
---> 58         c = c.expand(c.size(0), c.size(1), x.size(2), x.size(3))
     59         x = torch.cat([x, c], dim=1)
     60         return self.main(x)

RuntimeError: invalid argument 2: dimension 3 out of range of 3D tensor at /pytorch/torch/lib/TH/generic/THTensor.c:24

I assume this has to do with a wrong input of "target_c".

Any suggestions on how to solve this?

Thanks!

Peter

Always out of memory

@yunjey As the training continues, the memory it needs increased ,and stopped when it report the runtimeError said that out of memory.
I don't know why the memory increased all the time as training proceed, and I wonder if i modify the code to use muitl-gpu , whether i will also receive the same error or not?
btw, my gpu memory is 12G
thanks~

thanks for yours ! There is a error for pytorch 0.4?

Traceback (most recent call last):
File "main.py", line 110, in
main(config)
File "main.py", line 49, in main
solver.test()
File "D:\pytorchdemo\StarGAN-master\solver.py", line 526, in test
self.restore_model(self.test_iters)
File "D:\pytorchdemo\StarGAN-master\solver.py", line 103, in restore_model
self.G.load_state_dict(torch.load(G_path, map_location=lambda storage, loc: storage))
File "C:\Program Files\Python35\lib\site-packages\torch\nn\modules\module.py", line 721, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Generator:

Changing attributes

Hi,

great project!
Is there any easy way to change the attributes. It seems some part are hardcoded in solver.py but it is not clear to me what to change if I want to use different CelebA attributes.

Peter

May I ask how do you get the images attribute?

Hi, it is a wonderful project, I learned and got inspired from it. I see there are around 40 attributes in the file - list_attr_celeba.txt. May I ask how do you get the 40 attributes for each CelebA images? (there are over 200K images!!)

pretrained model can not download

hi, thanks for share the code, but the pretrained mode can not down load smoothly, Can someone share them in google drive ? thanks

RaFD dataset training

When training RaFD, you said the input domain is fixed as the "neutral" expression, the target domain varies among the seven remaining expressions.
But I have not found that the "neutral" is assigned in the training process.
out_src, out_cls = self.D(real_x) (in solver.py line 267 )
The real_x is read through torchvision.ImageFolder.
Could you help me? Thank you very much.

test problem on my own data

HI
thank you for share your work
I trained the celeba model,now I test the model. The test result is the mode has very good performance on training data, poor performance on my own production of data.
e.g. my test data:

The generate picture has two mouthes, It is looks terrible!
Can you help me If I want to has a good performance on my data,how can I do it?
I must train my data first?

when will update to pytorch0.4+？

Please help me

i try this code but i have this error

ConnectionRefusedError: [Errno 111] Connection refused

how can i do?

Question about the method

Congratulations. Impressive results!! After reading the paper I still didn't understand completely some aspects. For example, there is no noise input for the generator? Is it possible to generate an image from latent space like a regular GAN?

Thanks!!

Can't make project run without cuda

Hi,

I'm trying to run the model purely on my CPU (i only have an OpenCL supported GPU).

I install pytorch wihout cuda enabled and set

cudnn.benchmark = False

in line13 of main.py yet still get the error:

Traceback (most recent call last):
File "main.py", line 106, in
main(config)
File "main.py", line 41, in main
solver.train()
File "/Users/akbirkhan/Documents/CompVision/Assignment3/StarGAN/solver.py", line 299, in train
alpha = torch.rand(real_x.size(0), 1, 1, 1).cuda().expand_as(real_x)
File "/usr/local/lib/python3.6/site-packages/torch/_utils.py", line 69, in cuda
return new_type(self.size()).copy(self, async)
File "/usr/local/lib/python3.6/site-packages/torch/cuda/init.py", line 358, in _lazy_new
_lazy_init()
File "/usr/local/lib/python3.6/site-packages/torch/cuda/init.py", line 120, in _lazy_init
_check_driver()
File "/usr/local/lib/python3.6/site-packages/torch/cuda/init.py", line 55, in _check_driver
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

Generator Loss Function

In the paper it says, Generator loss function is

But in the code, it was g_loss = g_loss_fake + self.lambda_rec * g_loss_rec + self.lambda_cls * g_loss_cls

Why is g_loss_real excluded in this part?

Pre-trained models

Dear @yunjey,
Thank you for your nice work. Would you please release your pre-trained models?

AssertionError: MaskedFill can't differentiate the mask

hi, when i run the following command , i get a exception, can you help me?
@yunjey

python3 main.py --mode='train' --dataset='CelebA' --c_dim=5 --image_size=128 --sample_path='stargan_celebA/samples' --log_path='stargan_celebA/logs' --model_save_path='stargan_celebA/models' --result_path='stargan_celebA/results'

The number of parameters: 44762048
Traceback (most recent call last):
File "main.py", line 106, in
main(config)
File "main.py", line 41, in main
solver.train()
File "/data1/software/deeplearning/StarGAN/solver.py", line 279, in train
accuracies = self.compute_accuracy(out_cls, real_label, self.dataset)
File "/data1/software/deeplearning/StarGAN/solver.py", line 145, in compute_accuracy
predicted = self.threshold(x)
File "/data1/software/deeplearning/StarGAN/solver.py", line 138, in threshold
x[x >= 0.5] = 1
File "/usr/local/lib/python3.5/dist-packages/torch/autograd/variable.py", line 85, in setitem
return MaskedFill.apply(self, key, value, True)
File "/usr/local/lib/python3.5/dist-packages/torch/autograd/_functions/tensor.py", line 440, in forward
assert not ctx.needs_input_grad[1], "MaskedFill can't differentiate the mask"
AssertionError: MaskedFill can't differentiate the mask

How to generate a picture with two or three attributes

I train a model with 5 attributes:Black_Hair Blond_Hair Brown_Hair Male Young
But how can I generate a picture with two or three attributes like the paper's picture:

How to continue the interrupted training?

Hello, I started the training and the computer shutdown by accident, is there anyway I could continue the interrupted training? Doesn't have to start it over again? thanks.

solver.py problem

In solver.py ，if you have no GPU, the .cuda() will be runtime error
so you must judge the torch.cuda.is_available() first

` # Compute gradient penalty
alpha = torch.rand(real_x.size(0), 1, 1, 1).cuda().expand_as(real_x)
interpolated = Variable(alpha * real_x.data + (1 - alpha) * fake_x.data, requires_grad=True)
out, out_cls = self.D(interpolated)

            grad = torch.autograd.grad(outputs=out,
                                       inputs=interpolated,
                                       grad_outputs=torch.ones(out.size()).cuda(),
                                       retain_graph=True,
                                       create_graph=True,
                                       only_inputs=True)[0]`

number of parameters

Hello @yunjey ,
Thanks for the great project!
I've noticed, that the total number of parameters doesn't change in accordance with image_size. Why so?