yunjey / stargan Goto Github PK
View Code? Open in Web Editor NEWStarGAN - Official PyTorch Implementation (CVPR 2018)
License: MIT License
StarGAN - Official PyTorch Implementation (CVPR 2018)
License: MIT License
In solver.py ๏ผif you have no GPU, the .cuda() will be runtime error
so you must judge the torch.cuda.is_available() first
` # Compute gradient penalty
alpha = torch.rand(real_x.size(0), 1, 1, 1).cuda().expand_as(real_x)
interpolated = Variable(alpha * real_x.data + (1 - alpha) * fake_x.data, requires_grad=True)
out, out_cls = self.D(interpolated)
grad = torch.autograd.grad(outputs=out,
inputs=interpolated,
grad_outputs=torch.ones(out.size()).cuda(),
retain_graph=True,
create_graph=True,
only_inputs=True)[0]`
So far as I know, I see "--sample_dir" arg does not work as expected. It is not used in the code.
When training RaFD, you said the input domain is fixed as the "neutral" expression, the target domain varies among the seven remaining expressions.
But I have not found that the "neutral" is assigned in the training process.
out_src, out_cls = self.D(real_x) (in solver.py line 267 )
The real_x is read through torchvision.ImageFolder.
Could you help me? Thank you very much.
I want to use the pre-trained model for less than 5 attributes by using :
python main.py --mode test --dataset CelebA --image_size 256 --c_dim 2
--selected_attrs Blond_Hair Brown_Hair
--model_save_dir='stargan_celeba_256/models'
--result_dir='stargan_celeba_256/results'
but when I try it I get this:
Loading the trained models from step 200000...
Traceback (most recent call last):
File "main.py", line 110, in
main(config)
File "main.py", line 49, in main
solver.test()
File "C:\Users\gigi\PycharmProjects\StarGANPyTorch\venv\solver.py", line 526, in test
self.restore_model(self.test_iters)
File "C:\Users\gigi\PycharmProjects\StarGANPyTorch\venv\solver.py", line 103, in restore_model
self.G.load_state_dict(torch.load(G_path, map_location=lambda storage, loc: storage))
File "C:\Users\gigi\PycharmProjects\StarGANPyTorch\venv\lib\site-packages\torch\nn\modules\module.py", line 721, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Generator:
While copying the parameter named "main.0.weight", whose dimensions in the model are torch.Size([64, 5, 7, 7]) and whose dimensions in the checkpoint are torch.Size([64, 8, 7, 7]).
When I run the command with the 5 attributes (as you suggest in the README.md ) I do not get this error... What I could do to solve it??
Is anyone tried --Batch_szie=1?
Hi,
I'm trying to run the model purely on my CPU (i only have an OpenCL supported GPU).
I install pytorch wihout cuda enabled and set
cudnn.benchmark = False
in line13 of main.py yet still get the error:
Traceback (most recent call last):
File "main.py", line 106, in
main(config)
File "main.py", line 41, in main
solver.train()
File "/Users/akbirkhan/Documents/CompVision/Assignment3/StarGAN/solver.py", line 299, in train
alpha = torch.rand(real_x.size(0), 1, 1, 1).cuda().expand_as(real_x)
File "/usr/local/lib/python3.6/site-packages/torch/_utils.py", line 69, in cuda
return new_type(self.size()).copy(self, async)
File "/usr/local/lib/python3.6/site-packages/torch/cuda/init.py", line 358, in _lazy_new
_lazy_init()
File "/usr/local/lib/python3.6/site-packages/torch/cuda/init.py", line 120, in _lazy_init
_check_driver()
File "/usr/local/lib/python3.6/site-packages/torch/cuda/init.py", line 55, in _check_driver
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
Congratulations. Impressive results!! After reading the paper I still didn't understand completely some aspects. For example, there is no noise input for the generator? Is it possible to generate an image from latent space like a regular GAN?
Thanks!!
Hi, @yunjey
I am confused about the method of preprocessing RaFD, where the download images have initial size 618 x 1024. If I crop the images to 256 x 256, some images became partial absent, especially when the camera angle is 45 degree or 135 degree.
So I crop the initial images as follow code:
elif dataset == 'RaFD':
box = (100, 150, 600, 700)
if mode == 'train':
transform = transforms.Compose([
transforms.Lambda(lambda x: x.crop(box)),
transforms.Resize(image_size),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
else:
transform = transforms.Compose([
transforms.Lambda(lambda x: x.crop(box)),
transforms.Resize(image_size),
transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
dataset = ImageFolder(image_path, transform)
But I am not sure whether the method is right, could you help me? Thanks in advance.
Traceback (most recent call last):
File "main.py", line 110, in
main(config)
File "main.py", line 49, in main
solver.test()
File "D:\pytorchdemo\StarGAN-master\solver.py", line 526, in test
self.restore_model(self.test_iters)
File "D:\pytorchdemo\StarGAN-master\solver.py", line 103, in restore_model
self.G.load_state_dict(torch.load(G_path, map_location=lambda storage, loc: storage))
File "C:\Program Files\Python35\lib\site-packages\torch\nn\modules\module.py", line 721, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Generator:
When I use all attributes of celebA to train stargan, It dose not work.
Hi,
I try to write a method that allows me to do inference on one single input image and looked into the solver.test() method for that. My goal is to have a method fakeimage = process(inputimg,attributes).
From what I understand is, that I have to:
fakeimg = self.G(image, target_c)
I do not quite understand on how to setup target_c. Also I do not use a batch for inference like in the original test method but only want to pass one single image.
In step 4 I am stuck with this error:
/data/StarGAN/model.py in forward(self, x, c)
56 # replicate spatially and concatenate domain information
57 c = c.unsqueeze(2).unsqueeze(3)
---> 58 c = c.expand(c.size(0), c.size(1), x.size(2), x.size(3))
59 x = torch.cat([x, c], dim=1)
60 return self.main(x)
RuntimeError: invalid argument 2: dimension 3 out of range of 3D tensor at /pytorch/torch/lib/TH/generic/THTensor.c:24
I assume this has to do with a wrong input of "target_c".
Any suggestions on how to solve this?
Thanks!
Peter
Hi, I want to re-implement your work. Can you please let me know how you have created the attribute list? Is it manually or you used any code for this?
Please help me why it generate only one image 8 times during training samples
Test time also generate only image multiple times
Training info:
Training data: RaFD
Arranged according to your instructions
Used these configurations for train following your instructions
$ python main.py --mode train --dataset RaFD --image_size 128 --c_dim 8
--sample_dir stargan_rafd/samples --log_dir stargan_rafd/logs
--model_save_dir stargan_rafd/models --result_dir stargan_rafd/results
I am confused about the version of StarGAN, because when I running it on GPU, it said:
File "/home/willamyeoh/StarGAN/solver.py", line 53, in init
self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
AttributeError: module 'torch' has no attribute 'device'
in the version I download to GPU, the line 53 of solver.py is: self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
and I found there is another version which I cloned on my own Mac is :self.dtype = torch.cuda.FloatTensor if torch.cuda.is_available() else torch.FloatTensor
Does this has anything to do with the running problem? and could you give me a hint how to solve this problem?
Dear @yunjey,
Thank you for your nice work. Would you please release your pre-trained models?
hello, I saw your code of papar "StarGAN". I have some issues.
1 In data_loder.py line 43 (random.shuffle(lines)), each time you get_loader, the lines will be newly shuffled, so these train and test samples may be duplicate ?
2 in paper, equation (6), I saw the first item of loss G is L_{adv}, but in the code, I only saw loss of D(G(x,c)). I have understood that the first item of the equation 6 has no need to put into the loss function, but how about the last item.
Thank you.
At this point self.data_loader is not defined:
https://github.com/yunjey/StarGAN/blob/master/solver.py#L671
I solved it on my local machine doing:
# Set dataloader
if self.dataset == 'CelebA':
self.data_loader = self.celebA_loader
else:
self.data_loader = self.rafd_loader
at the start of test, but not sure how you want to handle 'Both' there so I didn't submit a PR.
Finally, the application states that:
Translated test images and saved into ./test/results..!
Translated test images and saved into ./test/results..!
Translated test images and saved into ./test/results..!
Translated test images and saved into ./test/results..!
Translated test images and saved into ./test/results..!
but it appears to actually be putting them into ./test/samples
Thanks for releasing this.
hi, when i run the following command , i get a exception, can you help me?
@yunjey
python3 main.py --mode='train' --dataset='CelebA' --c_dim=5 --image_size=128 --sample_path='stargan_celebA/samples' --log_path='stargan_celebA/logs' --model_save_path='stargan_celebA/models' --result_path='stargan_celebA/results'
The number of parameters: 44762048
Traceback (most recent call last):
File "main.py", line 106, in
main(config)
File "main.py", line 41, in main
solver.train()
File "/data1/software/deeplearning/StarGAN/solver.py", line 279, in train
accuracies = self.compute_accuracy(out_cls, real_label, self.dataset)
File "/data1/software/deeplearning/StarGAN/solver.py", line 145, in compute_accuracy
predicted = self.threshold(x)
File "/data1/software/deeplearning/StarGAN/solver.py", line 138, in threshold
x[x >= 0.5] = 1
File "/usr/local/lib/python3.5/dist-packages/torch/autograd/variable.py", line 85, in setitem
return MaskedFill.apply(self, key, value, True)
File "/usr/local/lib/python3.5/dist-packages/torch/autograd/_functions/tensor.py", line 440, in forward
assert not ctx.needs_input_grad[1], "MaskedFill can't differentiate the mask"
AssertionError: MaskedFill can't differentiate the mask
@yunjey As the training continues, the memory it needs increased ,and stopped when it report the runtimeError said that out of memory.
I don't know why the memory increased all the time as training proceed, and I wonder if i modify the code to use muitl-gpu , whether i will also receive the same error or not?
btw, my gpu memory is 12G
thanks~
Hi,
I set batch_size=3, and tried the code on GPU Titan X. The memory cost keeps on increasing, and soon causes out of memory. Must I use a GPU with 24 GB memory? Any suggestions are appreciated.
Thanks!
HI
thank you for share your work
I trained the celeba model,now I test the model. The test result is the mode has very good performance on training data, poor performance on my own production of data.
e.g. my test data:
/home/dms/eclipse/workspace/stargan_pytorch04/tangni/results/1-images.jpg
The generate picture has two mouthes, It is looks terrible!
Can you help me If I want to has a good performance on my data,how can I do it?
HI
thank you for share your work
I trained the celeba model,now I test the model. The test result is the mode has very good performance on training data, poor performance on my own production of data.
e.g. my test data:
The generate picture has two mouthes, It is looks terrible!
Can you help me If I want to has a good performance on my data,how can I do it?
I must train my data first?
Hello, thank you for your code.
I performed the training stage and got the model - works perfectly on the default datasets. Is it possible to cast the model on my custom imageset? I created a proper folder structure, created valid metafile, filled all the paths (as i think) and got error
Traceback (most recent call last):
File "main.py", line 106, in <module>
main(config)
File "main.py", line 46, in main
solver.test()
File "/home/username/Documents/StarGAN/solver.py", line 679, in test
target_c_list = self.make_celeb_labels(org_c)
File "/home/username/Documents/StarGAN/solver.py", line 181, in make_celeb_labels
c[i] = 0 if c[i] == 1 else 1 # opposite value
IndexError: index 4 is out of range for dimension 0 (of size 4)
Is it possible in general? Can you provide a kind of instruction, so people can use it without tough reverse engineering? Thanks
Hello, I started the training and the computer shutdown by accident, is there anyway I could continue the interrupted training? Doesn't have to start it over again? thanks.
python:3.6
pytorch:0.4.0
tensorflow:1.4.0
When running the following code , there is a problem:
python main.py --mode test --dataset CelebA --image_size 128 --c_dim 5 \ --sample_dir stargan_celeba/samples --log_dir stargan_celeba/logs \ --model_save_dir stargan_celeba/models --result_dir stargan_celeba/results \ --selected_attrs Black_Hair Blond_Hair Brown_Hair Male Young
RuntimeError: Error(s) in loading state_dict for Discriminator:
While copying the parameter named "conv2.weight", whose dimensions in the model are torch.Size([5, 2048, 2, 2]) and whose dimensions in the checkpoint are torch.Size([5, 2048, 4, 4]).
can anyone please let me know what is irafd_crop_size? when you tarin star gan on your dataset
Could you sharing training/testing splitting on RaFD dataset of your StarGAN paper?
Moreover, I download RaFD which has 8,040 images, while in your paper you reported it only has 4,824 images, may you also provide the images you used in your paper?
Dear @yunjey,
Thank you for your nice work. Would you please release your pre-trained models?
hi, thanks for share the code, but the pretrained mode can not down load smoothly, Can someone share them in google drive ? thanks
I wanted to know if is it possible to train my dataset, I want to convert eyes from opened to closed and also the mouth
Hello @yunjey ,
Thanks for the great project!
I've noticed, that the total number of parameters doesn't change in accordance with image_size. Why so?
Hello,
CelebA dataset is available in download.sh, but I couldn't find RaFD dataset anywhere.
Could you offer a method to get RaFD dataset?
Thanks a lot
Hi, it is a wonderful project, I learned and got inspired from it. I see there are around 40 attributes in the file - list_attr_celeba.txt. May I ask how do you get the 40 attributes for each CelebA images? (there are over 200K images!!)
Is there a way to interpolate/extrapolate the latent space? In order to see the distribution near the inferenced image y given input image x and condition label c.
Will interpolating on c work? The input is currently (h, w, 3 + nc). With the extra nc channels of all zeros or all ones, I'm thinking if there's a way to use them for interpolation.
i try this code but i have this error
ConnectionRefusedError: [Errno 111] Connection refused
how can i do?
Hi,
I am trying to train the model on RaFD using python3 and torch v3. I've already prepared the dataset but after one epoch, this is what I am getting. Can someone please help me solve this?
I printed the size of out_cls, real_label.
I also tried training on CelebA+RaFD but I got cuda memory error. My gpu has only 8Gb of memory space ( I have two of them). Any workaround?
Thanks!
D
Discriminator(
(main): Sequential(
(0): Conv2d (3, 64, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(1): LeakyReLU(0.01, inplace)
(2): Conv2d (64, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(3): LeakyReLU(0.01, inplace)
(4): Conv2d (128, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(5): LeakyReLU(0.01, inplace)
(6): Conv2d (256, 512, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(7): LeakyReLU(0.01, inplace)
(8): Conv2d (512, 1024, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(9): LeakyReLU(0.01, inplace)
(10): Conv2d (1024, 2048, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(11): LeakyReLU(0.01, inplace)
)
(conv1): Conv2d (2048, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(conv2): Conv2d (2048, 8, kernel_size=(2, 2), stride=(1, 1), bias=False)
)
The number of parameters: 44786624
Classification Acc (8 emotional expressions): ['12.50']
Elapsed
Classification Acc (8 emotional expressions): ['12.50']
Elapsed [0:00:04.683473], Epoch [1/200], Iter [10/31], G/loss_cls: 2.1469, G/loss_rec: 0.6333, D/loss_fake: 2.5479, D/loss_cls: 6.0393, D/loss_real: -26.6812, G/loss_fake: -1.5794, D/loss_gp: 0.3869
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
Classification Acc (8 emotional expressions): ['18.75']
Elapsed [0:00:07.881092], Epoch [1/200], Iter [20/31], G/loss_cls: 3.8383, G/loss_rec: 0.5828, D/loss_fake: -5.0278, D/loss_cls: 9.1143, D/loss_real: -30.6130, G/loss_fake: 4.7769, D/loss_gp: 0.7379
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
torch.Size([16, 8]) torch.Size([16])
Classification Acc (8 emotional expressions): ['31.25']
Elapsed [0:00:11.084384], Epoch [1/200], Iter [30/31], G/loss_cls: 3.3109, G/loss_rec: 0.5486, D/loss_fake: 3.4921, D/loss_cls: 4.8124, D/loss_real: -38.8579, G/loss_fake: -0.7082, D/loss_gp: 0.6353
torch.Size([8]) torch.Size([1])
Traceback (most recent call last):
File "main.py", line 106, in
main(config)
File "main.py", line 41, in main
solver.train()
File "/home/ramin/codes/StarGAN/solver.py", line 274, in train
d_loss_cls = F.cross_entropy(out_cls, real_label)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py", line 1140, in cross_entropy
return nll_loss(log_softmax(input, 1), target, weight, size_average, ignore_index, reduce)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py", line 786, in log_softmax
return torch._C._nn.log_softmax(input, dim)
RuntimeError: dimension out of range (expected to be in range of [-1, 0], but got 1)
Hi,
great project!
Is there any easy way to change the attributes. It seems some part are hardcoded in solver.py but it is not clear to me what to change if I want to use different CelebA attributes.
Peter
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.