browatbn2 / 3fabrec Goto Github PK

Hi, I am trying to use the provided model trained on 300w on a sample unseen image. However I am getting some artifacts in the reconstructions and as a result the predicted landmarks are also random.

Here is what I did:

• Face cropped the image
• Resized to (256, 256)
• Normalized between (0,1) and (-1, 1) with the same result

Am I missing anything here?

Hi @browatbn2

Hope all's well with you!

I have successfully trained and tested the existing pipeline of your work. I am quite excited about using it in my context. I have my own internal dataset with images having 38 landmarks on the face. I'm trying to modify your code to make it work. So far, I have prepared the annotation files in the exact format WFLW demands, and have copied and modified the wflw.py code for the specific number of landmarks. I can save the crops but as soon as training starts, the file 'landmarks/lmutils.py' throws errors. After closer inspection, I noticed that the number 68 is hardcoded at multiple places in lmutils.py.
Really hope if you could guide me with the necessary modifications! Thanks and Take Care!

Hi!
I tried to train the model with only 1 training image. I excuted the training script with the command

python3 train_aae_landmarks.py --sessionname one_shot_aae -r ./ --train-encoder 0 --dataset w300 --train-count 1 --batchsize 1 --epoch 10000 --eval-freq 1000 --save-freq 1000

However, there is still a gap between the trained model and the reported performance in the paper. I got nmes=15.20/18.11/15.93 with the above command. Did I miss any important detail when I trained this model? Or is it just simply because the epoch number is not enough?
Thanks!

Hi,

First, thank you for sharing the repo, it is very well coded and useful.

I have a question about the results reported on the paper when training with limited data (50 samples for instance).
How did you choose the samples to train on ? Was the results based on the first 50 samples of dataset or did you average over multiple random sets ?

Thank you

Hi,
it's a great project, and I run successfully!
Can I know your demo's model trained from which dataset, and is it the best one you mentioned in your paper?

@browatbn2 Hey.
For only the unsupervised reconstruction task, have you tried looking for the training/validation losses? Just plotting the reconstruction loss for all the three steps (recon, with gan, with gan & with ssim) the losses don't seem to converge and are jittery, although the reconstruction quality improves. Can you comment on this behavior of loss curves?

Hello, I have learned a lot from reading your paper. It is an excellent job. However, I met some problems in the process of reading the paper. I hope you can help me.
1、How is Fig. 3 and Fig.4 generated? I find it is not stated more in the paper. I would like to know more about it.
2、Since it is possible to generate many different poses or styles from the original input image, can these images be used as additional data augmentation to train other supervised models ？
Thanks！

Hello,
Could you share the alfw.pkl file, please? Thanks a lot!

Hi!
Thanks for your great work and detailed code sharing.
I noticed that you provided a pre-trained model, and I suppose this model is trained on the full WFLW, right?
Would you like to also share the pre-trained adversarial autoencoder? Since the model seems to be very useful to train on different datasets and tasks but the data preparation and training will take several days.
I would be grateful if you can share this interesting model.
Thx!

I was wondering where I can get this csv file. Grateful for your answers!

Hi, I have some questions about AFLW annotations.

In the AFLW dataset code, you merge old and new annotations. Is the only reason to keep the "yaw, pitch, roll" keys? These keys are not used anywhere else in the code so is this operation necessary?

Also, after the merge 10 annotations are missing, the training dataset has 19993 annotations instead of 20000, and the test dataset has 4383 instead of 4886. Did you encounter this issue?

“AttributeError: module 'os' has no attribute 'uname'” when running train_aae_landmarks.py，why？Thanks~

Thanks for your stimulating work!

I am curious about the unsupervised training process, so I read the related source code. But there is something I cannot understand and I will describe as below:
As I can see, the parameters updating follow the procedure: L_enc for updating D_z (4 iters once) and Encoder (2 iters once); L_adv for updating D_x (2 iters once); L_rec+L_cs+L_adv for updating Encoder & Generator (1 iter once). I hope this is right. 😄
Here comes the question.
Firstly, L_enc for updating Encoder leaves gradient information coming form L_enc which is not useful afterwards but I cannot find any code zero_grad the gradient in Encoder. (The leaving gradient may even damage the training after ❔ ) Since you have done
self.saae.zero_grad()
in train_aae_unsupervised.py in line 228, I think that self.saae.P.zero_grad() in line 262 can be changed to self.saae.Q.zero_grad(). Then I think it will make sense.
Secondly, I would like to ask why the updating frequency of Discriminator is set to be slower than that of Generator?

Hope to hear from you soon~ 👍

Hi @browatbn2 ! Thanks for sharing your work. I was trying to reproduce your results by training the autoencoder.
I get this error when I run python train_aae_unsupervised.py --sessionname my_model --daug 4 --input-size 256 --epoch 1

Traceback (most recent call last):
File "train_aae_unsupervised.py", line 462, in
run(args)
File "train_aae_unsupervised.py", line 403, in run
fntr.train(num_epochs=args.epochs)
File "train_aae_unsupervised.py", line 171, in train
self._run_epoch(self.datasets[TRAIN])
File "train_aae_unsupervised.py", line 214, in _run_epoch
self._run_batch(data, eval=eval)
File "train_aae_unsupervised.py", line 304, in _run_batch
loss.backward()
File "/home/aseem/anaconda3/envs/myenv/lib/python3.8/site-packages/torch/tensor.py", line 198, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/aseem/anaconda3/envs/myenv/lib/python3.8/site-packages/torch/autograd/init.py", line 98, in backward
Variable._execution_engine.run_backward(
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [512, 99]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

I understand that pytorch==1.1 has been tested for this version. Due to mismatch with CUDA versions and latest hardware (V100 doesn't support CUDA 10.0 while pytorch 1.1 requires cuda 10.0), I am unable to install pytorch 1.1. Please let me know if you could help out with the error. Best Regards,

Hi, excuse me.
I didn't find eval_aae_landmarks.py in the package.

hi~thanks for your work. I am very interested in it.
I want to run the demo, can you provide the trained model?
Thank you a lot!!!

Coud someone share the vggface2 with me please? Thanks a lot!
All the targets to vggface2 are dead.