hila-chefer / targetclip Goto Github PK

amazing work

let me congratulate you on this incredible job, unfortunately I don't have the computing power to train targets, I'd like you to add some celebrity targets like megan fox and emilia clacke, and other celebrities.
and add new ones, for people who don't have a gpu, every week if they have free time and no work is busy.

I ran this command to reproduce your results:
python3 optimization/find_dirs.py --target_path dirs/targets/avatar.jpg --dir_name results_folder_avatar --weight_decay 3e-3 --lambda_consistency 0.6 --step 1000 --lr 0.2 --num_directions 8 --num_images 8
But I got strange result as bellow: here is some results. Anything wrong with me?

I wonder how to encode target or source image in latent space. I usually use e4e, image2styleGAN, which one did you use?

Hello again,

in these marked lines you initialize a set of coefficients to optimize over. As far as I can see, these are not mentioned in the paper. The coefficients are multiplied by the direction per source image, so I get that you want to optimize for a different scale of the direction vector per source vector. I have some questions on this:

1. Did you try it without these coefficients?
2. To what values do the coefficients converge to? Do they stay close to 1?
3. You re-initialize the Adam optimizer for the coefficients for every step within the optimization, hence drastically changing the behavior of the optimizer. Is this intended or a misplacement? If it is intended, what is it used for?

Thanks again for your work! I hope I am not too picky on this - I'm just curious about the topic of semantics in these latent spaces :-)

Great work! I tried to train my own dirs, but failed in some case. Are you using the default parameters to get them?

steps to reproduce

!git clone  https://github.com/hila-chefer/TargetCLIP.git

%cd /content/TargetCLIP

!python /content/TargetCLIP/optimization/find_dirs.py

Traceback (most recent call last):
  File "/content/TargetCLIP/optimization/find_dirs.py", line 9, in <module>
    from criteria.clip_loss import CLIPLoss
ModuleNotFoundError: No module named 'criteria'

but criteria is in the root directory

In these lines the latents are recalculated in the nested loop in every inner loop. The latents themselves stay constant though, so it can be done once outside of these loops. Only the augmented latents (latents + direction) need to be recalculated here.