universome / epigraf Goto Github PK

Not an issue 😅.
Cool results! So refreshing to see 3D GAN results on something other than faces and toy-cars 😆!
Looking forward to your code.

Cheers!
@akanimax

P.S. To the present owners of GitHub, perhaps it's a good time to introduce a comment feature 😄...

Thank you for releasing the code for your awesome paper! I am trying to reproduce the results on the Plants dataset and get the following error message:

Traceback (most recent call last):
  File "/users/eldar/src/epigraf/src/train.py", line 291, in main
    launch_training(c=c, outdir=cfg.experiment_dir, dry_run=opts.dry_run)
  File "/users/eldar/src/epigraf/src/train.py", line 109, in launch_training
    subprocess_fn(rank=0, c=c, temp_dir=temp_dir)
  File "/users/eldar/src/epigraf/src/train.py", line 52, in subprocess_fn
    training_loop.training_loop(rank=rank, **c)
  File "/users/eldar/src/epigraf/src/training/training_loop.py", line 234, in training_loop
    images = torch.cat([G_ema(z=z, c=c, camera_angles=a, noise_mode='const').cpu() for z, c, a in zip(vis.grid_z, vis.grid_c, vis.grid_camera_angles)]).numpy()
  File "/users/eldar/src/epigraf/src/training/training_loop.py", line 234, in <listcomp>
    images = torch.cat([G_ema(z=z, c=c, camera_angles=a, noise_mode='const').cpu() for z, c, a in zip(vis.grid_z, vis.grid_c, vis.grid_camera_angles)]).numpy()
  File "/users/eldar/src/epigraf/env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/users/eldar/src/epigraf/src/training/networks_epigraf.py", line 494, in forward
    ws = self.mapping(z, c, camera_angles=camera_angles_cond, truncation_psi=truncation_psi, truncation_cutoff=truncation_cutoff, update_emas=update_emas)
  File "/users/eldar/src/epigraf/env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/users/eldar/src/epigraf/src/training/layers.py", line 148, in forward
    misc.assert_shape(c, [None, self.c_dim])
  File "/users/eldar/src/epigraf/src/torch_utils/misc.py", line 95, in assert_shape
    raise AssertionError(f'Wrong size for dimension {idx}: got {size}, expected {ref_size}')
AssertionError: Wrong size for dimension 1: got 0, expected 191

The problem seems to be in the training_loop.py where at test time it passes argument c into the mapping network with the shape (4, 0) instead of presumably (4, 191)

I launch training with the following command:

python src/infra/launch.py hydra.run.dir=. exp_suffix=plants dataset=megascans_plants dataset.resolution=256 training.gamma=0.05 num_gpus=1 +ignore_uncommited_changes=true

Update: even if I comment out this part https://github.com/universome/epigraf/blob/main/src/training/training_loop.py#L232-L236 it still fails further down https://github.com/universome/epigraf/blob/main/src/training/training_loop.py#L323-L334 with the same error, so it is not just a problem at test time.

Update 2: looks like for the Megascans datasets you need to additionally set training.use_labels=true. Will close the issue once I can run the training!

Thanks for the great work! I am wondering whether you have tried to increase the image resolution to 1024x1024?

Thanks for your excellent work!
I trained the model following
python src/infra/launch.py hydra.run.dir=. exp_suffix=<EXPERIMENT_NAME> dataset=<DATASET_NAME> dataset.resolution=<DATASET_RESOLUTION> model.training.gamma=0.1
in FFHQ dataset, but I can not get reasonable results.
Could you please provide configs for your training?
Thanks for your help!

Great work and exciting performance! BTW, How should I run the raw source code?

I have run the source code and it works! I wonder how should I reduce the initial peak gpu memory, so that I can increase the resolution to 512. Thank you in advance!

Hi !
Thank you for sharing this great work !
I was wondering if you could share the dataset.json corresponding to FFHQ (or how exactly to get the [yaw,pitch,roll] vector from the face_poses.zip from GRAM)
Thanks a lot !

Hi, I met some issues when training your model on Cats dataset.

I used Cats dataset which you uploaded with 128x128 resolution with default configuration following "cats_aligned.yaml"
For 128 resolution, I resize the image by PIL.Image.resize function with lanzcos resampling filter, and do not change any values in camera pos label in "dataset.json".
What I only changed is the resolution of triplane, which has a default setting of 512 to 256.
(by changing "configs/model/epigraf.yaml")

The results I got are as follows:

Can you give me some insights why model failed to learn the volume density?

thanks,

Hi, Thank you for your great work!

Do you have a plan for releasing code about the Megascan dataset?
It is a very interesting part of this paper.

Thank you.

Will you provide a pretrained model for your code??

Thanks for your excellent work!
I trained the model following
python src/infra/launch.py hydra.run.dir=. exp_suffix=<EXPERIMENT_NAME> dataset=ffhq_posed dataset.resolution=512 model.training.gamma=0.1 or python src/infra/launch.py hydra.run.dir=. exp_suffix=<EXPERIMENT_NAME> dataset=ffhq_posed dataset.resolution=512 model.training.gamma=0.1 model.discriminator.camera_cond=true model.discriminator.camera_cond_drop_p=0.5
based on the provided FFHQ dataset (https://disk.yandex.ru/d/UmglE8U3YVbuLg), but I can not get reasonable results. The FID@2k value is around 20 after three days's training on 8 A100.
Is there anything wrong with my implementation?
Thanks for your help!

When the custom ffhq dataset is adopted without camera pose information, the error exists in train.py (line 127, "Broken yaw angles (all zeros)"). So how can I reproduce the ffhq training result similar to the one in paper?

Hi, do you have plans to release the megascan checkpoints?
Thank you very much!

Hello there, this is a great work, I tried to reprodce the results on FFHQ, but met the error: "Broken yaw angles (all zeros)".
After that, I turned to reproduce the results on cats.
Unfortuntely, I could find the cats dataset, even searched in the CUHK MM home page.
Could you provide the downloading link of Cats dataset?