brummi / derender3d Goto Github PK

The script you've provided in setup/download_processed_celebahq.sh does not provide links for downloading the CelebAHQ dataset. What links should we use if we want to train the network on this dataset? Thanks!

Thanks for sharing your excellent work!
I want to test this model, but I can't download it by using set_up/download_model.sh(received nothing), so I tried with entering website https://www.robots.ox.ac.uk/~vgg/research/derender3d/data, but received You don't have permission to access this resource.
Can you help? thanks a lot.

Thanks for your work and code, I have a few questions about Extracting Coarse Light & Albedo.

1. How should the total variation regularization (TV) mentioned in the paper be calculated?
2. How large is the image size for which the calculation takes less than 1 second as mentioned in the paper?
3. if I want to estimate the coarse light parameters and albedo map on another dataset, what optimization method should I choose, can scipy.optimize.last_squares work?

Hi,

Thanks for the great work. When I try to inference the model and loading the checkpoints (CO3D), with whatever script I always get such errors: ModuleNotFoundError: No module named 'unsup3d.

@Brummi Could you please guide me on how to inference custom images using the pre-trained CO3D models? Thanks!

Amazing work there! Very glad you release the training code and the preprocessed Co3D datasets.

But I cannot reproduce the Co3D dataset or the CelebA-HQ dataset.

What is the “extracted” folder in the dataloader?

Why are the dimensions of light (1x3) and view matrix (2x3) different from those of unsup3d?

When will the data setup scripts be released?

How could I do the preprocessing step on my own or other published scanned object datasets such as GSO (Google Scanned Objects)?

Thanks again for this fantastic job!

Hi, thanks for the great work.

May I ask how can I train the model on my own dataset? From the paper, seems there is one step to extract coarse light and albedo information, just want to know how I can do that for my own dataset.

Thanks in advance.

Hey there,
one stupid question, I don't really understand the code of get_normal_from_depth and some related code like depth_to_3d_grid, can you please tell me where I can learn this(compute normal map from depth map)? It's quiet different from what I saw on https://stackoverflow.com/questions/34644101/calculate-surface-normals-from-depth-image-using-neighboring-pixels-cross-produc

If I get the optimised $L_c$ from Equation 3, then the aggregated shading map is very close to $2 B_u$.
Then watching Equation 4, why not just represent coarse albedo from $B_u$ ?
$$\tilde{A}=I_u (2 B_u)^{-1} $$

Hello @Brummi, I was trying to run inference using scripts/images_decomposition_co3d.py, however, it looks like the code always relies on precomputed stuff e.g., depth, albedo, normal map etc. For example when running for the category = 'bench', the test_path_precompute is always set to 'datasets/co3d/extracted_bench/precomputed/val' and inspecting the dictionary (data_dict) shows all the tensors are already set essentially to non-zero values:

print('data_dict.keys(): ', data_dict.keys())
print(data_dict['input_im'].shape)
print(data_dict['recon_albedo'].shape)
print(data_dict['recon_depth'].shape)
print(data_dict['recon_normal'].shape)
print(data_dict['recon_normal'][0,0,:10,:10])

Shouldn't the test image be run without relying on any pre-computed inputs and just the single RGB image? as that is the perception I had reading the paper. All these tensors should be initialized to zero except the input_im. May be having a single minimum inference example can help here: loading the model and running inference for a sample face/object image.

Excellent work!
I was wondering how to apply target lighting to single source image like Fig.6 in the paper?
Could you teach me how to do that? Thanks!

Hi, thanks for the reproducibility of your code.

A little question about the result of co3d and eval_cosy.py.
I got result as follow:

  35.5Hz       Normal_l1: 0.96366      Normal_mse: 0.17515     Normal_dot: 0.26274     Normal_deviation: 37.98527      Albedo_sie: 0.07594     Albedo_l1: 0.85689      Albedo_ssim: 0.76152    Spec_l1: 0.12306        Spec_mse: 0.07565       Spec_sie: 0.04684

from which Spec_l1: 0.12306 Spec_mse: 0.07565 are different from reported results in Table 2.

The distance from light source to object surface are related to depth. Why not consider it in your light model?

I tried this code and saved normal map from depth and refinement normal map, like this:
normal from depth map:

normal from network:

I'm confused, why is the refinement normal map half red half green? looking forward to your response!