zhaofang0627 / anchorudf

Learning Anchored Unsigned Distance Functions with Gradient Direction Alignment for Single-view Garment Reconstruction (ICCV 2021 Oral)

This repository contains the code for the paper:

Learning Anchored Unsigned Distance Functions with Gradient Direction Alignment for Single-view Garment Reconstruction
Fang Zhao, Wenhao Wang, Shengcai Liao and Ling Shao

Requirements

• Cuda 9.2
• Python 3.7
• Pytorch 1.6
• ChamferDistancePytorch

To install all python dependencies for this project:

conda env create -f env_anchor_udf.yml
conda activate AnchorUDF

Dataset

• Deep Fashion3D

We provide the preprocessed data used for model training and evaluation. You can prepare your own data by following the data generation steps of PIFu.

Demo

To do a quick test, download the trained models and run:

python -m apps.eval --results_path {path_of_output} --name {folder_of_output} --dataroot {path_of_dataset} --test_folder_path {folder_of_test_data, e.g., 290-1} --load_netG_checkpoint_path ./checkpoints/anchor_udf_df3d/netG_epoch_59 --anchor --num_steps 5 --filter_val 0.007

For the HD version:

python -m apps.eval_hd --results_path {path_of_output} --name {folder_of_output} --dataroot {path_of_dataset} --test_folder_path {folder_of_test_data, e.g., 290-1} --load_netMR_checkpoint_path ./checkpoints/anchor_udf_hd_df3d/netMR_epoch_14 --anchor --merge_layer 2 --joint_train --loadSize 1024 --num_steps 5 --filter_val 0.007

Optionally, you can remove outliers by statistical outlier removal:

python -m apps.remove_outlier --file_path {path_of_file} --nb_neighbors 5 --std_ratio 10.0

Training

To generate targets for training:

python -m apps.gen_targets --dataroot {path_of_dataset} --sigma {0.003, 0.02, AND 0.08, separately} --point_num 600

To train the model:

• First run:

python -m apps.train --dataroot {path_of_dataset} --random_flip --random_scale --random_trans --anchor --learning_rate 5e-5 --batch_size 4 --name {path_of_saved_model} --schedule 40 --num_epoch 50

• Then add gradient direction alignment:

python -m apps.train --dataroot {path_of_dataset} --random_flip --random_scale --random_trans --anchor --learning_rate 5e-6 --batch_size 4 --num_sample_inout 2000 --name {path_of_saved_model} --grad_constraint --backbone_detach --no_num_eval --continue_train --resume_epoch 49 --num_epoch 60

Evaluation

To evaluate the model:

• Obtain reconstruction results on the test set:

python -m apps.eval_all --dataroot {path_of_dataset} --results_path {path_of_output} --load_netG_checkpoint_path {path_of_model} --anchor --num_steps 5 --filter_val 0.007

• Compute Chamfer and P2S errors:

python -m apps.compute_errors --root_path {path_of_dataset} --results_path {path_of_output}

Acknowledgement

Our code is based on PIFu and NDF. We thank the authors for their excellent work!

Citation

If you use this code for your research, please consider citing:

@inproceedings{zhao2021learning,
  title={Learning Anchored Unsigned Distance Functions with Gradient Direction Alignment for Single-view Garment Reconstruction},
  author={Zhao, Fang and Wang, Wenhao and Liao, Shengcai and Shao, Ling},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={12674--12683},
  year={2021}
}

anchorudf's Issues

数据集处理中labels*缩放比例

您好，最近在阅读您的代码中发现数据及处理文件TrainDatasetDF3D.py中

第265行中对应的udf值labels乘以缩放比例scale_intrinsic，可以请问这样做的含义是什么吗

Missing data from the linked download page

Hi, it seems that the data downloading page lacks RENDER_512 and MASK_512 which lead to the FileNotFoundError: [Errno 2] No such file or directory: '/data/Datasets/AnchorUDF/new_data_222/MASK/290-1/0_0_00.png' when executing apps.eval. The apps.eval_hd works fine however, as the RENDER_1024 and MASK_1024 are provided on the download page.

Is it OK to directly resize the 1024*1024 rendered images and masks to 512*512 and set them as the prepared data?

gen_targets生成的数据可以直接分享一份吗？生成太慢了

gen_targets.py生成太慢了，希望能直接分享一下，真的非常感谢了。

UDF scaling during training

What is the reasoning behind scaling the UDF for training here?

AnchorUDF/lib/data/TrainDatasetDF3D.py

Line 265 in 0cbbe32

labels = torch.Tensor(distances).float() * scale_intrinsic

It doesn't make sense to me to change the scaling of the UDF with respect to the scaling of the image (also the random scaling). The scaling and translation are accounted for via the calib matrix and thus only directly affect the image encoder. Scaling the UDF without scaling the sample point space is causing inconsistency. What am I missing?

data_processing.

import data_processing.implicit_waterproofing as iw

I cant't get data_processing.implicit_waterproofing 。Is it convenient to provide?

Large gap between training/eval performance

@zhaofang0627
Hi! Thanks for your excellent research and implementation. However, when we try to reproduce the results following your instruction, we found the evaluation performance (on both training/test subsets) is far lower than that of training.

For example, the performance obtained at epoch 4 is listed as follows:
mean train L1Loss: 72.158565 ChamLoss: 3.533241 DirectLoss: 0.000000
eval test L1Loss: 205.070717 ChamLoss: 16.174886
eval train L1Loss: 209.616318 ChamLoss: 16.152732

We did not make any change to your implementation. Do you have any idea to solve this issue? Thank you.

off文件

你好，这个demo生成的结果是off文件，我应该怎么将他变成obj文件呢？

运行出错

opt missing for training code

similar to #6 in training there is also some obsolete options:

AnchorUDF/lib/data/TrainDatasetDF3D.py", line 53, in __init__
    self.num_sample_color = self.opt.num_sample_color
AttributeError: 'Namespace' object has no attribute 'num_sample_color'

AnchorUDF/lib/data/TrainDatasetDF3D.py", line 81, in __init__
    if not (self.opt.ndf or self.opt.ndf_pc):
AttributeError: 'Namespace' object has no attribute 'ndf'

neighborhood

Data preprocess problems. How did you choose the neighborhood of the sampled points when using grad_constraint ?

您好，想问一下quick demo生成的 文件

是不是只有.png 和 .off 没有.obj格式的文件吗？感谢您。，。

opt.ndf_pc does not exist in options

Thus I get the same error message for eval.py and eval_hd.py:

Is it meant to be if "ndf_pc" in opt and opt.ndf_pc is True ?

Some errors when running demo

trainning slow

The speed is slow when we train this interesting model. Could you tell the GPU parameters, for example the GPU card and memory？

TARGET folder missing in provided preprocessed data

... thus training can't be run. Can you please elaborate on what this TARGET folder should contain and how this data is generated?

how to dress SMPL ?

Hi,
This is a great project, how to use this project generate garment to dress SMPL model? Thanks

Multiple views support?

Hi, thank you for this amazing project.
I found in options.py there is an argument named num_views but you didn't use it. I wonder if the model supports multiple views?

multiple GPUs support?

Hi, thanks for your sharing. I found related codes using multiple gpus here, but not sure whether it works. I have tried it a few times, but the final reconstruction didn't seem to reach the level mentioned in the article. I would appreciate it if you give an answer.

Input mesh scaling

Am I geussing correctly that you are uniformly scaling and translating ground truth meshes such that they fit within a box of [-0.5, 0.5]? Is it the same as here: https://github.com/jchibane/ndf/blob/master/dataprocessing/convert_to_scaled_off.py?

How do you take care of scaling back up (and potentially translating) at test time?

issue with evaluation

i tried using the code with the dataset mentioned and the addons.
my code is stuck at at this step.
: "Jitting Chamfer 3D"

i see just one file added into the result folder which is the image of the sample 291 and not output since 12 hours
is there an error or does it take longer that that just to predict one 3d model of a image

Dataset preparation

in README.md you state that data preparation follows the original PIFu data preparation steps. This does not generate any anchor points though. In the paper you describe that you use k-means clustering on randomly sampled points on the surface and use the cluster centers as anchor points. Is that correct? Can you please provide the code you used to calculate the keypoints/anchorpoints?

get_targets部分生成较慢

您好，我按照您的办法，在get_targets部分生成sigma分别为0.02 0.08 0.003 的npz文件，但是生成过程较慢，平均6分钟一个，随机无序生成，一个2000个左右的文件，请问这是正常的吗？

When source code releasing?

Hi,
Thanks for great job to reconstruct garment 3d model from RGB image, I want to know when is source code released?

Thanks very much!
Baobei Xu

How to get reconstruction as shown in example images.

How to get reconstructed garment as shown in the results and anchor points ?

When doing:
python -m apps.eval_hd --results_path {path_of_output} --name {folder_of_output} --dataroot {path_of_dataset} --test_folder_path {folder_of_test_data, e.g., 290-1} --load_netMR_checkpoint_path ./checkpoints/anchor_udf_hd_df3d/netMR_epoch_14 --anchor --merge_layer 2 --joint_train --loadSize 1024 --num_steps 5 --filter_val 0.007

Getting .off mesh and anchor points, but no clarity.

Any suggestion will be helpful.

Thanks.

Target Sampling Distribution

In the supplemental of the paper you state

To generate training point sets, we randomly sample
points on the ground-truth surface and displace them with
Gaussian distribution N (0, σ) along x, y and z axis, where
1% of points are sampled from σ = 0.08, 49% of points
from σ = 0.02 and 50% of points from σ = 0.003 as sug-
gested by [13]

, yet the data preparation you provide only takes a single sigma. Is this intentional?

EDIT: (answering my own question)
The selection of samples according to the three different values of σ is taken care of during data loading - see

AnchorUDF/lib/data/TrainDatasetDF3D.py

Line 228 in 7df547f

def select_sampling_method(self, subject, scale_intrinsic):

I would suggest changing the README from

python -m apps.gen_targets --dataroot {path_of_dataset} --sigma {0.003, 0.02, or 0.08} --point_num 600

to at least

python -m apps.gen_targets --dataroot {path_of_dataset} --sigma {0.003, 0.02, AND 0.08} --point_num 600

to avoid confusion.

Maybe state that it needs to be run 3 times then.

failed:chamfer3D.cuda.0 想知道，cuda和cudnn的版本是什么呀，这个问题要怎么解决呢？