This repository contains training and inference code for the following paper:
TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style
Chaitanya Patel*, Zhouyingcheng Liao*, Gerard Pons-Moll
CVPR 2020 (ORAL)
[arxiv] [project website] [Dataset Repo] [Youtube]
python3
pytorch
scipy
chumpy
psbody.mesh
- Download and prepare SMPL model and TailorNet data from dataset repository.
- Set DATA_DIR and SMPL paths in
global_var.py
file accordingly. - Download trained models' weights and unzip it. Set paths of
LF_MODEL_PATH
,HF_MODEL_PATH
andSS2G_MODEL_PATH
variables inglobal_var.py
accordingly.- old-t-shirt_female_weights
- [Other garments coming soon]
- Set output path in
run_tailornet.py
and run it to predict garments on some random inputs. You can play with different inputs. You can also run inference on motion sequence data. - To visualize predicted garment using blender, run
python run_tailornet.py render
. (Blender needs to be installed.)
... then you can merge downloaded weights directories to follow a directory structure similar to the following.
weights_folder
----tn_orig_baseline
--------{garment_class}_{gender} (e.g. t-shirt_female)
------------lin.pth.tar (model weights)
------------params.json (some model params)
----tn_orig_lf
--------{garment_class}_{gender}
------------lin.pth.tar
------------params.json
----tn_orig_ss2g
--------{garment_class}_{gender}
------------lin.pth.tar
------------params.json
----tn_orig_hf
--------{garment_class}_{gender}
------------{shape_idx}_{style_idx} (e.g. 000_023 pivot)
----------------lin.pth.tar
----------------params.json
and then you won't need to change model checkpoint paths while dealing with multiple garments.
- Set global variables in
global_var.py
, especially LOG_DIR where training logs will be stored. - Set config variables like gender and garment class in
trainer/base_trainer.py
(or pass them via command line) and runpython trainer/base_trainer.py
to train TailorNet MLP baseline. - Similarly, run
python trainer/lf_trainer.py
to train low frequency predictor andtrainer/ss2g_trainer.py
to train shape-style-to-garment(in canonical pose) model. - Run
python trainer/hf_trainer.py --shape_style <shape1>_<style1> <shape2>_<style2> ...
to train pivot high frequency predictors for pivots<shape1>_<style1>
,<shape2>_<style2>
, and so on. SeeDATA_DIR/<garment_class>_<gender>/pivots.txt
to know available pivots. - Use
models.tailornet_model.TailorNetModel
with appropriate logdir arguments to do prediction.
Cite us if you use our model, code or data:
@inproceedings{patel20tailornet,
title = {TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style},
author = {Patel, Chaitanya and Liao, Zhouyingcheng and Pons-Moll, Gerard},
booktitle = {{IEEE} Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {jun},
organization = {{IEEE}},
year = {2020},
}
- Thanks to Bharat for many fruitful discussions and for
smpl_lib
library taken from his MultiGarmentNet repo's lib folder. - Thanks to Garvita for helping out during the onerous procedure of data generation.