magic-research / xagen

[NeurIPS 2023] XAGen: 3D Expressive Human Avatars Generation

XAGen: 3D Expressive Human Avatars Generation

Zhongcong Xu · Jianfeng Zhang · Jun Hao Liew · Jiashi Feng · Mike Zheng Shou

⚒️ Installation

prerequisites: python>=3.7, CUDA>=11.3.

Install with conda activated:

source ./install_env.sh

Follow the instructions in this repo and website to download parametric models and place the parametric models as follow:

xagen
|----smplx
  |----assets
    |----MANO_SMPLX_vertex_ids.pkl
    |----SMPL-X__FLAME_vertex_ids.npy
    |----smplx_canonical_body_sdf.pkl
    |----smplx_extra_joints.yaml
    |----SMPLX_NEUTRAL_2020.npz
    |----SMPLX_to_J14.pkl

🏃‍♂️ Getting Started

Due to the copyright issue, we are unable to release all the processed datasets, we provide a sampled dataset and all the dataset labels for inference. Please download the sampled datasets and pretrained checkpoints from release. Then modify the path to data and checkpoints in the scripts. Run training:

bash dist_train.sh

Run inference:

bash inference.sh

Note: We don't have the plan to release all the data preprocssing scripts, but please email me if you are interested.

Citing

If you find our work useful, please consider citing:

@inproceedings{XAGen2023,
    title={XAGen: 3D Expressive Human Avatars Generation},
    author={Xu, Zhongcong and Zhang, Jianfeng and Liew, Junhao and Feng, Jiashi and Shou, Mike Zheng},
    booktitle={NeurIPS},
    year={2023}
}

xagen's Issues

Training configs of SHHQ and deepfashion

Hi, thanks for releasing the code! Could you also share the training cofigs of SHHQ and deepfashion? I can't find related hyperparameters in dist_train.py. Thanks.

About smplx_canonical_body_sdf.pkl

This is such a great piece of work. May I know where I can download the file smplx_canical_body_sdf.pkl? Is it the all_means.pkl file on the ExPose website? Thanks！

Inference of audio-driven animation

May I ask if you will provide inference code for generating animated videos based on SMPL-X sequences in the future? Thanks!