Fabien Baradel*,
Matthieu Armando,
Salma Galaaoui,
Romain Brégier,
Philippe Weinzaepfel,
Grégory Rogez,
Thomas Lucas*
* equal contribution
![Multi-HMR illustration 1](assets/visu1.gif)
![Multi-HMR illustration 2](assets/visu2.gif)
Multi-HMR is a simple yet effective single-shot model for multi-person and expressive human mesh recovery. It takes as input a single RGB image and efficiently performs 3D reconstruction of multiple humans in camera space.
First, you need to clone the repo.
We recommand to use virtual enviroment for running MultiHMR.
Please run the following lines for creating the environment with venv
:
python3.9 -m venv .multihmr
source .multihmr/bin/activate
pip install -r requirements.txt
Otherwise you can also create a conda environment.
conda env create -f conda.yaml
conda activate multihmr
The installation has been tested with python3.9 and CUDA 11.7.
Checkpoints will automatically be downloaded to $HOME/models/multiHMR
the first time you run the demo code.
Besides these files, you also need to download the SMPLX model.
You will need the neutral model for running the demo code.
Please go to the corresponding website and register to get access to the downloads section.
Download the model and place SMPLX_NEUTRAL.npz
in ./models/smplx/
.
The following command will run Multi-HMR on all images in the specified --img_folder
, and save renderings of the reconstructions in --out_folder
.
The --model_name
flag specifies the model to use.
The --extra_views
flags additionally renders the side and bev view of the reconstructed scene, --save_mesh
saves meshes as in a '.npy' file.
python3.9 demo.py \
--img_folder example_data \
--out_folder demo_out \
--extra_views 1 \
--model_name multiHMR_896_L
We provide multiple pre-trained checkpoints.
Here is a list of their associated features.
Once downloaded you need to place them into $HOME/models/multiHMR
.
modelname | training data | backbone | resolution | runtime (ms) |
---|---|---|---|---|
multiHMR_896_L | BEDLAM+AGORA+CUFFS+UBody | ViT-L | 896x896 | 126 |
We compute the runtime on GPU V100-32GB.
The code is distributed under the CC BY-NC-SA 4.0 License.
See Multi-HMR LICENSE, Checkpoint LICENSE and Example Data LICENSE for more information.
If you find this code useful for your research, please consider citing the following paper:
@inproceedings{multi-hmr2024,
title={Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shot},
author={Baradel*, Fabien and
Armando, Matthieu and
Galaaoui, Salma and
Br{\'e}gier, Romain and
Weinzaepfel, Philippe and
Rogez, Gr{\'e}gory and
Lucas*, Thomas
},
booktitle={arXiv},
year={2024}
}