princeton-vl / pose-hg-demo

Code to test and use the model from "Stacked Hourglass Networks for Human Pose Estimation"

Stacked Hourglass Networks for Human Pose Estimation (Demo Code)

This repository includes Torch code for evaluation and visualization of the network presented in:

Alejandro Newell, Kaiyu Yang, and Jia Deng, Stacked Hourglass Networks for Human Pose Estimation, arXiv:1603.06937, 2016.

A pretrained model is available on the project site. Include the model in the main directory of this repository to run the demo code.

Check out the training and experimentation code now available at: https://github.com/anewell/pose-hg-train

In addition, if you download the full MPII Human Pose dataset and replace this repository's images directory you can generate full predictions on the validation and test sets.

To run this code, the following must be installed:

• Torch7
• hdf5 (and the torch-hdf5 package)
• cudnn
• qlua (for displaying results)

For displaying the demo images: qlua main.lua demo

For generating predictions: th main.lua predict-[valid or test]

For evaluation on a set of validation predictions: th main.lua eval

Testing your own images

To use the network off-the-shelf, it is critical that the target person is centered in the input image. There is some robustness to scale, but for best performance the person should be sized such that their full height is roughly three-quarters of the input height. Play around with different scale settings to see the impact it has on the network output. We offer a convenient function for generating an input image:

inputImg = crop(img, center, scale, rot, res)

res should be set to 256 for our network. rot is offered if you wish to rotate the image (in degrees). You can run the input image through the network, and get the (x,y) coordinates with:

outputHm = m:forward(inputImg:view(1,3,256,256):cuda())

predsHm,predsImg = getPreds(outputHm, center, scale)

The two outputs of getPreds are coordinates with respect to either the heatmap or the original image (using center and scale to apply the appropriate transformation back to the image space).

The MPII images come with center and scale annotations already. An important detail with regards to the annotations: we have modified their format slightly for ease of use with our code. In addition, we adjusted the original center and scale annotations uniformly across all images so as to reduce the chances of our function cropping out feet from the bottom of the image. This mostly involved moving the center down a fraction.

pose-hg-demo's Issues

What is the different between valid-ours.h5 and valid-example.h5

Hi, I notice that valid-example.h5 is better than valid-ours.h5. Would you please tell me the difference between these two results? Thank you.

A quarter of offest?

In the original paper, there goes: "the prediction is offset by a quarter of a pixel in the direction of its next highest neighbor before transforming back to the original coordinate space of the image"

Is it implemented in this project? Looking forward for replying. Thank you!

Running the demo without GPU

How can I run the demo without GPU as I wont have cudnn.

Pytorch version

Hi, is there a pytorch version for this?

About Testing my own cropped images

Thanks for sharing your code, It's an amazing work! But I'm a little confused.
Suppose My testing images are already cropped , and there is only one person that occupies the whole image. Thus I don't have the scale and center mentioned in your code, I only have the cropped images.
It seems that the input image of the model has been cropped, and the shape of the output is 1X16X64X64. How can I get the pose point in My cropped testing images?

about running demo

Hi
When I run qlua main.lua demo it gives some error
Unable to connect X11 server (continuing with -nographics)
qlua: cannot open <umich-stacked-hourglass.t7> in mode r at /home/jiahang/torch/pkg/torch/lib/TH/THDiskFile.c:668
stack traceback:
[C]: at 0x7fd1a79d79c0
[C]: at 0x7fd19f92eac0
[C]: in function 'DiskFile'
/home/jiahang/torch/install/share/lua/5.1/torch/File.lua:405: in function 'load'
main.lua:28: in main chunk
Segmentation fault (core dumped)

I hope I can get your help. Thank you

Question about center & scale

HI:
in your code:
inputImg = crop(img, center, scale, rot, res)
how should i set the value of "center" and "scale" if i know the bounding box of a single person in img?
thx~

python3 version?

As mentioned above,thx.

is there any caffe version of this?

Error about the HDF5 version

Hello, I have some problems about the code ,when just ran the qlua main.lua demo,
and I think my HDF5 is new enough,which version should I use.
qlua main.lua demo qlua: /home/zakeyao/torch/install/share/lua/5.1/hdf5/ffi.lua:71: Unsupported HDF5 version: 1.10.1 stack traceback: [C]: at 0x7ff9f2856f50 [C]: in function 'error' /home/zakeyao/torch/install/share/lua/5.1/hdf5/ffi.lua:71: in function 'checkHDF5Version' /home/zakeyao/torch/install/share/lua/5.1/hdf5/ffi.lua:78: in main chunk [C]: in function 'dofile' /home/zakeyao/torch/install/share/lua/5.1/torch/init.lua:54: in function 'include' /home/zakeyao/torch/install/share/lua/5.1/hdf5/init.lua:29: in main chunk [C]: in function 'require' /home/zakeyao/pose-hg-demo-master/util.lua:7: in main chunk [C]: in function 'dofile' main.lua:2: in main chunk

error running demo

HI there, I may have done something wrong when trying to install and run the code, but I think I followed all requirements, and am getting the following error when trying to execute the demo.

qlua main.lua demo

qlua: ...h/install/share/lua/5.1/nn/SpatialBatchNormalization.lua:83: attempt to index field 'running_std' (a nil value)

Am I missing a library? Thanks for the help!

About real-time

Issue summary

Is it possible to input videos as real-time detection? If can, how about its FPS?

Anyway to visualize the model structure

Is somebody able to create a visualziation for me to see the model structure? I am not able to make it work.

discrepancy between model architecture in .t7 model and in the demo code

Hi,
I am trying to reproduce the your demo code in tensorflow using the trained .t7 model.
But I guess the model in the demo code( residual.lua & stacked-hourglass-model.lua ) and the model used to trained umich-stacked-hourglass.t7 is different.

Could you provide the model architecture details used to train umich-stacked-hourglass.t7?

Thank you!

Torch max function update

Hi Alejandro,

I guess you updated the training code to work with the updated 'max' function which doesn't return index anymore. (with this commit)

But it doesn't seem to be updated in this demo repository. I haven't ran it myself, I just realized looking at the code.

accuracy in paper

Hi @anewell ,
Thanks for your paper, I have seen the paper results total accuracy PCKH @0.5 as 90.9 but when I compute the accuracy from prediction you provide in valid-ours.h5 it becomes about 88.0

What could cause this difference?
Thanks,

error running the demo code on nvidia-docker

I was trying to run the demo code on a nvidia-docker and I'm getting the following error

# qlua main.lua demo
Unable to connect X11 server (continuing with -nographics)
qlua: not loading module qtgui (running with -nographics)...............................]  ETA: 5s714ms | Step: 952ms  
qlua: qtwidget window functions will not be usable (running with -nographics)
qtwidget window functions will not be usable (running with -nographics)
qlua: not loading module qtuiloader (running with -nographics)
qlua: /root/torch/install/share/lua/5.1/image/init.lua:1647: attempt to index global 'qtuiloader' (a nil value)
stack traceback:
[C]: in function '__index'
/root/torch/install/share/lua/5.1/image/init.lua:1647: in function 'window'
/root/torch/install/share/lua/5.1/image/init.lua:1598: in function 'display'
main.lua:74: in main chunk
Segmentation fault (core dumped)

Can anyone help me out with this?

Thanks in Advance!!!

The pretrained model is unavailable

The file is not found on the given website. Cloud you please re-upload it? Thanks a lot.

some issues about running demo

when I run qlua main.lua demo
there are some errors,like:
cannot open <umich-stacked-hourglass.t7> in mode r at /home/master-grade2-1/linli/torch/torch/pkg/torch/lib/TH/THDiskFile.c:673
stack traceback:
[C]: at 0x7fca76426750
[C]: in function 'DiskFile'
...1/linli/torch/torch/install/share/lua/5.1/torch/File.lua:405: in function 'load'
main.lua:28: in main chunk
[C]: in function 'dofile'
...orch/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00406670

I'm so pleasure to get your help!Thanks

MPII images cropping

Hi,

You mentioned that you adjust the center and scale annotation in MPII dataset.
Would you mind to describe more details of the adjustment in addition to moving down the center?

Thanks!

A question about the input image

Hey,I read your paper and implemented it using tensorflow,and after train,I have a question.That is,if I don't know the scale and center of an image(the person maybe in the center,maybe not),what if I just resize a input image to 256x256 directly without scale and center information,and then do prediction?Would the result be bad? Do you have had a try?And if so,is there anyway to get a good result without scale and center?

demo error!Error: unable to locate HDF5 header file

thanks for your code!＼（＾ｏ＾）／～ It is so helpful!
when i run $ qlua main.lua demo
,i got errors .could you please give me some advice?

sun@sunwin:~/0newcodedown/pose-hg-demo-master$ qlua main.lua demo
qlua: /home/sun/torch/install/share/lua/5.1/hdf5/ffi.lua:42: Error: unable to locate HDF5 header file at /usr/local/HDF_Group/HDF5/1.8.18/include;/hdf5.h
stack traceback:
[C]: at 0x7fa4d28d1f50
[C]: in function 'error'
/home/sun/torch/install/share/lua/5.1/hdf5/ffi.lua:42: in function 'loadHDF5Header'
/home/sun/torch/install/share/lua/5.1/hdf5/ffi.lua:60: in main chunk
[C]: in function 'dofile'
/home/sun/torch/install/share/lua/5.1/torch/init.lua:54: in function 'include'
/home/sun/torch/install/share/lua/5.1/hdf5/init.lua:29: in main chunk
[C]: in function 'require'
/home/sun/0newcodedown/pose-hg-demo-master/util.lua:7: in main chunk
[C]: in function 'dofile'
main.lua:2: in main chunk
i have already installed hdf5 1.8.18 ,my system is ubuntu16.04 cuda8.0.

Demo results

Hi,

I've run your demo code and got pretty weird results:

method	Head	Knee	Ankle	Shoulder	Elbow	Wrist	Hip
valid-example	96.52	78.60	73.52	93.51	85.18	79.17	84.14
valid-ours	96.67	83.10	79.07	94.68	88.54	83.38	87.49

I downloaded your pre-trained model and MPII dataset and followed the instructions. Could you explain please why results are much worse than in #1 ? Have you updated this model since that time ?

Thanks!

Is .h5 pre-trained model available ?

Hello,
Thank you for sharing code
Is there .h5 pre-trained model of the Stacked Hourglass?
Or
Do you have any clue how should convert .t7 model to .h5 model in an easy way?
and also it will be very useful get an answer from the community.
Thanks in advance...