wrong output about glamr

[haofanwang]

It works! Thanks for your patient help! @Khrylx

[arakis92]

Hello, currently I'm having the same issue shared by @JonathanLehner , rendering of glamr is totally black. I've tried with mesa upgrading and pyvista upgrading.

[Khrylx]

The mesh visualization you are using seems different from what the demo uses.

Do the examples in readme work?

python global_recon/run_demo.py --cfg glamr_dynamic \
                                --video_path assets/dynamic/running.mp4 \
                                --out_dir out/glamr_dynamic/running \
                                --save_video

[JonathanLehner]

no, the output is from the example in the readme. It also does not work for the running scene. The output looks similar. Is there a specific python version? I used 3.8. Or are there additional installation requirements for the visualisation to work correctly? HybrIK works without problems.

[Khrylx]

could you try these commands and try rendering again?

# mesa
sudo apt update
sudo wget https://github.com/mmatl/travis_debs/raw/master/xenial/mesa_18.3.3-0.deb
sudo dpkg -i ./mesa_18.3.3-0.deb || true
sudo apt install -f

[Khrylx]

If this doesn't solve it, it might due to problems with the visualization package pyvista.
Could you try its example here: https://docs.pyvista.org/examples/02-plot/gif.html?highlight=off_screen?

[JonathanLehner]

Hi Yeyuan, thanks for your fast reply! I tried mesa but it did not help. I used pyvista 3.3 since not fixed in requirements, but also no difference.

The strange thing is that the demo in HybrIK works and in GLAMR the HybrIK output already is shifted. Could it be that there were breaking changes in the official HybrIK repo? It was updated recently

[Khrylx]

That's strange. Could you try my fork of HybrIK https://github.com/Khrylx/HybrIK?

[haofanwang]

I have met the same problem. The HybrIK result is already bad, and GLAMR works worse.

Here is the image saved in ./out/glamr_dynamic/running/pose_est/res_images/000089.jpg

This is my result

[Khrylx]

@haofanwang Could you try my fork of HybrIK? https://github.com/Khrylx/HybrIK

[haofanwang]

Hi, @Khrylx

It seems not to be the problem of update in HybrIK, but ./pose_est/hybrik_demo/demo.py.

If I comment out line 169-171, the visualization works fine. These lines are not existed in HybrIK. I'm not sure whether it is necessary in your method.

GLAMR/pose_est/hybrik_demo/demo.py

Line 169 in 45ce7ce

new_princpt = np.array([image.shape[1], image.shape[0]]) * 0.5

Besides, the following line should be bbox_3d_shape=(2.2, 2.2, 2.2), which should be a typo.

GLAMR/pose_est/hybrik_demo/demo.py

Line 85 in 45ce7ce

bbox_3d_shape=(2.2, 2,2, 2.2),

[Khrylx]

These lines are needed to move the camera coordinate from the bounding box center to the image center. I will also look into it.

[haofanwang]

Indeed, the modification above still does not work. Below is my current result for reference. Removing line 169-171 makes HybrIK right, but not the GLAMR.

[Khrylx]

@haofanwang if you don't make the change, does the demo work for basketball.mp4?

[Khrylx]

I have pushed a fix. At least on my end, both running.mp4 and basketball.mp4 work well. @haofanwang, also make sure for running you are using the dynamic config, which could be the reason for your wrong output.

python global_recon/run_demo.py --cfg glamr_dynamic \
                                --video_path assets/dynamic/running.mp4 \
                                --out_dir out/glamr_dynamic/running \
                                --save_video

[haofanwang]

Still stuck. The GLAMR fits at the first frame, but shifts later.

basketball.mp4 also doesn't work well. At the first frame, it looks fine, but all following frames keep same as the first one.

I will also check is there anything I have missed.

[Khrylx]

so you mean the GLAMR's (green) pose remains the same for all the frames? It would be also helpful if you can append your log.txt

[haofanwang]

For static video, yes.

I have uploaded the visualization results and logs to Google Drive.

[Khrylx]

The 2d keypoints look correct, so this is probably just a visualization issue. Could you try pyvista==0.31.3?

pip install pyvista==0.31.3

[Khrylx]

@JonathanLehner could you please try the above fix and let me know whether I can close this issue? thanks!

[JonathanLehner]

yes, sorry for delay. My PC has some issues right now..

[lucasjinreal]

I think pyvista only works on a specific version, this is sucks, since pyvista is just a visualization lib. Can we just please move render to pytorch3d?

[iulsko]

can attest to the same issue - totally blank GLAMR output!
have tried all the mentioned solutions

[JonathanLehner]

I also found I needed to install torchmetrics-0.6.0 for it to run

[JonathanLehner]

and downgrade to scipy==1.1.0

[JonathanLehner]

I had to upgrade pip to install it