akashsengupta1997 / hierarchicalprobabilistic3dhuman Goto Github PK

Hello
first of all - nice Job.

I am doing right now a research project regarding self collision and i would like to use your repo as a baseline to generate 3d avatars of people.

I ran your run_predict.py successfully but noticed that the output image contains multiple smaller, identical images.
I was expecting something similar as the teaser.png where u have the estimated 3d mesh of the person.

Could you please tell me how to reproduce this visualization? I would like to get the 3d mesh and use it as a mask to overlap with a pointcloud..

I would appreciate your help

Thanks

when i want to training straps,i find a problem,i put the vertices and cam_t in renderer,can not get the true shape ,the shape is not (B,N,W,H)，just a image(3,256,256),your code wirte that parts, _, mask = self.renderer(vertices, self.faces, self.textures, t=cam_ts),but i can not get three value,just one value which shape is 3,256,256

Hi! Thanks for your work. How can I save the output prediction of the body on .obj / .fbx formats?

Hi,

i would like to use your code for my research work but I encounter some problems and require some support.

My ultimate goal is to get the 3d coordinates of the vertices with my own camera matrix K. I want later on make selective body region selection and and need the 3d coordinates of the Human body to do so.

Acoording to your code, the SMPL vertices in the proxy representation and i would like to get the 3d coordinates somehow out of it.

Could you tell me how to do the projection from the proxy to the 3d coordinates using a specific camera matrix.

I would appreciate your help.

thanks

Hi,

Thank you for the amazing work!

I was wondering how long does the synthetic data training take, considering there is no DataParallel/DistributedDataParallel implementation.
Thank you in advance!

Hey,

After getting the STRAPS code to work i was highly impressed, and thought this Programm would be much better for the purpose of my Thesis. I was wondering if there was maybe a possibility to run HP3DH on a Headless ssh server the same as it is possible in STRAPS

Best

Hi! Thank for your work!
Do you know how to extract the 3d joints of the smpl model which correspond to frame coordinates?

Dear akash,

Thank you very much for the amazing work! When I tried to download the background files, it said I would probably need two weeks due to the slow downloading speed in google colab (I tried several different high-speed internets). Can I ask how you download them, or do you have a downloaded version already in your google drive that you can share?

Thanks again! :)

Hello,

When I run the Inference Step,
I got:
Traceback (most recent call last):
File "run_predict.py", line 126, in
gender=args.gender)
File "run_predict.py", line 90, in run_predict
visualise_samples=visualise_samples)
File "/Users/mac/Desktop/reconstruction/predict/predict_poseMF_shapeGaussian_net.py", line 197, in predict_poseMF_shapeGaussian_net
verts_features=vertex_var_colours)
File "/opt/anaconda3/envs/ll/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/Users/mac/Desktop/reconstruction/renderers/pytorch3d_textured_renderer.py", line 275, in forward
fragments = self.rasterizer(meshes_iuv, cameras=self.cameras)
File "/opt/anaconda3/envs/ll/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/opt/anaconda3/envs/ll/lib/python3.6/site-packages/pytorch3d/renderer/mesh/rasterizer.py", line 151, in forward
cull_backfaces=raster_settings.cull_backfaces,
File "/opt/anaconda3/envs/ll/lib/python3.6/site-packages/pytorch3d/renderer/mesh/rasterize_meshes.py", line 150, in rasterize_meshes
cull_backfaces,
File "/opt/anaconda3/envs/ll/lib/python3.6/site-packages/pytorch3d/renderer/mesh/rasterize_meshes.py", line 205, in forward
cull_backfaces,
RuntimeError: NOT IMPLEMENTED
"

Do you have any idea on it?

Thank you for the great work.
Is it possible to train with my own dataset?. If so, can you give me some prepossessing tips?

Hello, I am running prediction script with no errors but I only get hrnet pose heatmap. The other images do not have SMPL model overlaid. Am I missing something?

Thank you.

Dear Akash,

I read your paper, as well as other papers from the series ([2], [3], [4]). :) It's interesting that your attempts are one of the few trying to recover an accurate shape of the person, instead of mostly focusing on a 3D pose.

However, the crucial question that is still bothering me is why do you use unnormalized loss components, such verts3d and joints3d? I guess you already thought about that and decided to still force the model to learn the scale. The advantage that I see using this approach is the regularization effect, where you confuse the model and then hope that it will still manage to somehow figure out the scale, i.e., height of the person.

Thanks in advance!

Hi,

I'm interested in projecting the predicted vertices given the predicted camera. There is a a functionality in STRAPS for this:

                pred_vertices2d = orthographic_project_torch(pred_vertices, pred_cam_wp)
                pred_vertices2d = undo_keypoint_normalisation(pred_vertices2d,
                                                              proxy_rep_input_wh)

Is there something similar in HierarchicalProbabilistic3DHuman? I would like to avoid using body_vis_renderer.

Thanks!

HI! Great job with the project.
How can I extract data such as the blenshapes so I can export it?
Thanks for your help!

Hi...
Thanks for the great job!
And I have a question:
In part <3.4. Body shape and pose distribution prediction>,

Here, each joint is modelled independently of all the other joints. Thus, the matrix parameter of the i-th joint, F_i , is a function of the input X only.

As you mentioned in the paper that the probability density function of each joint’s relative rotation matrix is a matrix-Fisher
distribution conditioned on the parents of that joint in the kinematic tree.
So I am confused that why you can model each joint independently as "each joint is modelled independently of all the other joints" ....