Comments (9)
That is correct, we use rest shape of the sheep/human only to train PoseNet.
from banmo.
Hi, I think there is no off-the-shelf solution.
The most straightforward way to get vertex embedding is to follow the CSE solution, where one can label corresponding 3D keypoints on a canonical mesh, and label 2D keypoints on images, from which you can learn a vertex embedding that corresponds to pixel features.
Starting from a mesh with vertex features, you would be able to train banmo's posenet following the paper.
from banmo.
Hi, the details about PoseNet training are in the paper supplement B.1.
The training data are rendered on the fly. For more details, see here the training code of PoseNet, as well as the rendering + augmentation pipeline. Unfortunately, the sheep mesh is under commercial license and we are not allowed to release it.
occ
is the flow confidence computed by forward-backward check. It is renamed as cfd_at_samp
and used here to weight flow reconstruction loss.
from banmo.
Thanks for your reply!
I have a couple more follow-up questions
- Is there any way to get access to the sheep mesh and its surface features? Which SMPL mesh did you use?
- Whether human.pth and quad.pth were trained posenet using the script you mentioned here?
from banmo.
-
The sheep/SMPL mesh/surface features are from densepose-CSE. You may get the vertex features but I'm afraid it's not easy to get their vertex locations if you are outside the organization (even I don't have access to them now). A relevant issue is opened here. The SMPL mesh is a subdivided version of the original one as noted here.
-
Yes.
from banmo.
Hi @gengshan-y, after looking through the training code of PoseNet, I would like to ask that did you only use a single human mesh to train PoseNet for humans, and another sheep mesh to train PoseNet for quadruped animals ?
If we put aside that there are various types of humans and animals, that might be handled well by the pretrained CSE features. How can your PoseNet predict the root pose very well when the objects make various poses ?
from banmo.
Yes.
The initial poses passed into banmo are noisy indeed due to deformations/shape variations. BANMo is updating the root poses during optimization. See here.
from banmo.
Thank you for your quick response. But I would like to clarify a little bit that my question is about the function forward_warmup that you used to pre-train PoseNet, which is stored in human.pth and and quad.pth.
I was wondering if they were trained using a single mesh, e.g., the mesh of resting pose sheep (illustrated in Fig.12 of your paper), with random camera poses generated around it ?
from banmo.
Hi @gengshan-y . Thanks for your amazing work and for sharing the code! I see that the sheep mesh cannot be released. Can you suggest a way to train PoseNet on our custom mesh? I suppose we cannot get CSE embeddings for our custom mesh.
from banmo.
Related Issues (20)
- Could you provide a demo that can work with one-click HOT 1
- Running out of space when running AMA (Samba + Swing) HOT 2
- Questions about the code. HOT 2
- Mesh generated on AMA-swing and Eval code HOT 2
- About projecting 3D verteces to 2D images HOT 1
- Error when rendering AMA-swing dataset HOT 2
- unable to load materials from: ./wood.obj.mtl HOT 3
- nvs_iter.sh HOT 2
- Extracting root pose HOT 1
- Loading the ".npy" files such as "cat-pikachiu.npy" or "shiba-haru-1.npy" has been failed HOT 3
- Questions about output results HOT 8
- exporting 3D models with skinning HOT 5
- Can Banmo support torch 1.11+cu113? HOT 1
- BANMo VS. ViSER HOT 2
- Clarification of the Matching Loss HOT 3
- Ckpt Release HOT 1
- face_normals all zero,ignoring
- The problem of extracting masks for deformable dynamic target objects HOT 3
- Enhancing Mesh Quality and Texture in BANMo Extracted Models HOT 1
- Questions about PoseNet HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from banmo.