The code will be released no later than the acceptance of the paper.
jiahuilei / mosca Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
The code will be released no later than the acceptance of the paper.
Hello,
Thanks for your fancy work. I'm interested in the dynamic 3D from a monocular video. While reading the paper, I was curious about the part about optimizing camera pose. Since the current Gaussian Splatting renderer does not support gradient propagation of camera poses, it seems that you are not optimizing camera poses with the l1 loss of the rendered image and GT, as shown in Eq. 14 and 15.
However, I'm a little confused about "Note that we will also optimize the camera poses Wt throughout the rendering phases as well with
photometric losses to further adjust the camera". I saw this was also mentioned in another issue. Does this mean that the camera pose is also optimized using Eq. 13 as loss? If so, have you implemented a renderer that can pass the camera pose gradient? If so, that's so cool.
Hi, thanks for your great work.
I notice that you follow roboust dynamic radiance field to split the nvidia dynamic datasets. But I found the data downloaded from roboust dynamic radiance field(https://github.com/facebookresearch/robust-dynrf) only contains 11 images for each scene. Is this true?
It seems different from the split from DynIbar (https://github.com/google/dynibar).
Dear authors,
Thanks for your impressive work!
In the "Full Pipeline" section, you mentioned that MoSca performs "Initialize the dynamic Gaussians G and jointly optimize the complete representation including the static background Gaussians, dynamic foreground Gaussians, the deformation motion graph, and cameras".
Will you still optimize the camera poses after the BA process in section 3.5?
Also, in section 3.5, you mention that "we jointly optimize a correction to depth Da[pa], consisting of per-frame global scaling factors and small per-pixel corrections". Can you provide more detailed descriptions for your depth alignment implementation and the camera poses training and representation?
Great work!
Is your method applicable to monocular videos with fixed viewpoint?
Thanks.
Hi, thanks for your excellent work MoSca!
Currently I am trying to follow some techniques in MoSca to handle the monocular dynamic scene. And I meet a small problem, which I really hope that you could help me to solve it. Here is my question:
In MoSca, you use the epipolar error map to segment the dynamic region, which is really elegant. I found this part can be refered to the code in robust dynamic radiance field in CVPR2023. And when I use it to segment the dynamic part in nvidia dynamic datasets and iphone datasets, the results seem poor or instable. Here is some example in apple scene (iphone) and skating scene (nvidia). I guess that some parameters should be adjusted to adopt to these scenes. It would be so nice for you to share some experience.
Dear authors,
Thanks for your impressive work!
Can you provide detailed rendering FPS for both the iPhone and Nvidia datasets for comparison purposes?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.