monniert / differentiable-blocksworld Goto Github PK

Hi!

First of all thank you for the great work and codebase! 😃

I wanted to ask you about the design choice of adding the transparency value of the primitives to the differentiable rendering process. Specifically, in the paper you mention that it behaves better during optimization in comparison to the standard differentiable rendering pipeline. What do you mean by better behavior? How worse are the results when using directly the standard Pytorch3D renderer, are there any examples that you could share showing the difference? Finally, do you have any intuition why this is happening?

Best,
Konstantinos

Hi!

Very cool work! It would be nice to have the model checkpoints on the Hugging Face Hub.

Some of the benefits of sharing your models through the Hub would be:

• versioning, commit history and diffs
• repos provide useful metadata about their tasks, languages, metrics, etc that make them discoverable
• multiple features from TensorBoard visualizations, PapersWithCode integration, and more
• wider reach of your work to the ecosystem

Creating the repos and adding new models should be a relatively straightforward process if you've used Git before. This is a step-by-step guide explaining the process in case you're interested.

You can also create a research demo as well.

Please let us know if you would be interested and if you have any questions.

So in the line 422 in dbw.py
I notice that you define a val_blocks which is on cpu and the self.get_opacities() is on the gpu
so there will be a error raised
My personal solution is to revise the line 422 from val_blocks = torch.linspace(0, 1, self.n_blocks + 1)[1:]
to
val_blocks = torch.linspace(0, 1, self.n_blocks + 1)[1:].to(self.bkg.device)

and line 429 from values = torch.cat([torch.zeros(NFE) , val_blocks.repeat_interleave(self.BNF)])

to
values = torch.cat([torch.zeros(NFE).to(self.bkg.device) , val_blocks.repeat_interleave(self.BNF)])

and the problem is solved

i am not sure whether this is a common issue or just happened on my device
so just remark it here.

I am wondering that is it possible to use this method for Indoor scene reconstruction?

Hi, thanks for your nice work and concise repo!

I'm working on the project that needs to leverage on the reconstructed mesh generated from dbw, and I found the position and scale of mesh have certain gap compared to points computed from colmap.
I'm wondering if the mesh is not placed on the real position of world coordinates, and instead always placed at the origin?

Congratulations on a job well done. I want to run my custom data on your code. What are the S_world,R_world, and T_world parameters in the yml file under configs/nerfstudio, and what do they do?
Wish you all the best in your work and look forward to your reply!

Hi, thanks for the awesome work!
After reading the paper and codes I have one question about the model design. In section 3, you mention as

Note that compared to recent advances in neural volumetric representations [50, 45, 76], we do not use any neural network and directly optimize meshes, which are straightforward to use in computer graphic pipelines.

I was curious about the part about not using any neural networks because as the model is already light to train, I thought that attaching a small MLP layer after the primitive can highly increase the quality of the rendered images as from other NeRF type models. Although I do understand that attaching the MLP will make the entire model less practical in the sense that we cannot use it in computer graphics pipelines, I think this will make the model competitive with other existing MVS models. Have you ever conducted any experiments with neural networks?

Thanks!

Hi, thank you for sharing your great work!
I notice that your method can successfully decompose scenes into primitive 3D meshes, which is really cool. Since I only found logging codes for image/video rendering, I would like to know whether inference results can be exported as mesh files. If so, could you also share these mesh files with us?
Thank you very much!

In the multi_pipeline.sh, it seems like multiple seeds will be chosen in the training process. Is there any motivation behind this setting?