facebookresearch / donerf Goto Github PK

Hello, why is the depth generated by blender distorted, that is, circles appear from the origin to the distance.

Thanks for your great work!
I want to run DONeRF with my own dataset rendered from Blender, and I am stuck with how to get the 'depth_range_warped_log' by 'depth_range' that is measured by depth map.
Because I create my dataset by keeping 'depth_range_warped_log' the same as 'depth_range', and the output is not well.

Hello sir. :)

I have a question about section5.1 in your paper.

1. What is the classification class mean? In the paper, each class corresponds to discretized segment along the ray; for every discrete ray segment, a high value indicates it should receive multiple samples. I'm confusing about the "class ray segment classified" and "receiving multiple samples" meaning.

2. And how can multi-class target can be provided?.. The paper said it leads to binary classification.

Entirely, I can't understand the classification mechanism of the network.

Can you explain in detail?

Thanks!

Any idea what this problem might be? @thomasneff
Using the code out of the box with pavillon dataset for training.

WARNING! - import of cuda kernels for 'disc_depth_multiclass' failed - falling back to PyTorch
Training config: lo_l1.0_0.0_SpPoDir[128]-relu0(256x8)-CD-128-5-5_l1.0_10.0_RayMarchFromPoses_nSD[2_LSfCD_128_0.0](nerf(10-4))-relu1(256x80..63-7.63.)-RGBARayMarch (../configs/DONeRF_2_samples.ini)
no Checkpoints found
no Checkpoints found
epoch=21         loss=0.13672528:   0%|                                                                                    | 20/300001 [00:03<7:07:22, 11.70it/s]
Exception in thread Thread-4:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/opt/conda/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/pin_memory.py", line 28, in _pin_memory_loop
    idx, data = r
ValueError: not enough values to unpack (expected 2, got 0)
epoch=21         loss=0.13672528:   0%|                                                                                   | 21/300001 [00:08<34:03:39,  2.45it/s]
Traceback (most recent call last):
  File "train.py", line 344, in <module>
    main()
  File "train.py", line 329, in main
    pre_train(train_config)
  File "train.py", line 136, in pre_train
    batch_iterator = iter(train_config.train_data_loader)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 349, in __iter__
    self._iterator._reset(self)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 852, in _reset
    data = self._get_data()
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1029, in _get_data
    raise RuntimeError('Pin memory thread exited unexpectedly')
RuntimeError: Pin memory thread exited unexpectedly

How to use my own data set for training？

PyTorch's torch.eig was deprecated since version 1.9 and is being removed by pytorch/pytorch#70982. Please use the torch.linalg.eig function instead if you want your code to continue to work with the latest PyTorch.

Affected file:

Great work! I'm very interested in your work and want to test with your datasets. Unfortunately, I can't download your data at https://repository.tugraz.at/records/jjs3x-4f133. Could you please share the data on GoogleDrive? Thank you for your help!

WARNING! - import of cuda kernels for 'disc_depth_multiclass' failed - falling back to PyTorch

@thomasneff is there something missing in the repo? How did you make those kernels work?

Hey authors, great work! I was able to run the stock examples (forest for example).
How do I train this model over a colmap output? I can't find a way to get the .json format for the camera parameters etc.
I see a similar issue here: facebookresearch/nonrigid_nerf#7

Thanks for sharing this fantastic work. Would you consider releasing a Google Colab notebook for less technically inclined people like myself to to try this out? Thanks in advance!

Thanks for sharing your great work!

I wonder for a given dataset, how to define the view cell center and size?

Thank you very much for your excellent work!
While use your dataset, how can i get every image's camera to world pose?

How do I apply the transform I get from the .json？
Do I need to convert from blender coordinate system to opencv coordinate system like nerf?

Hi, I want to test your method's rendering speed. When I use your test.py it takes about 1.3 sec for each iteration on Nvidia V100 GPU using Pavillon data, whereas it is claimed about 50 ms/image in the paper. What's the easiest way to test your claimed rendering speed?