dberga / nerfstudio Goto Github PK
View Code? Open in Web Editor NEWThis project forked from nerfstudio-project/nerfstudio
A collaboration friendly studio for NeRFs
Home Page: https://docs.nerf.studio
License: Apache License 2.0
This project forked from nerfstudio-project/nerfstudio
A collaboration friendly studio for NeRFs
Home Page: https://docs.nerf.studio
License: Apache License 2.0
Make a bash or python script takes all results.json from output folders and creates a unified file.
Also, if rendered images exist, would be best also to make a table with all examples stacked up for each model.
Add detection of GPU numbers and therefore add multi-gpu usage (--machine.num-devices 1) in train.sh and eval.sh scripts, as shown in
https://docs.nerf.studio/quickstart/first_nerf.html#evaluating-runs
First check in pipelines where "encode_a" and "use_mask" is applied
https://github.com/rover-xingyu/Ha-NeRF/blob/main/models/networks.py
https://github.com/rover-xingyu/Ha-NeRF/blob/main/train_mask_grid_sample.py#L36
https://github.com/rover-xingyu/Ha-NeRF/blob/main/eval.py#L131
https://github.com/rover-xingyu/Ha-NeRF/blob/main/hallucinate.py#L178
Then re-implement a new case (take vanilla nerf as example)? Replacing the embeddings encoding and implicit mask
https://github.com/dberga/nerfstudio/blob/main/nerfstudio/models/vanilla_nerf.py
https://github.com/dberga/nerfstudio/blob/main/nerfstudio/fields/vanilla_nerf_field.py
https://github.com/dberga/nerfstudio/blob/main/nerfstudio/field_components/embedding.py
https://github.com/dberga/nerfstudio/blob/main/nerfstudio/field_components/encodings.py
After including the model in https://github.com/dberga/nerfstudio/blob/main/nerfstudio/models, you must include the model instruction name for "ns-train model" in https://github.com/dberga/nerfstudio/blob/main/nerfstudio/configs/method_configs.py
To do: include logs on each "outputs/SCENE/MODEL" when a case crashes.
Some examples and ideas:
Filling holes of meshes and point clouds
https://erkaman.github.io/posts/hole_filling.html
https://pymeshfix.pyvista.org/examples/repair_planar.html
https://pypi.org/project/meshlib/
http://www.open3d.org/docs/latest/tutorial/Advanced/surface_reconstruction.html
https://inria.hal.science/inria-00186820/document
https://github.com/jingdao/point_cloud_scene_completion
https://github.com/Geodan/fill-holes-pointcloud
https://github.com/pyvista/pymeshfix
pyvista/pymeshfix#13
Depth inpainting
https://www.casualganpapers.com/single_view_layered_depth_3d_inpainting/3D-Inpainting-explained.html
3D photo Inpainting
https://shihmengli.github.io/3D-Photo-Inpainting/
https://github.com/vt-vl-lab/3d-photo-inpainting
https://onlinelibrary.wiley.com/doi/10.1111/cgf.14735?af=R
https://www.youtube.com/watch?v=x54GdkpW9JY
Other links
https://towardsdatascience.com/the-ultimate-guide-to-3d-reconstruction-with-photogrammetry-56155516ddc4
https://towardsdatascience.com/python-libraries-for-mesh-and-point-cloud-visualization-part-1-daa2af36de30
https://towardsdatascience.com/how-to-voxelize-meshes-and-point-clouds-in-python-ca94d403f81d
https://towardsdatascience.com/5-step-guide-to-generate-3d-meshes-from-point-clouds-with-python-36bad397d8ba
Issues posted in:
Nerfstudio
nerfstudio-project#3059
nerfstudio-project#3057
SDFstudio
autonomousvision/sdfstudio#307
When calculating metrics for evaluation (nerf-rendered images vs real ground truth ones), we should be able to export the images
This is done in every model's function "get_image_metrics_and_images"
Alternative 1: Use database and open with Colmap
Alternative 2: code
https://nbviewer.org/github/cvg/Hierarchical-Localization/blob/master/pipeline_Aachen.ipynb
https://nbviewer.org/github/cvg/Hierarchical-Localization/blob/master/pipeline_InLoc.ipynb
Adding ref-nerf in nerfstudio has been already discussed here
nerfstudio-project#1469
Suggestions: i.e. take nerfacto or mip-nerf as example?
Note: the orientation calculation is already implemented in nerfacto (the Reflection Direction)
https://github.com/nerfstudio-project/nerfstudio/blob/main/nerfstudio/models/nerfacto.py
https://github.com/nerfstudio-project/nerfstudio/blob/main/nerfstudio/model_components/losses.py#L200
To do: the Directional encoding (Moses-Fischer vMF), but here we can utilize and modify the field processing, namely, the perceptron encoding (MLP) to generate that distribution instead:
https://github.com/nerfstudio-project/nerfstudio/blob/main/nerfstudio/fields/nerfacto_field.py
https://github.com/dberga/nerfstudio/blob/main/nerfstudio/fields/density_fields.py
https://github.com/dberga/nerfstudio/blob/main/nerfstudio/fields/vanilla_nerf_field.py
https://github.com/nerfstudio-project/nerfstudio/blob/main/nerfstudio/models/instant_ngp.py#L115
https://github.com/dberga/nerfstudio/blob/main/nerfstudio/field_components/encodings.py
https://github.com/nerfstudio-project/nerfstudio/blob/main/nerfstudio/field_components/mlp.py
After including the model in https://github.com/dberga/nerfstudio/blob/main/nerfstudio/models, you must include the model instruction name for "ns-train model" in https://github.com/dberga/nerfstudio/blob/main/nerfstudio/configs/method_configs.py
add another argument in the bash files for overwriting and so don't "exit" instead overwrite if required file is existing if overwrite flag is true
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.