Recommended Setup/Instructions to Run on Custom data? about 4dgaussians

[taoranyi]

Thank you for your attention. You can use the colab from nerfies.

[x-CK-x]

Hi, maybe like this:

1. Downloading nerfstudio `pip install nerfstudio`

2. using `ns-process-data` to convert data into `blender` format

3. split data into "transforms_train", "transforms_test", "transforms_val" ( I have not finished using data for only training yet, you can write the code or adding the guidance of using custom data and create a pr o.O)
   btw, the pose calculated by colmap is not precise so rendering quality may not good enough, free-view rendering may cannot work well as you imaging.

I've also noticed issues with colmap in the past regarding the pose. I don't suppose there's a direct replacement for colmap yet is there?

[guanjunwu]

Hi,
maybe like this:

1. Downloading nerfstudio pip install nerfstudio
2. using ns-process-data to convert data into blender format
3. split data into "transforms_train", "transforms_test", "transforms_val" ( I have not finished using data for only training yet, you can write the code or adding the guidance of using custom data and create a pr o.O)
   btw, the pose calculated by colmap is not precise so rendering quality may not good enough, free-view rendering may cannot work well as you imaging.

[guanjunwu]

Yes, thank you for the notation :)

[camtrik]

Hi, @taoranyi
It seems colab from nerfies can generate most of the required files from custom data, but not points.npy, which is required in 4D Gaussians code.
Any way to generate this file? Or is it necessary for training the 4D Gaussian?

[taoranyi]

Hello camtrik! You can use (this method)[https://github.com/hustvl/4DGaussians/blob/a2de1841756f49f7bf8d4b2f6fcad0ee5a8324ae/scene/dataset_readers.py#L449] to randomly initialize instead of using points.npy.

[Tejasdavande07]

Hi I was getting below mentioned error -
I installed whole set-up properly as below mentioned steps -

First check your installed cuda version typing in a command prompt;

nvcc --version.

do installation of proper cude version
create folder with proper destination.
open this location into cmd.
git clone https://github.com/hustvl/4DGaussians
cd 4DGaussians
git submodule update --init --recursive
conda create -n Gaussians4D python=3.10
conda activate Gaussians4D

Then run these commands, make sure you have activated your conda env first;

conda activate Gaussians4D
pip uninstall torch torchvision torchaudio

I used Python 3.10 and

pip install torch==2.1.0+cu118 torchvision==0.16.0+cu118 torchaudio==2.1.0+cu118 --extra-index-url https://download.pytorch.org/whl/cu118

pip install -e submodules/depth-diff-gaussian-rasterization
pip install -e submodules/simple-knn

pip install nerfstudio

till this I had done installation. please suggest further steps to check already available data-set and my own dataset.

(Gaussians4D) F:\Standalones\test\4DGaussians>python train.py -s data/hypernerf/interp/bouncingballs --port 6017 --expname "hypternerf/interp/bouncingballs" --configs arguments/hypernerf/default.py

Traceback (most recent call last):
File "F:\Standalones\test\4DGaussians\train.py", line 17, in
from gaussian_renderer import render, network_gui
File "F:\Standalones\test\4DGaussians\gaussian_renderer_init_.py", line 15, in
from scene.gaussian_model import GaussianModel
File "F:\Standalones\test\4DGaussians\scene_init_.py", line 16, in
from scene.dataset_readers import sceneLoadTypeCallbacks
File "F:\Standalones\test\4DGaussians\scene\dataset_readers.py", line 26, in
from plyfile import PlyData, PlyElement
ModuleNotFoundError: No module named 'plyfile'

[guanjunwu]

`

Hi I was getting below mentioned error -

(Gaussians4D) F:\Standalones\test\4DGaussians>python train.py -s data/hypernerf/interp/bouncingballs --port 6017 --expname "hypternerf/interp/bouncingballs" --configs arguments/hypernerf/default.py

Traceback (most recent call last): File "F:\Standalones\test\4DGaussians\train.py", line 17, in from gaussian_renderer import render, network_gui File "F:\Standalones\test\4DGaussians\gaussian_renderer__init__.py", line 15, in from scene.gaussian_model import GaussianModel File "F:\Standalones\test\4DGaussians\scene__init__.py", line 16, in from scene.dataset_readers import sceneLoadTypeCallbacks File "F:\Standalones\test\4DGaussians\scene\dataset_readers.py", line 26, in from plyfile import PlyData, PlyElement ModuleNotFoundError: No module named 'plyfile'

maybe pip install plyfile may work?

[Tejasdavande07]

pip install plyfile

Now getting this issue.

(Gaussians4D) F:\Standalones\test\4DGaussians>python train.py -s data/hypernerf/interp/bouncingballs --port 6017 --expname "hypternerf/interp/bouncingballs" --configs arguments/hypernerf/default.py
Traceback (most recent call last):
File "F:\Standalones\test\4DGaussians\train.py", line 328, in
import mmcv
ModuleNotFoundError: No module named 'mmcv'

[guanjunwu]

pip install plyfile

Now getting this issue.

(Gaussians4D) F:\Standalones\test\4DGaussians>python train.py -s data/hypernerf/interp/bouncingballs --port 6017 --expname "hypternerf/interp/bouncingballs" --configs arguments/hypernerf/default.py Traceback (most recent call last): File "F:\Standalones\test\4DGaussians\train.py", line 328, in import mmcv ModuleNotFoundError: No module named 'mmcv'

I guess that you didn't use pip install -r requirements.txt to install relative packages firstly :)

[Tejasdavande07]

pip install plyfile

Now getting this issue.
(Gaussians4D) F:\Standalones\test\4DGaussians>python train.py -s data/hypernerf/interp/bouncingballs --port 6017 --expname "hypternerf/interp/bouncingballs" --configs arguments/hypernerf/default.py Traceback (most recent call last): File "F:\Standalones\test\4DGaussians\train.py", line 328, in import mmcv ModuleNotFoundError: No module named 'mmcv'

I guess that you didn't use pip install -r requirements.txt to install relative packages firstly :)

done the changes but still getting below mentioned error -

(Gaussians4D) F:\Standalones\test\4DGaussians>python train.py -s data/hypernerf/interp/bouncingballs --port 6017 --expname "hypternerf/interp/bouncingballs" --configs arguments/hypernerf/default.py
Optimizing
Output folder: ./output/hypternerf/interp/bouncingballs [21/10 14:30:41]
feature_dim: 64 [21/10 14:30:41]
Traceback (most recent call last):
File "F:\Standalones\test\4DGaussians\train.py", line 340, in
training(lp.extract(args), hp.extract(args), op.extract(args), pp.extract(args), args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from, args.expname)
File "F:\Standalones\test\4DGaussians\train.py", line 224, in training
scene = Scene(dataset, gaussians, load_coarse=None)
File "F:\Standalones\test\4DGaussians\scene_init_.py", line 55, in init
assert False, "Could not recognize scene type!"
AssertionError: Could not recognize scene type!

Please suggest how to process for the data training if you have the file data testing please share so that I can check for the same.

[guanjunwu]

pip install plyfile

Now getting this issue.
(Gaussians4D) F:\Standalones\test\4DGaussians>python train.py -s data/hypernerf/interp/bouncingballs --port 6017 --expname "hypternerf/interp/bouncingballs" --configs arguments/hypernerf/default.py Traceback (most recent call last): File "F:\Standalones\test\4DGaussians\train.py", line 328, in import mmcv ModuleNotFoundError: No module named 'mmcv'

I guess that you didn't use pip install -r requirements.txt to install relative packages firstly :)

done the changes but still getting below mentioned error -

(Gaussians4D) F:\Standalones\test\4DGaussians>python train.py -s data/hypernerf/interp/bouncingballs --port 6017 --expname "hypternerf/interp/bouncingballs" --configs arguments/hypernerf/default.py Optimizing Output folder: ./output/hypternerf/interp/bouncingballs [21/10 14:30:41] feature_dim: 64 [21/10 14:30:41] Traceback (most recent call last): File "F:\Standalones\test\4DGaussians\train.py", line 340, in training(lp.extract(args), hp.extract(args), op.extract(args), pp.extract(args), args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from, args.expname) File "F:\Standalones\test\4DGaussians\train.py", line 224, in training scene = Scene(dataset, gaussians, load_coarse=None) File "F:\Standalones\test\4DGaussians\scene__init__.py", line 55, in init assert False, "Could not recognize scene type!" AssertionError: Could not recognize scene type!

Please suggest how to process for the data training if you have the file data testing please share so that I can check for the same.

Please check your input:
--expname "hypternerf/interp/bouncingballs"

[Tejasdavande07]

Now in my input side issue was their right?
please share file for input from data folder to run for testing by using command.
share the command to run.

[guanjunwu]

please follow readme.md to set up your environment and prepare your dataset step by step carefully :)
"bouncingballs" is in datasets of dnerf instead of "hypternerf" ("hypernerf" exactly)

[Tejasdavande07]

please follow readme.md to set up your environment and prepare your dataset step by step carefully :) "bouncingballs" is in datasets of dnerf instead of "hypternerf" ("hypernerf" exactly)

Thanks for suggestion.
actually I was quite confuse in which data-set i have to use thier is lot of reference things are available in readme.md.
for my better understanding please guide me the step for creation of data set for training to output of video.
It helps all those who are come here to see commet.

Thanks