GithubHelp home page GithubHelp logo

haomo-ai / cam4docc Goto Github PK

View Code? Open in Web Editor NEW
180.0 12.0 9.0 11.54 MB

[CVPR 2024] Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications

License: MIT License

Python 98.34% C++ 0.63% Cuda 0.81% Shell 0.22%
3d-occupancy 3d-occupancy-prediction occupancy-prediction 4d-occupancy occupancy-forecasting surrounded-camera

cam4docc's Issues

Error with running a baseline

I experience the following problem when using config OCFNet_in_Cam4DOcc_V1.2.py:

  File "/home/eitan/Cam4DOcc/projects/occ_plugin/occupancy/image2bev/ViewTransformerLSSBEVDepth.py", line 495, in forward
    depth = self.depth_conv(depth)
  File "/fs/scratch/rb_bd_dlp_rng-dl01_cr_AID_employees/users/eitan/envs/cam4docc/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/fs/scratch/rb_bd_dlp_rng-dl01_cr_AID_employees/users/eitan/envs/cam4docc/lib/python3.7/site-packages/torch/nn/modules/container.py", line 141, in forward
    input = module(input)
  File "/fs/scratch/rb_bd_dlp_rng-dl01_cr_AID_employees/users/eitan/envs/cam4docc/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/fs/scratch/rb_bd_dlp_rng-dl01_cr_AID_employees/users/eitan/envs/cam4docc/lib/python3.7/site-packages/mmcv/ops/deform_conv.py", line 378, in forward
    False, self.im2col_step)
  File "/fs/scratch/rb_bd_dlp_rng-dl01_cr_AID_employees/users/eitan/envs/cam4docc/lib/python3.7/site-packages/mmcv/ops/deform_conv.py", line 109, in forward
    im2col_step=cur_im2col_step)
RuntimeError: CUDA error: no kernel image is available for execution on the device

What could be the problem? The installation seemed to work fine without any error.

How many GPU resources does this task require in total?

Hi! I noticed that your script uses 8 GPUs for training and testing simultaneously. How many GPU resources are needed to reproduce the results of this task, training and testing, in totally? Thanks for noticing this issue！

about computation cost

Hi, thanks to your great work,

Can you tell me how much GPU memory is needed approximately for this task?

Wrap feature possible issue

In line 293 of ocfnet.py， I think the transformed_grid.shape[1]=4, which may indicate you only wrap 4 voxel grid feature to the wraped_x from x. I don't know if I understand right, I think the for loop should be the number of voxels. May you check again this part?

where is segmentation folder? 

Traceback (most recent call last):
  File "tools/train.py", line 204, in <module>
    main()
  File "tools/train.py", line 200, in main
    meta=meta)
  File "/home/glj/roseupram/Cam4DOcc/projects/occ_plugin/occupancy/apis/train.py", line 27, in custom_train_model
    meta=meta)
  File "/home/glj/roseupram/Cam4DOcc/projects/occ_plugin/occupancy/apis/mmdet_train.py", line 113, in custom_train_detector
    runner.run(data_loaders, cfg.workflow)
  File "/home/glj/miniconda3/envs/cam4docc/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/glj/miniconda3/envs/cam4docc/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 47, in train
    for i, data_batch in enumerate(self.data_loader):
  File "/home/glj/miniconda3/envs/cam4docc/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
    data = self._next_data()
  File "/home/glj/miniconda3/envs/cam4docc/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
    return self._process_data(data)
  File "/home/glj/miniconda3/envs/cam4docc/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
    data.reraise()
  File "/home/glj/miniconda3/envs/cam4docc/lib/python3.7/site-packages/torch/_utils.py", line 434, in reraise
    raise exception
FileNotFoundError: Caught FileNotFoundError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/glj/miniconda3/envs/cam4docc/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/glj/miniconda3/envs/cam4docc/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/glj/miniconda3/envs/cam4docc/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/glj/roseupram/Cam4DOcc/projects/occ_plugin/datasets/cam4docc_dataset.py", line 100, in __getitem__
    data = self.prepare_train_data(idx)
  File "/home/glj/roseupram/Cam4DOcc/projects/occ_plugin/datasets/cam4docc_dataset.py", line 312, in prepare_train_data
    example = self.prepare_sequential_data(index)
  File "/home/glj/roseupram/Cam4DOcc/projects/occ_plugin/datasets/cam4docc_dataset.py", line 370, in prepare_sequential_data
    example = self.pipeline(input_seq_data)
  File "/home/glj/miniconda3/envs/cam4docc/lib/python3.7/site-packages/mmdet/datasets/pipelines/compose.py", line 40, in __call__
    data = t(data)
  File "/home/glj/roseupram/Cam4DOcc/projects/occ_plugin/datasets/pipelines/loading_instance.py", line 392, in __call__
    np.savez(seg_label_path, segmentation_saved_list2)
  File "<__array_function__ internals>", line 6, in savez
  File "/home/glj/miniconda3/envs/cam4docc/lib/python3.7/site-packages/numpy/lib/npyio.py", line 616, in savez
    _savez(file, args, kwds, False)
  File "/home/glj/miniconda3/envs/cam4docc/lib/python3.7/site-packages/numpy/lib/npyio.py", line 712, in _savez
    zipf = zipfile_factory(file, mode="w", compression=compression)
  File "/home/glj/miniconda3/envs/cam4docc/lib/python3.7/site-packages/numpy/lib/npyio.py", line 112, in zipfile_factory
    return zipfile.ZipFile(file, *args, **kwargs)
  File "/home/glj/miniconda3/envs/cam4docc/lib/python3.7/zipfile.py", line 1240, in __init__
    self.fp = io.open(file, filemode)
FileNotFoundError: [Errno 2] No such file or directory: './data/segmentation/2175d0e84f224ea69907e5c338bde395_b60a2bc82d88419497510082d9301a59.npz'

where is segmentation folder?

How to change the grid_size?

Hello! I have read this paper, which is an impressive work. However, I want to run your code on my computer (RTX4090) because the default grid size is 512 x 512 x 40, which is too big.

I have tried to change the grid_size to 200 x 200 x 16 and it can run appropriately. However, when I run the vis_gt.py, I get the below result:

242ac1b59fec4e0c9c200ffca81775e2_7c3bbed918f14e58b0ab7dd94bb4216d

Uploading eae0cfe3be5f44a5be4f3e2c961397ed_72a725a46e374dd281c955b5da4cd603.png…

About training time

Hi，
I am trying to run training code in this repository with 5 GPUs, but as training continues, more and more training time is required. Can you help me with it?

87697c6fafa8904dbd40ca7abc17d861

how much GPU memory do we need?

thanks for your release
I only have one gpu, memory is 48G, so i run this

run.sh ./projects/configs/baselines/OCFNet_in_Cam4DOcc_V1.1.py 1

but seems my gpu is not enough

RuntimeError: CUDA out of memory. Tried to allocate 198.00 MiB (GPU 0; 47.54 GiB total capacity; 45.25 GiB already allocated; 18.19 MiB free; 45.46 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Is there a way to run this code with one 48G gpu

4D annotations

Hi, could you provide the 4D annotations as zip files?
Generating the dataset is taking longer than a week.

Thanks

An question about Depth loss

Dear Authors,

I noticed that the numerical results provided in your paper seem to have been obtained using depth loss. However, the source code you have provided does not include the implementation of depth loss. Could you please clarify if the results in the paper were indeed obtained using depth loss, and if so, could you provide the implementation details or update the source code to include it?

Thank you!

Best regards,

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.