opendrivelab / elm Goto Github PK

Thank you for your outstanding contributions. How to activate the result of 'B, C, R' of surrounding narration, Action & Decision in the paper? Looking forward to your reply.

Below attaching the full log:

Using downloaded and verified file: /home/pravaig-20/ELM/lavis/.././data/drivelm/train.json
Using downloaded and verified file: /home/pravaig-20/ELM/lavis/.././data/drivelm/val.json
Using downloaded and verified file: /home/pravaig-20/ELM/lavis/.././data/drivelm/val.json
2024-04-03 10:44:16,614 [INFO] Building datasets...
Traceback (most recent call last):
  File "scripts/train.py", line 112, in <module>
    main()
  File "scripts/train.py", line 104, in main
    datasets = task.build_datasets(cfg)
  File "/home/pravaig-20/ELM/lavis/tasks/vqa.py", line 74, in build_datasets
    datasets = super().build_datasets(cfg)
  File "/home/pravaig-20/ELM/lavis/tasks/base_task.py", line 57, in build_datasets
    dataset = builder.build_datasets()
  File "/home/pravaig-20/ELM/lavis/datasets/builders/base_dataset_builder.py", line 58, in build_datasets
    datasets = self.build()  # dataset['train'/'val'/'test']
  File "/home/pravaig-20/ELM/lavis/datasets/builders/base_dataset_builder.py", line 228, in build
    datasets[split] = dataset_cls(
  File "/home/pravaig-20/ELM/lavis/datasets/datasets/elm_datasets.py", line 84, in __init__
    self.default_drivelm()
TypeError: default_drivelm() missing 1 required positional argument: 'ann_paths'

Great work！
I have some questions to ask you.
I would like to know your training details during the pre-train phase, such as the amount of training data and training configuration.
Can you tell me the GPU hours for pre-train and fine-tuned?
It seems that you are only using the front view video and not the multi view. I don't know if I understand correctly

How to obtain the Pr@k indicators proposed in the paper? I can't seem to find the relevant code.

No such file or directory: 'data/vocab.txt'

one typo in tools/video_clip_processor

from lavis.datasets.data_utils load_pickle

to

from lavis.datasets.data_utils import load_pickle

I found that the responses to keyword I found that the responses to keyword A were all invalid:
were all invalid in drivelm_train.json:
like: Apologies, but I do not have information about the surrounding objects at this time.

What do I do if I want to compare these metrics about surrounding narration, Action & Decision?

hi, nice work!
I wanna know how you produce the planning_train/val.json? Is it the same way as UniAD does?
best!
@ZhouYunsong-SJTU

How can I get 'data/embodied/BOXQA_train_v3.json' in this line ?

What is self.temporal_length mean in lavis/datasets/datasets/elm_datasets.py ?

How can I get self.traffic_element_dict in this line?

When will you release file used in this function?

In lavis/datasets/datasets/elm_datasets.py and lavis/projects/blip2/train/advqa_t5_elm.yaml, I found some necessary files that hasn't been mentioned in README.

• How to get bevdetv2-nuscenes_infos_trainval.pkl?
• I've seen drivelm_train.json in this link , but how can I get 'data/drivelm/val.json' in advqa_t5_elm.yaml? Will this file be available in the future?

I try to reproduce the code but I find the model use temporal image lists in default, which I think is the settings of default_box_qa but not default_drivelm. Nowhere says the temporal length issue, however.

Hi, it's a nice work!
I notice that you test the planning on waymo dataset. It's structure is differrent from nuscenes, so can you provide the scripts to get waymo trajectory gt and eval ? thanks so much