Request permission for checkpoint unitr_pretrain. about unitr

[synsin0]

Okay, I will reproduce the training process as the instructions says. Considers the preprocessing and training time cost. I will report two or three days later to see if the eval goes normally.

[Haiyang-W]

Thanks a lot! If you have any problems, please feel free to let me know.

Notably, we observed that the fluctuation of NuScenes on our unitr is relatively large, and the NDS of Unitr+LSS is between 72.8 and 73.3. The performance reported is the best among the three times. We also provide the best-performing model parameters and logs.

Hope everything goes well with you. :)

Best,
Haiyang

[synsin0]

My reproduced results for the default unitr.yaml without any modification is: mAP: 0.6958 NDS: 0.7269. The result is 0.5% slightly lower than yours best. My training log is provided as follows:
train_20230924-190858.log

[Haiyang-W]

Sorry for that, we will update the pre-trained weights.

We self-pretrain the unitr on ImageNet and nuImage. The structure of Unitr is basically the same as DSVT.

[Haiyang-W]

The Google Drive link is updated.

By the way, could you mention it in this issue if you can reproduce the same performance easily?
We have checked the code in our workspace. However, your double-check will be important to us and helpful to others who use this code. :)

Thanks!

Best,
Haiyang

[Haiyang-W]

Here is the referenced training cost, which could be of assistance to you.

The overall training time of UniTR+LSS is 24h on 8 A100 GPUs(40GB) with shared memory (gt sampling) and closed torch checkpoints here set to [ ]. Enabling the torch checkpoints will extend the training time by an hour. We haven't experimented with the version that doesn't share memory of gt sampling, so we're unable to provide a reference cost regarding this variant.

Enabling torch checkpoints will consume 20GB per GPU card, while disabling them will require over 35GB per GPU card. All batch size are set to 3 by default.

Hope the above information might be helpful to you. :)

[Haiyang-W]

If you have any questions, please feel free to let us know. We will improve and update this code based on your feedback to make it more user-friendly and provide a better experience for everyone.

[synsin0]

Well, I encoutered a problem at the end of pre-processing, which is , Ran out of memory for RAM=120G in my environment. The gt_database creation process costs too much memory. I'd like to confirm the size of two files. I see in my environment , nuscenes_10sweeps_withvelo_lidar.npy = 12G, is that true, may you check the file size to see if mine is correctly generated.

[Haiyang-W]

Well, I encoutered a problem at the end of pre-processing, which is , Ran out of memory for RAM=120G in my environment. The gt_database creation process costs too much memory. I'd like to confirm the size of two files. I see in my environment , nuscenes_10sweeps_withvelo_lidar.npy = 12G, is that true, may you check the file size to see if mine is correctly generated.

• This 12GB is correct; it may vary on different machines and environments. In my case, some machines have 12GB, and some have 24GB. I believe that 12GB should be fine.

• However, when enabling 'share mem,' it does require a significant amount of memory. I have 256GB of RAM on my end. Regarding your question, I would recommend using a machine with sufficient memory to generate the 'share mem' file because training is faster with it, and all of our experiments have 'share mem' enabled. However, turning off 'share mem' should also be fine, and that way, you won't have any memory issues.

[Haiyang-W]

Well, I encoutered a problem at the end of pre-processing, which is , Ran out of memory for RAM=120G in my environment. The gt_database creation process costs too much memory. I'd like to confirm the size of two files. I see in my environment , nuscenes_10sweeps_withvelo_lidar.npy = 12G, is that true, may you check the file size to see if mine is correctly generated.

If you feel it's necessary, you can email me and then add me on WeChat, so we can have a more convenient way to contact and resolve issues.

My email is [email protected]

[Haiyang-W]

Thank you very much for your prompt response. Based on your results, our code should be fine.

I think these results to be reasonably sound; we conducted the experiment thrice and selected the highest score.
The performance of unitr we provided is 73.0 NDS and 70.1 mAP, your result is 72.7 NDS and 69.6 mAP.

The NDS scores across the three runs were 72.6, 73.0, and 72.9 for unitr.yaml without LSS. Notably, there is still a significant fluctuation in the NuScenes dataset, particularly in the mAP metric, where fluctuations are even more pronounced.

Additionally, we have also provided checkpoints for you to verify on your end to ensure consistent evaluation with ours.
Switching to the LSS version can lead to an improvement of 0.3 to 0.5 points.

Thank you once again for your prompt response and for helping us validate the correctness of our code.
I hope UNITR can be helpful for your future research. Wishing you all the best.

Haiyang,

[Haiyang-W]

This issue will be closed as it appears to have been resolved. If you have any further questions, please feel free to ask.

[Looottch]

The Google Drive link is updated.

By the way, could you mention it in this issue if you can reproduce the same performance easily? We have checked the code in our workspace. However, your double-check will be important to us and helpful to others who use this code. :)

Thanks!

Best, Haiyang

link of google drive is gone, would you mind put a new link here

[Haiyang-W]

The Google Drive link is updated.
By the way, could you mention it in this issue if you can reproduce the same performance easily? We have checked the code in our workspace. However, your double-check will be important to us and helpful to others who use this code. :)
Thanks!
Best, Haiyang

link of google drive is gone, would you mind put a new link here

Can you tell me which link is missing? Everything I checked seems to be working.

[Looottch]

The Google Drive link is updated.
By the way, could you mention it in this issue if you can reproduce the same performance easily? We have checked the code in our workspace. However, your double-check will be important to us and helpful to others who use this code. :)
Thanks!
Best, Haiyang

link of google drive is gone, would you mind put a new link here

Can you tell me which link is missing? Everything I checked seems to be working.

ckpt of Bev Map Segmentation in the following pic

eg: https://drive.google.com/file/d/1x189DFgx04SeoyVDkDoZj-FpMPzgYkcn/view?usp=sharing

[nnnth]

I checked my sharing permissions and there seems to be no problem, can you show us what happens after clicking the link?

[Looottch]

I checked my sharing permissions and there seems to be no problem, can you show us what happens after clicking the link?

like this, (maybe u moved to other folder?

[nnnth]

I asked other people to try clicking this link and they had no problem. Maybe you need to change the network, browser, device, or ask someone else to download it for you.

[Haiyang-W]

I checked my sharing permissions and there seems to be no problem, can you show us what happens after clicking the link?

like this, (maybe u moved to other folder?

I think it might be an issue with your internet connection, for example, if you are in mainland China, Google Drive requires using a VPN.