Comments (2)
Thanks for releasing the nuscenes dataset code support. I have some questions about the implement of the multi-tasks. I see in the code that you define obj_num=500 for each task and then the task_id will be added to the pos embedding to identify each task in rpn transformer. But unfortunately, the computation increases, and my machine directly throw the error that the cuda memory OOM. As for the implement of multi-task, my intuitive idea is that each task has its own head during the generation of heatmap. Then, all heatmaps are contacted to one tensor and generate top500 center queries, then sent to rpn transformer, Meanwhile, the pos feature is also the regular x and y coordinates. In the final output detection head, each task have their own detection head applying to transformer output features, which can reduce the increasing computation in transformer layer. This is my first thought, I wonder if you has experimented this way, is there any drawbacks? Could you share the effects or conclusions or something like that? It is very important to me. Thank you ~
Hi, sorry for the late reply. I agree with you that the current method is a bit cumbersome. Some tasks may not need that much of center candidates. But there will be some issues if you select the top K centers from a merged heatmap:
- It is hard to merge the scores or select a suitable threshold for the center candidates. Some tasks may have lower heatmap scores than others.
- Different tasks may have the same high response region. I found it has better results if each task is dealt with separately.
I also found the computation cost increase is relatively small since the transformer part of CenterFormer is already lightweight. Hence, I choose to implement it in this way. If you still have the memory issue, consider reducing the batch size or obj_num
.
from centerformer.
By the way, have you experimented the time sequence fusion through the rpn transformer in nuscenes dataset? How does it work?
from centerformer.
Related Issues (20)
- About `disable_dbsampler` HOT 2
- About Lidar and image fusion
- The effect of deformable attention HOT 1
- Question about why the add&norm structure of the tranformer network differ from the typical transformer one HOT 3
- If the positions "x_coor" and "y_coor" should be swapped in Line 466 and 468 det3d/models/necks/rpn_transformer? HOT 7
- Use CenterFormer on other datasets HOT 2
- CenterFormer on kitti
- Evaluation on waymo opendataset HOT 8
- waymo coordinates HOT 2
- previous frame transformed to current frame? HOT 1
- Positional embedding in RPN_transformer_deformable_multitask
- Is it correct that Nan appears in the loss? HOT 5
- AUTOMATIC MIXED PRECISION
- /usr/include/stdio.h(189): error: attribute "__malloc__" does not take arguments
- Redundant boxes after post processing
- Implementation of CorssAttention
- Are the x, y, z values included in the value of the variable center_pos?
- nuScenes result?
- Issue in points np.concatenate(s_points_list, axis=0) in centerformer-master/det3d/core/sampler/sample_ops.py HOT 2
- issues about testing on nuscenes testset
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from centerformer.