Comments (15)
@MadhavEsDios
When add new class to model, you need to change:
- model.second.num_class
- add new anchor_generators
- train_input_reader.class_names
- eval_input_reader.class_names (I will move this to the model proto in future.)
I have fixed the assign problem by assign targets independently for each class and trained a multi-class (car, ped, cyc, van) model:
Car [email protected], 0.70, 0.70:
bbox AP:90.84, 89.53, 88.40
bev AP:90.09, 86.85, 86.83
3d AP:88.35, 77.96, 76.03
aos AP:90.80, 89.19, 87.70
Cyclist [email protected], 0.50, 0.50:
bbox AP:83.99, 70.85, 64.79
bev AP:80.24, 65.75, 60.44
3d AP:76.89, 59.41, 57.49
aos AP:83.75, 69.74, 63.86
Pedestrian [email protected], 0.50, 0.50:
bbox AP:68.37, 62.37, 59.95
bev AP:62.24, 57.79, 52.19
3d AP:58.00, 51.32, 48.36
aos AP:60.23, 54.72, 52.04
Van [email protected], 0.70, 0.70:
bbox AP:47.23, 41.50, 35.88
bev AP:47.41, 41.47, 36.03
3d AP:41.79, 33.79, 28.65
aos AP:47.20, 41.41, 35.31
This model use a slightly different architecture and will be released soon. This assign problem will be fixed after the PointPillars authors send a merge pull request to this repo.
from second.pytorch.
I will upload a sample config. you can see the following example first.
model: {
second: {
...
num_class: 3
target_assigner: {
anchor_generators: {
# fixed-size anchor generator for EVERY class.
anchor_generators: { # cars
anchor_generator_range: {
sizes: [1.6, 3.9, 1.56] # wlh
anchor_ranges: [0, -40, -1.78, 70.4, 40, -1.78] # carefully set z center
rotations: [0, 1.57] # DON'T modify this unless you are very familiar with my code.
matched_threshold : 0.6
unmatched_threshold : 0.45
}
}
anchor_generators: { # cyclist
anchor_generator_stride: {
sizes: [0.6, 1.76, 1.73] # wlh
strides: [0.2, 0.2, 0.0] # if generate only 1 z_center, z_stride will be ignored
offsets: [0.1, -19.9, -1.465] # origin_offset + strides / 2
rotations: [0, 1.57]
matched_threshold : 0.5
unmatched_threshold : 0.35
}
}
anchor_generators: { # pedestrian
anchor_generator_stride: {
sizes: [0.6, 0.8, 1.73] # wlh
strides: [0.2, 0.2, 0.0] # if generate only 1 z_center, z_stride will be ignored
offsets: [0.1, -19.9, -1.465] # origin_offset + strides / 2
rotations: [0, 1.57]
matched_threshold : 0.5
unmatched_threshold : 0.35
}
}
...
}
}
}
train_input_reader: {
# the order of class_names MUST match order of anchor generators
class_names: ["Car", "Cyclist", "Pedestrian"]
...
database_sampler {
database_info_path: "/media/yy/960evo/datasets/kitti/kitti_dbinfos_train.pkl"
sample_groups {
name_to_max_num {
key: "Car"
value: 8
}
}
sample_groups {
name_to_max_num {
key: "Pedestrian"
value: 4
}
}
sample_groups {
name_to_max_num {
key: "Cyclist"
value: 4
}
}
database_prep_steps {
filter_by_min_num_points {
min_num_point_pairs {
key: "Pedestrian"
value: 5
}
min_num_point_pairs {
key: "Cyclist"
value: 5
}
}
}
database_prep_steps {
filter_by_difficulty {
removed_difficulties: [-1]
}
}
global_random_rotation_range_per_object: [0, 0]
rate: 1.0
}
...
}
eval_input_reader: {
class_names: ["Car", "Cyclist", "Pedestrian"]
...
}
from second.pytorch.
@traveller59 Thanks for you advice. I shall try it out
from second.pytorch.
Any update for multi-class detection?
from second.pytorch.
@traveller59
In order to train multi classes(I only tried 3, car, van, truck), I go through the KITTI dataset and get the average dimensions wlh for each class. The problem now is, if I only change the num_class
, class_names
and the parameters sizes
in the anchor_generator_range
of each class, the training accuracy is still quite low (car is OK with AP 0.80+, but van is ~0.4) after 150000+ steps, especially for the Truck which remains 0 all the time. So, I am wondering is there any other parameter that you suggest me to modify? I am not that familiar with your code.
thanks for your suggestions in advance
from second.pytorch.
I tried to train the model with people.config, but the result is not good. Waiting for multi-class detection.
from second.pytorch.
Has anyone tried out the sample config ?
from second.pytorch.
@MadhavEsDios there is a problem in target assign when training with multi-class, lots of people and cyclist anchors are assigned to car target. expected behavior is every class only be assigned to one kind of anchors. I need time to fix this.
from second.pytorch.
Thank you so much @traveller59 for looking into this.
So when I was comparing the config files for both classes i.e cars and ped-cycle, I found that apart from the anchor-generator, there were some additional differences namely :-> point_cloud_range, voxel_size and layer_strides. Is this a possible reason for the issues ?
from second.pytorch.
Thanks a lot @traveller59, looking forward to the new commit.
from second.pytorch.
Could you please upload sample.config file to train simulatanously for multiclass detection
from second.pytorch.
@traveller59 Any updates on this?
from second.pytorch.
@chowkamlee81 there is a sample config all.fhd.config
in this project.
@jinfagang The people detection need small receptive field. The performance of people detection in car-like config leads to bad performance. we may need a other head from sparse feature extractor for people detection, or a RCNN structure, point cloud of pedestrian can be scaled before get into RCNN.
There is another strange thing: I have submit people detection network result to KITTI test server (for my master thesis) and get bad test performance (a little worse than SECOND v1), but the val performance is better than v1.
from second.pytorch.
@traveller59
Is there any pretrained model for pointpillars? It would be great if there is a single model for all classes.
from second.pytorch.
@traveller59 i have following queries
- can you give a small description of what does the function anchor_generator_stride() and anchor_generator_range() do
- for custom dataset for custom class should we have to use anchor_generator_range() with values sizes: [1.6, 3.9, 1.56] and anchor_ranges[], is it default since the custom class object is small object
thanks in advance
from second.pytorch.
Related Issues (20)
- KeyError: 'annotations' when using nuscenes dataset
- KeyError: 'annotations' when using nuscenes dataset HOT 3
- How to start training from interrupted step HOT 1
- About gt_sampling
- ModuleNotFoundError: No module named 'second' HOT 1
- Kitti web viewer backend issue HOT 1
- second.pytorch 1.6.0 Alpha and spconv 2.1 HOT 2
- Convert custom Lidar point cloud data to KITTI format HOT 1
- Summary name eval.kitti/official/Car/[email protected]/1 is illegal; using eval.kitti/official/Car/3d_0.70/1 instead.
- Issues while using Kitti viewer. HOT 1
- Need tips on improving performance on custom dataset
- Need suggestions on how to generate onnx files using this repo
- Source of torchplus package
- OSS License compatibility question
- About kitti viewer
- second.data HOT 2
- I would like to know how many samples should be used for validation if 3517 samples are used for training?
- Is the sparse library writed by the autuor?
- Input only the desired scene
- 有人试过把这个代码在windows上运行吗
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from second.pytorch.