Comments (5)
Oh hooray! There's no docs yet, sorry, I might write them this weekend but I've written up the description of each of the states below as well as a pointer to the chunk of code where they're computed. All the feature extraction code is in lines 142-203 in https://github.com/facebookresearch/nocturne/blob/main/nocturne/cpp/src/scenario.cc
The 500 is a fixed number of road points that are viewed, controlled by the scenario.max_visible_road_points
value in the config. We use a fixed value that is padded to ensure that a fixed size state is provided to users; if you don't want padding you can set padding=false
in the call. As for the column values, I'm detailing them below.
For notation I'll call the vehicle whose feature is being constructed the "ego" agent. Everything where possible is constructed in ego-centric frame.
Object features (zero-indexed):
Element 0: a 1 if it's a valid feature i.e. not padding, 0 otherwise
Element 1: distance between the ego and the agent
Element 2: angle between the positions of the ego and the agent with angle increasing counter-clockwise
Element 3: length of the object
Element 4: width of the object
Element 5: angle between the headings of the agents
Element 6: angle between the velocities of the agents
Element 7: norm of the relative velocities of the agents
Element 8-12: a one hot vector indicating the type of the object. In order these are unset, vehicle, pedestrian, cyclists, other.
Since road points are connected, we refer to the next road point in the road line as the "neighbor road point" i.e. the road line is a series of connected points and there's a direction we traverse the road line in. Note that all of these are also in ego-centric frame.
road point features (zero-indexed):
Element 0: a 1 if it's a valid feature i.e. not padding, 0 otherwise
Element 1: distance between the ego and the road point
Element 2: angle between the positions of the ego and the road point increasing counter-clockwise
Element 3: distance between the road point and its neighbor point in the road.
Element 4: angle between the road point and its neighbor point in the road.
Element 5-12: one hot vector indicating the type of the road point. In order these are none, lane, road-line, road-edge (these can't be crossed without colliding with them), stop-sign, crosswalk, speedbump, other. Note that there's a tiny bug here where stop-sign is included as a possible type but shouldn't be because stop signs are a distinct object from road points.
stop-sign features: also in ego-centric coordinates
Element 0: a 1 if it's a valid feature i.e. not padding, 0 otherwise
Element 1: distance between the ego and the stop sign
Element 2: angle between the positions of the ego and the stop sign
ego features: these are features describing the ego vehicle itself
Element 0: length of ego vehicle
Element 1: width of ego vehicle
Element 2: speed of ego vehicle
Element 3: distance to goal
Element 4: angle to goal
Element 5: desired heading at goal
Element 6: desired speed at goal
Element 7: current acceleration
Element 8: current steering angle
Element 9: current angle of head
Hope that helps! I'll write it up into a nice doc shortly.
from nocturne.
Btw, do feel free to message if more issues arise. I'd really like to see this feature work.
from nocturne.
I am looking into doing this :)
However, I am struggling to understand the data format. For example, the road points (accessed via the Python wrapper via scenario.visible_state(obj, view_dist, view_angle)['road_points']
) are a 2 -dimensional array - as an example, of shape (500, 13). How is this data to be interpreted? I read in the white paper that the data is in VectorNet format - can each row then be decoded into (start, end, features, idx)? Could you point me to some processing code or docs to understand this? Thanks.
from nocturne.
Hi Eugene,
Question about the road points - my understanding is that in the single road_points array are all the road points for all the different line segments of road visible to that agent. For example, both the left and right lines of the road would have points in there. Is there any index indicating which line segment each point belongs to? I am asking because I want to embed line segments separately rather than embedding all the road points together.
Thanks for your help!
from nocturne.
Hi Kate,
Unfortunately there isn't, there's an element indicating what the neighboring point in the line (elements 3 and 4 of the vector) is but no direct pointer to the entire line itself. I admit that does seem useful so we could provide such a functionality but we would have to write it, it doesn't already exist
from nocturne.
Related Issues (20)
- [Bug] Dependencies not listed in environment file HOT 1
- [Bug] Problem installing it on Windows 10 HOT 1
- [Bug] Mistake in continuous action space definition HOT 2
- [Question] Optimal hyperparameters and scripts to reach 2000 steps/sec training speed HOT 3
- Comparison of missing driving simulators in the nocturne paper[Question] HOT 3
- [Question] SFML/Graphics.hpp file not found HOT 4
- [Feature] Is it possible to support user-specific map? HOT 1
- [Feature] Don't install `cfgs` module as a top level Python import HOT 4
- [Question] How is the masking achieved? HOT 4
- [Question] Difference in active_masks copying in shared_buffer and separated_buffer files HOT 2
- [Question] How can I get the VisibleObjects? HOT 2
- The Dropbox abort the download HOT 4
- [Question] Compatibility with VectorNet HOT 8
- [Bug] APPO
- [Bug] python examples/imitation_learning/train.py HOT 1
- [Bug] eval_sample_factory.py
- [Question] How to import the provided expert trajectory and integrate it into the state?
- [Question] Usage issues HOT 1
- [Question] HOT 2
- [Question] HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nocturne.