Comments (4)
Currently, if a new metric needs to be added, the name of the metric must be added at various places in the code. This process can be made simpler.
from localplannerbench.
Developer notes on the current implementation:
- To create a new metric, inherit from the metric class and implement the computemetric() method in metrics.py
- In order to use the metric for benchmarking purposes for a specific type of robot, it must be instantiated for that type of robot in the postprocessor script.
self._metrics['time2Goal'] = TimeToReachGoalMetric(
"time2Goal",
["q0", "q1", "goal_0_0", "goal_1_0", "t"],
{"m": 2, "des_distance": self._experiment.primeGoal().epsilon()},
)
- The fist argument is a string to represent the type of metric. It doesn't need to be passed as an argument here, it can be an instance attribute of the metric class.
- The 2nd argument is a list of column indices in res.csv that needs to be selected for postprocessing. The columns chosen for each type of robot and metric combination will differ which requires a unique instantiation for each metric and robot combination, but this issue can be solved by renaming the columns in the res.csv to represent the parameters type and not the specific variable names.
- The 3rd argument comes from experiemnt.py; this can be standardized too. in the above manner.
-
Requirements for creating a new metric
- Knowledge of the computation logic for the metric; this is needed to implement the computemetric method()
- Knowledge of how the new metric will be instantiated for a particular type of robot. This is needed to specify which column indices from res.sv are required for metric computation.
-
The arguments of the class instantiation of the metrics in the postprocessor script can be standardized so that they can be embedded in the metric class itself.
-
self.experiment.robot_type() can be used to construct 2nd and 3rd argument of the class instantiation
-
A new metric may not apply to all the robot types; it would depend on the what is being computed in the metric.
from localplannerbench.
Expected functionality (tentative): A metric class should be instantiated for a particular type of robot based on the arguments passed in the postprocessor command. Currently, all metric class are being instantiated for all types of robots in advance.
from localplannerbench.
This could be done with the registry pattern. Similar to #25 for planners.
from localplannerbench.
Related Issues (20)
- Make use of property for private variables.
- Variable namings
- Split config file loading from class abstraction HOT 1
- Vulnerabilities from pillow <9.0.1
- KeyError running postProcessor tutorial example HOT 2
- Unit test metrics
- Add at least one additional planner that is fully open source. HOT 2
- Add section to documentation that lists steps to add your own planner (walkthrough)
- Document in one place all available yaml parameters with their types and descriptions. i.e. an API for using localPlannerBench
- All dependent packages should be on pip in a first release. HOT 1
- KeyError forward kinematics and goal in postProcessor HOT 1
- Fabrics planner set again with multiple runs HOT 1
- GNUplot scripts requires hardcoded planner names
- Installation as a package (through PyPi) is not well thought through
- Rerunning experiment from study HOT 5
- [Feature Request] Simulate uncertainty of state and observations HOT 1
- [Feature Request] Add wrappers/interfaces to opensource State-of-the-art
- [Feature Request] Support more fidelity in observations
- [Feature Request] Interface with global planner
- Integrating Social forces model for obstacles
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from localplannerbench.