Comments (3)
Approach to writing the integration/blackbox test: Compare the output of running an experiment with a specific type of planner and robot with the expected output.
The planners are deterministic, so the planners are expected to produce the same output for the same input configuration. Some of the fields are non-deterministic; they produce values that depend on the platform or OS that the experiment is run on. For the integration tests only the deterministic output fields can be compared each time.
Requirements
- Input configurations and results produced by the planners for those configurations.
- Pseudo code/logic to run experiments, store results in temporary directories, compare them against expected output, and delete the temporary result directories.
Workflow actions to be tested:
- Test if the runner command resulted in creation of a results folder with the expected no. and names of the files: res.csv, exp.yml, etc.
- test if the deterministic contents of the results produced are the same for a specific robot type and experiment.
- The test can be made more robust by testing runner execution and results produced for different combinations such as one planner and one robot type, two panners and one robot type, one planner and two robot typs, etc.
The integration test can be developed for the example planner now. It is not possible to use fabric and mpc planner in the tests because these planner are not open source; they can't be installed and run by GitHub actions. In future, open source versions of these two planners could be developed.
from localplannerbench.
Introduced initial tests in a1c69c6
from localplannerbench.
See #12 for switching from bash script to pytest only syntax.
from localplannerbench.
Related Issues (20)
- Make use of property for private variables.
- Variable namings
- Split config file loading from class abstraction HOT 1
- Vulnerabilities from pillow <9.0.1
- KeyError running postProcessor tutorial example HOT 2
- Unit test metrics
- Add at least one additional planner that is fully open source. HOT 2
- Add section to documentation that lists steps to add your own planner (walkthrough)
- Document in one place all available yaml parameters with their types and descriptions. i.e. an API for using localPlannerBench
- All dependent packages should be on pip in a first release. HOT 1
- KeyError forward kinematics and goal in postProcessor HOT 1
- Fabrics planner set again with multiple runs HOT 1
- GNUplot scripts requires hardcoded planner names
- Installation as a package (through PyPi) is not well thought through
- Rerunning experiment from study HOT 5
- [Feature Request] Simulate uncertainty of state and observations HOT 1
- [Feature Request] Add wrappers/interfaces to opensource State-of-the-art
- [Feature Request] Support more fidelity in observations
- [Feature Request] Interface with global planner
- Integrating Social forces model for obstacles
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from localplannerbench.