Execution Monitoring Framework

Fully integrated plan execution, monitoring and resolution framework capable of overcoming some of the typical limitations of long-term autonomous mobile outdoor robots. Detailed background information is available in my master's thesis, in the context of which this framework was developed.

General Dependencies

plan_generation: ROS node that generates / provides action plans based on CSV data
mongodb_store: MongoDB tools for storing and analyzing runs of ROS systems
- when using the logging component
pointcloud_to_laserscan: Converts 3D point clouds into 2D laser scans
- when working with PointCloud2
psutil: Cross-platform library for process and system monitoring in Python
- for data management monitoring
speedtest-cli: Command line interface for testing internet bandwidth using speedtest.net
- for internet connection monitoring
PyOWM: Python wrapper around OpenWeatherMap API
- for weather monitoring
- set up src/execution_monitoring/secret_config.py containing OWM_API_KEY = YOUR_OWM_API_KEY

Dependencies for the Prototype Scenario in the Simulation

arox_docker: Dockerization of the AROX (Autonomous Robotic Experimentation Platform)
- branch: noetic
- packages and compatible branches within the docker container:
  - arox_navigation_flex: feature_msc_setup_tim
  - arox_launch: feature_msc_setup_tim
  - arox_indoor_navi: sim_launch_detection_config
  - arox_engine: feature_lta_engine
  - arox_performance_parameters: feature_msc_setup_tim
  - map_langsenkamp: feature_lta_map
  - arox_docking: feature/python2_compatible
arox_description: ROS launch files and URDF model for the AROX system
- branch: feature_msc_setup_tim
container_description: ROS launch files and URDF model for the mobile container (charge station)
- branch: feature_msc_setup_tim
innok_heros_description: URDF description for Innok Heros robot
- branch: arox_noetic
innok_heros_driver: ROS driver for the Innok Heros robot platform
- branch: master
velodyne_simulator: URDF and gazebo plugin to provide simulated data from Velodyne laser scanners
- branch: master
gazebo_langsenkamp: Langsenkamp world (test field)
- branch: feature_msc_setup_tim
hector_gazebo_plugins: contains a 6wd differential drive plugin, an IMU sensor plugin, an earth magnetic field sensor plugin, a GPS sensor plugin and a sonar ranger plugin
- install: apt-get install ros-noetic-hector-gazebo

Gazebo Models

Models to be placed in /.gazebo/models/:

charger_ground_patch, Corn, langsenkamp_simulation, Pillar_1
additionally: stop_sign and jersey_barrier from the /models directory of this repository

Usage - Prototype Scenario in the Simulation

run simulation (with GUI): roslaunch arox_description launch_arox_sim.launch gui:=true
optional [spawn container: roslaunch container_description spawn.launch]
spawn AROX: roslaunch arox_description spawn.launch
run AROX controllers: roslaunch arox_description run_controllers.launch
run docker container named 'arox_msc': aroxstartdocker arox_msc (alias)
- launch outdoor simulation: roslaunch arox_launch arox_sim_outdoor.launch

Usage - Execution Monitoring Framework (Within Docker Container)

$ roslaunch execution_monitoring execution_monitoring.launch

Components that can be (de)activated in the launch file:

plan generation
autonomous (un)docking
database logging
dummy scanner
failure resolution
failure simulation
monitoring nodes (individually)
energy consumption model
LTA experiments
pointcloud to laserscan converter

Robot-Human Communication Module

$ rosrun execution_monitoring operator_communication.py

Configuration of Setup, Monitoring, Resolution and Simulation

src/execution_monitoring/config.py

Architecture

high level (plan execution + monitoring):

low level (operational model):

Show Database Entries

Complete database:

$ rostopic pub -1 /show_db_entries std_msgs/String "complete"

Entries for one category:

$ rostopic pub -1 /show_db_entries std_msgs/String "CATEGORY_NAME"

Access database after mission is completed:

path for the database PATH_TO_DB can be configured in the launch file
run MongoDB server with the generated database: mongod --dbpath PATH_TO_DB --storageEngine=wiredTiger
use MongoDB client to connect and examine the logged data

Exploration GUI

http://localhost/exploration_gui/

Settings for AROX Battery (Non-Defaults)

discharge_rate: 0.1
battery_charging_level: 100

Configurable via:

$ rosrun rqt_reconfigure rqt_reconfigure

Useful Topics to Monitor

state of mission: rostopic echo /arox/ongoing_operation
battery charge level: rostopic echo /arox/battery_param

Control AROX

Launch keyboard control:

$ rosrun teleop_twist_keyboard teleop_twist_keyboard.py

Visualize Sensor Data

rViz
- fixed frame: map
- add sensors by topic, e.g., /velodyne_points to visualize the point cloud recorded by the Velodyne 3D lidar sensor
- or open the provided config msc_conf.rviz

Open Container (Lower Ramp)

$ rostopic pub -1 /container/rampA_position_controller/command std_msgs/Float64 "data: 1.57"

Clear Costmaps (`move_base_flex`)

$ rosservice call /move_base_flex/clear_costmaps "{}"

Send 'End-of-Episode' Signal

$ rostopic pub -1 /end_of_episode std_msgs/String "EOE"

Monitoring States

Manually trigger CONTINGENCY or CATASTROPHE:

$ rostopic pub -1 "/contingency_preemption" std_msgs/String contingency
$ rostopic pub -1 "/catastrophe_preemption" std_msgs/String catastrophe

Visualize State of Hierarchical SMACH (Introspection Server)

$ rosrun smach_viewer smach_viewer.py

Application to other ROS Systems / Extensibility

The framework is in principle applicable to other ROS systems if the introduced slight constraints in terms of information provision based on the defined ROS messages and topics are met. In addition, some minor requirements must be fulfilled in order to be able to utilize the resolution attempts. To get a detailed overview of the requirements, it is advisable to read the relevant sections in the thesis (cf. sec. 3.5, 3.6, 6.1).

A further important note with regard to the general applicability of the framework is that each monitoring node can easily be removed, i.e., deactivated. Presently, this can be achieved by simply removing the option from the framework’s launch file. Analogously, custom monitoring nodes can be added to the framework by simply adding the nodes to the launch file and supporting the described architecture by publishing to /contingency_preemption / /catastrophe_preemption in case of failure, etc. Thus, the idea is that context-dependent special solutions can be easily added, removed and replaced by other specific solutions.

The general way of responding to new custom failure cases included this way would be to define custom error cases in config.py, check for those in the execution of CATASTROPHE / CONTINGENCY in high_level_smach.py and add a custom resolver to resolver.py.

It is planned to further simplify the customization options in the future.

TL;DR

Requirements for monitoring:

Power Management
- battery watchdog module that publishes the expected states "CONT", "CATO", "NORM" on /watchdog
- indicate fully charged battery via /fully_charged
Drastic Weather Change
- provide NavSatFix data on /fix
- internet connection for API access
Sensor (Perception) Failure
- indicate scan action via /scan_action
- provide configurable topic (config.py) at which LaserScan / PointCloud2 data is to be expected
WiFi Connection Failures
- if not using a Linux system, provide OS-specific WiFi monitor equivalent to the one in /os_specific that creates WiFi.msg data and publishes on /wifi_connectivity_info
Internet Connection Failures
- internet connection
GNSS Connection Failures
- publish NavSatFix messages on /fix
Data Management
- indicate scan actions via /scan_action and their completion via /scan_completed
- config.py: configure MONITOR_DRIVE as well as SCAN_PATH
Plan Deployment Failure
- information whether a plan has been received, i.e., a mission is being executed, is retrieved via a configurable topic, by default /arox/ongoing_operation (cf. config.py)
- communicate explicit plan retrieval fails via /plan_retrieval_failure
  - pre-defined codes:
    - 0: plan retrieval service unavailable
    - 1: empty plan
    - 2: infeasible plan
- use plan_generation and configure the list of actions in config.py
Navigation Failure
- any navigation framework that is compatible with the general actionlib_msgs/GoalStatus is compatible with the introduced monitoring
- specify GOAL_STATUS_TOPIC under which the GoalStatusArray messages appear
- NavSatFix messages are expected to appear on /fix to track progress in cases of sustained recovery
- /explicit_nav_failure can be used to report explicit navigation errors
Incorrect or Inaccurate Localization
- localization must be based on sensor fusion of IMU, odometry and GNSS data
- provide the corresponding sensor information on /imu_data, /odom and /odometry/gps (types: sensor_msgs/Imu.msg and nav_msgs/Odometry.msg)
- /odometry/gps - GNSS information converted to the same format as the odometry data
- navigation based on actionlib_msgs/GoalStatus.msg, e.g., move_base or move_base_flex
- Kinematics also play a role; in general, it should be a wheeled robot with differential drive. While an Ackermann drive would also be permissible, others, such as omni-drive robots, are not compatible because they would violate certain assumptions used in the monitoring approaches, such as in the orientation interpolation based on GNSS data.

More detailed explanations as well as the requirements for the resolution procedures can be found in the corresponding sections of the thesis (cf. sec. 3.5, 3.6, 6.1).

Replacement of the Operational State Machine

In order for another robotic system to use the monitoring framework without using the operational model used in this thesis, the embedded OPERATION state machine shown above would have to be replaced by the system’s own operational state machine, which is either also implemented as SMACH or at least wrapped by one. There are some assumptions that such an operational state machine must satisfy:

Communicate information about the mode of the system via /arox/ongoing_operation in the following form:

arox_operational_param.msg

string operation_mode
int32 total_tasks
int32 ongoing_task
int32 rewards_gained

[naming scheme due to compatibility with the battery watchdog module]

Available modes:

scanning, traversing, waiting, docking, undocking, charging, contingency, catastrophe

all active goals should be interruptible via a message on /interrupt_active_goals, e.g., by implementing the actions as SimpleActionServer
the outcomes must be compatible with the architecture shown above, i.e., minor_complication, critical_complication, end_of_episode and preempted

Experiments - Simulation of LTA Challenges

Each failure case can be simulated by publishing on the respective topic, e.g.:

$ rostopic pub -1 /toggle_simulated_total_sensor_failure std_msgs/String fail

Category	Failure	Topic
Sensor Failures	total sensor failure	`/toggle_simulated_total_sensor_failure`
Sensor Failures	empty list of range values	`/toggle_simulated_empty_ranges`
Sensor Failures	predominantly impermissible values	`/toggle_simulated_impermissible_ranges`
Sensor Failures	repeated scan	`/toggle_simulated_scan_repetition`
WiFi Failures	poor link quality	`/toggle_simulated_bad_wifi_link`
WiFi Failures	poor signal	`/toggle_simulated_bad_wifi_signal`
WiFi Failures	poor bit rate	`/toggle_simulated_bad_wifi_bit_rate`
WiFi Failures	disconnect	`/toggle_simulated_wifi_disconnect`
Internet Failures	disconnect	`/sim_internet_connection_failure`
Internet Failures	bad download	`/toggle_simulated_bad_download`
Internet Failures	bad upload	`/toggle_simulated_bad_upload`
GNSS Failures	disconnect / timeout	`/toggle_simulated_timeout_failure`
GNSS Failures	good quality	`/set_simulated_good_quality`
GNSS Failures	medium quality	`/set_simulated_med_quality`
GNSS Failures	low quality	`/set_simulated_low_quality`
GNSS Failures	unknown status	`/set_simulated_unknown_status`
GNSS Failures	no fix	`/set_simulated_no_fix`
GNSS Failures	no RTK	`/set_simulated_no_rtk`
GNSS Failures	unknown service	`/toggle_simulated_unknown_service`
GNSS Failures	infeasible lat / lng	`/toggle_simulated_infeasible_lat_lng`
GNSS Failures	variance history failure	`/toggle_simulated_variance_history_failure`
GNSS Failures	high deviation	`/toggle_simulated_high_deviation`
Drastic Weather Changes	heavy rain	`/toggle_rain_sim`
Drastic Weather Changes	heavy snow	`/toggle_snow_sim`
Drastic Weather Changes	gale	`/toggle_wind_sim`
Drastic Weather Changes	low temperature	`/toggle_low_temp_sim`
Drastic Weather Changes	thunderstorm	`/toggle_thuderstorm_sim`
Drastic Weather Changes	sunset	`/toggle_sunset_sim`
Data Management Failures	full memory	`/sim_full_disk_failure`
Data Management Failures	scan logging failure	`/toggle_simulated_scan_logging_failure`
Localization Failures	odometry-GNSS distance divergence (type 1)	`/wheel_movement_without_pos_change`
Localization Failures	odometry-GNSS distance divergence (type 2)	`/pos_change_without_wheel_movement`
Localization Failures	interpolated GNSS and IMU / odometry yaw divergence	`/yaw_divergence`
Localization Failures	IMU acceleration although no active nav goal	`/moving_although_standing_still_imu`
Localization Failures	odometry twist although no active nav goal	`/moving_although_standing_still_odom`
Plan Deployment Failures	extended idle time	`/sim_extended_idle_time`
Plan Deployment Failures	unavailable plan service	`/toggle_unavailable_plan_service`
Plan Deployment Failures	empty plan	`/sim_empty_plan`
Plan Deployment Failures	infeasible plan	`/sim_infeasible_plan`
Navigation Failures	static obstacles	`/spawn_static_obstacles std_msgs/String scene_[one, two, three, four]`
Navigation Failures	robot prison	`/spawn_robot_prison`
Navigation Failures	navigation failure	`/trigger_nav_fail`
Charging Failures	undocking failure (raised ramp)	`/sim_undocking_failure`
Charging Failures	docking failure (raised ramp)	`/sim_docking_failure_raised_ramp`
Charging Failures	docking failure (wrong base pose)	`/sim_docking_failure_base_pose`
Charging Failures	charging failure	`/sim_charging_failure`
Power Management Failures	contingency	`/sim_power_management_contingency`
Power Management Failures	catastrophe	`/sim_power_management_catastrophe`

Notes:

IMU acceleration although no active nav goal:
- only working if both the active and passive history is complete (e.g. 1500)
full memory:
- prepare full USB flash drive
- find out device: fdisk -l, e.g., /dev/sdd1
- create directory for flash drive: mkdir /mnt/usb
- mount flash drive: mount /dev/sdd1 /mnt/usb
- set MONITOR_DRIVE to /mnt/usb

Related Publications

@inproceedings{Bohne:06_2023,
    author = {Bohne, Tim and Kisliuk, Benjamin},
    title = {Execution Monitoring for Long-Term Autonomous Mobile Robots in Outdoor Scenarios},
    booktitle = {Workshop on Robot Execution Failures and Failure Management Strategies at ICRA 2023. IEEE International Conference on Robotics and Automation (ICRA-2023), Embracing the future: Making robots for humans, befindet sich ICRA 2023, May 29-June 2, London, United Kingdom},
    year = {2023},
    month = {6},
    organization = {IEEE}
}

@inproceedings{Bohne:02_2023,
    author = {Tim Bohne and Gurunatraj Parthasarathy and Benjamin Kisliuk},
    title = {A systematic approach to the development of long-term autonomous robotic systems for agriculture},
    booktitle = {43. GIL-Jahrestagung, Resiliente Agri-Food-Systeme, 13.-14. Februar 2023, Osnabr{\"{u}}ck, Germany},
    series = {{LNI}},
    volume = {{P-330}},
    pages = {285--290},
    publisher = {Gesellschaft f{\"{u}}r Informatik e.V.},
    year = {2023},
    url = {https://dl.gi.de/handle/20.500.12116/40260},
    biburl = {https://dblp.org/rec/conf/gil/BohnePK23.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

@mastersthesis{Bohne:2022,
    author = {Bohne, Tim},
    year = {2022},
    month = {04},
    title = {Execution Monitoring for Long-Term Autonomous Plant Observation with a Mobile Robot}
}

tbohne / execution_monitoring Goto Github PK

execution_monitoring's Introduction