ivoire / reactobus

https://git.lavasoftware.org/ReactOBus/ReactOBus

ReactOBus

ReactOBus is a message broker that helps to create software bus over the network and to react to some messages.

Features

ReactOBus is able to:

• collect events (as network messages) from different sources
• publish the stream of events
• store the events in a database
• launch specific commands when an event match some conditions

In a near future, ReactOBus will be able to:

• filter-out some events
• modify on-the-fly event format
• ...

Using ReactOBus

Requirements

ReactOBus is known to work with Python3.4 and Python3.5 under Linux.

It depends on (see requirements.txt):

• pyzmq
• pyYAML
• SQLAlchemy (if you wish to store events in a database)
• setproctitle

Installing

ReactOBus is available on pypi and can be installed with:

pip install ReactOBus

By default, pip will not install SQLAlchemy. If need, install it manually afterward or ask pip to include the right ReactOBus variant with:

pip install ReactOBus[db]

It's also possible to execute ReactOBus directly from the sources:

git clone https://git.lavasoftware.org/ReactOBus/ReactOBus
cd ReactOBus
virtualenv -p python3 venv
source venv/bin/activate
pip install -r requirements.txt
python reactobus --level DEBUG --conf share/examples/reactobus.yaml

Configuration

The configuration file is a YAML dictionary with:

• inputs: a list of input streams
• outputs: a list of output streams
• core: configuration of the internal sockets
• reactor: the reacting part of ReactOBus
• db: the database configuration

All keys except core and inputs are optional. If the optional keys are not found in the configuration, the corresponding modules won't be loaded.

Message format

For the moment, ReactOBus only accepts one type of messages. The messages should be multipart ZMQ messages with the following meaning:

• topic
• uuid (as generated by uuid.uuid1() in Python)
• datetime when the message was generated (isoformat)
• username of the sending process or user
• data as JSON

Testing ReactOBus

In order to run ReactOBus automatic tests, you will have to install py.test:

pip install pytest

Then run the tests using:

py.test tests -v
[...]
tests/test_core.py::test_core PASSED
tests/test_db.py::test_run PASSED
tests/test_db.py::test_errors PASSED
[...]

reactobus's Issues

ROUTER socket

Add a ROUTER socket for the outgoing messages to push to many listening clients.

Create a Debian package

Make usage examples

Examples:

• LAVA device usage reports
• desktop notification

Something else ?

Improve documentation

To be documented

• creating a CURVE certificate (zmq)
• how to use secure sockets
• the configuration file (with all options)
• the reactor

Try asyncio for reactor

See https://hackernoon.com/threaded-asynchronous-magic-and-how-to-wield-it-bba9ed602c32#.n8otbup6y

ZMQ PUB/SUB heartbeating

Add an heartbeat mechanism for the PUB and SUB sockets to reconnect the SUB socket in case of network failure.
Also add a way to grab the missing messages

Currently implemented

• send heartbeat in PUB
• filter-out heartbeats in SUB
• force SUB reconnection when missing heartbeats
• grab missing messages

Matching in a list

Allow the matching value to be a list of values, not only one pattern.

Create an init script for deamonising reactobus

To use this project in a production level environment, it would be best to have init start/respawn the script to increase reliability. Create an example init script which could be used now, or for when ReactOBus gets packaged.

Handle PubSubHubBub

See https://github.com/pubsubhubbub/

Filter topic at ZMQSub level using ZMQ feature