dizhaung / monitoring_flux

An example of a complete flow monitoring system

An example of a complete monitoring system.

Detailed description of all components can be found here in French.

Vagrant installation

You can use vagrant and ansible scripts to create a virtual machine with all application configured.

Manual installation

Requirements

To run all the elements you will need:

• Java 8

• Maven

• Ruby or JRuby

• Bundler

• ZeroMQ

• Redis

• Elasticsearch

• Kibana

Installation

After all the required elements are installed:

• Create the index in ElasticSearch (see below)

• Run mvn install in the root directory to build all the Java components

• Run bundle install in the frontend directory to install all the frontend’s dependencies

• Optionally import the kibana-dashboards.json data in the .kibana index in Elasticsearch to get a sample dashboard

Starting the applications

First start the external components:

• Redis

• Elasticsearch

• Kibana

Check each application contains the information to start it:

• the frontend

• the middle end

• the backend

• the cep

Setup the index in ElasticSearch

Even if Elasticsearch can auto-create indexes, it’s best to setup it manually so fields are analyzed the right way:

curl -XPOST 'http://localhost:9200/monitoring' -d '{
  "settings": {
    "number_of_shards": 1
  },
  "mappings": {
    "cep_to_elasticsearch": {
      "properties": {
        "correlation_id": {
          "type": "string",
          "index": "not_analyzed"
        },
        "module_type": {
          "type": "string",
          "index": "not_analyzed",
          "doc_values": true
        },
        "module_id": {
          "type": "string",
          "index": "not_analyzed"
        },
        "endpoint": {
          "type": "string",
          "index": "not_analyzed",
          "doc_values": true
        },
        "message_type": {
          "type": "string",
          "index": "not_analyzed",
          "doc_values": true
        },
        "timestamp": {
          "type": "date",
          "format": "dateOptionalTime"
        },
        "begin_timestamp": {
          "type": "date",
          "format": "dateOptionalTime"
        },
        "end_timestamp": {
          "type": "date",
          "format": "dateOptionalTime"
        },
        "elapsed_time": {
          "type": "double"
        },
        "service_params": {
          "type": "object"
        },
        "headers": {
          "type": "object"
        },
        "result": {
          "type": "object"
        }
      }
    },
    "throttling_violation": {
      "properties": {
        "alert_type": {
          "type": "string",
          "index": "not_analyzed"
        },
        "timestamp": {
          "type": "date",
          "format": "dateOptionalTime"
        },
        "avg_cnt": {
          "type": "double"

        },
        "correlation_id": {
          "type": "string",
          "index": "not_analyzed"
        }
      }
    },
    "unit_sla_violation": {
      "properties": {
        "alert_type": {
          "type": "string",
          "index": "not_analyzed"
        },
        "timestamp": {
          "type": "date",
          "format": "dateOptionalTime"
        },
        "module": {
          "type": "string",
          "index": "not_analyzed"
        },
        "time": {
          "type": "double"
        }
      }
    },
    "global_sla_violation": {
      "properties": {
        "alert_type": {
          "type": "string",
          "index": "not_analyzed"
        },
        "timestamp": {
          "type": "date",
          "format": "dateOptionalTime"
        },
        "correlation_id": {
          "type": "string",
          "index": "not_analyzed"
        },
        "count": {
          "type": "double"
        }
      }
    }
  }
}'

Monitoring message format

{
    "correlation_id": "octo.local_MonitoringBase_24389_2015-01-30 11:05:29 UTC_36cddd01-7bcd-4ced-8024-919ff1dbe6ca",  // correlation id
    "timestamp": "2015-01-30T12:05:29.230+01:00", // message timestamp

    "module_type": "FrontendApp", // module type sending the message
    "module_id": "FrontendApp_octo.local_001", // module identifier
    "endpoint": "GET /messages",
    "message_type": "Send message to backend", // message type

    "begin_timestamp": "2015-02-19T22:11:15.939+01:00", // optional: timestamp of the beginning of the current action
    "end_timestamp": "2015-02-19T22:11:15.959+01:00", // optional: timestamp of the end of the current action
    "elapsed_time": 0.020169, // optional: elapsed time of the current action in second

    "service_params": {
        // optional: service parameters
    },

    "headers": {
        // optional: current headers
    }

    "result": {
        // optional: result of current action
    }
}

Call service from the command line

If you want to watch the system running from some time you can schedule services call from the command line:

while sleep 5; do curl -X POST --data 'numberOfMessages=10&timeToSpend=1' 'http://localhost:9292/messages' ; done

License

Code © 2015 released under the MIT license.