ivan-tamayo / elk-docker

Docker configuration for ELK monitoring stack with Curator and Beats data shippers support

Elasticsearch, Logstash, Kibana with Curator and Beats support

Ready to go Docker configuration for set up ELK stack in a minutes

Description

• Elasticsearch - official image with data volume in elasticsearch/data directory
• Logstash - official image + custom configuration which takes care about Filebeat, Topbeat and Packetbeat index templates for Elasticsearch + multiline option for correct stacktraces representation
• Kibana - official image
• Curator - lightweight 50mb container which could run scheduled tasks against Elasticsearch to manage its indices (delete, optimize, snapshot, etc)

Setup

1. Install required Beats shippers on the host which should be monitored
2. Install Docker and Docker Compose on the ELK host
3. Clone this repository and hit docker-compose build

Usage

Start everything with one command:

docker-compose up -d

Keep track of your containers execution. For example, controll Curator scheduled tasks:

docker-compose logs curator

Scaling up

This simple configuration will run very happily on your laptop, but it can be easely scaled up for highload production servers with a huge amount of logs and monitoring data.

• Learn about Elasticsearch cluster and horizontal scaling
• Consider Logstash scaling with multiple shipping and indexing instances with MQ in the middle

Notes

• You may want to add Kibana Shield plugin for users authentication
• It might be really helpful to use Elasticsearch Watcher or Yelp ElastAlert to get notified on significant events or anomalies in your data
• You can specify resource limits (like CPU and memory allocation) for each docker container

Feel free to contact me with any issues and questions