Guidance on adding plugins and enabling logstash web interface about elk-docker

[spujadas]

@Analect Great questions! I'll update the README with the answers below as they're bound to interest others faced with the same problems.

Elasticsearch's home directory in the image is /usr/share/elasticsearch, plugins live in the plugins subdirectory and the plugin tool resides in bin.

The most straightforward way to install a plugin is to extend the base image by creating a Dockerfile like this:

FROM sebp/elk

ENV ES_HOME /usr/share/elasticsearch
WORKDIR ${ES_HOME}
RUN bin/plugin -i elasticsearch/elasticsearch-mapper-attachments/2.6.0

Then build the new image as you usually would (e.g. docker build -t analect/elk-and-plugins .) and run it (e.g. docker run ... analect/elk-and-plugins).

You could also install a plugin manually by downloading its binary package (e.g. the 'hidden' http://download.elasticsearch.org/elasticsearch/elasticsearch-mapper-attachments/elasticsearch-mapper-attachments-2.6.0.zip, not the source at https://github.com/elastic/elasticsearch-mapper-attachments/releases) and unzip-/untargz-ing it in the /usr/share/elasticsearch/plugins directory, but it's more of a hassle than just using plugin -install.

In the same way, create a Dockerfile like this to install a Logstash plugin:

FROM sebp/elk

# ... other stuff, e.g. install Elasticsearch plugins ... 

## the LOGSTASH_HOME env var is created in sebp/elk, no need to redefine it
#ENV LOGSTASH_HOME /opt/logstash

WORKDIR ${LOGSTASH_HOME}
RUN bin/plugin install logstash-output-kafka

If you want to start Elasticsearch's web interface, then you need to override the start.sh script from the base sebp/elk image to start the logstash-web service.

To do that:

1. Grab the original start.sh script, and add this line in it (say just after service logstash start):

   service logstash-web start
   

2. Create a new Dockerfile next to the amended start.sh script, and put this in:

   FROM sebp/elk
   
   # ... other stuff, e.g. install Elasticsearch and Logstash plugins ... 
   
   ADD ./start.sh /usr/local/bin/start.sh
   EXPOSE 9292
   

3. Build the image as usual.

4. Start the image with port 9292 published (i.e. docker run ... -p 9292:9292 ...).

As for leveraging logstash-forwarder, I'm not quite sure what you're looking for, so for the time being I can refer you to the Logstash book for an end-to-end description of setting up and using Logstash, and to the Logstash Reference for the details. If you're after something in particular, let me know and I'll see if I can point you to a specific write-up.

[Analect]

@spujadas
Thanks for the comprehensive instructions. Much appreciated.