Updating LS_HEAP_SIZE about elk-docker

[spujadas]

Hi,

Thanks for the kind words!

You're right, you can't set the env var directly as the service won't see it (as was pointed out to me in #25 (comment)).

The most straightforward way to proceed would be to extend the image using a Dockerfile like this (untested, so make sure you cross-check the syntax):

FROM sebp/elk

RUN sed -i -e 's#^LS_HEAP_SIZE="500m"$#LS_HEAP_SIZE="1g"#' /etc/init.d/logstash

The idea is that it replaces the LS_HEAP_SIZE line in Logstash's init file before starting the Logstash service.

As long as I don't touch the LS_HEAP_SIZE="500m" line in the init file (which I have no foreseeable intention of doing!), you can simply rebuild your image when I publish a new one (hooks might come in handy to automate this) and your image will be aligned with mine in terms of features and bugfixes.

Having said that, I think it would be a good idea to natively expose LS_HEAP_SIZE as an env var of the image (i.e. have the sed magic done by the start.sh at runtime if a LS_HEAP_SIZE env var has been passed to docker run), so I'll probably work that feature in the next update. Will add a reference to this issue when I do so that you're automatically notified by GitHub.

[tallavi]

Thanks!

[tallavi]

@spujadas, I read your last paragraph again. Why would you want the start.sh to perform sed magic instead of just changing logstash-init so it checks if the env var is defined and act upon it?

[spujadas]

Because it turns out that services don't see env vars (apart from a select few), so some trickery is needed to feed them to the services.

[tallavi]

Hmm, didn't know that.

BTW, the correct command is:

sed -i -e 's/LS_HEAP_SIZE=\"500m\"/LS_HEAP_SIZE=\"1g\"/g' /etc/init.d/logstash

explanations:

-i : replace in file (and not in another file)
-e : 's/string1/string2/g' - replace 'string1' with 'string2' on all occurances

[spujadas]

Cheers.

[kenwdelong]

Thanks for this! I've been struggling to up the ES heap size or max mem size as well. I'll try the sed trick until you figure out how to wedge those env vars in there!

Great project!

[tallavi]

Eventually for me it didn't do the trick. Even setting it up to 2g still OOM after a while.. Something is leaking bad there..

[spujadas]

@kenwdelong Thanks. Just pushed the update, had postponed it in the hope of releasing it at the same time as a new version of Elasticsearch, Logstash or Kibana, but by popular demand 😄 here it is.

@tallavi Haven't heard about memory leak issues with Logstash in the default image, so may be Logstash-related (had a brief look at https://github.com/elastic/logstash/search?p=1&q=memory+leak&type=Issues&utf8=%E2%9C%93 there might be some things to look into).

Closing this issue following commit c731456