Can't get Elk started about elk-docker

[YouveGotMeowxy]

Thank you!

I just upgraded and so far, so good!

I had a couple small issues when using the latest tag, like:

java.lang.IllegalStateException: cannot upgrade a node from version [7.16.3] directly to version [8.1.0], upgrade to version [7.17.0] first.

but just doing a quick restart using the 7.17.1 tag, and then back to the latest tag solved that.

and then I got this error:

ERROR: [1] bootstrap checks failed. You must address the points described in the following [1] lines before starting Elasticsearch.
today at 3:38:15 PMbootstrap check failure [1] of [1]: The default value for [xpack.security.enabled] has changed in the current version. Security features were implicitly disabled for this node but they would now be enabled, possibly preventing access to the node. See https://www.elastic.co/guide/en/elasticsearch/reference/8.1/security-minimal-setup.html to configure security, or explicitly disable security by setting [xpack.security.enabled] to "false" in elasticsearch.yml before restarting the node.
today at 3:38:15 PMERROR: Elasticsearch did not exit normally - check the logs at /var/log/domain/domain-cloud.log

But for now (until I wrestle with security) manually setting it to false in the config solved it.

Then, I was getting a page full of these in the log:

[2022-03-23T15:40:12,783][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"http://localhost:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401' contacting Elasticsearch at URL 'http://localhost:9200/'"}

No idea what that is or was, but it seems to have sorted itself out (at least for now; no idea if it'll show up again at some point)

Other than that, everything seems to be going great, ty for maintaining this! :)

[spujadas]

My first thought is that the local volumes that you’re bind-mounting could be missing files that are needed to start the services.

Specifically, this error message

today at 11:22:40 AM**Exception in thread "main" java.nio.file.NoSuchFileException: /etc/elasticsearch/jvm.options**

makes this line of your compose suspicious:

    - /opt/docker/configs/elasticsearch:/etc/elasticsearch:rw   # Elasticsearch config         

In the default image, the /etc/elasticsearch directory contains quite a few configuration files:

root@d546cc5e58c3:/etc/elasticsearch# ll
total 52
drwxr-xr-x 1 elasticsearch elasticsearch  4096 Mar 21 19:28 ./
drwxr-xr-x 1 root          root           4096 Mar 21 19:28 ../
-rw-rw---- 1 elasticsearch elasticsearch   199 Mar 21 19:28 elasticsearch.keystore
-rwxrwxrwx 1 elasticsearch elasticsearch  2900 Jan  5 19:26 elasticsearch.yml*
-rw-r--r-- 1 elasticsearch elasticsearch  3257 Mar 21 19:28 jvm.options
drwxr-xr-x 1 elasticsearch elasticsearch  4096 Mar 21 19:27 jvm.options.d/
-rw-r--r-- 1 elasticsearch elasticsearch 19150 Mar 21 19:28 log4j2.properties
drwxr-xr-x 1 elasticsearch elasticsearch  4096 Mar 21 19:27 scripts/

I’m guessing that your host’s /opt/docker/configs/elasticsearch is missing (at least) one these files.
If you only want to override Elasticsearch’s elasticsearch.yml config file whilst keeping all the other files as is, then I’d suggest changing your config to something like:

    - /opt/docker/configs/elasticsearch/elasticsearch.yml:/etc/elasticsearch/elasticsearch.yml:rw   # Elasticsearch config

If nothing else, at least it should sort out the very first error message that you’re seeing 😄

[spujadas]

Shouldn't the config files, including the jvm.options file automatically get written/copied to the mounted volumes?

Sorry, missed your question. The way Docker’s volume mounting system works, it’s actually the other way round, so to speak: the host’s files/directories replace the ones in the image.

[YouveGotMeowxy]

@spujadas TY, I got it up! : D

I don't know why, but I had already tried that with the standalone ES container and it wouldn't work. I decided to try your ELK thinking it was all already gonna work out of the box, and copy basic files into my mounted folders and i could go from there (it didnt, as per my original question). I copied the same es.yml I was already using and mounted it in elk ... and it worked! lol

I just have a couple of quick questions now, if you dont mind?

• What if we want to upgrade ES to the latest 8.1.0? (Normally in standalone containers we can easily update but just pulling the new container; I'm not sure how it works in ELK?)
• Same question for any of the others, if ever needed? (kibana, logstash)
• How should I go about mounting my personal kibana.yml? Is there a path inside the running elk that I should copy it out of first, and then mount it with my tweaks?
• *I got this while visiting one of the pages in kibaba, is this a big deal?

I apologize for so many questions, but I'm new to all of this, and I figure you know this stuff like the back of your hand so the questions would be easy. :)

*I found this, so the last question is answered :)

[spujadas]

Glad to hear you got it working 👍

What if we want to upgrade ES to the latest 8.1.0? (Normally in standalone containers we can easily update but just pulling the new container; I'm not sure how it works in ELK?)

Same for ELK, you can just pull sebp/elk:xyz where xyz is the tag of one of the pre-built versions of the image (see https://github.com/spujadas/elk-docker or https://hub.docker.com/r/sebp/elk for the complete list).
As it turns out I haven’t built the image since February so version 8.1.0 isn’t available yet, but I’ll look into it shortly.

Same question for any of the others, if ever needed? (kibana, logstash)

Since version 5, each image contains the same version of Elasticsearch, Logstash, and Kibana, so same answer as above.

How should I go about mounting my personal kibana.yml? Is there a path inside the running elk that I should copy it out of first, and then mount it with my tweaks?

In the image, Kibana’s config file is ${KIBANA_HOME}/config/kibana.yml, which resolves to /opt/kibana/config/kibana.yml.
Instead of copying it out of the image (which would work, of course), you could copy/download the source file from the GitHub repo (https://github.com/spujadas/elk-docker/blob/master/kibana.yml), which is used to build the image.

• *I got this while visiting one of the pages in kibaba, is this a big deal?

The link that you found points to my outdated ELKX image, so not sure if that will actually help you.
What you’re seeing is not a big deal if you don’t need security. If you do want to use security features, then you can just follow the instructions in the message and tweak the security configuration to your heart’s content 😉

[YouveGotMeowxy]

OK, again, TYVM for the help!

I wasn't sure of copying the kibana.yml from the repo, since I tried that with the jv.options file while messing with the standalone ES, and got errors, because I think the file gets some values written to it when it starts running and the "raw" file only has tokens as placeholders in it.

Yes, my next step is to add security. I also had issues trying to do THAT in standalone, due to having to mount the .keystore, and then persimmons for it, and ... on and on and on .... ugggghhh, lol.

[spujadas]

Yep, setting up security is definitely not for the faint of heart :laugh:

I’ve just updated the repository and the image with the latest version of the ELK stack, namely 8.1.0, if you want to give it a spin.

(And closing this issue.)