Unable to open Log file for Write about kibana-docker

[dliappis]

@aconite33 The images you are using in your compose files (e.g. elasticsearch:latest) are not the ones maintained by Elastic, built from this repo. The images can be found in www.docker.elastic.co or in the documentation pages e.g. https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html

[aconite33]

Some additional testing:

I was able to create a logstash-plain.log and a kibana.log file under my ./log/[kibana|logstash] directory. I set 777 on those files, and they were able to be written to.

Elasticsearch still continues to write without any issue, but logstash/kibana need 777 permissions. It seems that setting 664 666 or any other combination will not yield the same result.

[aconite33]

@dliappis oh boy. Well, I've been doing that wrong.

I updated to use the proper docker images.

I think the issue is still the same, but now it's across all my elastic docker's. I have to set file permissions to 777 or else I get write errors. Is there a proper way to allow host to docker without setting such insecure permissions?

[dliappis]

@aconite33 Sure thing. If you are able to chgrp -R 1000 <yourhostdir> it will grant RW access to the container (e.g. elasticsearch) that you are bind mounting it to. This assumes <yourhostdir has at least g+rx permissions, which typically is the case with the default unix umask. chmod -R o+r <yourhostdir> will typically suffice for read access (e.g. conf files etc.)

Some details here: https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#_b_bind_mounted_configuration

For elasticsearch in particular, I strongly recommend using docker named volumes that don't have any permissions issues. We also have a repo with a full stack example (switch to the corresponding branch for the version you are after e.g. 5.6).

Also note that the elasticsearch|kibana|logstash images from docker.elastic.co have a number of different ways to configure parameters, please see the documentation e.g. for kibana and elasticsearch

Finally some links to suggestions we wrote in the past:

• elastic/elasticsearch-docker#49 (comment) (after the bold section)
• elastic/elasticsearch-docker#86 (comment)

[aconite33]

@dliappis Oh man. Thanks for all the input! I've revamped my docker-compose.yml and basically have everything up and running again. I took your advice and ran everything as the 1000:1000 user, made sure the permissions were properly setup.

One thing on the permissions, since I didn't have the complete path created, docker went ahead and made it for me. However, it made it with the root user, even though I'm running as a user for the docker-compose command. I just changed the owner back to my user, re-ran and everything came up.

I did have additional questions on the xpack plugins. I figured out how to disable the xpack security package, but I keep getting errors regarding the monitoring. Is there a way to disable the monitoring extension in elasticsearch? I added a config in my logstash.yml to disable the plugin, but I haven't figured out how to disable it in elasticsearch yet.

Here is the error:

elasticsearch3 | org.elasticsearch.xpack.monitoring.exporter.ExportException: Exception when closing export bulk

[jarpy]

If you don't want Monitoring, then:

xpack.monitoring.enabled: false

should do the trick for you.

https://www.elastic.co/guide/en/elasticsearch/reference/5.6/monitoring-settings.html#general-monitoring-settings

[aconite33]

Okay great. Is there a way to just disable all addons? I'm trying to get a strip down version with just the basics and no addons. Need to get a handle on just working with the bare minimum so I can see where I can include which addons make sense for me. Seeing alot of errors for watch errors etc. I could just remove all the plugins from the images, but I'd like to keep the images clean as I can, and just work with configurations as opposed to the file system.

[Shifter2600]

/bin/bash -c '/usr/share/kibana/bin/kibana-plugin remove x-pack;/usr/local/bin/kibana-docker'
will remove xpack add it to your docker run.

or
XPACK_GRAPH_ENABLED: "false"
XPACK_MONITORING_ENABLED: "false"
XPACK_REPORTING_ENABLED: "false"
XPACK_SECURITY_ENABLED: "true"
XPACK_ML.ENABLED: "false"

I have found if you are running against elastic cloud you need the Xpack enabled and still there but you can turn off the others.

[jarpy]

Seems quiet around here. Closing.