bakhti / ansible-elk Goto Github PK
View Code? Open in Web Editor NEWAnsible role for Elasticsearch ELK Stack
Ansible role for Elasticsearch ELK Stack
Thanks for putting together a cool package.
I've had a crack at updating this, by just changing the version numbers to 2.1 but it seems to cause issues like this:
Fetched 250 kB in 4s (55.5 kB/s)
W: Failed to fetch http://packages.elastic.co/logstash/2.x/debian/dists/stable/main/binary-amd64/Packages 404 Not Found [IP: 23.21.242.59 80]
It looks like they have a debian/ubuntu repo for this now, does it make sense to use this instead? https://www.elastic.co/guide/en/logstash/current/package-repositories.html
Can you add some usage sample and a rundown of the variables with a single example?
Result:
failed: [172.32.1.82] => (item=openntpd,python-apt,python-pycurl,python-pip,supervisor,monit,git-core) => {"failed": true, "item": "openntpd,python-apt,python-pycurl,python-pip,supervisor,monit,git-core"}
stderr: /etc/openntpd/ntpd.conf: Permission denied
invoke-rc.d: initscript openntpd, action "start" failed.
dpkg: error processing package openntpd (--configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
openntpd
E: Sub-process /usr/bin/dpkg returned an error code (1)
stdout: Reading package lists...
Building dependency tree...
Reading state information...
The following package was automatically installed and is no longer required:
libopts25
Use 'apt-get autoremove' to remove it.
The following extra packages will be installed:
git git-man liberror-perl python-meld3
Suggested packages:
git-daemon-run git-daemon-sysvinit git-doc git-el git-email git-gui gitk
gitweb git-arch git-bzr git-cvs git-mediawiki git-svn exim4 postfix
mail-transport-agent
The following packages will be REMOVED:
ntp
The following NEW packages will be installed:
git git-core git-man liberror-perl monit openntpd python-meld3 supervisor
0 upgraded, 8 newly installed, 1 to remove and 145 not upgraded.
Need to get 4021 kB of archives.
After this operation, 22.8 MB of additional disk space will be used.
Get:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ trusty/main liberror-perl all 0.17-1.1 [21.1 kB]
Get:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ trusty-updates/main git-man all 1:1.9.1-1ubuntu0.1 [698 kB]
Get:3 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ trusty-updates/main git amd64 1:1.9.1-1ubuntu0.1 [2627 kB]
Get:4 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ trusty-updates/main git-core all 1:1.9.1-1ubuntu0.1 [1458 B]
Get:5 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ trusty/universe openntpd amd64 20080406p-7 [33.9 kB]
Get:6 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ trusty/universe monit amd64 1:5.6-2 [286 kB]
Get:7 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ trusty/universe python-meld3 amd64 0.6.10-1 [39.0 kB]
Get:8 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ trusty/universe supervisor all 3.0b2-1 [314 kB]
Fetched 4021 kB in 0s (11.6 MB/s)
(Reading database ... 56888 files and directories currently installed.)
Removing ntp (1:4.2.6.p5+dfsg-3ubuntu2.14.04.3) ...
msg: '/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" install 'openntpd' 'supervisor' 'monit' 'git-core'' failed: /etc/openntpd/ntpd.conf: Permission denied
invoke-rc.d: initscript openntpd, action "start" failed.
dpkg: error processing package openntpd (--configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
openntpd
E: Sub-process /usr/bin/dpkg returned an error code (1)
This repo seems abandoned, but still.
The version of Elasticsearch which is embedded in logstash (currently 1.1.1) doesn't match with the one expected by Kibana (>1.4), making the latter fail. I workaround this by manually doing cp -R /usr/share/elasticsearch/ /opt/logstash/vendor/jar/elasticsearch-1.4.4
and then removing the old ES version in /opt/logstash/vendor/jar
.
Maybe it's possible to add this task to the role so it keeps everything working? Do you think of a better approach for solving this? Let me know and I can work on a PR.
They are not available yet upstream apparently, I wonder if there's a clean way for ansible to fall back on older distro repos such as trusty
automatically, without spilling out and halting with this error.
TASK: [ansible-elk | Add nginx apt repo] **************************************
failed: [23.253.172.105] => {"failed": true, "parsed": false}
Traceback (most recent call last):
File "/root/.ansible/tmp/ansible-tmp-1421313402.51-120126372595310/apt_repository", line 2466, in <module>
main()
File "/root/.ansible/tmp/ansible-tmp-1421313402.51-120126372595310/apt_repository", line 436, in main
cache.update()
File "/usr/lib/python2.7/dist-packages/apt/cache.py", line 440, in update
raise FetchFailedException(e)
apt.cache.FetchFailedException: W:Failed to fetch http://nginx.org/packages/ubuntu/dists/utopic/nginx/binary-amd64/Packages 404 Not Found
, W:Failed to fetch http://nginx.org/packages/ubuntu/dists/utopic/nginx/binary-i386/Packages 404 Not Found
, E:Some index files failed to download. They have been ignored, or old ones used instead.
FATAL: all hosts have already failed -- aborting
https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html
Now it's packages.elastic.co
instead of packages.elasticsearch.com
, which 404s. Everything else appears to be the same.
Also, using the playbook with debian/openJDK, it breaks at the 'accept oracle license' step - need to change
when: webupd8team.stdout != 'true'
to
"when: elk_java.flavor == 'oracle' and webupd8team.stdout != 'true' and ansible_distribution == 'Ubuntu'"
(better matches the previous steps)
Thanks for the playbook, by the way.
failed: [XXXXXX] => (item=openntpd,python-apt,python-pycurl,python-pip,supervisor,monit,git-core,jq) => {"failed": true, "item": "openntpd,python-apt,python-pycurl,python-pip,supervisor,monit,git-core,jq"}
msg: No package matching 'jq' is available
The json CLI is present only on Ubuntu
Is it mandatory ?
The change is substantial, now it runs on a dedicated script instead of having it deployed in a regular www
server.
Currently logstash and elasticsearch can be notified to restart after e.g. config change:
- name: Restart logstash
service:
name=logstash
enabled=yes state=restarted
sudo: yes
But that will actually spawn another logstash process in addition to the one monit handles. I think it needs to be:
- name: Restart logstash
monit:
name=logstash
state=restarted
sudo: yes
Keep up the good work,
Martin
Using the following ubuntu image in a VirtualBox Vagrantfile
:
config.vm.box_url = "https://cloud-images.ubuntu.com/vagrant/trusty/current/trusty-server-cloudimg-amd64-vagrant-disk1.box"
TASK: [elk | Modify config.js] ************************************************
failed: [default] => {"failed": true, "rc": 257}
msg: Destination /var/www/kibana/src/config.js does not exist !
FATAL: all hosts have already failed -- aborting
This role doesn't work with Ansible 2.
With default settings I'm getting the following and there is no way to override elk_java.flavor
TASK [elk : Java apt packages] *************************************************
failed: [localhost] => (item=[u'elk_java.openjdk']) => {"failed": true, "item": ["elk_java.openjdk"], "msg": "No package matching 'elk_java.openjdk' is available"}
logstash 1.5 releases don't have logstash-contrib package anymore, which cause the install to fail.
vars:
elk_logstash:
version: 1.5
TASK: [bakhti.elk | Install logstash apt packages] ****************************
<xxx.xxx.xxx.xxx> REMOTE_MODULE apt pkg=logstash,logstash-contrib update_cache=yes state=present
failed: [xxx.xxx.xxx.xxx] => (item=logstash,logstash-contrib) => {"failed": true, "item": "logstash,logstash-contrib"}
msg: No package matching 'logstash-contrib' is available
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.