GithubHelp home page GithubHelp logo

ansible-kafka's People

Contributors

asaladin avatar bharkreader avatar eladamitpxi avatar gbolo avatar hamnis avatar henridf avatar jaytaylor avatar jordiclariana avatar kcdbaba avatar kknd22 avatar ledroide avatar vincepii avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ansible-kafka's Issues

Upstart configuration fails with CentOS 6.8 (upstart 0.6.5)

There is an unsupported syntax in /etc/init/kafka.conf

serge:~$ cat /etc/centos-release 
CentOS release 6.8 (Final)
serge:~$ initctl version
init (upstart 0.6.5)
serge:~$ initctl status kafka
initctl: Unknown job: kafka
serge:~$ initctl start kafka 
initctl: Unknown job: kafka
serge:~$ initctl list | grep kafka ; echo $?
1
serge:~$ initctl log-priority debug && initctl --verbose reload-configuration
serge:~$ tail -n 4 -f /var/log/messages
Jun  2 03:42:18 serge-opo-1 init: Reloading configuration
Jun  2 03:42:18 serge-opo-1 init: Loading configuration from /etc/init.conf
Jun  2 03:42:18 serge-opo-1 init: Loading configuration from /etc/init
Jun  2 03:42:18 serge-opo-1 init: /etc/init/kafka.conf:9: Unknown stanza

running this role on vagrant vm fails during "apt-get"ting java

I think the Vagrant VM is irrelevant, or at least much less relevant than updating apt's cache before trying to install the java package.

$ vagrant provision
==> deployml: Running provisioner: ansible...
    deployml: Running ansible-playbook...
statically included: /usr/local/etc/ansible/roles/ansible-kafka/tasks/check-env.yml
statically included: /usr/local/etc/ansible/roles/ansible-kafka/tasks/java.yml
[DEPRECATION WARNING]: Instead of sudo/sudo_user, use become/become_user and
make sure become_method is 'sudo' (default).
This feature will be removed in a
future release. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
statically included: /usr/local/etc/ansible/roles/ansible-kafka/tasks/system.yml
statically included: /usr/local/etc/ansible/roles/ansible-kafka/tasks/limits.yml
statically included: /usr/local/etc/ansible/roles/ansible-kafka/tasks/kafka-install.yml
statically included: /usr/local/etc/ansible/roles/ansible-kafka/tasks/kafka-cfg.yml

PLAY [Apply configuration to virtual machine] **********************************

TASK [setup] *******************************************************************
ok: [deployml]

TASK [ansible-kafka : Check 'kafka_zookeeper_hosts' variable] ******************
skipping: [deployml]

TASK [ansible-kafka : Check 'kafka_hosts' variable] ****************************
skipping: [deployml]

TASK [ansible-kafka : Check 'kafka_generate_broker_id' variable] ***************
skipping: [deployml]

TASK [ansible-kafka : Check if Java is installed] ******************************
fatal: [deployml]: FAILED! => {"changed": false, "cmd": "command -v java", "delta": "0:00:00.001418", "end": "2016-11-03 20:17:58.563578", "failed": true, "rc": 127, "start": "2016-11-03 20:17:58.562160", "stderr": "", "stdout": "", "stdout_lines": [], "warnings": []}
...ignoring

TASK [ansible-kafka : Install OpenJDK] *****************************************
fatal: [deployml]: FAILED! => {"cache_update_time": 0, "cache_updated": false, "changed": false, "failed": true, "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\"     install 'openjdk-7-jre-headless'' failed: E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/t/tzdata/tzdata-java_2016f-0ubuntu0.14.04_all.deb  404  Not Found [IP: 91.189.91.23 80]\n\nE: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?\n", "stderr": "E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/t/tzdata/tzdata-java_2016f-0ubuntu0.14.04_all.deb  404  Not Found [IP: 91.189.91.23 80]\n\nE: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?\n", "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n  acl at-spi2-core colord dconf-gsettings-backend dconf-service fontconfig\n  hicolor-icon-theme libasound2 libasound2-data libatk-bridge2.0-0 libatk1.0-0\n  libatk1.0-data libatspi2.0-0 libcairo-gobject2 libcairo2 libcanberra-gtk3-0\n  libcanberra-gtk3-module libcanberra0 libcolord1 libcolorhug1 libdatrie1\n  libdconf1 libdrm-intel1 libdrm-nouveau2 libdrm-radeon1 libexif12 libfontenc1\n  libgd3 libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-common libgl1-mesa-dri\n  libgl1-mesa-glx libglapi-mesa libgphoto2-6 libgphoto2-l10n libgphoto2-port10\n  libgraphite2-3 libgtk-3-0 libgtk-3-bin libgtk-3-common libgudev-1.0-0\n  libgusb2 libharfbuzz0b libice6 libieee1284-3 libjasper1 libjbig0 liblcms2-2\n  libllvm3.4 libltdl7 libnotify-bin libnotify4 libpango-1.0-0\n  libpangocairo-1.0-0 libpangoft2-1.0-0 libpciaccess0 libpixman-1-0 libsane\n  libsane-common libsm6 libtdb1 libthai-data libthai0 libtiff5\n  libtxc-dxtn-s2tc0 libv4l-0 libv4lconvert0 libvorbisfile3 libvpx1\n  libwayland-client0 libwayland-cursor0 libx11-xcb1 libxaw7 libxcb-dri2-0\n  libxcb-dri3-0 libxcb-glx0 libxcb-present0 libxcb-render0 libxcb-shm0\n  libxcb-sync1 libxcomposite1 libxcursor1 libxdamage1 libxfixes3 libxfont1\n  libxi6 libxinerama1 libxkbcommon0 libxkbfile1 libxmu6 libxpm4 libxrandr2\n  libxrender1 libxshmfence1 libxt6 libxtst6 libxxf86vm1 notification-daemon\n  sound-theme-freedesktop x11-common x11-xkb-utils xfonts-base\n  xfonts-encodings xfonts-utils xserver-common xserver-xorg-core\nUse 'apt-get autoremove' to remove them.\nThe following extra packages will be installed:\n  ca-certificates-java java-common libasyncns0 libflac8 libnspr4 libnss3\n  libnss3-nssdb libpcsclite1 libpulse0 libsctp1 libsndfile1 libvorbisenc2\n  lksctp-tools tzdata-java\nSuggested packages:\n  default-jre equivs pcscd pulseaudio icedtea-7-jre-jamvm libnss-mdns\n  sun-java6-fonts fonts-dejavu-extra fonts-ipafont-gothic fonts-ipafont-mincho\n  ttf-wqy-microhei ttf-wqy-zenhei ttf-indic-fonts-core ttf-telugu-fonts\n  ttf-oriya-fonts ttf-kannada-fonts ttf-bengali-fonts\nThe following NEW packages will be installed:\n  ca-certificates-java java-common libasyncns0 libflac8 libnspr4 libnss3\n  libnss3-nssdb libpcsclite1 libpulse0 libsctp1 libsndfile1 libvorbisenc2\n  lksctp-tools openjdk-7-jre-headless tzdata-java\n0 upgraded, 15 newly installed, 0 to remove and 0 not upgraded.\nNeed to get 69.6 kB/41.4 MB of archives.\nAfter this operation, 66.0 MB of additional disk space will be used.\nErr http://archive.ubuntu.com/ubuntu/ trusty-updates/main tzdata-java all 2016f-0ubuntu0.14.04\n  404  Not Found [IP: 91.189.88.162 80]\nErr http://security.ubuntu.com/ubuntu/ trusty-security/main tzdata-java all 2016f-0ubuntu0.14.04\n  404  Not Found [IP: 91.189.91.23 80]\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", "  acl at-spi2-core colord dconf-gsettings-backend dconf-service fontconfig", "  hicolor-icon-theme libasound2 libasound2-data libatk-bridge2.0-0 libatk1.0-0", "  libatk1.0-data libatspi2.0-0 libcairo-gobject2 libcairo2 libcanberra-gtk3-0", "  libcanberra-gtk3-module libcanberra0 libcolord1 libcolorhug1 libdatrie1", "  libdconf1 libdrm-intel1 libdrm-nouveau2 libdrm-radeon1 libexif12 libfontenc1", "  libgd3 libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-common libgl1-mesa-dri", "  libgl1-mesa-glx libglapi-mesa libgphoto2-6 libgphoto2-l10n libgphoto2-port10", "  libgraphite2-3 libgtk-3-0 libgtk-3-bin libgtk-3-common libgudev-1.0-0", "  libgusb2 libharfbuzz0b libice6 libieee1284-3 libjasper1 libjbig0 liblcms2-2", "  libllvm3.4 libltdl7 libnotify-bin libnotify4 libpango-1.0-0", "  libpangocairo-1.0-0 libpangoft2-1.0-0 libpciaccess0 libpixman-1-0 libsane", "  libsane-common libsm6 libtdb1 libthai-data libthai0 libtiff5", "  libtxc-dxtn-s2tc0 libv4l-0 libv4lconvert0 libvorbisfile3 libvpx1", "  libwayland-client0 libwayland-cursor0 libx11-xcb1 libxaw7 libxcb-dri2-0", "  libxcb-dri3-0 libxcb-glx0 libxcb-present0 libxcb-render0 libxcb-shm0", "  libxcb-sync1 libxcomposite1 libxcursor1 libxdamage1 libxfixes3 libxfont1", "  libxi6 libxinerama1 libxkbcommon0 libxkbfile1 libxmu6 libxpm4 libxrandr2", "  libxrender1 libxshmfence1 libxt6 libxtst6 libxxf86vm1 notification-daemon", "  sound-theme-freedesktop x11-common x11-xkb-utils xfonts-base", "  xfonts-encodings xfonts-utils xserver-common xserver-xorg-core", "Use 'apt-get autoremove' to remove them.", "The following extra packages will be installed:", "  ca-certificates-java java-common libasyncns0 libflac8 libnspr4 libnss3", "  libnss3-nssdb libpcsclite1 libpulse0 libsctp1 libsndfile1 libvorbisenc2", "  lksctp-tools tzdata-java", "Suggested packages:", "  default-jre equivs pcscd pulseaudio icedtea-7-jre-jamvm libnss-mdns", "  sun-java6-fonts fonts-dejavu-extra fonts-ipafont-gothic fonts-ipafont-mincho", "  ttf-wqy-microhei ttf-wqy-zenhei ttf-indic-fonts-core ttf-telugu-fonts", "  ttf-oriya-fonts ttf-kannada-fonts ttf-bengali-fonts", "The following NEW packages will be installed:", "  ca-certificates-java java-common libasyncns0 libflac8 libnspr4 libnss3", "  libnss3-nssdb libpcsclite1 libpulse0 libsctp1 libsndfile1 libvorbisenc2", "  lksctp-tools openjdk-7-jre-headless tzdata-java", "0 upgraded, 15 newly installed, 0 to remove and 0 not upgraded.", "Need to get 69.6 kB/41.4 MB of archives.", "After this operation, 66.0 MB of additional disk space will be used.", "Err http://archive.ubuntu.com/ubuntu/ trusty-updates/main tzdata-java all 2016f-0ubuntu0.14.04", "  404  Not Found [IP: 91.189.88.162 80]", "Err http://security.ubuntu.com/ubuntu/ trusty-security/main tzdata-java all 2016f-0ubuntu0.14.04", "  404  Not Found [IP: 91.189.91.23 80]"]}

NO MORE HOSTS LEFT *************************************************************
	to retry, use: --limit @/vagrant_setup.retry

PLAY RECAP *********************************************************************
deployml                   : ok=6    changed=0    unreachable=0    failed=1

Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

I believe this issue can easily be fixed by editing this step in tasks/java.yml:

- name: "Install OpenJDK"
  apt: name=openjdk-7-jre-headless state=present update_cache=yes cache_valid_time=3600
  sudo: yes
  when: check_java.rc != 0
  tags:
    - kafka-install
    - java

Unpacking Kafka archive does not work with differing version

TASK: [jaytaylor.kafka | Extract downloaded kafka archive] ********************
ok: [52.31.200.214] => {"changed": false, "check_results": {"cmd": "/bin/tar -C \"/usr/local/\" --diff -zf \"/tmp/kafka.tar.gz\"", "err": "", "out": "", "rc": 0, "unarchived": true}, "dest": "/usr/local/", "gid": 0, "group": "root", "handler": "TgzArchive", "mode": "0755", "owner": "root", "size": 4096, "src": "/tmp/kafka.tar.gz", "state": "directory", "uid": 0}
ok: [52.48.44.129] => {"changed": false, "check_results": {"cmd": "/bin/tar -C \"/usr/local/\" --diff -zf \"/tmp/kafka.tar.gz\"", "err": "", "out": "", "rc": 0, "unarchived": true}, "dest": "/usr/local/", "gid": 0, "group": "root", "handler": "TgzArchive", "mode": "0755", "owner": "root", "size": 4096, "src": "/tmp/kafka.tar.gz", "state": "directory", "uid": 0}
ok: [52.48.12.7] => {"changed": false, "check_results": {"cmd": "/bin/tar -C \"/usr/local/\" --diff -zf \"/tmp/kafka.tar.gz\"", "err": "", "out": "", "rc": 0, "unarchived": true}, "dest": "/usr/local/", "gid": 0, "group": "root", "handler": "TgzArchive", "mode": "0755", "owner": "root", "size": 4096, "src": "/tmp/kafka.tar.gz", "state": "directory", "uid": 0}

TASK: [jaytaylor.kafka | Link /usr/local/kafka to the right version] **********
failed: [52.48.44.129] => {"failed": true, "path": "/usr/local/kafka", "src": "/usr/local/kafka_2.11-0.9.0.0", "state": "absent"}
msg: src file does not exist, use "force=yes" if you really want to create the link: /usr/local/kafka_2.11-0.9.0.0
failed: [52.48.12.7] => {"failed": true, "gid": 0, "group": "root", "mode": "0777", "owner": "root", "path": "/usr/local/kafka", "size": 29, "src": "/usr/local/kafka_2.11-0.9.0.0", "state": "link", "uid": 0}
msg: src file does not exist, use "force=yes" if you really want to create the link: /usr/local/kafka_2.11-0.9.0.0
failed: [52.31.200.214] => {"failed": true, "gid": 0, "group": "root", "mode": "0777", "owner": "root", "path": "/usr/local/kafka", "size": 29, "src": "/usr/local/kafka_2.11-0.9.0.0", "state": "link", "uid": 0}
msg: src file does not exist, use "force=yes" if you really want to create the link: /usr/local/kafka_2.11-0.9.0.0

I had 0.8.2.0 installed and tried to install 0.9.0.0 on top of that. The unpack module was unable to change state.

'dict object' has no attribute 'port'

i'm trying to run the ansible-kafka playbook clone from the github repo and happen to be receiving below 'dict object' has no attribute 'por

playbook.yml

---

- hosts: dev

  roles:
   - ansible-kafka

  vars:
    zookeeper_hosts: "kafka-d1:2181,kafka-d2:2181,kafka-d3:2181"
    kafka_zookeeper_hosts: [ 'kafka-d1','kafka-d2','kafka-d3' ]
    kafka_zookeeper_port: 2181
    kafka_hosts:
      - dev
    kafka_healthcheck_address: "kafka-d1:2181,kafka-d2:2181,kafka-d3:2181/kafka"
    kafka_server_port: 9092
    kafka_connection_string: [ 'kafka-d1','kafka-d2','kafka-d3' ]

TASK [ansible-kafka : Generate the kafka hosts connection string] ******************************************************************************************** fatal: [kafka-d1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'port'\n\nThe error appears to have been in '/home/ahshan.md/ansible/ansible-kafka/tasks/kafka-cfg.yml': line 31, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears

Wrong path added to profile.d

In the ansible-kafka/tasks/system.yml, we have the following task:

- name: "Add kafka utilities to /etc/profile* so they'll be available in $PATH"
  lineinfile: dest=/etc/profile.d/Z99-kafka.sh regexp="^export PATH=.*$" line='export PATH="${PATH}:/usr/local/lib/kafka/bin"' state=present create=yes
sudo: yes

Please bear in mind that /usr/local/lib/kafka/bin is wrong. It should be /usr/local/kafka/bin instead.

Bug with machineidinteger when using LVM partionning (kafka-cfg.yml)

File : kafka-cfg.yml
task name: "Generate generic unique machine id integer"
shell command : ( test -r /etc/fstab && ls -l /dev/disk/by-uuid/ | grep $(mount | grep ' / ' | cut -d' ' -f1 | cut -d'/' -f3) | grep --ignore-case --only-matching --extended-regexp --max 1 '[0-9a-f]{3,}[0-9a-f-]+' | tr -d '-' || echo '0' ; ifconfig | grep --ignore-case --only-matching --extended-regexp '([0-9a-f]{2}:){5}[0-9a-f]{2}' | tr -d ':' | tr -d '\n') | python -c 'import sys; x, y = sys.stdin.read().split(chr(10))[0:2]; x = int(x, 16); y = int(y, 16); sys.stdout.write((str(x + y)[-9:])); sys.exit(1 if x == 0 and y == 0 else 0)'

When I use a non-LVM partition, it works because of :

$ mount | grep ' / ' | cut -d' ' -f1
/dev/sda2

And this pattern matches with UUID symlink, so you can pick its UUID :

$ ls -l /dev/disk/by-uuid/ | grep $(mount | grep ' / ' | cut -d' ' -f1 | cut -d'/' -f3) ; echo $?
lrwxrwxrwx 1 root root 10 avril 28 11:48 5c0dc710-1247-402a-b3c7-b6a263c02cc3 -> ../../sda2
0

Output code is 0, and that's okay

But with LVM partitioning :

$ mount | grep ' / ' | cut -d' ' -f1 
/dev/mapper/vg_covfefe-lv_root
$ ls -l /dev/disk/by-uuid/
total 0
lrwxrwxrwx 1 root root 10 28 avril 09:12 1815e505-77d6-47e5-aecf-653404cb8933 -> ../../sda1
lrwxrwxrwx 1 root root 10 28 avril 09:12 9b40461e-6840-4225-8899-421c0e5dcc39 -> ../../dm-0
lrwxrwxrwx 1 root root 10 28 avril 09:12 e63ff395-4c7b-4e14-b690-1cb5c7da371e -> ../../dm-1
$ ls -l /dev/disk/by-uuid/ | grep $(mount | grep ' / ' | cut -d' ' -f1 | cut -d'/' -f3) ; echo $?
1

Output code is 1, and string is empty.

So here is the result of the whole shell command with LVM partitioning :

$  ( test -r /etc/fstab && ls -l /dev/disk/by-uuid/ | grep $(mount | grep ' / ' | cut -d' ' -f1 | cut -d'/' -f3) | grep --ignore-case --only-matching --extended-regexp --max 1 '[0-9a-f]{3,}[0-9a-f-]+' | tr -d '-' || echo '0' ; ifconfig | grep --ignore-case --only-matching --extended-regexp '([0-9a-f]{2}:){5}[0-9a-f]{2}' | tr -d ':' | tr -d '\n') | python -c 'import sys; x, y = sys.stdin.read().split(chr(10))[0:2]; x = int(x, 16); y = int(y, 16); sys.stdout.write((str(x + y)[-9:])); sys.exit(1 if x == 0 and y == 0 else 0)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError: need more than 1 value to unpack

And here is the error when playing role :

TASK [jaytaylor.ansible-kafka : Generate generic unique machine id integer] ****
fatal: [sh-opo-1]: FAILED! => {"changed": false, "cmd": "( test -r /etc/fstab && ls -l /dev/disk/by-uuid/ | grep $(mount | grep ' / ' | cut -d' ' -f1 | cut -d'/' -f3) | grep --ignore-case --only-matching --extended-regexp --max 1 '[0-9a-f]{3,}[0-9a-f-]+' | tr -d '-' || echo '0' ; ifconfig | grep --ignore-case --only-matching --extended-regexp '([0-9a-f]{2}:){5}[0-9a-f]{2}' | tr -d ':' | tr -d '\\n') | python -c 'import sys; x, y = sys.stdin.read().split(chr(10))[0:2]; x = int(x, 16); y = int(y, 16); sys.stdout.write((str(x + y)[-9:])); sys.exit(1 if x == 0 and y == 0 else 0)'", "delta": "0:00:00.018923", "end": "2017-05-28 08:49:15.438258", "failed": true, "rc": 1, "start": "2017-05-28 08:49:15.419335", "stderr": "Traceback (most recent call last):\n  File \"<string>\", line 1, in <module>\nValueError: need more than 1 value to unpack", "stdout": "", "stdout_lines": [], "warnings": []} 

Hope it helps you to solve this issue.

Regards,
Serge

latest version 0.10.2.0 signed by different key

this role fails on latest version unless you change the keys url to:
http://kafka.apache.org/KEYS

This file contains one extra key:

diff KEYS KEYS-NEW
101d100
< 
544a544
> 
604,713d603
< pub   4096R/BE7EFC73 2015-01-14
< uid       [ultimate] Ewen Cheslack-Postava <[email protected]>
< sig 3        BE7EFC73 2015-01-14  Ewen Cheslack-Postava <[email protected]>
< sub   4096R/08EA97A4 2015-01-14
< sig          BE7EFC73 2015-01-14  Ewen Cheslack-Postava <[email protected]>
< 
< pub   4096R/5234D94F 2014-08-22
< uid       [ultimate] Ewen Cheslack-Postava <[email protected]>
< sig 3        5234D94F 2014-08-22  Ewen Cheslack-Postava <[email protected]>
< sub   4096R/964F16A8 2014-08-22
< sig          5234D94F 2014-08-22  Ewen Cheslack-Postava <[email protected]>
< 
< -----BEGIN PGP PUBLIC KEY BLOCK-----
< Version: GnuPG v2
< 
< mQINBFS2qt4BEACksfgjdHfD2io0WZsc8jXHzx6oXE6lWnARo4hLgf1q32adgeRz
< ofGV5AdiP1N5Pwus0o8PEa2UN9/rOIXavw4W0xMlWRiZlavYw6wR7zOGuFINRZJm
< LFvPyIuTtAwdrsuUdOqAcw4hrz0j/V7xg7l79CalVpzO4JKnBYRtNSbg0FeVpk+0
< AEt6e3nlA/5WZLRnNwJTkSpwVkgRgmIjJh889xCtXtxRkjNoTq0H+ks12xdQyruL
< nFRzQmrBCsIS9C+VCmVu8r/OBVVtwulaIYCcIxu213a23XmTmlfmHQXnvwdMPBT3
< me/2ubm6Xqon92Nc08AhxzzWp/rh9SceNWBkFzMdYnXpCIwWGRcvzJ4M+WhL5ho2
< SjaPoKj1LqFhzpeNA5F4SRPOfqvkEkGIl3L3qxXlrax2yOuJMv9sb6op01JIrQY6
< c0oQCzX5vxRdDimZfHnrcZnlbn7Wzs3N8hNwPn0+0ZGLTn//5/esNWtUkf/5cK8R
< afqe0bntiNFLxvR1ysMYJf02FXRJ7rVaS02KZISrc/64bpE5GG5v6iklHrx3SJ4B
< HykbITfN9n0uO2B4iivb86OnV/X5bjv52aJ3llwmsQtTyMrr+JOeccjYv+bw6SK+
< xi/t8BWlwfxa7+//5XrTgNaINE96R9EtDlMMnEU+zk0EFFQxXmTx2384DwARAQAB
< tClFd2VuIENoZXNsYWNrLVBvc3RhdmEgPGV3ZW5AY29uZmx1ZW50LmlvPokCOAQT
< AQIAIgUCVLaq3gIbAwYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQk07sfb5+
< /HNJIg/9GW2s+0EpKyIcvyp46Hw4y0ruOKz++JZs/vbHGWCbYQbC80gKDHxCQxvH
< FgK+s4xFe9hsbD/36FBKE9Jbit581F983/KPgoD0bvX2yum6X6T/TaGFSfPkO6hw
< qCbLSswLo5QJV527yrvbKe2dpL7v6E6ZOLZiP8vYJMIS30UyV40gPJp7q6yyT8gH
< xEKW5aGDuCHCzRlRB+jNFUzsB6Q5MeOTVHfTvjPhHLbFhGBmU7buqxtT7EREAze5
< 8I8JUFJkHtdf32a463a+2qjll6AEkWt2iBVuRilkgOnxuX/xNdb7H/6iFCT6UNCb
< pMrM3RLzh0iklT5BNeZ5mMXY8FnOqBy/fFGvWedb6T9Im4ERgWrVq14X8b+uUaP9
< URRvBczXvTfSb4Lajt5BvOvgleOF7iQhzFO1byzo2HD5CArv0HJEIP7GYNDnp14g
< GjQehaqmHP7/RBRDw/Hg0gy828KmfwD3K6VSWpwHOPaas84O6waxRsCKW6Ng/PpM
< FWF5NfPm4sVsuK/QiCpc/hpDzpDVxLL9Z7cOG6APZKFv+g9isUTtP2sn7tDoas0Y
< BvgpPz9gpASF2Y3/3wgyitzY7YO1w36jwgSqkTud8Lknc6tGOStGqclqmt4Ep/OB
< 43i7nbYTo12Udn37WotuDstOm8NVZWxsjXVBb7QW8Rn/fJnolZq5Ag0EVLaq3gEQ
< AKN94fP55s47wo31KQhRgQH7AsLQcMh2bWcJDchIvMsF6oB582T7uHqU3AYFvJNP
< FSkPc0k4XetyZTBTtbnw0bRM1UQECDvQnB4M+HpqlAM3LJce3oPzyVYZ0vxpQRcc
< Y/bCyEc9j3U2BxtbHfZgCTv+keSNsE6i5dpZ92KRw/PQAyth6fDDM40DR8SiXJR2
< tIaTmXoa0JOoECPwDLQqccXmpL+5Vl4xcjrPBZ9BIiA0WBm2dvSk4Nt+8xz8FjNh
< M+slsTUiI20yfKEXN9mM9pPsS7gnngZR96QJBWOBersf3L/aO1z1ywZORyYFED86
< U+I67IIHHHYVC+CsRTOLHEeSNGnYKAmuY9aQnnFkE5fkGQ46mx2eE05Od9xiciy7
< U+XKJ0Pz/yBaiSoLg5TMHo+33giHCh/ESrc+aiohjXKA6Z6cqkmQIg19sisNmZqJ
< 0IBNZrvNJHCxolN8VSvz1lfAyEkbYQ6L4JGdnp5Gu0DWaaJcQ2BoE/i2uD0xjNCv
< o+GuEF66YI2Gm0d1oePsy+jkgrw0sZetaCccX08dJOcNklKz6nQYjFT2C43zSlzP
< jkDEA9sgEabDN0SyYAnbDR+ZW0g+/B/VrWS+j+apNQAckYWAIQCzyMSbkrYJRAdO
< vEXKo+rQxm/LyBIq5pMwqy1nlx87LP9h/AgtGJN6Qc0PABEBAAGJAh8EGAECAAkF
< AlS2qt4CGwwACgkQk07sfb5+/HOqHw/+MhfNz5I8n7xmBvVln7sujPkiA6heWbxe
< Ga+zZnqRTFFsULaMgz4dIdXwXI2Sgym3WKangiJZr3OEpLPlXSM/FS+GAruRH+/y
< 2CPfkqxd1jG13Q1/BWBHbtxJizWuZ+UtK+/wqWc099/4oTnkckchtZBJQ5gdGoio
< swqZdj9rguXROAtW2Fmc/5K+vxgGTuE8cu4TxjA0ZEWdMaJYI6dlLg9LjPpGzs4L
< jDn2dItasnsZ6irhXISYhV9WJbus2H9kYMjZxdJhY6HzkWVaUVrdtiGlWjf6fQ9N
< e0X6DN9kd3YMNhiJ8JExxrB//2JPr/3B5nc/Z0zKmwn1znUz04WWKQ1QM9Irszrr
< 5fWAFy0LgsiT0922hH0f9puss5+irQUSIXDaW2iaCdNszZiTZxNwJDYu8m0pj7eP
< KgkKutptC/1on/mE9IeOkQ5acj2XfgKvljMP9VBX5PrRoRjJ/PMxinZPbzAHZy20
< 7TYMbJH5dsqFIeh66jycn2IDKElvk7A9Wk98XbaxEFq8KhxblG+glV+kz2NWm3x6
< wJ/WFx5uXvKXuXrir92HGgzC6hy+v7m1DPCBlqM3+qrXTcyhbqeC52pjj4AAFnaF
< G9VPAuAnKuZSCkxGuiEgxxjDdikTkYmxbqpel+m0T6DLLgNX58M2yURAIaDajLW/
< gozBq5tLHNiZAg0EU/a4MQEQAMohxV5xUHkknnH4h3xr6tOczInCW3xObdUlsZh9
< mLhiU4+0kzQtNbjKK2qgNgmQzL1p+VYuQ2XQJmhxynXAcc9sFag7kOnzrVvC+HGb
< gZRr79YrtGqEFwU/a3YommEVAFdVvBe0HM+24pbBMoiHV1OaabnSmAYX2Fmpuh+t
< xo7FvPZL6/XqPs3lNQiOR10HU5ZaRajfjok8X5w10X5nOUr0jlj3hhE2usb5/tII
< 6WX2LaYwiC0nSu61TlbijN4m0ikbY3L1ycZirGGbn9eziTfK36itznrNF3DaARSW
< 2rodDKiISiln92QGXnV9rA+C/+bcSUpd8KFsswjfuWCUVzhS3h3xnRZ9UBhvex69
< soLa0TAZPzGqyoRwQF0gTTmRDkwOebzop/t1ZfaQWDaNfEqAK7MD72fmqGQX43dj
< DdgMg9dpmxi3EMSLuk082NqffL1bTv3wW6SFpyra3iRLfz1TJISuszU/Z+TNRZTi
< 8v/U5X5VH5TBRLSRaLGbAfq3z0Ro1XmK+opccbwKMpQ1j40KXTy1kWgt/RYNQWZ/
< nSZ4sNSwko6sFN8F6pvLcjyHZCFpc3aSdTdsv/c539qgEF+2+b/Epus9GEiYvyE+
< mxUqxR37jLXF6lnkHOtJBc1FEiUNDLupdzGU5XRvXDmpxJQY8tgSALloiwneVhpL
< U8E/ABEBAAG0JUV3ZW4gQ2hlc2xhY2stUG9zdGF2YSA8bWVAZXdlbmNwLm9yZz6J
< AjgEEwECACIFAlP2uDECGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEN2q
< NFJSNNlPvpwP/1wypIiAYYn32A/FEbRP6XZ9IjTOBL3HXUqj1FpZm+kNorkUoBk5
< EyMV9O4emzhrxqBlvsgHG9OM+Zw+oHNcdw3bcSI08+ckIFIRqVgRYe9ghEZZgbZs
< SmupaW/wr0jcBQSWLW7VEMlFleTZjrGhL6rP0sIXm4cqTiJJAzDF6Ngf1slfZ2xd
< /eZxHII7mNizFLtLuA57+aBhekxCX2i7n13vM8l7cr+9uXUeeUwxh0HFlpy+24sf
< yqrm261jE186YPGkrB6oFTa3XfP1JsLx1yTO2dXUqYzroMzh7jM8Ohv16U/2PWIf
< 5UKskEnQqv9sqPOKWDhWoWtpk92bYWj4mglPpqk0s7A+gIgCq8ODICNMFKIYl/EO
< +zKftuGLZnAOOqJ6yUz+ZWOQHbDpKNCPmtTxj5QZtaWeZEghCtysITFTvZNLkex3
< xAJo659/k3LJqphHn3RXR9u694ELxNoHCD5ZPSO4nW8zcBcqway4lEZV6LBxE+A8
< oKNKzWOpdnxwussK120PXIR4rO+V/x6rEh4pcQzzgD1dmkaXXtBsFh/dykgT0Abd
< M0qHQO0uJmZkhh/H80mH9IBGrU5QWxspwmfcLOZfx0lHsUkXZJcW1EhPSRC2hIGm
< gYm3wHrBcbXHHKH4f0XsBQSM6MASC1GseZJOR6qcs0oqbAfK3sQDFoyIuQINBFP2
< uDEBEAC3lnj+gp6swX14ZG/vHBmC8TrSi0AdHQRA5p3eYwYCo5lxjdYfzhYSXW8p
< /5M9i4x3M7MQodKqNTmG7w8w7EJhDUxR6j5dUcIRtt1oPbKcjazzYt3GOChyY8lI
< dt6eXsdZ+s0lFujDu7lyJXVbR3A7/HSjy75ExH03Dtct20yawO8yibZOvs9A2QeR
< B689GSp6d1xp2iO8g66zrRGWjiJka7Uxy9hlr0TQ1/9uovhlcj+3ISJwDs3J/ZUc
< dra7fKnr3aanGBJ6fVeKh6ymz8AfuecXRrIza1tx0Hp2K+JcZDcNa+gYjfjWjV0l
< W8M/cxDJpusLbu0szP4B5lPqMGzU/NMjVkV3BOMHRGZDRmGxP/GeTEmjjmLiiVz/
< nsg1n6sGIVwKhpy/c7kcxzJUTA+/+pSLT6H7TvKo6K9GYmb/EJ/GLwobZePo09h0
< pH5uVjKEKCoa83Nr0ZYyanN3GHQIgm5ZbsjEUeiRUGnDSo6dHfJzvPUFFHwpaYHa
< kfSg+kBmti+7igCbqbF2jMAaaiChSQDc4BXsgj77RDM81e10ZqK0Ju3sKyVvwBK2
< KjrQXqJcesxRPpg8GjLbCDw5W7W1yOvtRyhhxKXR4hGMSFVxGdQ1XAbdVb0BQEHb
< rPDh4VanzX9ih0q1bLLdXlRMy88AqW2pOa3x92amDgTEXpi+RQARAQABiQIfBBgB
< AgAJBQJT9rgxAhsMAAoJEN2qNFJSNNlPHRIP/336jYm0s4n5usCiGj3sI8e9EQXh
< QBm0lvScLZAwFESkGIWk1Zs09E5hy9iZlMENQnOrMLCmvwj7T7YJq8ctMz+5R2tB
< a5xCMvmc+D2WfFQzHY4b9AsBddkEAt88izZEpZZszR3ikAtMzO2pw+HEd6ATW7pI
< cBGeJ9INInpwz9wjKW6mjWtGuSDAsVtSvnDShjxOnbw0/lJvGUNRvqkJ7TFeU0wl
< YkrLuQgry2tzAqaYzTq6cA7P9aYQd0dVjjhnAIv5ZU1Bq5vYQnoF6cJZRtnmBNaq
< GwHGE/wpkLD/Mxu/gCvjB1eT7UqMPsdxTqAKlyFKPDqpggLqNjcUVVckUXO94P67
< 3R/fOWSCvPtwEioVSUaT3Y2EQcdO7nyGzydSYFhlvJf1foNhmabXHebyZ7SzwDX2
< EW/9IzyvZCeiduCu3zCiRD5GHiKpHwTUSsHxg/evmhZQlq8P6HGme94IP1VZzxAy
< jqUFRnVEc12qWtYKXROF5VsZ8HAyil8O4rlzstv5eFzB1HHBUOMONFsVyMqSHybW
< EbtR59HTeakJ/3W1aNPK0niadVYhR583XHoOZlKLXq2AVd5PeSgRANI3c9847zsY
< U0PdwcFXT1sU3rtuhREDUTagZ3FmwIOse/NXbN5dHcnnZMWn+qqzMxiutb4MuN01
< qUAdpxxKffByAEiV
< =LJo0
< -----END PGP PUBLIC KEY BLOCK-----

override default server variable

Hi,

I had to override server.advertised_listeners, which is defined in the default role variables, and it was quite a difficult task.

I asked a coworker which knows Ansible much better than I and the best method we found so far is to use pre_tasks as bellow:

- hosts: kafka
  roles:
    - role: ansible-kafka
      kafka_hosts: "{{ ansible_play_batch }}"
      kafka_version: 1.0.0     # Kafka version override.

  vars:
    zookeeper_hosts: "kafka1-1:2181,kafka1-2:2181,kafka1-3:2181"
    kafka_zookeeper_hosts: ['kafka1-1', 'kafka1-2', 'kafka1-3']

  pre_tasks:
    - name: set advertised listeners
      set_fact:
        server: '{{ server | combine({"advertised_listeners": "PLAINTEXT://"+ansible_hostname+":9092" }) }}'

The thing is we either need to completely redefine the server dict in the ansible play, or change the Ansible merge strategy in ansible.cfg (which is generally a bad idea) or use this set_fact trick.

Are you aware of a better method to redefine those variables ? If not, would you like to add an example in the role documentation?

Cheers,

incorrect link to ansible galaxy project

Hi,
It seems that the current link to galaxy in the project description is outdated. Looks like it should now be https://galaxy.ansible.com/jaytaylor/kafka/.

I cannot make a PR for this one ;)

Regards,

"Generate generic unique machine id integer" fails on LVM root

Problem is caused by fact that tr on empty string is ok (command return 0). So tr -d '-' || echo '0' behaves different then expected.

Proper order of commands below

  • name: "Generate generic unique machine id integer"

    NB: This uses a combination of root partition UUID + network interface MAC address.

    shell: ( (test -r /etc/fstab && ls -l /dev/disk/by-uuid/ | grep $(mount | grep ' / ' | cut -d' ' -f1 | cut -d'/' -f3) | grep --ignore-case --only-matching --extended-regexp --max 1 '[0-9a-f]{3,}[0-9a-f-]+' || echo '0') | tr -d '-' ; ifconfig | grep --ignore-case --only-matching --extended-regexp '([0-9a-f]{2}:){5}[0-9a-f]{2}' | tr -d ':' | tr -d '\n') | python -c 'import sys; x, y = sys.stdin.read().split(chr(10))[0:2]; x = int(x, 16); y = int(y, 16); sys.stdout.write((str(x + y)[-9:])); sys.exit(1 if x == 0 and y == 0 else 0)'
    register: machineidinteger
    changed_when: False
    when: kafka_generate_broker_id | bool
    tags:
    • kafka-cfg

With generated machine id, this seems to exceed reserved.broker.max.id in 0.9.0.0

reserved.broker.max.id = 1000

[2016-01-06 10:46:40,824] FATAL  (kafka.Kafka$)
java.lang.IllegalArgumentException: requirement failed: broker.id must be equal or greater than -1 and not greater than reserved.broker.max.id
        at scala.Predef$.require(Predef.scala:219)
        at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:945)
        at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:942)
        at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:691)
        at kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:28)
        at kafka.Kafka$.main(Kafka.scala:58)
        at kafka.Kafka.main(Kafka.scala)

set_fact: kafka_zookeeper_connection_string= wrong string

Hi. Thanks for perfect role, like it a lot, however:

This is my playbook:

- hosts: kafka
  vars:
    zookeeper_hosts: 172.31.32.93
    kafka_zookeeper_hosts:
      - 172.31.32.93
    kafka_hosts:
      - 172.31.46.135
      - 172.31.47.248
  roles:
    - {role: ansible_kafka ,kafka_hosts: "{{ groups.kafka | list }}", kafka_zookeeper_hosts: "{{ zookeeper_hosts | list }}", kafka_version: "0.11.0.2", kafka_scala_serverion: "2.10" }

There is a line in kafka-cfg.yml which suppose to create zookeeper connection string.
set_fact: kafka_zookeeper_connection_string="{{ kafka_zookeeper_hosts | join(':' ~ kafka_zookeeper_port ~ ',') }}:{{ kafka_zookeeper_port }}"

Instead of creating string like 172.31.32.93:2181 it creates something like
1:2181,7:2181,2:2181,.:2181,3:2181,1:2181,.:2181,3:2181,2:2181,.:2181,9:2181,3:2181

Could You please suggest a fix?

Be able to specify JMX_PORT

Would have been nice to specify JMX_PORT in the role. Seems like Kafka won't start the JMX service as default if the variable isn't set, and if it's not specified in ZK.

Can't install 0.10.1.0

This role fails with kafka_version: 0.10.1.0, because it has been signed with another key than the one specified in defaults.yml (kafka_sig_id: E0A61EEA).

We should instead follow the instructions at the bottom of https://www.apache.org/dyn/closer.cgi and download and import the KEYS file.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.