start: Unknown job: kafka about kafka-debian

[drewbroadley]

Zookeeper and Kakfa can be started fine without init.d script:

$ /usr/lib/kafka/bin/zookeeper-server-start.sh /etc/kafka/zookeeper.properties &
$ /usr/lib/kafka/bin/kafka-server-start.sh /etc/kafka/server.properties &

[ottomata]

Hm, did you paste the wrong output? You pasted the dpkg -i output, which doesn't say anything about starting kafka.

[drewbroadley]

:That is all the output there is, nothing in any system log.

Here's the strace of running /etc/init.d/kafka start if it will help:

execve("/etc/init.d/kafka", ["/etc/init.d/kafka", "start"], [/* 20 vars
*/]) = 0
brk(0) = 0x235f000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f290106f000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or
directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=40641, ...}) = 0
mmap(NULL, 40641, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f2901065000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
directory)
open("/lib/libc.so.6", O_RDONLY) = 3
read(3,
"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\355\1\0\0\0\0\0"..., 832)
= 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1584520, ...}) = 0
mmap(NULL, 3692584, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0x7f2900acc000
mprotect(0x7f2900c49000, 2093056, PROT_NONE) = 0
mmap(0x7f2900e48000, 20480, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17c000) = 0x7f2900e48000
mmap(0x7f2900e4d000, 18472, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f2900e4d000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f2901064000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f2901063000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f2901062000
arch_prctl(ARCH_SET_FS, 0x7f2901063700) = 0
mprotect(0x7f2900e48000, 16384, PROT_READ) = 0
mprotect(0x617000, 4096, PROT_READ) = 0
mprotect(0x7f2901071000, 4096, PROT_READ) = 0
munmap(0x7f2901065000, 40641) = 0
getpid() = 16937
rt_sigaction(SIGCHLD, {SIG_DFL, [CHLD], SA_RESTORER|SA_RESTART,
0x7f2900affba0}, {SIG_DFL, [], 0}, 8) = 0
geteuid() = 0
brk(0) = 0x235f000
brk(0x2380000) = 0x2380000
getppid() = 16936
stat("/home/drew", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat(".", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/etc/init.d/kafka", O_RDONLY) = 3
fcntl(3, F_DUPFD, 10) = 10
close(3) = 0
fcntl(10, F_SETFD, FD_CLOEXEC) = 0
rt_sigaction(SIGINT, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGINT, {0x408189, ~[RTMIN RT_1], SA_RESTORER,
0x7f2900affba0}, NULL, 8) = 0
rt_sigaction(SIGQUIT, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGQUIT, {SIG_DFL, ~[RTMIN RT_1], SA_RESTORER,
0x7f2900affba0}, NULL, 8) = 0
rt_sigaction(SIGTERM, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGTERM, {SIG_DFL, ~[RTMIN RT_1], SA_RESTORER,
0x7f2900affba0}, NULL, 8) = 0
read(10, "#!/bin/sh -e\n# upstart-job\n#\n# S"..., 8192) = 1830
pipe([3, 4]) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7f29010639d0) = 16938
close(4) = 0
read(3, "kafka\n", 128) = 6
read(3, "", 128) = 0
close(3) = 0
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 16938
--- SIGCHLD (Child exited) @ 0 (0) ---
write(1, "Rather than invoking init script"..., 74) = 74
write(1, "utility, e.g. service kafka star"..., 34) = 34
write(1, "\n", 1) = 1
write(1, "Since the script you are attempt"..., 71) = 71
write(1, "Upstart job, you may also use th"..., 69) = 69
pipe([3, 4]) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7f29010639d0) = 16939
close(4) = 0
read(3, "", 128) = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
close(3) = 0
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 16939
stat("/usr/local/sbin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or
directory)
stat("/usr/local/bin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or
directory)
stat("/usr/sbin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or
directory)
stat("/usr/bin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or
directory)
stat("/sbin/start", {st_mode=S_IFREG|0755, st_size=114056, ...}) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7f29010639d0) = 16942
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0, NULL) = 16942
--- SIGCHLD (Child exited) @ 0 (0) ---
exit_group(1)

On Tue, Jan 29, 2013 at 3:43 AM, Andrew Otto [email protected]:

Hm, did you paste the wrong output? You pasted the dpkg -i output, which
doesn't say anything about starting kafka.

—
Reply to this email directly or view it on GitHubhttps://github.com/wmf-analytics/kafka-debian/issues/1#issuecomment-12784538.

[ottomata]

Hm,

If you are using the init/upstart script to start kafka, you should be
Try starting it this way:

sudo service kafka start

/etc/init.d/kafka is just a symlink to upstart. /etc/init/kafka.conf is the real service script. Do you have a system with upstart installed?

On Jan 28, 2013, at 9:19 PM, Drew Broadley [email protected] wrote:

:That is all the output there is, nothing in any system log.

Here's the strace of running /etc/init.d/kafka start if it will help:

execve("/etc/init.d/kafka", ["/etc/init.d/kafka", "start"], [/* 20 vars
*/]) = 0
brk(0) = 0x235f000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f290106f000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or
directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=40641, ...}) = 0
mmap(NULL, 40641, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f2901065000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
directory)
open("/lib/libc.so.6", O_RDONLY) = 3
read(3,
"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\355\1\0\0\0\0\0"..., 832)
= 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1584520, ...}) = 0
mmap(NULL, 3692584, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0x7f2900acc000
mprotect(0x7f2900c49000, 2093056, PROT_NONE) = 0
mmap(0x7f2900e48000, 20480, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17c000) = 0x7f2900e48000
mmap(0x7f2900e4d000, 18472, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f2900e4d000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f2901064000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f2901063000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f2901062000
arch_prctl(ARCH_SET_FS, 0x7f2901063700) = 0
mprotect(0x7f2900e48000, 16384, PROT_READ) = 0
mprotect(0x617000, 4096, PROT_READ) = 0
mprotect(0x7f2901071000, 4096, PROT_READ) = 0
munmap(0x7f2901065000, 40641) = 0
getpid() = 16937
rt_sigaction(SIGCHLD, {SIG_DFL, [CHLD], SA_RESTORER|SA_RESTART,
0x7f2900affba0}, {SIG_DFL, [], 0}, 8) = 0
geteuid() = 0
brk(0) = 0x235f000
brk(0x2380000) = 0x2380000
getppid() = 16936
stat("/home/drew", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat(".", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/etc/init.d/kafka", O_RDONLY) = 3
fcntl(3, F_DUPFD, 10) = 10
close(3) = 0
fcntl(10, F_SETFD, FD_CLOEXEC) = 0
rt_sigaction(SIGINT, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGINT, {0x408189, ~[RTMIN RT_1], SA_RESTORER,
0x7f2900affba0}, NULL, 8) = 0
rt_sigaction(SIGQUIT, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGQUIT, {SIG_DFL, ~[RTMIN RT_1], SA_RESTORER,
0x7f2900affba0}, NULL, 8) = 0
rt_sigaction(SIGTERM, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGTERM, {SIG_DFL, ~[RTMIN RT_1], SA_RESTORER,
0x7f2900affba0}, NULL, 8) = 0
read(10, "#!/bin/sh -e\n# upstart-job\n#\n# S"..., 8192) = 1830
pipe([3, 4]) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7f29010639d0) = 16938
close(4) = 0
read(3, "kafka\n", 128) = 6
read(3, "", 128) = 0
close(3) = 0
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 16938
--- SIGCHLD (Child exited) @ 0 (0) ---
write(1, "Rather than invoking init script"..., 74) = 74
write(1, "utility, e.g. service kafka star"..., 34) = 34
write(1, "\n", 1) = 1
write(1, "Since the script you are attempt"..., 71) = 71
write(1, "Upstart job, you may also use th"..., 69) = 69
pipe([3, 4]) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7f29010639d0) = 16939
close(4) = 0
read(3, "", 128) = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
close(3) = 0
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 16939
stat("/usr/local/sbin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or
directory)
stat("/usr/local/bin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or
directory)
stat("/usr/sbin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or
directory)
stat("/usr/bin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or
directory)
stat("/sbin/start", {st_mode=S_IFREG|0755, st_size=114056, ...}) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7f29010639d0) = 16942
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0, NULL) = 16942
--- SIGCHLD (Child exited) @ 0 (0) ---
exit_group(1)

On Tue, Jan 29, 2013 at 3:43 AM, Andrew Otto [email protected]:

Hm, did you paste the wrong output? You pasted the dpkg -i output, which
doesn't say anything about starting kafka.

—
Reply to this email directly or view it on GitHubhttps://github.com/wmf-analytics/kafka-debian/issues/1#issuecomment-12784538.

—
Reply to this email directly or view it on GitHub.

[drewbroadley]

I gave both a go, same output:

/etc/init.d/kafka start

start: Unknown job: kafka

sudo service kafka start

start: Unknown job: kafka

upstart is installed:

apt-get install upstart

Reading package lists... Done
Building dependency tree
Reading state information... Done
upstart is already the newest version.

Let me know what else I can do to help.

On Wed, Jan 30, 2013 at 3:33 AM, Andrew Otto [email protected]:

Hm,

If you are using the init/upstart script to start kafka, you should be
Try starting it this way:

sudo service kafka start

/etc/init.d/kafka is just a symlink to upstart. /etc/init/kafka.conf is
the real service script. Do you have a system with upstart installed?

On Jan 28, 2013, at 9:19 PM, Drew Broadley [email protected]
wrote:

:That is all the output there is, nothing in any system log.

Here's the strace of running /etc/init.d/kafka start if it will help:

execve("/etc/init.d/kafka", ["/etc/init.d/kafka", "start"], [/* 20 vars
*/]) = 0
brk(0) = 0x235f000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
directory)

mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)

0x7f290106f000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or
directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=40641, ...}) = 0
mmap(NULL, 40641, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f2901065000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
directory)
open("/lib/libc.so.6", O_RDONLY) = 3
read(3,
"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\355\1\0\0\0\0\0"...,
832)
= 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1584520, ...}) = 0
mmap(NULL, 3692584, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
0) =
0x7f2900acc000
mprotect(0x7f2900c49000, 2093056, PROT_NONE) = 0
mmap(0x7f2900e48000, 20480, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17c000) = 0x7f2900e48000
mmap(0x7f2900e4d000, 18472, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f2900e4d000
close(3) = 0

mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)

0x7f2901064000

mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)

0x7f2901063000

mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)

0x7f2901062000
arch_prctl(ARCH_SET_FS, 0x7f2901063700) = 0
mprotect(0x7f2900e48000, 16384, PROT_READ) = 0
mprotect(0x617000, 4096, PROT_READ) = 0
mprotect(0x7f2901071000, 4096, PROT_READ) = 0
munmap(0x7f2901065000, 40641) = 0
getpid() = 16937
rt_sigaction(SIGCHLD, {SIG_DFL, [CHLD], SA_RESTORER|SA_RESTART,
0x7f2900affba0}, {SIG_DFL, [], 0}, 8) = 0
geteuid() = 0
brk(0) = 0x235f000
brk(0x2380000) = 0x2380000
getppid() = 16936
stat("/home/drew", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat(".", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/etc/init.d/kafka", O_RDONLY) = 3
fcntl(3, F_DUPFD, 10) = 10
close(3) = 0
fcntl(10, F_SETFD, FD_CLOEXEC) = 0
rt_sigaction(SIGINT, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGINT, {0x408189, ~[RTMIN RT_1], SA_RESTORER,
0x7f2900affba0}, NULL, 8) = 0
rt_sigaction(SIGQUIT, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGQUIT, {SIG_DFL, ~[RTMIN RT_1], SA_RESTORER,
0x7f2900affba0}, NULL, 8) = 0
rt_sigaction(SIGTERM, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGTERM, {SIG_DFL, ~[RTMIN RT_1], SA_RESTORER,
0x7f2900affba0}, NULL, 8) = 0
read(10, "#!/bin/sh -e\n# upstart-job\n#\n# S"..., 8192) = 1830
pipe([3, 4]) = 0
clone(child_stack=0,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7f29010639d0) = 16938
close(4) = 0
read(3, "kafka\n", 128) = 6
read(3, "", 128) = 0
close(3) = 0
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 16938
--- SIGCHLD (Child exited) @ 0 (0) ---
write(1, "Rather than invoking init script"..., 74) = 74
write(1, "utility, e.g. service kafka star"..., 34) = 34
write(1, "\n", 1) = 1
write(1, "Since the script you are attempt"..., 71) = 71
write(1, "Upstart job, you may also use th"..., 69) = 69
pipe([3, 4]) = 0
clone(child_stack=0,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7f29010639d0) = 16939
close(4) = 0
read(3, "", 128) = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
close(3) = 0
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 16939
stat("/usr/local/sbin/start", 0x7fffb88ff070) = -1 ENOENT (No such file
or
directory)
stat("/usr/local/bin/start", 0x7fffb88ff070) = -1 ENOENT (No such file
or
directory)
stat("/usr/sbin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or
directory)
stat("/usr/bin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or
directory)
stat("/sbin/start", {st_mode=S_IFREG|0755, st_size=114056, ...}) = 0
clone(child_stack=0,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7f29010639d0) = 16942
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0, NULL) = 16942
--- SIGCHLD (Child exited) @ 0 (0) ---
exit_group(1)

On Tue, Jan 29, 2013 at 3:43 AM, Andrew Otto [email protected]:

Hm, did you paste the wrong output? You pasted the dpkg -i output,
which
doesn't say anything about starting kafka.

—
Reply to this email directly or view it on GitHub<
https://github.com/wmf-analytics/kafka-debian/issues/1#issuecomment-12784538>.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHubhttps://github.com/wmf-analytics/kafka-debian/issues/1#issuecomment-12836978.

[vide]

It happens here the same on Debian Squeeze with upstart 0.6.6 installed

start kafka

start: Unknown job: kafka

(which is what the legacy init script is executing)

[dastra]

I had the same trouble - so forked this project and wrote a standard init.d script for it. See https://github.com/dastra/kafka-debian

[vide]

Oh dastra, I was just about to do the same :)

[ottomata]

Cool! Sorry guys, I've been super busy with other stuff and haven't been able to replicate this myself. init.d script may be the way to go.

[vide]

Don't worry, anyway even if it worked, upstart for Debian makes you change a critical system component which is not a good idea in a server.

[drdee]

What's the status of this? Otto, do we need to fix this or can we close it?

[ottomata]

I suppose it is still an issue, but this repository can probably be removed, as we will be hosting it here:

https://github.com/wikimedia/operations-debs-kafka