gearman / gearmand

gearmand

The latest version of gearmand source code and versions 1.1.13 and later can be found at GitHub Repository. Older versions released before 1.1.13 can be found at Launchpad Repository.

You can grab the latest release distribution of Gearman from https://github.com/gearman/gearmand/releases. Unless otherwise specified, releases are GPG-signed by Clint Byrum, whose public key can be found at https://fewbar.com/clint-byrum-public-key/.

What Is Gearman?

Gearman provides a generic application framework to farm out work to other machines or processes that are better suited to do the work. It allows you to do work in parallel, to load balance processing, and to call functions between languages. Gearman is the nervous system for how distributed processing communicates.

If you downloaded this package as a tar.gz distribution, you'll want to read the Getting Started section below or visit the more detailed web page Getting Started.

If you are interested in developing or submitting patches to the project, read the Contributing section below and check out the CONTRIBUTING.md file for Coding Style and COPYING for details on licensing.

Getting Started

If you want to work on the latest code, please read the file CONTRIBUTING.md.

To build a release version from a tarball (.tar.gz or .tgz), you can follow the normal:

Change into the directory where you saved the tarball and run:

tar xzf gearmand-X.Y.tar.gz
cd gearmand-X.Y

Then run the usual autoconfigure style build (you may need to use sudo to install):

./configure
make
make install

There are various dependencies that may be satisfied on Ubuntu by installing these packages:

sudo apt install automake autoconf libtool make curl gcc g++ git gperf     \ 
    libssl-dev libboost-all-dev libevent-dev libhiredis-dev libpq-dev      \ 
    libtokyocabinet-dev python3-sphinx uuid-dev

You can also run make test before installing to make sure everything checks out ok. You can also streamline the process of building and testing by running:

./configure && make && make test
make install

Once you have it installed, you can start the Gearman job server with:

gearmand --verbose INFO

This will start it while printing some verbose messages. To try running a job through it, look in the examples/ directory of this source and run:

./reverse_worker

Once that is running, you can run your first job with:

./reverse_client "Hello, Gearman!"

If all goes well, the reverse_worker application should have output:

Job=H:lap:1 Workload=Hello, Gearman! Result=!namraeG ,olleH

While the reverse_client returned:

Result=!namraeG ,olleH

There are a lot more details about gearmand at Getting Started.

If you want to start writing your own client and workers, be sure to check out the Developer API documentation.

For using and configuring the gearmand server, please check out the latest documentation.

There are also many other useful resources to help you put gearmand to work for you!

Enjoy!

Contributing

The current versions of geamand are maintained on our GitHub Repo for gearmand.

If you are not familiar with git, you can find more info at Getting Started with Git.

Please follow these instructions to clone, create a branch, and generate a pull request on that branch. More details on using GitHub can be found at GitHub Help.

1. Clone the GitHub repository to your local file system:

    git clone https://github.com/gearman/gearmand
   

   If you do not have access to create branches in the gearmand GitHub repository, you should fork the repository and clone your fork instead. Refer to Contributing to Open Source on GitHub for details.

2. Next, think of a clear, descriptive branch name and then create a new branch and change to it:

    cd gearmand
    git checkout -b DESCRIPTIVE_BRANCH_NAME
   

3. Once the tree is branched you will need to generate the "configure" script for autoconfigure.

    ./bootstrap.sh -a
   

4. Finally, you are ready to run tests, make changes to the code, commit and push them to GitHub, and generate a pull request on your branch so we can consider your changes.

You can learn more about how to Create a Pull Request and Create a Pull Request from a Fork.

But Wait! There's More!

Once you have made your changes there are two additional make targets to build release ready distributions:

To generate a tarball distribution of your code:

make dist

Or to generate an RPM distribution use:

make rpm

Thanks and keep hacking!

Cheers,
-Brian
Seattle, WA.

gearmand's Issues

Client connected to multiple servers doesn't gracefully handle a server being unavailable

If a gearman client is connected to multiple servers and one of the servers 'goes away', the client doesn't gracefully move on to the next server. Instead the next attempt or two (the example below shows two attempts, but I have seen it reconnect after one) to queue a job result in an error before the client connects to another server.

$client = new GearmanClient();
$client->addServer('127.0.0.1','4730');
$client->addServer('127.0.0.1','4731');
$client->doBackground('test-func','data'); //Job added to gearmand on 4731
// gearmand on 4731 stopped
$client->doBackground('test-func','data'); //Job not added
PHP Warning:  GearmanClient::doBackground(): recv_socket(GEARMAN_LOST_CONNECTION) lost connection to server (EOF) -> libgearman/connection.cc:1181: pid(15120) in php shell code on line 1
$client->doBackground('test-func','data'); //Job not added
PHP Warning:  GearmanClient::doBackground(): _client_run_tasks(GEARMAN_LOST_CONNECTION) detected lost connection in _client_run_tasks() -> libgearman/client.cc:1495: pid(15120) in php shell code on line 1
$client->doBackground('test-func','data'); //Job added to gearmand on 4730

This was produced using gearmand 1.1.14, libgearman 1.1.14 and PHP gearman extension version 2.0.2.

static analyze report created with PVS-Studio

Here is a static analyze report created for gearmand at 15 October 2016 with PVS Studio

A large memory leak

Hello,

We have a large memory leak issue regarding the "malloc" line 90 of file "packet.cc".
We lost 4MB, 8MB of RAM in 15 minutes of use.

I did not find in your memory the release of this code "malloc".
Will you help us fix this problem?

We found this problem by using the tool Valgrind.

No changelog for the 1.1.13 release

The 1.1.13 release has no entry inside the changelog file nor on the github release page.

Server wake up worker twice at same time

I ran into a problem:
worker: CAN_DO
worker: PRE_SLEEP
worker: CAN_DO
worker: PRE_SLEEP // two different functions registered completed
server: NOOP
worker: GRAB_JOB
server: NOOP
worker: GRAB_JOB
server: JOB_ASSIGN
server: JOB_ASSIGN

above is the communication between Server and Worker, in the course of nature, Server would wake up the worker use NOOP just once, and waiting Worker to send GRAB_JOB request. What is strange is the worker received two Jobs, say a and b, with different payload content meanwhile the same Job Id(in Gearman server is named Handle like: H:localhost:5113). That is the problem, Server deleted the Job Id after received the first WORK_COMPLETE message, then Server cannot get second job since it is deleted. Maybe I can figure out in gearman_server_job_get function, but there is no debug information:

gearman_server_job_st *gearman_server_job_get(gearman_server_st *server,
                                              const char *job_handle,
                                              const size_t job_handle_length,
                                              gearman_server_con_st *worker_con)
{
  uint32_t key= _server_job_hash(job_handle, job_handle_length);

  for (gearman_server_job_st *server_job= server->job_hash[key % server->hashtable_buckets];
       server_job != NULL; server_job= server_job->next)
  {
    if (server_job->job_handle_key == key and
        strncmp(server_job->job_handle, job_handle, GEARMAND_JOB_HANDLE_SIZE) == 0)
    {
      /* Check to make sure the worker asking for the job still owns the job. */
      if (worker_con != NULL and
          (server_job->worker == NULL or server_job->worker->con != worker_con))
      {
        return NULL;
      }

      return server_job;
    }
  }

  return NULL;
}

Server return error message is:

code=JOB_NOT_FOUND, msg=Job does not exist on server
code=JOB_NOT_FOUND, msg=Job given in work result not found

Any idea?

"./configure --enable-debug=yes" causes make failure

• Ubuntu 14.04

...
/usr/include/x86_64-linux-gnu/gnu/stubs-64.h
/usr/include/x86_64-linux-gnu/gnu/stubs.h
make[2]: *** [libhashkit/libhashkit_libhashkitinc_la-strerror.lo] Error 1
make[2]: Leaving directory `/vagrant/gearmand'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/vagrant/gearmand'
make: *** [all] Error 2

• Ubuntu 16.04

...
/usr/include/x86_64-linux-gnu/gnu/stubs-64.h
/usr/include/x86_64-linux-gnu/gnu/stubs.h
Makefile:6207: recipe for target 'libgearman/libgearman_libgearman_la-actions.lo' failed
make[2]: *** [libgearman/libgearman_libgearman_la-actions.lo] Error 1
make[2]: Leaving directory '/home/palik/workspace/Gearman/gearmand'
Makefile:8369: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/home/palik/workspace/Gearman/gearmand'
Makefile:3088: recipe for target 'all' failed
make: *** [all] Error 2

mac os build failed

➜  ./configure                                                                                                            ruby-2.3.1 

checking for cc... cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether cc accepts -g... yes
checking for cc option to accept ISO C89... none needed
checking whether cc understands -c and -o together... yes
checking for c++... c++
checking whether we are using the GNU C++ compiler... yes
checking whether c++ accepts -g... yes
checking build system type... x86_64-apple-darwin16.4.0
checking host system type... x86_64-apple-darwin16.4.0
checking how to run the C preprocessor... cc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/gear_config.h usability... no
checking minix/gear_config.h presence... no
checking for minix/gear_config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking for git... /usr/bin/git
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... build-aux/install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking for style of include used by make... GNU
checking whether make supports nested variables... yes
checking whether UID '501' is supported by ustar format... yes
checking whether GID '20' is supported by ustar format... yes
checking how to create a ustar tar archive... gnutar
checking dependency style of cc... gcc3
checking dependency style of c++... gcc3
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for fgrep... /usr/bin/grep -F
checking for ld used by cc... /Library/Developer/CommandLineTools/usr/bin/ld
checking if the linker (/Library/Developer/CommandLineTools/usr/bin/ld) is GNU ld... no
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 196608
checking how to convert x86_64-apple-darwin16.4.0 file names to x86_64-apple-darwin16.4.0 format... func_convert_file_noop
checking how to convert x86_64-apple-darwin16.4.0 file names to toolchain format... func_convert_file_noop
checking for /Library/Developer/CommandLineTools/usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... no
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from cc object... ok
checking for sysroot... no
checking for a working dd... /bin/dd
checking how to truncate binary pipes... /bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for dsymutil... dsymutil
checking for nmedit... nmedit
checking for lipo... lipo
checking for otool... otool
checking for otool64... no
checking for -single_module linker flag... yes
checking for -exported_symbols_list linker flag... yes
checking for -force_load linker flag... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if cc supports -fno-rtti -fno-exceptions... yes
checking for cc option to produce PIC... -fno-common -DPIC
checking if cc PIC flag -fno-common -DPIC works... yes
checking if cc static flag -static works... no
checking if cc supports -c -o file.o... yes
checking if cc supports -c -o file.o... (cached) yes
checking whether the cc linker (/Library/Developer/CommandLineTools/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin16.4.0 dyld
checking how to hardcode library paths into programs... immediate
checking for dlopen in -ldl... yes
checking whether a program can dlopen itself... yes
checking whether a statically linked program can dlopen itself... yes
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... c++ -E
checking for ld used by c++... /Library/Developer/CommandLineTools/usr/bin/ld
checking if the linker (/Library/Developer/CommandLineTools/usr/bin/ld) is GNU ld... no
checking whether the c++ linker (/Library/Developer/CommandLineTools/usr/bin/ld) supports shared libraries... yes
checking for c++ option to produce PIC... -fno-common -DPIC
checking if c++ PIC flag -fno-common -DPIC works... yes
checking if c++ static flag -static works... no
checking if c++ supports -c -o file.o... yes
checking if c++ supports -c -o file.o... (cached) yes
checking whether the c++ linker (/Library/Developer/CommandLineTools/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin16.4.0 dyld
checking how to hardcode library paths into programs... immediate
checking for cc option to accept ISO C99... none needed
checking whether C++ compiler accepts -std=c++0x... yes
checking whether make supports nested variables... (cached) yes
checking for boostlib >= 1.39... yes
checking whether the Boost::Program_Options library is available... yes
checking for rand in -lboost_program_options-mt... yes
checking for location of cstdint... <cstdint>
<cstdint>
checking for location of cinttypes... <cinttypes>
<cinttypes>
checking for mysqld... /usr/local/bin/mysqld
checking for mysql_config... /usr/local/bin/mysql_config
checking if MySQL version is >= 5.0... yes
checking valgrind/valgrind.h usability... no
checking valgrind/valgrind.h presence... no
checking for valgrind/valgrind.h... no
checking for curl-config... /usr/bin/curl-config
checking for curl (libcurl 7.51.0) >= 7.21.7... yes
checking curl/curl.h usability... yes
checking curl/curl.h presence... yes
checking for curl/curl.h... yes
checking for main in -lcurl... yes
checking whether drizzled executable path has been provided... no
checking for drizzled... no
checking libdrizzle-5.1/drizzle_client.h usability... no
checking libdrizzle-5.1/drizzle_client.h presence... no
checking for libdrizzle-5.1/drizzle_client.h... no
checking for SQLite3 library >= 3.0.0... yes
checking for pg_config... /usr/local/bin/pg_config
checking for PostgreSQL libraries... yes
checking tcadb.h usability... no
checking tcadb.h presence... no
checking for tcadb.h... no
checking whether strerror_r is declared... yes
checking for strerror_r... yes
checking whether strerror_r returns char *... no
checking for MSG_NOSIGNAL... 
checking for MSG_DONTWAIT... yes
checking for MSG_MORE... 
checking for O_CLOEXEC... yes
checking hiredis/hiredis.h usability... no
checking hiredis/hiredis.h presence... no
checking for hiredis/hiredis.h... no
checking for dpkg-gensymbols... no
checking for openssl... openssl
checking for valgrind... no
checking for wine... no
checking for rpmbuild... no
checking for --nocheck... no
checking for rpmdev-setuptree... no
checking for rpm... no
checking for gperf... /usr/bin/gperf
checking for gawk... (cached) awk
checking whether ln -s works... yes
checking whether make sets $(MAKE)... (cached) yes
checking whether memcached executable path has been provided... no
checking for memcached... unknown
checking whether sphinx-build executable path has been provided... no
checking for sphinx-build... :
checking whether lcov executable path has been provided... no
checking for lcov... no
checking whether genhtml executable path has been provided... no
checking for genhtml... no
checking for ANSI C header files... (cached) yes
checking for sys/wait.h that is POSIX.1 compatible... yes
checking for arpa/inet.h... yes
checking for errno.h... yes
checking for execinfo.h... yes
checking for fcntl.h... yes
checking for features.h... no
checking for fnmatch.h... yes
checking for getopt.h... yes
checking for inttypes.h... (cached) yes
checking for limits.h... yes
checking for mach/mach.h... yes
checking for netdb.h... yes
checking for netinet/in.h... yes
checking for netinet/tcp.h... yes
checking for poll.h... yes
checking for pwd.h... yes
checking for signal.h... yes
checking for spawn.h... yes
checking for stdarg.h... yes
checking for stddef.h... yes
checking for stdint.h... (cached) yes
checking for stdio.h... yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking for strings.h... (cached) yes
checking for sys/epoll.h... no
checking for sys/resource.h... yes
checking for sys/socket.h... yes
checking for sys/stat.h... (cached) yes
checking for sys/time.h... yes
checking for sys/types.h... (cached) yes
checking for sys/utsname.h... yes
checking for sys/wait.h... (cached) yes
checking for syslog.h... yes
checking for unistd.h... (cached) yes
checking for winsock2.h... no
checking for libmemcached-1.0/types/return.h... no
checking for ptrdiff_t... yes
checking for an ANSI C-conforming const... yes
checking for inline... inline
checking for working volatile... yes
checking for stdbool.h that conforms to C99... yes
checking for _Bool... yes
checking whether time.h and sys/time.h may both be included... yes
checking for int32_t... yes
checking for int64_t... yes
checking for off_t... yes
checking for pid_t... yes
checking for size_t... yes
checking for ssize_t... yes
checking for uid_t in sys/types.h... yes
checking for uint16_t... yes
checking for uint32_t... yes
checking for uint64_t... yes
checking for uint8_t... yes
checking for error_at_line... no
checking vfork.h usability... no
checking vfork.h presence... no
checking for vfork.h... no
checking for fork... yes
checking for vfork... yes
checking for working fork... yes
checking for working vfork... (cached) yes
checking for working memcmp... yes
checking whether strerror_r is declared... (cached) yes
checking for strerror_r... (cached) yes
checking whether strerror_r returns char *... (cached) no
checking for vprintf... yes
checking for _doprnt... no
checking for working alloca.h... yes
checking for alloca... yes
checking for mbstate_t... yes
checking for working POSIX fnmatch... yes
checking for accept4... no
checking for dup2... yes
checking for fork... (cached) yes
checking for gettimeofday... yes
checking for memchr... yes
checking for memmove... yes
checking for memset... yes
checking for pipe... yes
checking for pipe2... no
checking for select... yes
checking for setenv... yes
checking for socket... yes
checking for strcasecmp... yes
checking for strdup... yes
checking for strerror... yes
checking for strncasecmp... yes
checking for strtol... yes
checking for strtoul... yes
checking for uname... yes
checking for atexit... yes
checking for getcwd... yes
checking for strstr... yes
checking for setsockopt... yes
checking for bind... yes
checking for clock_gettime in -lrt... no
checking for clock_gettime... yes
checking for pkg-config... /usr/local/bin/pkg-config
checking whether compiling and linking against OpenSSL works... no
checking cyassl/ssl.h usability... no
checking cyassl/ssl.h presence... no
checking for cyassl/ssl.h... no
no
checking libmemcached-1.0/memcached.h usability... no
checking libmemcached-1.0/memcached.h presence... no
checking for libmemcached-1.0/memcached.h... no
checking test for a working libevent... yes
checking uuid/uuid.h usability... yes
checking uuid/uuid.h presence... yes
checking for uuid/uuid.h... yes
checking check to see if -luuid is needed... no
checking for uuid/uuid.h... (cached) yes
checking check to see if -luuid is needed... (cached) no
checking for uuid_generate_time_safe... no
checking the number of available CPUs... 4 
configure: adding automake macro support
configure: creating aminclude.am
configure: added jobserver support to make for 5 jobs
checking if compiler needs -Werror to reject unknown flags... yes
checking whether pthreads work without any flags... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... -D_THREAD_SAFE
checking for PTHREAD_PRIO_INHERIT... yes
checking check for pthread_timedjoin_np... 
checking whether byte ordering is bigendian... no
checking for C compiler vendor... clang
checking C Compiler version... "Apple LLVM version 8.0.0 (clang-800.0.42.1)"
checking C++ Compiler version... "Apple LLVM version 8.0.0 (clang-800.0.42.1)"
checking MINGW... checking whether __MINGW32__ is declared... no
no
checking whether the linker accepts ... yes
checking for debug... no
checking for vcs system... none
checking for vcs checkout... no
checking for assert... no
checking whether the -Werror option is usable... yes
checking for simple visibility declarations... yes
checking if all warnings into errors... no
checking if all warnings into errors... (cached) no
checking whether the linker accepts -z... no
checking whether the linker accepts relro... no
checking whether the linker accepts -z... (cached) no
checking whether the linker accepts now... no
checking whether C compiler accepts -g... yes
checking whether C compiler accepts -O2... yes
checking whether C compiler accepts -Wno-unknown-pragmas... yes
checking whether C compiler accepts -Wno-pragmas... yes
checking whether C compiler accepts -Wall... yes
checking whether C compiler accepts -Wextra... yes
checking whether C compiler accepts -Weverything... no
checking whether C compiler accepts -Wthis-test-should-fail... no
checking whether C compiler accepts -Wunsuffixed-float-constants... no
checking whether C compiler accepts -Wjump-misses-init... no
checking whether C compiler accepts -Wno-attributes... yes
checking whether C compiler accepts -Waddress... yes
checking whether C compiler accepts -Wvarargs... yes
checking whether C compiler accepts -Warray-bounds... yes
checking whether C compiler accepts -Wbad-function-cast... yes
checking whether C compiler accepts -Wchar-subscripts... yes
checking whether C compiler accepts -Wcomment... yes
checking whether C compiler accepts -Wfloat-equal... yes
checking whether C compiler accepts -Wformat-security... yes
checking whether C compiler accepts -Wformat=2... yes
checking whether C compiler accepts -Wformat-y2k... yes
checking whether C compiler accepts -Wlogical-op... no
checking whether C compiler accepts -Wmaybe-uninitialized... no
checking whether C compiler accepts -Wmissing-field-initializers... yes
checking whether C compiler accepts -Wmissing-noreturn... yes
checking whether C compiler accepts -Wmissing-prototypes... yes
checking whether C compiler accepts -Wnested-externs... yes
checking whether C compiler accepts -Wnormalized=id... no
checking whether C compiler accepts -Woverride-init... no
checking whether C compiler accepts -Wpointer-arith... yes
checking whether C compiler accepts -Wpointer-sign... yes
checking whether C compiler accepts -Wredundant-decls... yes
checking whether C compiler accepts -Wshadow... yes
checking whether C compiler accepts -Wshorten-64-to-32... yes
checking whether C compiler accepts -Wsign-compare... yes
checking whether C compiler accepts -Wstrict-overflow=1... yes
checking whether C compiler accepts -Wstrict-prototypes... yes
checking whether C compiler accepts -Wswitch-enum... yes
checking whether C compiler accepts -Wtrampolines... no
checking whether C compiler accepts -Wundef... yes
checking whether C compiler accepts -funsafe-loop-optimizations... no
checking whether C compiler accepts -Wclobbered... no
checking whether C compiler accepts -Wunused... yes
checking whether C compiler accepts -Wunused-result... yes
checking whether C compiler accepts -Wunused-variable... yes
checking whether C compiler accepts -Wunused-parameter... yes
checking whether C compiler accepts -Wunused-local-typedefs... yes
checking whether C compiler accepts -Wwrite-strings... yes
checking whether C compiler accepts -fwrapv... yes
checking whether C compiler accepts -fmudflapt... no
checking whether C compiler accepts -pipe... yes
checking whether C compiler accepts -fPIE... yes
checking whether C compiler accepts -pie... no
checking whether C compiler accepts -Wsizeof-pointer-memaccess... yes
checking whether C compiler accepts -Wpacked... yes
checking whether C++ compiler accepts -g... yes
checking whether C++ compiler accepts -O2... yes
checking whether C++ compiler accepts -Wno-unknown-pragmas... yes
checking whether C++ compiler accepts -Wno-pragmas... yes
checking whether C++ compiler accepts -Wall... yes
checking whether C++ compiler accepts -Wextra... yes
checking whether C++ compiler accepts -Weverything... no
checking whether C++ compiler accepts -Wthis-test-should-fail... no
checking whether C++ compiler accepts -Wno-attributes... yes
checking whether C++ compiler accepts -Wvarargs... yes
checking whether C++ compiler accepts -Waddress... yes
checking whether C++ compiler accepts -Warray-bounds... yes
checking whether C++ compiler accepts -Wchar-subscripts... yes
checking whether C++ compiler accepts -Wcomment... yes
checking whether C++ compiler accepts -Wctor-dtor-privacy... yes
checking whether C++ compiler accepts -Wfloat-equal... yes
checking whether C++ compiler accepts -Wformat=2... yes
checking whether C++ compiler accepts -Wformat-y2k... yes
checking whether C++ compiler accepts -Wmaybe-uninitialized... no
checking whether C++ compiler accepts -Wmissing-field-initializers... yes
checking whether C++ compiler accepts -Wlogical-op... no
checking whether C++ compiler accepts -Wnon-virtual-dtor... yes
checking whether C++ compiler accepts -Wnormalized=id... no
checking whether C++ compiler accepts -Woverloaded-virtual... yes
checking whether C++ compiler accepts -Wpointer-arith... yes
checking whether C++ compiler accepts -Wredundant-decls... yes
checking whether C++ compiler accepts -Wshadow... yes
checking whether C++ compiler accepts -Wshorten-64-to-32... yes
checking whether C++ compiler accepts -Wsign-compare... yes
checking whether C++ compiler accepts -Wstrict-overflow=1... yes
checking whether C++ compiler accepts -Wswitch-enum... yes
checking whether C++ compiler accepts -Wtrampolines... no
checking whether C++ compiler accepts -Wundef... yes
checking whether C++ compiler accepts -funsafe-loop-optimizations... no
checking whether C++ compiler accepts -Wc++11-compat... yes
checking whether C++ compiler accepts -Wclobbered... no
checking whether C++ compiler accepts -Wunused... yes
checking whether C++ compiler accepts -Wunused-result... yes
checking whether C++ compiler accepts -Wunused-variable... yes
checking whether C++ compiler accepts -Wunused-parameter... yes
checking whether C++ compiler accepts -Wunused-local-typedefs... yes
checking whether C++ compiler accepts -Wwrite-strings... yes
checking whether C++ compiler accepts -Wformat-security... yes
checking whether C++ compiler accepts -fwrapv... yes
checking whether C++ compiler accepts -fmudflapt... no
checking whether C++ compiler accepts -pipe... yes
checking whether C++ compiler accepts -fPIE... yes
checking whether C++ compiler accepts -pie... no
checking whether C++ compiler accepts -Wsizeof-pointer-memaccess... yes
checking whether C++ compiler accepts -Wpacked... yes
checking whether the compiler supports GCC C++ ABI name demangling... yes
checking checking for cxx_gcc_abi_demangle... "yes"
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating libhashkit/hashkitcon.h
config.status: creating libhashkit-1.0/configure.h
config.status: creating libtest/version.h
config.status: creating libtest/yatlcon.h
config.status: creating Makefile
config.status: creating docs/source/conf.py
config.status: creating libgearman-1.0/version.h
config.status: creating scripts/gearmand
config.status: creating scripts/gearmand.xml
config.status: creating scripts/smf_install.sh
config.status: creating support/gearmand.init
config.status: creating support/gearmand.pc
config.status: creating support/gearmand.spec
config.status: creating gear_config.h
config.status: gear_config.h is unchanged
config.status: executing depfiles commands
config.status: executing libtool commands
---
Configuration summary for gearmand version 1.1.14

   * Installation prefix:       /usr/local
   * System type:               apple-darwin16.4.0
   * Host CPU:                  x86_64
   * C Compiler:                cc Apple LLVM version 8.0.0 (clang-800.0.42.1)
   * C Flags:                   -g -O2  -Wno-unknown-pragmas -Wno-pragmas -Wall -Wextra -Wno-attributes -Waddress -Wvarargs -Warray-bounds -Wbad-function-cast -Wchar-subscripts -Wcomment -Wfloat-equal -Wformat-security -Wformat=2 -Wformat-y2k -Wmissing-field-initializers -Wmissing-noreturn -Wmissing-prototypes -Wnested-externs -Wpointer-arith -Wpointer-sign -Wredundant-decls -Wshadow -Wshorten-64-to-32 -Wsign-compare -Wstrict-overflow=1 -Wstrict-prototypes -Wswitch-enum -Wundef -Wunused -Wunused-result -Wunused-variable -Wunused-parameter -Wunused-local-typedefs -Wwrite-strings -fwrapv -pipe -fPIE -Wsizeof-pointer-memaccess -Wpacked
   * C++ Compiler:              c++ Apple LLVM version 8.0.0 (clang-800.0.42.1)
   * C++ Flags:                 -g -O2 -std=c++0x -Wno-unknown-pragmas -Wno-pragmas -Wall -Wextra -Wno-attributes -Wvarargs -Waddress -Warray-bounds -Wchar-subscripts -Wcomment -Wctor-dtor-privacy -Wfloat-equal -Wformat=2 -Wformat-y2k -Wmissing-field-initializers -Wnon-virtual-dtor -Woverloaded-virtual -Wpointer-arith -Wredundant-decls -Wshadow -Wshorten-64-to-32 -Wsign-compare -Wstrict-overflow=1 -Wswitch-enum -Wundef -Wc++11-compat -Wunused -Wunused-result -Wunused-variable -Wunused-parameter -Wunused-local-typedefs -Wwrite-strings -Wformat-security -fwrapv -pipe -fPIE -Wsizeof-pointer-memaccess -Wpacked
   * CPP Flags:                  -fvisibility=hidden
   * LIBS:                      
   * LDFLAGS Flags:             
   * Assertions enabled:        no
   * Debug enabled:             no
   * Warnings as failure:       no
   * Building with libsqlite3   yes
   * Building with libdrizzle   yes
   * Building with libmemcached not found
   * Building with libpq        yes
   * Building with tokyocabinet no
   * Building with libmysql     yes
   * SSL enabled:               no
   * cyassl found:              no
   * openssl found:             no
   * make -j:                   5
   * VCS checkout:              no
   * sphinx-build:              :

---
kota@iMac—Kota ~/Downloads/gearmand-1.1.14
➜  make                                                                                                                   ruby-2.3.1 
/Library/Developer/CommandLineTools/usr/bin/make -j5  all-recursive
make[2]: warning: -jN forced in submake: disabling jobserver mode.
  CXX      libgearman/libgearman_libgearman_la-check.lo
  CXX      libgearman/libgearman_libgearman_la-actions.lo
  CXX      libgearman/libgearman_libgearman_la-aggregator.lo
  CXX      libgearman/libgearman_libgearman_la-allocator.lo
  CXX      libgearman/libgearman_libgearman_la-argument.lo
In file included from libgearman/actions.cc:39:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:46:10: error: expected ')'
uint64_t ntohll(uint64_t);
         ^
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:30: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
                             ^
./libgearman/byteorder.h:46:10: note: to match this '('
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:5: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
    ^
In file included from libgearman/actions.cc:39:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:46:10: error: functions that differ only in their return type cannot be overloaded
uint64_t ntohll(uint64_t);
         ^
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
./libgearman/byteorder.h:46:10: note: previous implicit declaration is here
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
In file included from libgearman/actions.cc:39:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:49:10: error: expected ')'
uint64_t htonll(uint64_t);
         ^
/usr/include/sys/_endian.h:141:25: note: expanded from macro 'htonll'
#define htonll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:30: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
                             ^
./libgearman/byteorder.h:49:10: note: to match this '('
/usr/include/sys/_endian.h:141:25: note: expanded from macro 'htonll'
#define htonll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:5: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
    ^
In file included from libgearman/actions.cc:39:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:49:10: error: functions that differ only in their return type cannot be overloaded
uint64_t htonll(uint64_t);
         ^
/usr/include/sys/_endian.h:141:25: note: expanded from macro 'htonll'
#define htonll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
./libgearman/byteorder.h:46:10: note: previous implicit declaration is here
uint64_t ntohll(uint64_t);
         ^
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
In file included from libgearman/check.cc:40:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:46:10: error: expected ')'
uint64_t ntohll(uint64_t);
         ^
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:30: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
                             ^
./libgearman/byteorder.h:46:10: note: to match this '('
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:5: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
    ^
In file included from libgearman/check.cc:40:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:46:10: error: functions that differ only in their return type cannot be overloaded
uint64_t ntohll(uint64_t);
         ^
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
./libgearman/byteorder.h:46:10: note: previous implicit declaration is here
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
In file included from libgearman/check.cc:40:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:49:10: error: expected ')'
uint64_t htonll(uint64_t);
         ^
/usr/include/sys/_endian.h:141:25: note: expanded from macro 'htonll'
#define htonll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:30: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
                             ^
./libgearman/byteorder.h:49:10: note: to match this '('
/usr/include/sys/_endian.h:141:25: note: expanded from macro 'htonll'
#define htonll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:5: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
    ^
In file included from libgearman/check.cc:40:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:49:10: error: functions that differ only in their return type cannot be overloaded
uint64_t htonll(uint64_t);
         ^
/usr/include/sys/_endian.h:141:25: note: expanded from macro 'htonll'
#define htonll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
./libgearman/byteorder.h:46:10: note: previous implicit declaration is here
uint64_t ntohll(uint64_t);
         ^
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
In file included from libgearman/aggregator.cc:39:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:46:10: error: expected ')'
uint64_t ntohll(uint64_t);
         ^
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:30: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
                             ^
./libgearman/byteorder.h:46:10: note: to match this '('
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:5: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
    ^
In file included from libgearman/aggregator.cc:39:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:46:10: error: functions that differ only in their return type cannot be overloaded
uint64_t ntohll(uint64_t);
         ^
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
./libgearman/byteorder.h:46:10: note: previous implicit declaration is here
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
In file included from libgearman/aggregator.cc:39:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:49:10: error: expected ')'
uint64_t htonll(uint64_t);
         ^
/usr/include/sys/_endian.h:141:25: note: expanded from macro 'htonll'
#define htonll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:30: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
                             ^
./libgearman/byteorder.h:49:10: note: to match this '('
/usr/include/sys/_endian.h:141:25: note: expanded from macro 'htonll'
#define htonll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:5: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
    ^
In file included from libgearman/aggregator.cc:39:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:49:10: error: functions that differ only in their return type cannot be overloaded
uint64_t htonll(uint64_t);
         ^
/usr/include/sys/_endian.h:141:25: note: expanded from macro 'htonll'
#define htonll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
./libgearman/byteorder.h:46:10: note: previous implicit declaration is here
uint64_t ntohll(uint64_t);
         ^
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
4 errors generated.
make[2]: *** [libgearman/libgearman_libgearman_la-actions.lo] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from libgearman/allocator.cc:38:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:46:10: error: expected ')'
uint64_t ntohll(uint64_t);
         ^
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:30: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
                             ^
./libgearman/byteorder.h:46:10: note: to match this '('
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:5: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
    ^
In file included from libgearman/allocator.cc:38:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:46:10: error: functions that differ only in their return type cannot be overloaded
uint64_t ntohll(uint64_t);
         ^
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
./libgearman/byteorder.h:46:10: note: previous implicit declaration is here
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
In file included from libgearman/allocator.cc:38:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:49:10: error: expected ')'
uint64_t htonll(uint64_t);
         ^
/usr/include/sys/_endian.h:141:25: note: expanded from macro 'htonll'
#define htonll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:30: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
                             ^
./libgearman/byteorder.h:49:10: note: to match this '('
/usr/include/sys/_endian.h:141:25: note: expanded from macro 'htonll'
#define htonll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:5: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
    ^
In file included from libgearman/allocator.cc:38:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:49:10: error: functions that differ only in their return type cannot be overloaded
uint64_t htonll(uint64_t);
         ^
/usr/include/sys/_endian.h:141:25: note: expanded from macro 'htonll'
#define htonll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
./libgearman/byteorder.h:46:10: note: previous implicit declaration is here
uint64_t ntohll(uint64_t);
         ^
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
In file included from libgearman/argument.cc:39:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:46:10: error: expected ')'
uint64_t ntohll(uint64_t);
         ^
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:30: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
                             ^
./libgearman/byteorder.h:46:10: note: to match this '('
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:5: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
    ^
In file included from libgearman/argument.cc:39:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:46:10: error: functions that differ only in their return type cannot be overloaded
uint64_t ntohll(uint64_t);
         ^
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
./libgearman/byteorder.h:46:10: note: previous implicit declaration is here
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
In file included from libgearman/argument.cc:39:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:49:10: error: expected ')'
uint64_t htonll(uint64_t);
         ^
/usr/include/sys/_endian.h:141:25: note: expanded from macro 'htonll'
#define htonll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:30: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
                             ^
./libgearman/byteorder.h:49:10: note: to match this '('
/usr/include/sys/_endian.h:141:25: note: expanded from macro 'htonll'
#define htonll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:5: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
    ^
In file included from libgearman/argument.cc:39:
In file included from ./libgearman/common.h:64:
./libgearman/byteorder.h:49:10: error: functions that differ only in their return type cannot be overloaded
uint64_t htonll(uint64_t);
         ^
/usr/include/sys/_endian.h:141:25: note: expanded from macro 'htonll'
#define htonll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
./libgearman/byteorder.h:46:10: note: previous implicit declaration is here
uint64_t ntohll(uint64_t);
         ^
/usr/include/sys/_endian.h:140:25: note: expanded from macro 'ntohll'
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
                        ^
/usr/include/libkern/_OSByteOrder.h:78:6: note: expanded from macro '__DARWIN_OSSwapInt64'
    (__builtin_constant_p(x) ? __DARWIN_OSSwapConstInt64(x) : _OSSwapInt64(x))
     ^
4 errors generated.
4 errors generated.
4 errors generated.
make[2]: *** [libgearman/libgearman_libgearman_la-allocator.lo] Error 1
make[2]: *** [libgearman/libgearman_libgearman_la-argument.lo] Error 1
make[2]: *** [libgearman/libgearman_libgearman_la-check.lo] Error 1
4 errors generated.
make[2]: *** [libgearman/libgearman_libgearman_la-aggregator.lo] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
kota@iMac—Kota ~/Downloads/gearmand-1.1.14
➜                                         

A free license for docs ?

Hi All,
Is there a way to change the license for the documentation to a more permissive one ?
The current license forbids Non-Commercial [0] use and thus the documentation can not be included into distributions such as Debian.
Moreover the man pages are build from the rst files as far as I see and are also non-free due to the source license.

[0] http://gearman.info/license.html

Thank you,
Alex

Feature: Job Scheduling

Hi,
We are working on a Golang implementation of Gearman Server (https://github.com/appscode/g2). Our backend requires the option of job scheduling. In our implementation, we are using SUBMIT_JOB_SCHED code to schedule job. But Gearman protocol does not define any REQ/RES types for canceling schedule jobs. We think 2 new codes are needed for that:

• REQ: CANCEL_JOB_SCHED
• RES: JOB_CANCELLED

Can these codes be added to the gearman protocol spec?

cc: @sadlil @ashiquzzaman33

client.fork.fork() test fails in some envorinments

@esabol mentioned the issue in the group.

I tested it on Ubuntu

• 14.04 gcc 4.8.4 test passed
• 16.04 5.4.0 test failed

Putting blamed ASSERT_EQ in try block

diff --git a/tests/libgearman-1.0/fork.cc b/tests/libgearman-1.0/fork.cc
index 87d5a2c..2d5a88a 100644
--- a/tests/libgearman-1.0/fork.cc
+++ b/tests/libgearman-1.0/fork.cc
@@ -105,8 +105,12 @@ test_return_t check_client_fork_TEST(void*)
             }
             ASSERT_EQ(EXIT_SUCCESS, exit_code);
           }
-
-          ASSERT_EQ(GEARMAN_SUCCESS, gearman_client_echo(&client, test_literal_param("parent forked")));
+          try{
+                ASSERT_EQ(GEARMAN_SUCCESS, gearman_client_echo(&client, test_literal_param("parent forked")));
+          } catch (const libtest::__test_result& err)
+          {
+                  Error << gearman_client_error(&client);
+          }
         }
     }
     return TEST_SUCCESS;

shows the reason:
tests/libgearman-1.0/fork.cc:113: in check_client_fork_TEST() pid(31365) flush(Broken pipe) lost connection to server during send -> libgearman/connection.cc:903: pid(31365)

Paradoxically neither gearman_client_echo in try block nor ASSERT_EQ failed if I change fork.cc in this way:

index 87d5a2c..0cb174f 100644
--- tests/libgearman-1.0/fork.cc
+++ tests/libgearman-1.0/fork.cc
@@ -105,7 +105,12 @@ test_return_t check_client_fork_TEST(void*)
             }
             ASSERT_EQ(EXIT_SUCCESS, exit_code);
           }
-
+          try{
+                gearman_client_echo(&client, test_literal_param("parent forked"));
+          } catch (const libtest::__test_result& err)
+          {
+                  Error << gearman_client_error(&client);
+          }
           ASSERT_EQ(GEARMAN_SUCCESS, gearman_client_echo(&client, test_literal_param("parent forked")));
         }
     }

ChangeLog in 'make dist' tarballs does not show the top tag

Add "prioritystatus" to report status broken down by priority

For monitoring our gearman server we would like to know how many jobs are currently queued, broken down by priority. Then we can alert quickly if high prio jobs are piling up.

Usage could look like this:

$ gearadmin --priority-status
HIGH	12	7
NORMAL	14	3
LOW	0	0

With these columns:

• Priority name
• Number in queue
• Number of jobs running

@SpamapS wrote on the mailing list:

we'd love to see a patch for that as an addition to the admin protocol. The information is definitely readily available in the data structure, since each priority is in fact its own linked list.

Any patch would need to include updates to gearmand/PROTOCOL to explain the difference in the Administrative protocol.

cc @ppp0 @kris-lab @fvovan maybe something for the next Research Day (December 14th)?

risk of data truncation and/or overrun when logging from a buffer bigger than GEARMAN_MAX_ERROR_SIZE

There is a risk of data truncation.
`
gearmand_log(....)
{
....
char log_buffer[GEARMAN_MAX_ERROR_SIZE*2] = { 0 };
......
if (Gearmand() and Gearmand()->log_fn)
{
Gearmand()->log_fn(log_buffer, verbose, (void *)Gearmand()->log_context);
}
}

The callback function:
static void _log(const char *mesg, gearmand_verbose_t verbose, void *context)
{
gearmand_log_info_st *log_info= static_cast<gearmand_log_info_st *>(context);

log_info->write(verbose, mesg);
}

void write(gearmand_verbose_t verbose, const char *mesg)
{
if (opt_file)
{
char buffer[GEARMAN_MAX_ERROR_SIZE];
int buffer_length= snprintf(buffer, GEARMAN_MAX_ERROR_SIZE, "%7s %s\n", gearmand_verbose_name(verbose), mesg);
.....
}
}
`

libhostile test is non-deterministic and does not provide a random seed to reproduce fails

Every once in a while, we do see a failure of the libhostile tests. But it seems that it runs with randomized data, so there is a problem, but we'll have to run thousands of times again to see it, and then we'll need to get the random seed out to reproduce over and over.

"make test" doesn't explicitly test SSL connections

Yes, there are SSL tests in the test suite, but they're part of the existing tests. It would be nice if there was an explicit "ssl" test that would SKIP if the compilation did not have "SSL" enabled and "PASS" if SSL connections are working.

I spent a couple hours yesterday trying to debug why my SSL connections weren't working before I realized that SSL wasn't enabled (even though I compiled with "--enable-ssl"). It turns out my Ubuntu Docker container did not have the libssl-dev package installed, so "--enable-ssl" failed to actually enable SSL. If there was an explicit SSL test in the test suite, I would have noticed it a lot sooner.

Cannot ./configure

vagrant@vagrant-ubuntu-trusty-64:/tmp/gearmand-1.1.12$ sudo ./configure
checking for cc... cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether cc accepts -g... yes
checking for cc option to accept ISO C89... none needed
checking for c++... no
checking for g++... no
checking for clang++... no
checking whether we are using the GNU C++ compiler... no
checking whether g++ accepts -g... no
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to run the C preprocessor... cc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/gear_config.h usability... no
checking minix/gear_config.h presence... no
checking for minix/gear_config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for style of include used by make... GNU
checking whether make supports nested variables... yes
checking whether UID '0' is supported by ustar format... yes
checking whether GID '0' is supported by ustar format... yes
checking how to create a ustar tar archive... gnutar
checking dependency style of cc... gcc3
checking dependency style of g++... none
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for fgrep... /bin/grep -F
checking for ld used by cc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from cc object... ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking for dlfcn.h... yes
checking for objdir... .libs
checking if cc supports -fno-rtti -fno-exceptions... no
checking for cc option to produce PIC... -fPIC -DPIC
checking if cc PIC flag -fPIC -DPIC works... yes
checking if cc static flag -static works... yes
checking if cc supports -c -o file.o... yes
checking if cc supports -c -o file.o... (cached) yes
checking whether the cc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for shl_load... no
checking for shl_load in -ldld... no
checking for dlopen... no
checking for dlopen in -ldl... yes
checking whether a program can dlopen itself... yes
checking whether a statically linked program can dlopen itself... no
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking for cos in -lm... yes
checking for cc option to accept ISO C99... -std=gnu99
checking whether C++ compiler accepts -std=c++0x... no
checking whether make supports nested variables... (cached) yes
checking for boostlib >= 1.39... configure: We could not detect the boost libraries (version 1.39 or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option.  If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.
configure: error: could not find boost

vagrant@vagrant-ubuntu-trusty-64:/tmp/gearmand-1.1.12$ sudo apt-get install libboost-all-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
libboost-all-dev is already the newest version.
The following packages were automatically installed and are no longer required:
  libnotify-bin libxaw7 libxkbfile1 libxmu6 x11-xkb-utils xfonts-base
  xserver-common xserver-xorg-core
Use 'apt-get autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.

vagrant@vagrant-ubuntu-trusty-64:/tmp/gearmand-1.1.12$ ls /usr/lib/x86_64-linux-gnu/libboost*
/usr/lib/x86_64-linux-gnu/libboost_atomic.a
/usr/lib/x86_64-linux-gnu/libboost_atomic.so
/usr/lib/x86_64-linux-gnu/libboost_atomic.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_chrono.a
/usr/lib/x86_64-linux-gnu/libboost_chrono.so
/usr/lib/x86_64-linux-gnu/libboost_chrono.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_context.a
/usr/lib/x86_64-linux-gnu/libboost_context.so
/usr/lib/x86_64-linux-gnu/libboost_context.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_coroutine.a
/usr/lib/x86_64-linux-gnu/libboost_date_time.a
/usr/lib/x86_64-linux-gnu/libboost_date_time.so
/usr/lib/x86_64-linux-gnu/libboost_date_time.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_exception.a
/usr/lib/x86_64-linux-gnu/libboost_filesystem.a
/usr/lib/x86_64-linux-gnu/libboost_filesystem.so
/usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_graph.a
/usr/lib/x86_64-linux-gnu/libboost_graph_parallel.a
/usr/lib/x86_64-linux-gnu/libboost_graph_parallel.so
/usr/lib/x86_64-linux-gnu/libboost_graph_parallel.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_graph.so
/usr/lib/x86_64-linux-gnu/libboost_graph.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_iostreams.a
/usr/lib/x86_64-linux-gnu/libboost_iostreams.so
/usr/lib/x86_64-linux-gnu/libboost_iostreams.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_locale.a
/usr/lib/x86_64-linux-gnu/libboost_locale.so
/usr/lib/x86_64-linux-gnu/libboost_locale.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_log.a
/usr/lib/x86_64-linux-gnu/libboost_log_setup.a
/usr/lib/x86_64-linux-gnu/libboost_log_setup.so
/usr/lib/x86_64-linux-gnu/libboost_log_setup.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_log.so
/usr/lib/x86_64-linux-gnu/libboost_log.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_math_c99.a
/usr/lib/x86_64-linux-gnu/libboost_math_c99f.a
/usr/lib/x86_64-linux-gnu/libboost_math_c99f.so
/usr/lib/x86_64-linux-gnu/libboost_math_c99f.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_math_c99l.a
/usr/lib/x86_64-linux-gnu/libboost_math_c99l.so
/usr/lib/x86_64-linux-gnu/libboost_math_c99l.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_math_c99.so
/usr/lib/x86_64-linux-gnu/libboost_math_c99.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_math_tr1.a
/usr/lib/x86_64-linux-gnu/libboost_math_tr1f.a
/usr/lib/x86_64-linux-gnu/libboost_math_tr1f.so
/usr/lib/x86_64-linux-gnu/libboost_math_tr1f.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_math_tr1l.a
/usr/lib/x86_64-linux-gnu/libboost_math_tr1l.so
/usr/lib/x86_64-linux-gnu/libboost_math_tr1l.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_math_tr1.so
/usr/lib/x86_64-linux-gnu/libboost_math_tr1.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_mpi.a
/usr/lib/x86_64-linux-gnu/libboost_mpi_python.a
/usr/lib/x86_64-linux-gnu/libboost_mpi_python-py27.a
/usr/lib/x86_64-linux-gnu/libboost_mpi_python-py27.so
/usr/lib/x86_64-linux-gnu/libboost_mpi_python-py27.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_mpi_python-py34.a
/usr/lib/x86_64-linux-gnu/libboost_mpi_python-py34.so
/usr/lib/x86_64-linux-gnu/libboost_mpi_python-py34.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_mpi_python.so
/usr/lib/x86_64-linux-gnu/libboost_mpi.so
/usr/lib/x86_64-linux-gnu/libboost_mpi.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_prg_exec_monitor.a
/usr/lib/x86_64-linux-gnu/libboost_prg_exec_monitor.so
/usr/lib/x86_64-linux-gnu/libboost_prg_exec_monitor.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_program_options.a
/usr/lib/x86_64-linux-gnu/libboost_program_options.so
/usr/lib/x86_64-linux-gnu/libboost_program_options.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_python.a
/usr/lib/x86_64-linux-gnu/libboost_python-py27.a
/usr/lib/x86_64-linux-gnu/libboost_python-py27.so
/usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_python-py34.a
/usr/lib/x86_64-linux-gnu/libboost_python-py34.so
/usr/lib/x86_64-linux-gnu/libboost_python-py34.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_python.so
/usr/lib/x86_64-linux-gnu/libboost_random.a
/usr/lib/x86_64-linux-gnu/libboost_random.so
/usr/lib/x86_64-linux-gnu/libboost_random.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_regex.a
/usr/lib/x86_64-linux-gnu/libboost_regex.so
/usr/lib/x86_64-linux-gnu/libboost_regex.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_serialization.a
/usr/lib/x86_64-linux-gnu/libboost_serialization.so
/usr/lib/x86_64-linux-gnu/libboost_serialization.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_signals.a
/usr/lib/x86_64-linux-gnu/libboost_signals.so
/usr/lib/x86_64-linux-gnu/libboost_signals.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_system.a
/usr/lib/x86_64-linux-gnu/libboost_system.so
/usr/lib/x86_64-linux-gnu/libboost_system.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_test_exec_monitor.a
/usr/lib/x86_64-linux-gnu/libboost_thread.a
/usr/lib/x86_64-linux-gnu/libboost_thread.so
/usr/lib/x86_64-linux-gnu/libboost_thread.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_timer.a
/usr/lib/x86_64-linux-gnu/libboost_timer.so
/usr/lib/x86_64-linux-gnu/libboost_timer.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_unit_test_framework.a
/usr/lib/x86_64-linux-gnu/libboost_unit_test_framework.so
/usr/lib/x86_64-linux-gnu/libboost_unit_test_framework.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_wave.a
/usr/lib/x86_64-linux-gnu/libboost_wave.so
/usr/lib/x86_64-linux-gnu/libboost_wave.so.1.54.0
/usr/lib/x86_64-linux-gnu/libboost_wserialization.a
/usr/lib/x86_64-linux-gnu/libboost_wserialization.so
/usr/lib/x86_64-linux-gnu/libboost_wserialization.so.1.54.0

vagrant@vagrant-ubuntu-trusty-64:/tmp/gearmand-1.1.12$ uname -a
Linux vagrant-ubuntu-trusty-64 3.13.0-91-generic #138-Ubuntu SMP Fri Jun 24 17:00:34 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

./configure --with-boost-libdir=/usr/lib/x86_64-linux-gnu gives same error

any ideas?

Implement priorities for redis queues

Currently the priority argument for a job is silently ignored for redis and jobs replayed from redis are all added at the normal priority.

A possible solution would be to use hashes. If instead of calling SET key data the code called HMSET key data "data" priority "priority", and HGETALL key in the replay method, it would be possible to store and retrieve the priority from redis.

make test-stress never ends

make test-stress
stress.echo.echo                                        0:000422052 [ ok ]

The same result on Ubuntu 14.04 and 16.04.

SSL failure(error:00000005:lib(0):func(0):DH lib) if client/worker closes socket connection

As mentioned in groups gearmand running with ssl support complains permanently about failed DH key exchange.

SSL failure(error:00000005:lib(0):func(0):DH lib) errno:0

SSL_CTX_set_cipher_list should be implemented in gearmand in proper way to solve the issue

gearman.errors.ServerUnavailable: Found no valid connections in list

I use the centOS6.5 and python2.7.9, start up with the command of gearmand -d , I 'm failed all the time.

log:
File "schedulerTask.py", line 345, in
gm_worker.work()
File "/usr/local/lib/python2.7/site-packages/gearman/worker.py", line 83, in work
continue_working = self.poll_connections_until_stopped(worker_connections, continue_while_connections_alive, timeout=poll_timeout)
File "/usr/local/lib/python2.7/site-packages/gearman/connection_manager.py", line 203, in poll_connections_until_stopped
raise ServerUnavailable('Found no valid connections in list: %r' % self.connection_list)
gearman.errors.ServerUnavailable: Found no valid connections in list: []

what'w wrong with me ?
can you give me some advice?

libgearman isn't hashing submitted jobs based on unique

In the original gearman client, the client would take the list of servers it knows about, hash the unique ID of each submitted job, and choose one of the servers in the list based on the hash of the unique.

This code seems to have been lost, and no tests enforce it. This is an important part of what makes gearman awesome, so it needs to be resurrected.

make a protocol asynchronous

I'm implementing Typescript client/worker library from scratch and I don't like the idea to "lock" a socket to do not mess the package order.

Please adjust the protocol to have a way to tie a request to a response.
Example:
Client sends:
SUBMIT_JOB (functionName, UniqueId, Data);
Server responses
JOB_CREATED(handle)

This forcing a client to "lock" after request and wait for "JOB_CREATED"
OR queue all the requests and rely on request order (if it is reliable).

Instead I would like to have this:
I would like to send an GUID with each request, and get it back in the response.
Client sends:
SUBMIT_JOB (functionName, UniqueId, GUID, Data);
Server responses
JOB_CREATED(handle, GUID)

This is true for each (request,response) pair.

Please also update a protocol documentation:
Do you guaranty the request/response order to be the same in single/multi-threaded environment?

No output has been received in the last 10m0s

Hi
I change pull request, and build got the error:

https://travis-ci.org/gearman/gearmand/jobs/177850613

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received

gearmand can't start

[root@localhost sbin]# gearmand -d -p 4730
[root@localhost sbin]# lsof -i:4730
[root@localhost sbin]# uname -a
Linux localhost.localdomain 3.10.0-327.36.3.el7.x86_64 #1 SMP Mon Oct 24 16:09:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
[root@localhost sbin]# gearmand

Begin stack trace, frames found: 7
?0 0x7fbff1a74f23 in gearmand(+0x1af23) [0x7fbff1a74f23]
?1 0x7fbff1a66147 in gearmand(+0xc147) [0x7fbff1a66147]
?2 0x7fbff0677100 in /lib64/libpthread.so.0(+0xf100) [0x7fbff0677100]
?3 0x7fbff1a777d3 in gearmand(+0x1d7d3) [0x7fbff1a777d3]
?4 0x7fbff1a659bf in gearmand(+0xb9bf) [0x7fbff1a659bf]
#5 0x7fbfef429b15 in __libc_start_main at /lib64/libc.so.6
?6 0x7fbff1a65fc5 in gearmand(+0xbfc5) [0x7fbff1a65fc5]
[root@localhost sbin]# gearadmin --status
Error: Error connecting to localhost.

test t/cycle sometimes gets stuck waiting for a gearmand to die

While investigating #50 I ran into this, which we've seen in travis as well. Sometimes the test just hangs forever. It seems to get stuck waiting for a gearmand to die after a SIGTERM, so I suspect there are instances where gearmand misses SIGTERM's:

Here is the output of t/cycle while stuck. Note that it does not receive my ^C:

cycle.kill.kill					2:000106912 [ ok ]
cycle.worker.single startup/shutdown					0:002458791 [ ok ]
cycle.server_startup().server_startup(1)					1:001918107 [ ok ]
^C

Here is a backtrace of t/cycle while it is stuck:

#0  0x00007f2abdb0cfcb in __waitpid (pid=7175, stat_loc=stat_loc@entry=0x557300461728, options=options@entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
#1  0x00005572ff235c07 in libtest::Application::join (this=this@entry=0x5573004615f8) at libtest/cmdline.cc:443
#2  0x00005572ff237ed2 in libtest::Application::murder (this=this@entry=0x5573004615f8) at libtest/cmdline.cc:327
#3  0x00005572ff245f13 in libtest::Server::kill (this=0x5573004614c0) at libtest/server.cc:532
#4  0x00005572ff2465ae in libtest::Server::kill (this=<optimized out>) at libtest/server.cc:530
#5  0x00005572ff225cda in libtest::server_startup_st::shutdown (this=0x557300452658) at libtest/server_container.cc:143
#6  0x00005572ff21afb0 in __server_startup_TEST (count=count@entry=20, context=<optimized out>) at tests/cycle.cc:100
#7  0x00005572ff21b094 in server_startup_multiple_TEST (obj=<optimized out>) at tests/cycle.cc:113
#8  0x00005572ff21a10e in libtest::Runner::main (this=<optimized out>, func=<optimized out>, object=<optimized out>) at libtest/runner.cc:51
#9  0x00005572ff23ae03 in runner_code (run=<optimized out>, _timer=..., frame=0x557300452610) at libtest/collection.cc:59
#10 libtest::Collection::exec (this=0x557300452dc0) at libtest/collection.cc:210
#11 0x00005572ff21ee48 in libtest::Framework::exec (this=0x557300452610) at libtest/framework.cc:217
#12 0x00005572ff219c8c in main (argc=<optimized out>, argv=<optimized out>, environ_=<optimized out>) at libtest/main.cc:382

And here is pid 7175 from above's backtrace:

#0  0x00007f0ce1e07e23 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84
#1  0x00007f0ce315b138 in ?? () from /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5
#2  0x00007f0ce314638a in event_base_loop () from /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5
#3  0x000055cbbc17319d in gearmand_run (gearmand=0x55cbbdb85950) at libgearman-server/gearmand.cc:408
#4  0x000055cbbc15e7cd in main (argc=6, argv=0x7ffc3a983088) at gearmand/gearmand.cc:454

The file for 7175 was this:

   INFO 2016-11-30 01:50:32.834979 [  main ] Initializing Gear on port 32903 with SSL: false
   INFO 2016-11-30 01:50:32.000000 [  main ] Starting up with pid 7175, verbose is set to INFO
   INFO 2016-11-30 01:50:32.000000 [  main ] Retrying bind(Address already in use) on 127.0.0.1:32903 0 + 1 >= 30
   INFO 2016-11-30 01:50:32.000000 [  main ] Retrying bind(Address already in use) on 127.0.0.1:32903 1 + 2 >= 30
   INFO 2016-11-30 01:50:34.000000 [  main ] Retrying bind(Address already in use) on 127.0.0.1:32903 3 + 4 >= 30
   INFO 2016-11-30 01:50:38.000000 [  main ] Listening on 127.0.0.1:32903 (12)
   INFO 2016-11-30 01:50:38.000000 [  main ] Adding event for listening socket (12)
   INFO 2016-11-30 05:21:33.000000 [  main ] Clearing event for listening socket (12)
   INFO 2016-11-30 05:21:33.000000 [  main ] Closing listening socket (12)
   INFO 2016-11-30 05:21:33.000000 [  main ] Shutdown complete

Note that I left it running for a few hours, and came back, and manually sent it a SIGTERM at 05:21:33 after creating the backtrace from above.

Dist tarballs do not include any docs

Hey,

your .gitignore contains:

Makefile

instead of

/Makefile

and thus /docs/Makefile gets killed when doing release tarball.

redis queue is not support reconnect redis server

Hi
Everyone

I use redis as persistence queue, and have a problem that redis queue is not support reconnect redis server.

• If gearmand is running , The network is disconnect or redis server restart.

This can lead to got error:

failed to insert 'gear-reverse-c9984240-b20d-11e6-9924-080027596944' into redis _hiredis_add(QUEUE_ERROR)

./configure file missing from 1.1.13

The ./configure file and the build-aux directories are missing from the 1.1.13 release.

Running the ./bootstrap.sh autoreconf won't work when the directory is not a git checkout (which is the case when building the openSUSE package).

Would it be possible to re-add these files?

Add a config option for a redis key prefix to allow multiple gearmand instances to use the same redis server

Currently it's not possible to have two (or more) separate gearmand instances use the same redis server as a queue backend as the keys may conflict, and restarting either gearmand instance will cause it to load all the jobs created on both instances.

A solution to this would be to add a config option to specify a key prefix, e.g. --redis-key-prefix, and prefix all of the entries in redis with this prefix.

"configure --enable-ssl" does not fail if SSL libraries/headers are not found

If you do something like ./configure --enable-ssl && make, configure should exit with a non-zero exit status if the appropriate SSL libraries and headers cannot be found. Otherwise, gearmand will build, and there will be the expectation that gearmand was built with SSL support when it wasn't.

Sporadic SSL connection and handshake failures

If I start up 10 workers and they quasi-simultaneously try to connect to gearmand, a couple (but not all) of the workers inevitably report one of the following SSL errors:

error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
error:140940F6:SSL routines:SSL3_READ_BYTES:unknown alert type

The gearmand logs will typically show something like:

ERROR 2016-10-20 20:12:31.000000 [ 4 ] Error occured on SSL_accept()(Broken pipe) -> libgearman-server/plugins/protocol/gear/protocol.cc:391
ERROR 2016-10-20 20:12:31.000000 [ 4 ] ::3530:3135:3200:0:50154 _con_add() has failed, please report any crashes that occur immediatly after this. gearmand_con_check_queue(ERRNO) -> libgearman-server/gearmand_con.cc:774
ERROR 2016-10-20 20:12:31.000000 [ 4 ] con 140361343196112 is already cleaned-up. returning -> libgearman-server/connection.cc:225

Based on my web searches, I see reports of other multithreaded applications which have encountered these same errors. For example, an Ichinga developer at https://dev.icinga.org/issues/6981 wrote:

"Alright, so the problem here is that when SSL_read/SSL_write fail because they can't read/write from/to the socket you must call SSL_read/SSL_write again. However in Icinga there exists a race condition where SSL_read -> SSL_read calls might be interleaved with SSL_write calls from another thread. The first fix for this seemingly fixed this however it introduced a potential deadlock. The new fix should work properly though."

Icinga uses Boost, same as (parts of?) gearmand. If you take a look at
https://dev.icinga.org/projects/i2/repository/revisions/9133c01f008cdb729a4d8081ff7d9fac8452a319/diff/ , it appears the solution for Icinga was to add boost::mutex::scoped_locks in all of their routines that call either SSL_read() or SSL_write() in order to prevent one thread from calling SSL_write() while the other thread is in the middle of a SSL_read() operation.

Protocol addition to bulk register jobs (x CAN_DO -> 1 MASS_DO)

OpenStack Zuul relies on Gearman to run jobs on a farm of 800 nodes each having 20000 functions. The massive amount of CAN_DO cause stress to the server. @jeblair went with an expansion of the Gearman protocol that let ones register multiple functions in a single MASS_DO command.

Their software is in python and the protocol expansion is handled via child class: openstack-infra/zuul@d437159 . For the worker:

class GearWorker(gear.Worker):
    MASS_DO = 101

    def sendMassDo(self, functions):
        data = b'\x00'.join([gear.convert_to_bytes(x) for x in functions])
        self.broadcast_lock.acquire()
        try:
            p = gear.Packet(gear.constants.REQ, self.MASS_DO, data)
            self.broadcast(p)
        finally:
            self.broadcast_lock.release()

On the server side:

    def handleMassDo(self, packet):
        packet.connection.functions = set()
        for name in packet.data.split(b'\x00'):
            self.log.debug("Adding function %s to %s" % (
                name, packet.connection))
            packet.connection.functions.add(name)
            self.functions.add(name)

So that is merely \x00 joining the payload of several CAN_DO packets in a single one. That saves you all the overhead of a packet handling.

Would that be a feature that could be added to the reference Gearman protocol?

A bulk equivalent for CAN_DO_TIMEOUT could be of interest as well.

gearman can't use redis as persistence queue

Version:
gearmand 1.1.13

Start gearmand command:
exec start-stop-daemon --start --chuid gearman --exec /usr/sbin/gearmand -- --log-file=/var/log/gearman-job-server/gearman.log -q redis --redis-server 192.168.99.100 --redis-port 32768

Got error:
ERROR 2016-11-16 10:24:38.000000 [ main ] Failed to call KEYS during QUEUE replay _hiredis_replay(QUEUE_ERROR) -> libgearman-server/plugins/queue/redis/queue.cc:258
ERROR 2016-11-16 10:24:38.000000 [ main ] failed to reload queue gearmand_run(QUEUE_ERROR) -> libgearman-server/gearmand.cc:395
ERROR 2016-11-16 10:24:38.000000 [ main ] Failed to call KEYS during QUEUE replay _hiredis_replay(QUEUE_ERROR) -> libgearman-server/plugins/queue/redis/queue.cc:258
ERROR 2016-11-16 10:24:38.000000 [ main ] failed to reload queue gearmand_run(QUEUE_ERROR) -> libgearman-server/gearmand.cc:395
ERROR 2016-11-16 10:24:38.000000 [ main ] Failed to call KEYS during QUEUE replay _hiredis_replay(QUEUE_ERROR) -> libgearman-server/plugins/queue/redis/queue.cc:258
ERROR 2016-11-16 10:24:38.000000 [ main ] failed to reload queue gearmand_run(QUEUE_ERROR) -> libgearman-server/gearmand.cc:395
ERROR 2016-11-16 10:24:38.000000 [ main ] Failed to call KEYS during QUEUE replay _hiredis_replay(QUEUE_ERROR) -> libgearman-server/plugins/queue/redis/queue.cc:258
ERROR 2016-11-16 10:24:38.000000 [ main ] failed to reload queue gearmand_run(QUEUE_ERROR) -> libgearman-server/gearmand.cc:395
ERROR 2016-11-16 10:24:38.000000 [ main ] Failed to call KEYS during QUEUE replay _hiredis_replay(QUEUE_ERROR) -> libgearman-server/plugins/queue/redis/queue.cc:258
ERROR 2016-11-16 10:24:38.000000 [ main ] failed to reload queue gearmand_run(QUEUE_ERROR) -> libgearman-server/gearmand.cc:395
ERROR 2016-11-16 10:24:38.000000 [ main ] Failed to call KEYS during QUEUE replay _hiredis_replay(QUEUE_ERROR) -> libgearman-server/plugins/queue/redis/queue.cc:258
ERROR 2016-11-16 10:24:38.000000 [ main ] failed to reload queue gearmand_run(QUEUE_ERROR) -> libgearman-server/gearmand.cc:395
ERROR 2016-11-16 10:24:38.000000 [ main ] Failed to call KEYS during QUEUE replay _hiredis_replay(QUEUE_ERROR) -> libgearman-server/plugins/queue/redis/queue.cc:258
ERROR 2016-11-16 10:24:38.000000 [ main ] failed to reload queue gearmand_run(QUEUE_ERROR) -> libgearman-server/gearmand.cc:395
ERROR 2016-11-16 10:24:38.000000 [ main ] Failed to call KEYS during QUEUE replay _hiredis_replay(QUEUE_ERROR) -> libgearman-server/plugins/queue/redis/queue.cc:258
ERROR 2016-11-16 10:24:38.000000 [ main ] failed to reload queue gearmand_run(QUEUE_ERROR) -> libgearman-server/gearmand.cc:395

SO, I can't start up gearmand server!

Error checking version boostlib

I am trying to install an extension Gearman:

Operating System: Centos 7
PHP: 5.6.28

Perform the following steps:

wget https://launchpad.net/gearmand/1.2/1.1.11/+download/gearmand-1.1.12.tar.gz
tar -zxvf gearmand-1.1.12.tar.gz
cd gearmand-1.1.12
./configure

However, getting the error:
checking for boostlib> = 1.39 ... configure: We could not detect the boost libraries (version 1.39 or higher). If you have a staged boost library (still not installed) please specify $ BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost / version.hpp>. See http://randspringer.de/boost for more documentation.
configure: error: could not find boost

More details: http://pastebin.com/NFpbq20c

However, I have set BOOST 1.53

cat /usr/include/boost/version.hpp | grep "BOOST_LIB_VERSION"
//  BOOST_LIB_VERSION must be defined to be the same as BOOST_VERSION
#define BOOST_LIB_VERSION "1_53"

Why is this happening and how can I fix this?

test t/cycle is racey and fails periodically

This is one of the causes of spurious retries on our CI tests. I've been doing some analysis, and it was relatively easy to get the problem to repeat. After 14 tries, this happened:

tried 13 times
cycle.kill.kill					2:000140750 [ ok ]
cycle.worker.single startup/shutdown					0:001611553 [ ok ]
cycle.server_startup().server_startup(1)					1:001624276 [ ok ]
cycle.server_startup().server_startup(many)					20:047278546 [ ok ]
cycle.server_startup().shutdown_and_remove()					0:000085085 [ ok ]
cycle.server_startup().server_startup(many)					20:049186806 [ ok ]
cycle.server_startup().server_startup() with bind() conflict					0:000000629 [ ok ]
tried 14 times
cycle.kill.kill					2:000180426 [ ok ]
cycle.worker.single startup/shutdown					0:003570175 [ ok ]
cycle.server_startup().server_startup(1)					1:003090167 [ ok ]
libtest/client.cc:268: in start() pid(8858) localhost:54428 ping(libtest/client.cc:268: Connection refused), additionally pid: 9031 is alive: true waited: 17 server started. exec: /home/clint/src/gearman/gearmand/libtool --mode=execute /home/clint/src/gearman/gearmand/./gearmand/gearmand --verbose=INFO --log-file=var/log/gearmand.logNt7BSZ --pid-file=var/run/gearmand.pidLHPdDC --port=54428 --listen=localhost  stderr:

tests/cycle.cc:113: in server_startup_multiple_TEST() pid(8858) Assertion '__server_startup_TEST((cycle_context_st*)obj, 20)' != 'TEST_SUCCESS'
cycle.server_startup().server_startup(many)					[ failed ]
cycle.server_startup().shutdown_and_remove()					0:016349708 [ ok ]
cycle.server_startup().server_startup(many)					20:037825710 [ ok ]
cycle.server_startup().server_startup() with bind() conflict					0:000000453 [ ok ]

The mentioned log file shows this:

   INFO 2016-11-29 23:49:39.253396 [  main ] Initializing Gear on port 54428 with SSL: false
   INFO 2016-11-29 23:49:39.000000 [  main ] Starting up with pid 9031, verbose is set to INFO
  ERROR 2016-11-29 23:50:00.000000 [  main ] Timeout occurred when calling bind() for 127.0.0.1:54428 -> libgearman-server/gearmand.cc:688
   INFO 2016-11-29 23:50:00.000000 [  main ] Shutdown complete

Normal runs show this:

   INFO 2016-11-29 23:45:36.616470 [  main ] Initializing Gear on port 22942 with SSL: false
   INFO 2016-11-29 23:45:36.000000 [  main ] Starting up with pid 5764, verbose is set to INFO
   INFO 2016-11-29 23:45:36.000000 [  main ] Listening on 127.0.0.1:22942 (13)
   INFO 2016-11-29 23:45:36.000000 [  main ] Adding event for listening socket (13)
   INFO 2016-11-29 23:45:36.000000 [  main ] Accepted connection from 127.0.0.1:49006
   INFO 2016-11-29 23:45:36.000000 [     4 ] Peer connection has called close()
   INFO 2016-11-29 23:45:36.000000 [     4 ] Disconnected 127.0.0.1:49006
   INFO 2016-11-29 23:45:36.000000 [     4 ] Gear connection disconnected: -:-
   INFO 2016-11-29 23:45:43.000000 [  main ] Clearing event for listening socket (13)
   INFO 2016-11-29 23:45:43.000000 [  main ] Closing listening socket (13)
   INFO 2016-11-29 23:45:43.000000 [  main ] Shutdown complete

My guess is that the function that detects free ports doesn't hang on to that port, and so other things happening on the box take that port, causing an error/timeout in binding to it.

centOS6.5 , mysql57 conflicts with gearmand

Transaction Check Error:
file /usr/share/mysql/charsets/Index.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/armscii8.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/ascii.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/cp1250.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/cp1256.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/cp1257.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/cp850.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/cp852.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/cp866.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/dec8.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/geostd8.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/greek.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/hebrew.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/hp8.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/keybcs2.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/koi8r.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/koi8u.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/latin1.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/latin2.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/latin5.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/latin7.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/macce.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/macroman.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/charsets/swe7.xml from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/czech/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/danish/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/dutch/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/english/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/estonian/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/french/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/german/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/greek/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/hungarian/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/italian/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/japanese/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/korean/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/norwegian-ny/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/norwegian/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/polish/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/portuguese/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/romanian/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/russian/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/serbian/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/slovak/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/spanish/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/swedish/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /usr/share/mysql/ukrainian/errmsg.sys from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-common-5.7.15-1.el6.x86_64
file /etc/my.cnf from install of mysql-libs-5.1.73-7.el6.x86_64 conflicts with file from package mysql-community-server-5.7.13-1.el6.x86_64

I have no idea with the trouble, give me some tip to kill the error.

server does not log the text sent back to users when sending GEARMAN_COMMAND_ERROR

It's hard to debug from the server side without the text of the errors.

unreliable HAVE_PTHREAD_TIMEDJOIN_NP check

build on FreeBSD 11 ends in exception:
error: pthread_timedjoin_np' was not declared in this scope`
The author of this blog described the same pain.

Is it an issue for m4/ax_pthread_timedjoin_np.m4?

Compilation instructions outdated

I tried to compile following the instructions here and here.

It failed until I run ./bootstrap.sh -a. It's a problem on my side or documentation is outdated?

When restarting gearmand it will block redis if using it as the queue backend

When replaying the jobs back into gearmand from redis it uses the KEYS command, which is a blocking operation in redis that the redis documentation recommend not be used in production:

Don't use KEYS in your regular application code.

If gearmand is hooked up to it's own redis instance this may not be an issue, but if other parts of your application share redis this is likely to be painful if gearmand restarts.

The quick 'fix' to this is to move from using KEYS to using SCAN, however this still has to scan over all keys in the redis instance which will be potentially slow if there are other keys being stored in redis from other applications.

A more complete fix is to track all of the keys added in a set, e.g. when calling SET key data, also do SADD gearman_jobs key, and the complementary DEL and SREM. This then allows for jobs to be reloaded using SMEMBERS which only looks through relevant keys contained within the set, instead of all keys.

I cannot use libmemcached as the "Persistent Queues"

I have installed the libmemcached library, but when I install gearmand and run "./configure", it's show info like below :

LDFLAGS Flags:

• Assertions enabled: no
• Debug enabled: no
• Warnings as failure: no
• Building with libsqlite3 no
• Building with libdrizzle yes
• Building with libmemcached not found
• Building with libpq no
• Building with tokyocabinet no
• Building with libmysql no
• SSL enabled: no
• cyassl found: no
• openssl found: yes
• make -j: 2
• VCS checkout: no
• sphinx-build: :

Thus I finish to install gearmand , "libmemched persistence" cannot be used. below is the error:

[root@localhost gearmand-1.1.12]# /usr/local/gearmand/sbin/gearmand -q libmemcached --libmemcached-servers=127.0.0.1
/usr/local/gearmand/sbin/gearmand: unknown option libmemcached-servers

This is my libmemcached installed
[root@localhost libmemcached]# ls /usr/local/libmemcached/
bin include lib share

I now have no idear to solve this issue, is anyone can help me!

A couple of typos

Hi, while packaging gearmand for Debian, linitian ( debian package checker tool ) has detected a number of typos. Please see the attached patch.
typos.patch.txt

Thanks!

gearmand 1.1.12 does not work with SQL persistence on Ubuntu 14.04

Is there any good instructions to make it ork in Ubuntu 14.04?
Is possible with MySql persistence.
I have tried with the instructions provided in the website and looks to be very old, they don't work.

Does 'WORK_EXCEPTION' work for now?

Hi, I notice that https://bugs.launchpad.net/gearmand/+bug/405732 said the 'work_exception' did not behavior like the protocol description.

WORK_EXCEPTION

    This is to notify the server (and any listening clients) that 
    the job failed with the given exception.

    Arguments:
    - NULL byte terminated job handle.
    - Opaque data that is returned to the client as an exception.

I want client to know the error detail for a failed job. But 'WORK_FAIL' cannot carry the information, do you have any idea? thanks.

libgearman patch to support SSL connections in Gearman PHP library

There's a patch to add SSL support to the the Gearman PHP library that's been languishing for over 2 years over at http://bugs.php.net/bug.php?id=67623. The reason for that is that it relies on some (relatively straightforward?) changes to libgearman. The libgearman patch can be found at http://bugs.launchpad.net/gearmand/+bug/1338861. The patch (to gearmand 1.1.12) needs to be rebased, obviously, and it looks like it could use a little more polish. I'm willing to take a crack at that. If anyone has any comments on the patch or changes you feel need to be made to the patch in order to get it accepted, I'd like to hear them. It would be really nice to get this committed here, so that some future release of the Gearman PHP library could support SSL connections.

Server event error.

I have tested a lot in my program, and always found there are several jobs in Server's queue, but all workers are sleeping. So I set verbose to DEBUG on Server to see what happened, I found something strange when there was an ERROR, below is the log:

INFO 2016-09-19 06:26:14.403382 [     1 ] Gear unpack
  DEBUG 2016-09-19 06:26:14.403385 [     1 ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_PRE_SLEEP -> libgearman-server/plugins/protocol/gear/protocol.cc:248
  DEBUG 2016-09-19 06:26:14.403389 [     1 ] Received PRE_SLEEP 192.168.98.116:3229621481 -> libgearman-server/thread.cc:318
  DEBUG 2016-09-19 06:26:14.403399 [     1 ]  192.168.98.116:46681 Watching  POLLIN  -> libgearman-server/gearmand_thread.cc:151
  DEBUG 2016-09-19 06:26:14.403457 [     1 ] Received RUN wakeup event -> libgearman-server/gearmand_thread.cc:607
  DEBUG 2016-09-19 06:26:14.403478 [     1 ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_NOOP -> libgearman-server/plugins/protocol/gear/protocol.cc:274
  DEBUG 2016-09-19 06:26:14.403492 [     1 ] send() 12 bytes to peer 192.168.98.116:46681 -> libgearman-server/io.cc:275
  DEBUG 2016-09-19 06:26:14.403494 [     1 ] Sent NOOP to 192.168.98.116:-1065345815 -> libgearman-server/thread.cc:365
  DEBUG 2016-09-19 06:26:14.403974 [     1 ] 192.168.98.116:46681 Ready     POLLIN  -> libgearman-server/gearmand_con.cc:118
  DEBUG 2016-09-19 06:26:14.403999 [     1 ] read 12 bytes -> libgearman-server/io.cc:686
   INFO 2016-09-19 06:26:14.404003 [     1 ] Gear unpack
  DEBUG 2016-09-19 06:26:14.404007 [     1 ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_GRAB_JOB -> libgearman-server/plugins/protocol/gear/protocol.cc:248
  DEBUG 2016-09-19 06:26:14.404010 [     1 ] Received GRAB_JOB 192.168.98.116:3229621481 -> libgearman-server/thread.cc:318
  DEBUG 2016-09-19 06:26:14.404018 [     1 ]  192.168.98.116:46681 Watching  POLLIN  -> libgearman-server/gearmand_thread.cc:151
  DEBUG 2016-09-19 06:26:14.404073 [     1 ] Received RUN wakeup event -> libgearman-server/gearmand_thread.cc:607
  DEBUG 2016-09-19 06:26:14.404104 [     1 ] GEAR length: 941 gearmand_command_t: GEARMAN_COMMAND_JOB_ASSIGN -> libgearman-server/plugins/protocol/gear/protocol.cc:274
  DEBUG 2016-09-19 06:26:14.404119 [     1 ] send() 985 bytes to peer 192.168.98.116:46681 -> libgearman-server/io.cc:275
  DEBUG 2016-09-19 06:26:14.404122 [     1 ] Sent JOB_ASSIGN to 192.168.98.116:-1065345815 -> libgearman-server/thread.cc:365
  DEBUG 2016-09-19 06:26:14.496260 [     1 ] 192.168.98.116:46681 Ready     POLLIN  -> libgearman-server/gearmand_con.cc:118
  DEBUG 2016-09-19 06:26:14.496286 [     1 ] read 38 bytes -> libgearman-server/io.cc:686
   INFO 2016-09-19 06:26:14.496290 [     1 ] Gear unpack
  DEBUG 2016-09-19 06:26:14.496294 [     1 ] GEAR length: 12 gearmand_command_t: GEARMAN_COMMAND_WORK_COMPLETE -> libgearman-server/plugins/protocol/gear/protocol.cc:248
  DEBUG 2016-09-19 06:26:14.496299 [     1 ] Received WORK_COMPLETE 192.168.98.116:3229621481 -> libgearman-server/thread.cc:318
  DEBUG 2016-09-19 06:26:14.496307 [     1 ]  192.168.98.116:46681 Watching  POLLIN  -> libgearman-server/gearmand_thread.cc:151
  DEBUG 2016-09-19 06:26:14.496357 [     1 ] Received RUN wakeup event -> libgearman-server/gearmand_thread.cc:607
  DEBUG 2016-09-19 06:26:14.496365 [     1 ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_ERROR -> libgearman-server/plugins/protocol/gear/protocol.cc:274
  DEBUG 2016-09-19 06:26:14.496380 [     1 ] send() 60 bytes to peer 192.168.98.116:46681 -> libgearman-server/io.cc:275
  DEBUG 2016-09-19 06:26:14.496383 [     1 ] Sent ERROR to 192.168.98.116:-1065345815 -> libgearman-server/thread.cc:365
  DEBUG 2016-09-19 06:26:14.496670 [     1 ] 192.168.98.116:46681 Ready     POLLIN  -> libgearman-server/gearmand_con.cc:118

I have no idea when the Server fire up Recevied RUN wakeup event, but actually under normal conditions it will happen only after PRE_SLEEP or GRAB_JOB, But before GEARMAN_COMMAND_ERROR, this event fire up after WORK_COMPLETE, I think that's the problem, what you guys think of this bug?

and BTW, why the port of worker is a negative number?

There's no way to limit acces to administrative commands

Is there a way to limit the access to the text admin protocol ? It looks strange to me that a client can shutdown server for example.
Why not to use a different port for the admin protocol ?
Thanks!

Pull qequests gcc5 build error

Hi
I try to pull qequests, but gcc5 build error.

https://travis-ci.org/gearman/gearmand/jobs/177226646

But I can be build in the local success.

My local compilation environment:

Using built-in specs.
COLLECT_GCC=gcc-5
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/5/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 5.4.1-2ubuntu114.04' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=gcc4-compatible --disable-libstdcxx-dual-abi --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 5.4.1 20160904 (Ubuntu 5.4.1-2ubuntu114.04)

So, I close pull request.

make fail :bin/gearadmin.cc:84:37: fatal error: boost/program_options.hpp: No such file or directory

gearmand version: 1.1.13
./configure --prefix=/opt/app/gearmand1.1.13 --with-boost=/opt/app/boost --with-sqlite3 --disable-libdrizzle --disable-libpq --disable-libtokyocabinet --disable-hiredis --with-mysql=no --with-memcached=/opt/app/libmemcachedmake

root@ifchange-dev:/opt/app/boost# find -name program_options.hpp
./include/boost/program_options.hpp

system :CentOS release 6.8 (Final)

The new client can't addservers when one server of gearman cluster died

Look at this, maybe do some helps

<?php

try{
    // "192.168.0.1" is died, It will throw exception
    $client= new GearmanClient();
    $client->setTimeout(1000);
    $client->addServer("192.168.0.1", 4730);
}catch (Exception $e1) {
    // ...
    // "192.168.0.2" is ok, the new client was able to start work
    try{
        $client= new GearmanClient();
        $client->setTimeout(1000);
        $client->addServer("192.168.0.2", 4730);
    }catch (Exception $e2) {
        // ...
    }
}