While trying to build client or server on a FreeBSD 10.1 host, both fail with error error: unknown warning option '-Wlogical-op'; did you mean '-Wlong-long'?

Server

[freebsd@dbbench ~/bruce]$ source bash_defs
[freebsd@dbbench ~/bruce]$ cd src/bruce/
[freebsd@dbbench ~/bruce/src/bruce]$ build --release bruce
scons: Entering directory `/usr/home/freebsd/bruce'
c++ -o out/release/base/backoff_rate_limiter.o -c -std=c++11 -Wold-style-cast -Wall -Wextra -Werror -Wformat=2 -Winit-self -Wunused-parameter -Wshadow -Wpointer-arith -Wcast-align -Wlogical-op -O2 -DNDEBUG -Wno-unused -Wno-unused-parameter -flto -fvisibility=hidden -DSRC_ROOT='"/usr/home/freebsd/bruce/src"' -Isrc -Isrc/third_party/tclap-1.2.0/include src/base/backoff_rate_limiter.cc
error: unknown warning option '-Wlogical-op'; did you mean '-Wlong-long'?
      [-Werror,-Wunknown-warning-option]
scons: *** [out/release/base/backoff_rate_limiter.o] Error 1
[freebsd@dbbench ~/bruce/src/bruce]$

Client

[freebsd@dbbench ~/bruce]$ source bash_defs
[freebsd@dbbench ~/bruce]$ cd src/bruce/client/
[freebsd@dbbench ~/bruce/src/bruce/client]$ build --release libbruce_client.so
scons: Entering directory `/usr/home/freebsd/bruce'
cc -o out/release/bruce/client/libbruce_client.os -c -Wwrite-strings -Wall -Wextra -Werror -Wformat=2 -Winit-self -Wunused-parameter -Wshadow -Wpointer-arith -Wcast-align -Wlogical-op -O2 -DNDEBUG -Wno-unused -Wno-unused-parameter -flto -fvisibility=hidden -fPIC -DSRC_ROOT='"/usr/home/freebsd/bruce/src"' -Isrc -Isrc/third_party/tclap-1.2.0/include src/bruce/client/libbruce_client.c
error: unknown warning option '-Wlogical-op'; did you mean '-Wlong-long'?
      [-Werror,-Wunknown-warning-option]
scons: *** [out/release/bruce/client/libbruce_client.os] Error 1
[freebsd@dbbench ~/bruce/src/bruce/client]$

Hi there, I'm trying to troubleshoot a bruce behavior which I cannot explain.
For some reason, bruce works well with topic autocreate only if there's already at least one topic existing on kafka broker. After firing a few events into bruce for several topics nothing happens and topics are not auto created. Only as I create a dummy topic, that the other topics get created and the event are coming in.
Am I missing something?

*running with both --topic_autocreate & auto.create.topics.enable=true
Thanks in advance,
Oren

Hello,

Any chance of implementing SSL support?
https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka

Thanks for the software!

I have a little qurstion about how to guarantee message delivery to Kafka. If there are a large stream of important messages, how can we be sure of delivery?
For example, we run Bruce in docker in write to socket message. Bruce read it and temporary store in memory for batch send. What will happens if at that moment docker container crashed (killed)? Will messages from memory be missed? If Yes, how can we guarantee message delivery?

While there is a great writeup on building gcc 4.8.2 and then building bruce with it, I was wondering if there was a way to build bruce statically. If so, what would be needed to get that accomplished in RHEL 6.x and derivatives (OEL 6.6 in my case).

I've got a usecase of running bruce on a set of locked down machine with little remote access due to strict compliance issues, and looking for an easier deployment options.

Hi,

I am build bruce in ubuntu

cd bruce
source bash_defs
cd src/bruce
build --release bruce
But getting error :
g++ -o out/release/bruce/debug/debug_setup.o -c -std=c++11 -Wold-style-cast -Wall -Wextra -Werror -O2 -DNDEBUG -Wno-unused -Wno-unused-parameter -flto -fvisibility=hidden -DSRC_ROOT='"/home/stuser/RESTClients/bruce/src"' -Isrc -Isrc/third_party/tclap-1.2.0/include src/bruce/debug/debug_setup.cc
src/bruce/debug/debug_setup.cc: In member function 'void Bruce::Debug::TDebugSetup::CreateDebugDir()':
src/bruce/debug/debug_setup.cc:175:32: error: format not a string literal and no format arguments [-Werror=format-security]
syslog(LOG_ERR, msg.c_str());
^
cc1plus: all warnings being treated as errors
scons: *** [out/release/bruce/debug/debug_setup.o] Error 1

Any idea?

thanks

AL

Hi. I wanted to make sure I understand the "Design Overview" correctly. Is it true I cannot use Bruce for a use case where messages of size over ~426KB needs to be send to Kafka?

If I want to overcome this limitation I need to cut the big messages into smaller once prior to the to_bruce call, is that correct?

When trying to build a specific version of the code (based on the tags) from Jenkins, the build version always gets the version closest to the head.

The line of code that appears to be doing this is in: src/bruce/scripts/gen_version
The line of code appears to be on line 132, below are lines 131 - 134:

131         try:
132             p = subprocess.Popen(['git', 'describe', 'master'],
133                                  stdout=subprocess.PIPE)
134             version, err = p.communicate()

Removing the 'master' parameter (as seen below) happens to solve the problem I'm seeing, but I'm not an expert in git. It worked for building 1.0.12, and 1.0.16, as well as 1.0.17.

132             p = subprocess.Popen(['git', 'describe'],

When I made that change the build was able compile successfully and the version number of bruce output correctly when running "bruce --version". Without the change, the version of bruce was always labeled as 1.0.17 even though the code base that was checked out (and therefore compiled) was 1.0.12 or 1.0.16.

I used 1.0.22 version, whether build rpm package or build directly, there are some errors as follow:

(bruce_env)[root@host02 bruce]# source bash_defs
(bruce_env)[root@host02 bruce]# cd src/bruce/client/
(bruce_env)[root@host02 client]# build --release libbruce_client.so
scons: Entering directory `/home/cquan/bruce'
File "/home/cquan/bruce/SConstruct", line 135

prog_libs = {'pthread', 'dl', 'rt'}

                      ^

SyntaxError: invalid syntax

what happened? BTW, I use centos 6.5, also follow your suggestion steps.

Hello,

I have 2 test machines. One of them (host A) has 3 kafka broker instances (9092,9093,9094) + 1 zookeeper instance (2181)...

bruce is working on host B. And I want to push messages to unix domain socket on host B. And I want to consume them on host A.

What kind of configuration do we need to do that?

my configuration details:

I added:

to /etc/bruce/bruce_conf.xml file on host B. (bruce host)

Our network team open ports 9092,9093,9094 and 2181 between host A and host B.
Is it enough for communication or do we need another open ports?

bruce --msg_buffer_max 65536 --receive_socket_name /var/run/bruce/bruce.socket --config_path /etc/bruce/bruce_conf.xml --log_echo --log_level LOG_DEBUG --topic_autocreate --status_port 2122

bruce[21788]: Log started
bruce[21788]: Using "bruce" for client ID since none was specified with --client_id option. This is a workaround for a bug in Kafka 0.9.0.0 that causes broker to crash on receipt of produce request with empty client ID. See https://issues.apache.org/jira/browse/KAFKA-3088 for details.
bruce[21788]: Version: [1.0.28]
bruce[21788]: Config file: [/etc/bruce/bruce_conf.xml]
bruce[21788]: UNIX domain datagram input socket [/var/run/bruce/bruce.socket]
bruce[21788]: UNIX domain datagram input socket mode
bruce[21788]: Using Kafka protocol version [0]
bruce[21788]: Listening on status port 2122
bruce[21788]: Web interface loopback only: false
bruce[21788]: Buffered message limit 65536 kbytes
bruce[21788]: Max input message size 65536 bytes
bruce[21788]: Allow large UNIX datagrams: false
bruce[21788]: Max failed delivery attempts 5
bruce[21788]: Not running as daemon
bruce[21788]: Client ID [bruce]
bruce[21788]: Required ACKs -1
bruce[21788]: Replication timeout 10000 milliseconds
bruce[21788]: Shutdown send grace period 30000 milliseconds
bruce[21788]: Kafka dispatch restart grace period 5000 milliseconds
bruce[21788]: Metadata refresh interval 15 minutes
bruce[21788]: Kafka socket timeout 60 seconds
bruce[21788]: Pause rate limit initial 5000 milliseconds
bruce[21788]: Pause rate limit max double 4
bruce[21788]: Minimum pause delay 5000 milliseconds
bruce[21788]: Discard reporting interval 600 seconds
bruce[21788]: Debug directory [/home/bruce/debug]
bruce[21788]: Message debug time limit 3600 seconds
bruce[21788]: Message debug byte limit 2147483648
bruce[21788]: Skip comparing metadata on refresh: false
bruce[21788]: Discard logfile creation is disabled
bruce[21788]: Discard report bad msg prefix size: 256 bytes
bruce[21788]: Automatic topic creation enabled
bruce[21788]: Using new input datagram format
bruce[21788]: Using new output format
bruce[21788]: Pool block size is 128 bytes
bruce[21788]: Server started
bruce[21788]: Starting input thread
bruce[21788]: Starting router thread
bruce[21788]: Input thread 21789 started
bruce[21788]: Router thread 21790 startedbruce[21788]: Input thread opening UNIX domain datagram socket

bruce[21788]: Router thread sending initial metadata request
bruce[21788]: Input thread finished initialization, forwarding messages
bruce[21788]: Main thread detected successful input thread initializationbruce[21788]: Router thread getting metadata from broker broker_host_1 port 9092

bruce[21788]: Started web interface, waiting for signals and errors
bruce[21788]: Metadata sanity check passed
bruce[21788]: Router thread starting dispatcher during initialization
bruce[21788]: Starting connector thread for broker index 0 (Kafka ID 1)
bruce[21788]: Router thread finished initialization
bruce[21788]: Connector thread 21793 (index 0 broker 1) started
bruce[21788]: Connector thread 21793 (index 0 broker 1) connecting to host localhost port 9092
bruce[21788]: Starting pause on failure to connect to broker localhost port 9092
bruce[21788]: Connector thread 21793 (index 0 broker 1) finished on error
bruce[21788]: Router thread detected pause: waiting 5000 milliseconds before responding
bruce[21788]: All connector threads finished shutting down
bruce[21788]: Router thread shutting down dispatcher on pause
bruce[21788]: Sending fast shutdown request to connector thread (index 0 broker 1)
bruce[21788]: Waiting for shutdown ACK from connector thread (index 0 broker 1)
bruce[21788]: Got shutdown finished notification from connector thread (index 0 broker 1)
bruce[21788]: Router thread waiting for dispatcher shutdown
bruce[21788]: Start waiting for dispatcher shutdown status
bruce[21788]: Finished waiting for dispatcher shutdown status
bruce[21788]: Dispatcher terminated on error
bruce[21788]: Router thread getting metadata in response to pause
bruce[21788]: Router thread getting metadata from broker localhost port 9092
bruce[21788]: Router thread failed to connect to broker for metadata
bruce[21788]: Metadata request failed for all known brokers, waiting 5000 milliseconds before retry (1)
bruce[21788]: Router thread getting metadata from broker localhost port 9092
bruce[21788]: Router thread failed to connect to broker for metadata
bruce[21788]: Metadata request failed for all known brokers, waiting 6579 milliseconds before retry (1)

I defined broker_host_1 for connection but it tries to connect localhost why?

I'm getting the following error on Amazon Linux 2015.03 (It is a RHEL clone)

[ec2-user@ip-172-31-34-58 bruce]$ cat /etc/issue
Amazon Linux AMI release 2015.03
Kernel \r on an \m

[ec2-user@ip-172-31-34-58 bruce]$ uname -a
Linux ip-172-31-34-58 3.14.27-25.47.amzn1.x86_64 #1 SMP Wed Dec 17 18:36:15 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

[ec2-user@ip-172-31-34-58 bruce]$ python --version
Python 2.6.9

Build error:

Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.eVtNVZ
+ umask 022
+ cd /home/ec2-user/bruce/bruce/out/pkg/release/rpm/build/BUILD
+ cd /home/ec2-user/bruce/bruce/out/pkg/release/rpm/build/BUILD
+ rm -rf bruce-1.0.19.9.ga652bd3
+ /bin/tar -xf -
+ /usr/bin/gzip -dc /home/ec2-user/bruce/bruce/out/pkg/release/rpm/build/SOURCES/bruce-1.0.19.9.ga652bd3.tar.gz
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd bruce-1.0.19.9.ga652bd3
+ /bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ exit 0
Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.gSIYOd
+ umask 022
+ cd /home/ec2-user/bruce/bruce/out/pkg/release/rpm/build/BUILD
+ cd bruce-1.0.19.9.ga652bd3
+ IMPORT_PATH=
+ ./build_all -m release --asan no -f bruce/bruce apps
  File "./build_all", line 181
    target_set = {target for target in args.target}
                           ^
SyntaxError: invalid syntax
error: Bad exit status from /var/tmp/rpm-tmp.gSIYOd (%build)


RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.gSIYOd (%build)
RPM build failed