I'm attempting to send sflow counters at 1s intervals to pmacct, and have pmacct forward these to kafka.
I have pmacct (sfacctd) successfully receiving the sflow data, and sending data to kafka. However, sending each counter to kafka takes a long time, and while the counters are being sent to kafka any incoming sflow packets are not captured.
i.e.
Jan 10 10:07:48 INFO ( default/core ): sFlow Accounting Daemon, sfacctd 1.6.1 (20161001-00+c5)
Jan 10 10:07:48 INFO ( default/core ): '--enable-jansson' '--enable-rabbitmq' '--enable-kafka' '--enable-ipv6' '--enable-plabel' '--enable-64bit' '--enable-threads' '--prefix=/opt/pmacct'
Jan 10 10:07:48 INFO ( default/core ): Reading configuration file '/etc/pmacct/sfacctd-23503.conf'.
Jan 10 10:07:48 WARN ( default/kafka ): defaulting to SRC HOST aggregation.
Jan 10 10:07:48 INFO ( default/kafka ): plugin_pipe_size=9096000 bytes plugin_buffer_size=4096 bytes
Jan 10 10:07:48 INFO ( default/kafka ): ctrl channel: obtained=212992 bytes target=17760 bytes
Jan 10 10:07:48 INFO ( default/core ): waiting for sFlow data on :::23503
Jan 10 10:07:48 INFO ( default/kafka ): cache entries=16411 base cache memory=53434216 bytes
Jan 10 10:07:48 INFO ( default/core ): 3 brokers successfully added.
Jan 10 10:07:49 DEBUG ( default/core ): Received sFlow packet from [<ipaddress>:8888] version [5] seqno [1471283419]
Jan 10 10:07:49 DEBUG ( default/core ): Received sFlow packet from [<ipaddress>:8888] version [5] seqno [1471283420]
Jan 10 10:07:49 DEBUG ( default/core ): Received sFlow packet from [<ipaddress>:8888] version [5] seqno [1471283421]
Jan 10 10:07:49 DEBUG ( default/core ): Received sFlow packet from [<ipaddress>:8888] version [5] seqno [1471283422]
...
<snip> - lots more sflow packets
...
Jan 10 10:07:50 DEBUG ( default/core ): readv5CountersSample(): element tag 0:1.
Jan 10 10:07:51 DEBUG ( default/core ): Kafka message delivery successful (111 bytes): {"seq": 0, "timestamp": "2017-01-10 10:07:49.869972", "peer_src_ip": "<ipaddress>", "event_type": "log_init"}
Jan 10 10:07:51 DEBUG ( default/core ): readv5CountersSample(): element tag 0:2.
Jan 10 10:07:52 DEBUG ( default/core ): Kafka message delivery successful (665 bytes): {"seq": 1, "timestamp": "2017-01-10 10:07:50.395681", "peer_ip_src": "<ipaddress>", "event_type": "log", "source_id_in
dex": 1, "sflow_seq": 1471283580, "sflow_cnt_seq": 2022332, "sf_cnt_type": "sflow_cnt_generic", "ifIndex": 1, "ifType": 6, "ifSpeed": 1000000000, "ifDirection": 1, "ifStatus": 3, "ifInOctets": 15076004816257
, "ifInUcastPkts": 2506335772, "ifInMulticastPkts": 18978967, "ifInBroadcastPkts": 2041661, "ifInDiscards": 0, "ifInErrors": 0, "ifInUnknownProtos": 0, "ifOutOctets": 25695757687509, "ifOutUcastPkts": 279292
9131, "ifOutMulticastPkts": 19847164, "ifOutBroadcastPkts": 15577452, "ifOutDiscards": 0, "ifOutErrors": 0, "ifPromiscuousMode": 1}
Jan 10 10:07:52 DEBUG ( default/core ): readv5CountersSample(): element tag 0:1.
Jan 10 10:07:53 DEBUG ( default/core ): Kafka message delivery successful (651 bytes): {"seq": 2, "timestamp": "2017-01-10 10:07:50.395681", "peer_ip_src": "<ipaddress>", "event_type": "log", "source_id_in
dex": 1, "sflow_seq": 1471283580, "sflow_cnt_seq": 2022332, "sf_cnt_type": "sflow_cnt_ethernet", "dot3StatsAlignmentErrors": 0, "dot3StatsFCSErrors": 0, "dot3StatsSingleCollisionFrames": 0, "dot3StatsMultipl
eCollisionFrames": 0, "dot3StatsSQETestErrors": 0, "dot3StatsDeferredTransmissions": 0, "dot3StatsLateCollisions": 0, "dot3StatsExcessiveCollisions": 0, "dot3StatsInternalMacTransmitErrors": 0, "dot3StatsCar
rierSenseErrors": 0, "dot3StatsFrameTooLongs": 0, "dot3StatsInternalMacReceiveErrors": 0, "dot3StatsSymbolErrors": 0}
Jan 10 10:07:53 DEBUG ( default/core ): readv5CountersSample(): element tag 0:2.
Jan 10 10:07:54 DEBUG ( default/core ): Kafka message delivery successful (665 bytes): {"seq": 3, "timestamp": "2017-01-10 10:07:50.395681", "peer_ip_src": "<ipaddress>", "event_type": "log", "source_id_in
dex": 2, "sflow_seq": 1471283580, "sflow_cnt_seq": 2022332, "sf_cnt_type": "sflow_cnt_generic", "ifIndex": 2, "ifType": 6, "ifSpeed": 1000000000, "ifDirection": 1, "ifStatus": 3, "ifInOctets": 23306961007723
1, "ifInUcastPkts": 2013855039, "ifInMulticastPkts": 12462760, "ifInBroadcastPkts": 4173839, "ifInDiscards": 0, "ifInErrors": 0, "ifInUnknownProtos": 0, "ifOutOctets": 59364348042262, "ifOutUcastPkts": 23465
75762, "ifOutMulticastPkts": 37158770, "ifOutBroadcastPkts": 5023965, "ifOutDiscards": 0, "ifOutErrors": 0, "ifPromiscuousMode": 1}
Jan 10 10:07:54 DEBUG ( default/core ): readv5CountersSample(): element tag 0:1.
Jan 10 10:07:55 DEBUG ( default/core ): Kafka message delivery successful (651 bytes): {"seq": 4, "timestamp": "2017-01-10 10:07:50.395681", "peer_ip_src": "<ipaddress>", "event_type": "log", "source_id_in
dex": 2, "sflow_seq": 1471283580, "sflow_cnt_seq": 2022332, "sf_cnt_type": "sflow_cnt_ethernet", "dot3StatsAlignmentErrors": 0, "dot3StatsFCSErrors": 0, "dot3StatsSingleCollisionFrames": 0, "dot3StatsMultipl
eCollisionFrames": 0, "dot3StatsSQETestErrors": 0, "dot3StatsDeferredTransmissions": 0, "dot3StatsLateCollisions": 0, "dot3StatsExcessiveCollisions": 0, "dot3StatsInternalMacTransmitErrors": 0, "dot3StatsCar
rierSenseErrors": 0, "dot3StatsFrameTooLongs": 0, "dot3StatsInternalMacReceiveErrors": 0, "dot3StatsSymbolErrors": 0}
Jan 10 10:07:55 DEBUG ( default/core ): readv5CountersSample(): element tag 0:2.
Jan 10 10:07:56 DEBUG ( default/core ): Kafka message delivery successful (628 bytes): {"seq": 5, "timestamp": "2017-01-10 10:07:50.395681", "peer_ip_src": "<ipaddress>", "event_type": "log", "source_id_in
dex": 3, "sflow_seq": 1471283580, "sflow_cnt_seq": 2022332, "sf_cnt_type": "sflow_cnt_generic", "ifIndex": 3, "ifType": 6, "ifSpeed": 0, "ifDirection": 1, "ifStatus": 1, "ifInOctets": 120758166, "ifInUcastPk
ts": 416574, "ifInMulticastPkts": 0, "ifInBroadcastPkts": 1188, "ifInDiscards": 0, "ifInErrors": 0, "ifInUnknownProtos": 0, "ifOutOctets": 1052639508, "ifOutUcastPkts": 493524, "ifOutMulticastPkts": 10931783
, "ifOutBroadcastPkts": 4193566, "ifOutDiscards": 0, "ifOutErrors": 0, "ifPromiscuousMode": 1}
...
<snip> and so on, until
...
Jan 10 10:08:02 DEBUG ( default/core ): readv5CountersSample(): element tag 0:2.
Jan 10 10:08:02 DEBUG ( default/core ): Kafka message delivery successful (658 bytes): {"seq": 11, "timestamp": "2017-01-10 10:07:50.395681", "peer_ip_src": "<ipaddress>", "event_type": "log", "source_id_i
ndex": 6, "sflow_seq": 1471283580, "sflow_cnt_seq": 2022332, "sf_cnt_type": "sflow_cnt_generic", "ifIndex": 6, "ifType": 6, "ifSpeed": 1000000000, "ifDirection": 1, "ifStatus": 3, "ifInOctets": 144448666074,
"ifInUcastPkts": 425681217, "ifInMulticastPkts": 244209791, "ifInBroadcastPkts": 1955105, "ifInDiscards": 0, "ifInErrors": 0, "ifInUnknownProtos": 0, "ifOutOctets": 66168815367, "ifOutUcastPkts": 441955780,
"ifOutMulticastPkts": 9140271, "ifOutBroadcastPkts": 8642416, "ifOutDiscards": 0, "ifOutErrors": 0, "ifPromiscuousMode": 1}
Jan 10 10:08:02 DEBUG ( default/core ): Received sFlow packet from [<ipaddress>:8888] version [5] seqno [1471283585]
Jan 10 10:08:02 DEBUG ( default/core ): Received sFlow packet from [<ipaddress>:8888] version [5] seqno [1471283586]
Jan 10 10:08:02 DEBUG ( default/core ): Received sFlow packet from [<ipaddress>:8888] version [5] seqno [1471283587]
If I write to a file instead of kafka, everything gets collected and written out successfully, so this looks to be just a problem with writing to kafka.
Is there any other tuning/configuration available which could speed up writing out to kafka? My config looks like
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent