Comments (5)
from kafka.
Initial code done, but rd_kafka_produce_batch isn't in older librdkafka packages (build fails on travis cos librdkafka too old).
Find release it appeared to see if it may be possible to add condition to add when build with newer release.
from kafka.
Added in librdkafka release 0.8.4 https://github.com/edenhill/librdkafka/releases/tag/0.8.4
from kafka.
Looks like batch supported added in 0.8.4 but didn't support RD_KAFKA_MSG_F_PARTITION at this stage …
confluentinc/librdkafka@b8d0b84
Looks like 0.11.4
from kafka.
Support for batch publish.
Example use:
/ send 2 msgs to any partition for topic, use default key generation
.kfk.BatchPub[;.kfk.PARTITION_UA;batchMsg;""]each(topic1;topic2);
/ send 2 msgs to partition 1 for topic, use default key generation
.kfk.BatchPub[;1i;batchMsg;""]each(topic1;topic2);
/ send 2 msgs. First to partition 0 and second to partition 1. Use keys provided for each msg.
.kfk.BatchPub[;(0i;1i);batchMsg;batchKeys]each(topic1;topic2);
* @param x Topic Index (prev created)
* @param y Partition to use for all message (int) or partition per message (list of ints)
* @param z Payload for all messages (mixed list containing either bytes or string).
* @param r Key. Empty string to use auto key for all messages, or key per message (mixed list containing either bytes or string)
* @returns Integer list with a status for each send message (zero indicates success)
* reference: https://github.com/edenhill/librdkafka/blob/master/src/rdkafka.h (rd_kafka_resp_err_t)
Requires librdkafka be release 0.11.4 and above.
If older, function still exists but returns an error saying its unsupported.
from kafka.
Related Issues (20)
- Bad config can cause consumer not to get data HOT 4
- Kfk.sub HOT 6
- Updating librdkafka (2.2.0 to 2.3.0) can present build error
- Metadata on consumer can present UNKNOWN_TOPIC_OR_PART error HOT 6
- .kfk.CommitOffsets option for a single partition and offset HOT 1
- Consumer not working on osx (librdkafka 1.5.2) due to invalid config
- q process can hang on exit if kafka topic/client is not cleaned up HOT 1
- calling .kfk.ClientMemberId on a producer client causes segfault HOT 1
- Stale consumers after commit
- kafka using mTLS for kdb+ HOT 1
- Application of AssignOffsets results in purge of current assignment. HOT 1
- SASL/Kerberos authentication HOT 4
- Stale consumers after commit offsets HOT 2
- .kfk.Metadata returns topic information as lists of dictionaries instead of tables HOT 4
- Getting error when trying to load kfk.q HOT 6
- Issue with connecting AWS MSK kafka broker URL HOT 4
- Issue with special character HOT 1
- Failed to get current position offset HOT 6
- Docs should be included with release HOT 1
- Update docs with latest info
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kafka.