Before running any application or Spring test producing sample messages, run
docker-compose up
to start a local Kafka deployment.
Then, create a partitioned topic, e.g.:
./kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 3 --topic some_topic
Run plain Kafka consumer by starting KafkaConsumerApp
. Configuration is held by application.properties
file.
You can start it using BootApplication
class. Various containers and message listeners have some parts of the code commented out, so only one type of listener can be started at once.
test
package contains a sample test running a simple Kafka producer to feed required topic with some (partitioned) data.
The Micronaut version of Kafka consumer can be started by running MicronautApplicaiton
class. As in Sppring Boot example, here you can find some code commented out as well.
Implementation for Akka Streams contains three apps.
PlainRecordApp
uses a plain source mechanism, consuming records one by one on the same thread.
PartitionedApp
process messages in parallel, with a dedicated thread per partition.
CommittableApp
provides an example of an application with manual commits.
Kafka vertices are implemented by vertx-kafka-client component.
Build project using goal from vertx-gradle-plugin:
./gradlew :vert-x:build
You can then start the application by calling java -jar vert-x-1.0-all.jar
. It starts single-threaded Kafka consumer. You can run thread-per-partition consumer by providing -Dconsumer.type=multi
parameter.