hatamiarash7 / barmak

Benchmark of Kafka libraries for Go

Barmak

This is a simple project to test 4 popular Kafka libraries for Go. Our libraries are :

• Confluent
• Sarama
• Goka
• Kafka-Go

How to

We can get a simple benchmark from these libraries.

func BenchmarkProduceConfluent(b *testing.B) {
    count := b.N

    messages := make([]Message, count)

    for i := 0; i < count; i++ {
        messages[i] = Message{Key: "Confluent", Value: fmt.Sprint(i)}
    }

    Confluent(messages)
}

Run

There is a Makefile for you to run all commands.

Usage: make [target]

  help        Show this help message
  build       Build app's binary
  run         Run the app
  test        Run unit tests
  benchmark   Run benchmark tests

Change configuration in kafka/produce.go :

var (
    host   string   = "localhost:9092"
    topic  string   = "benchmark"
)

Then, execute the following command to run the benchmark:

$ make benchmark

goos: darwin
goarch: arm64
pkg: arash-hatami.ir/Barmak/kafka
BenchmarkProduceSegmentio-8          367           3235080 ns/op
BenchmarkProduceConfluent-8         1137           1006395 ns/op
BenchmarkProduceSarama-8             496           2136077 ns/op
BenchmarkProduceGoka-8                10         104631917 ns/op
PASS
ok      arash-hatami.ir/Barmak/kafka    6.299s
'graphic/operations.png' and 'graphic/time_operations.png' graphics were generated.

And you can see the results in graphic directory.

Total number of operations :

Operation in nanoseconds :

Default process will be handled by the Go testing package. We can also use -benchtime=100x argument to set the minimum amount of time that the benchmark function will run.

Note: You can change the range for values in y axis. Set y_range_max in Makefile as needed.

---

Support 💛

Contributing 🤝

Don't be shy and reach out to us if you want to contribute 😉

1. Fork it !
2. Create your feature branch : git checkout -b my-new-feature
3. Commit your changes : git commit -am 'Add some feature'
4. Push to the branch : git push origin my-new-feature
5. Submit a pull request

Issues

Each project may have many problems. Contributing to the better development of this project by reporting them. 👍

barmak's Issues

Update charts using new AsyncProducer

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update module github.com/lovoo/goka to v1.1.11
• Update module github.com/segmentio/kafka-go to v0.4.47
• Update module github.com/sirupsen/logrus to v1.9.3
• Update module github.com/Shopify/sarama to v1.43.2
• Update module github.com/confluentinc/confluent-kafka-go to v2
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

gomod

go.mod

• go 1.19
• github.com/Shopify/sarama v1.36.0
• github.com/confluentinc/confluent-kafka-go v1.9.2
• github.com/lovoo/goka v1.1.7
• github.com/segmentio/kafka-go v0.4.34
• github.com/sirupsen/logrus v1.9.0

---

• Check this box to trigger a request for Renovate to run again on this repository