jeremyvdw / jackdaw

A Clojure library for the Apache Kafka distributed streaming platform.

Jackdaw ·

Jackdaw is a Clojure library for the Apache Kafka distributed streaming platform. With Jackdaw, you can create and list topics using the AdminClient API, produce and consume records using the Producer and Consumer APIs, and create stream processing applications using the Streams API. Jackdaw also contains functions to serialize and deserialize records as JSON, EDN, and Avro, as well as functions for writing unit and integration tests.

Supported versions

Jackdaw currently only works with Clojure >= 1.10. This is because we are using the datafy protocol which was only introduced in 1.10.

Documentation

You can find all the documentation on cljdoc.

Examples

• Pipe
• Word Count
• Simple Ledger
• Roll Dice

Contributing

We welcome any thoughts or patches. You can reach us in #jackdaw (or open an issue).

Related projects

If you want to get more insight about your topologies, you can use the Topology Grapher library to generate graphs. See an example using jackdaw to check how to integrate it with your topology.

Releasing

Anyone with the appropriate credentials can "cut a release" of jackdaw

• Tags are protected, you will need to be a maintainer or admin to apply them
• Clojars deployment uses a protected env, once a tag is created, the CI job to deploy will need to be reviewed by the clojars env owner

Steps:

1. Review the diff of master vs the latest released tag (e.g. while preparing 0.7.0, I looked at https://github.com/FundingCircle/jackdaw/compare/0.6.9...master to see what was actually merged vs what was in the Changelog). Make a PR to put a date on the version being released and if necessary ensure completeness and consistency of the Changelog
2. Use the Draft a new release feature in github to prepare a release
3. In the "tag version" field, enter the proposed version
4. In the "release title" field, enter "v[version]"
5. In the "describe this release" field, enter the contents of the Changelog and add a credit to the contributors of the release
6. When happy, use the "Publish Release" button to publish the release in github which creates a corresponding git tag
7. Once the tag is seen by circleci, a deployment build is triggered which builds the project and deploys a release to clojars

Steps 2 to 6 is essentially git tag $version -m "$title\n\n$description" && git push --tags

Snapshot release

Snapshot releases can be created by pushing a tag with the format publish-snapshot-semver

License

Copyright © 2017 Funding Circle

Distributed under the BSD 3-Clause License.