Can I use dynamic librdkafka on my system? about rust-rdkafka

[fede1024]

I made a new branch, dyn_lib, that will check whether the right version of librdkafka is installed and if that's the case will link it dynamically. I've tested on my ubuntu machine and it seems to work.

Please let me know if it works on yours as well.

[fede1024]

It bundles librdkafka 0.9.5, as stated in the readme. The git commands are actually fairly common in other libraries as well (see libgit2-sys for example), and the SHA is defined by the git submodule, currently pinned to 0d540ab4e78a3e36.

It's common for other Rust wrappers to check if the C library is installed on the system first, and compile from sources if that's not the case. However in case of librdkafka is unlikely to be already installed on the system, and to have the latest version (the one we normally target).

I can give it a try on Arch and see if there are ways to make it work. Suggestions are welcome! :)

[lilydjwg]

I'll appreciate if you check my system first, or use a feature. It's likely to be there where people develop kafka tools.

I dug a little further, and found that it's the build script of rdkafka that requires static lz4 no matter what. (Fortunately it doesn't require static openssl things as I though previously.) It's strange that the AUR package builds fine for me (and link to dynamic lz4).

FYI, replacing lz4 with a custom build is dangerous for Arch users because systemd stuff (among a lot others) depends on it, and it's known to break compatibility without bumping soname version (so when it breaks you won't know by checking statically).

[lilydjwg]

The branch works well on my Arch.

[fbernier]

I also use Arch and was experiencing the same issue. I tried the dyn_lib branch with success.

Unfortunately, when specifying rust-rdkafka as a dependency in a project pointing to the git repo and using the dyn_lib branch, it still tried to compile and fails.

I'm still trying to figure out why.

[fbernier]

The error I was getting was this one, and using the env variable described here fixed it: https://github.com/alexcrichton/pkg-config-rs/blob/3cf9b12d06b2991cb3af2218d413539b0f62b2a7/src/lib.rs#L144

[fede1024]

Thanks! Do you think there is anything we can add to the documentation for the next person that will run into the same issue? Or maybe in the build code?

[fede1024]

The dyn_lib branch has been merged to master.

[fbernier]

@fede1024 I don't think anything more should be changed in the code, but I don't fully understand this pkg-config cross compilation thing so I can't recommend something yet.

Thanks for the work on this PR.