Comments (17)
I'm currently working on Windows support for node-rdkafka
. Right now it's building, but there are some linking issues to be resolved, and I don't have WITH_SASL
working just yet.
However, it's only been a day so far that I've been working on it, so I expect to be making headway with some time.
from node-rdkafka.
Windows support is not currently a priority for us. That said, we'd be happy to accept a pull request for it once a release of librdkafka has been made that includes confluentinc/librdkafka#982.
from node-rdkafka.
Hello,
As you can see in this thread : confluentinc/librdkafka#982, librdkafka can now run on Windows platform (master at this time). Can you tell me when you think you can add this feature in node-rdkafka ?
I'm working for a major insurance company and node-rdkafka had been chosen for our node applications, and we prefer to use the same library on Windows and Linux.
Thx
from node-rdkafka.
@mattness now that RC3 of librdkafka is released is there any possibility of providing a install for windows?
from node-rdkafka.
The reason it doesn't work on Windows is not because the compile of librdkafka didn't work on Windows: we just aren't able to prioritize its implementation.
As @mattness said we are happy to accept a PR to implement the changes, but it isn't the librdkafka source that will be the work - it is adapting the gypfile to accommodate for all windows platform specific things, it is generating the header files required for the application to work (config.h) without assuming the presence of (ba)sh. It is testing the software on windows servers, and potentially integrating its testing into the CI pipeline.
I'd be happy to help anyone willing to take a crack at these problems so we could have windows support.
from node-rdkafka.
@webmakersteve Can you provide a very rough estimate of how many developer weeks you'd expect that effort to take?
from node-rdkafka.
Just out of curiosity, is it even necessary to build librdkafka for Windows or would it suffice to simply use the pre-built redistributable?
from node-rdkafka.
There is potential for that pre-built binary to work on Windows, but you would need to configure node-gyp
to look for the library dependency there. It, too, would probably require changes to the code base to work, but may satisfy some of the complexity.
However, I do not think librdkafka
's compilation step is the problem stopping windows support from advancing. The vast majority of the problems relate to the gypfile and how it is generating the Visual Studio solution file given the current config. I imagine many windows specific pieces need to be added to the gypfile regarding that, especially regarding linking, to make this work.
from node-rdkafka.
Just a quick addition to what @webmakersteve said: In addition, node-rdkafka
depends on unistd.h
, which isn't a part of Visual C/C++, so a shim is needed (in my copy I am using an shim by Microsoft under the Apache licence). There are a few more details from what I remember, but I got sidelined with an important project that I need to take care of first before I can return to node-rdkafka
.
from node-rdkafka.
@webmakersteve Thanks for the quick response!
@vsimonian Any idea how long you'll be sidelined? If you expect it will be a while, would you be able to check in what you've got so far to your branch? I'd be happy to take a look at what you have and see if I'm able to move things a bit further down the field.
from node-rdkafka.
@vsimonian The only thing I'm using from unistd is usleep. You can just check what platform the code is running on and use the Windows equivalenet for that particular function instead of importing an entire polyfill.
from node-rdkafka.
@webmakersteve Only thing from unistd.h
being used is usleep
, yes, so I trimmed it down. However, Microsoft's stdlib.h
also does not have rand_r
, because their implementation of rand
is thread-safe unlike GNU rand
. I macro'd it as a replacement in the header file. And it looks like the polyfill I was talking about was actually for getopt
, but there's no use of getopt
in node-rdkafka
, so I removed it.
@RyanMelenaNoesis It might be a while, so I totally don't mind posting up my work so far if you'd like to take a crack at it and get it up and running! Keep in mind I haven't had an opportunity to touch the code too much so you might find some questionable parts. But still I think it's a step forward, just needs to be brought to completion and then cleaned up.
from node-rdkafka.
@RyanMelenaNoesis I have the code up at https://github.com/vsimonian/node-rdkafka/tree/vsimonian-windows
I've sent you an invite for write access.
from node-rdkafka.
@vsimonian Thanks for setting up access. I ended up taking some of what you did but skipping the librdkafka compilation (for the time being) and instead utilizing the pre-built librdkafka redistributable binary. I branched the the Blizzard repo and created a pull request from there. Please let me know if you'd like me to integrate any of the changes back to your fork.
from node-rdkafka.
@RyanMelenaNoesis Glad it was of use! Your PR looks great! Sorry I don't have time to do a more thorough look through it, I only had time to give it a very quick glance.
from node-rdkafka.
@vsimonian Unfortunately, my PR was denied because it does not build librdkafka from source. A decision I find bewildering but that is how it goes.
For anyone interested in using node-rdkafka on Windows immediately, you can point your package.json
dependency to our fork "node-rdkafka": "NoesisLabs/node-rdkafka"
until such a time that someone chooses to implement a build-librdkafka-from-source-on-windows implementation.
from node-rdkafka.
#248 - Prelim pass at windows support that compiles librdkafka from source
from node-rdkafka.
Related Issues (20)
- npm i node-rdkafka Fails installing on github runner HOT 1
- installation gives an error "cp: illegal option -- -" on Mac
- No such configuration property "connections.max.reauth.ms"
- Consumer is picking messages from single partition after adding 3 hours of delayed in consuming. HOT 1
- Some message is missing
- OpenSSL not available at build time HOT 1
- idempotence setting does not seem to work HOT 1
- Thread 15 "rdk:broker-1" received signal SIGSEGV, Segmentation fault. HOT 1
- consume callback has an incorrect typing
- Installation fails if any parent dir contains space
- Segfault when disconnecting right after consuming
- Build error when installing the module HOT 5
- Schema registry options
- _ZN2v88internal20CanHaveInternalFieldEi: symbol not found
- Cannot find ZSTD library on Mac M1 machine HOT 1
- Consuming messages from each partition separately HOT 2
- Denied Operation is getting triggered in node-rdkafka with incorrect principal
- Question: handling consumer stream errors gracefully (stream error, event.error)
- [FEATURE REQUEST] Allow to pass arbitrary compilation options to librdkafka via env variables
- [FEATURE REQUEST] Provide prebuilt binaries for each target triples HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from node-rdkafka.