paddor / cztop Goto Github PK
View Code? Open in Web Editor NEWCZMQ binding for Ruby
License: ISC License
CZMQ binding for Ruby
License: ISC License
my env
macOS Sierra
$ ruby -v
ruby 2.3.3p222 (2016-11-21 revision 56859) [x86_64-darwin16]
$ brew info zmq
zeromq: stable 4.2.2 (bottled), HEAD
$ brew info czmq
czmq: stable 4.0.2 (bottled), HEAD
$ gem list
czmq-ffi-gen (0.13.0)
cztop (0.11.4)
what's this mean?
$ ruby examples/cztop/simple_req_rep/rep.rb
dyld: lazy symbol binding failed: Symbol not found: _zsys_handler_reset
Referenced from: /usr/local/lib/libczmq.dylib
Expected in: flat namespace
dyld: Symbol not found: _zsys_handler_reset
Referenced from: /usr/local/lib/libczmq.dylib
Expected in: flat namespace
Abort trap: 6
I've been getting this failure about every 1 in 5 runs of a multi-threaded program I'm writing. It occurs whether I'm using CURVE_server/client or not. I'm using PUSH/PULL sockets. I'm wondering what would be most useful to provide to illustrate the issue or if you have any recommendations.
Thanks,
-joe
This morning I started to replace my gem ffi-rzmq
in the celluloid-zmq
gem with rbczmq
. Then @digitalextremist pointed me to your repository.
Interested in collaborating on this work?
If so, I think @digitalextremist could invite you into our chatrooms on slack where we are doing a lot of celluloid work. Let one of us know. Thanks!
Solving this issue will help #1.
It seems that I cannot install correctly cztop 0.13.0 nor 0.13.1 on Windows with Ruby 2.2.6. It works when I install cztop 0.12.2 though.
I found that out via this issue I opened on iRuby: SciRuby/iruby#148
You can find my detailed system information in this linked issue.
When I require cztop, it seems to be looking for Unix paths instead of Windows.
Here's what I get for 0.13.0 (or 0.13.1 for that matter):
(base) C:\Users\julien>gem install cztop -v 0.13.0
Fetching: ffi-1.9.23-x64-mingw32.gem (100%)
Successfully installed ffi-1.9.23-x64-mingw32
Fetching: czmq-ffi-gen-0.15.0.gem (100%)
Successfully installed czmq-ffi-gen-0.15.0
Fetching: cztop-0.13.0.gem (100%)
Successfully installed cztop-0.13.0
Parsing documentation for ffi-1.9.23-x64-mingw32
Installing ri documentation for ffi-1.9.23-x64-mingw32
Parsing documentation for czmq-ffi-gen-0.15.0
Installing ri documentation for czmq-ffi-gen-0.15.0
Parsing documentation for cztop-0.13.0
Installing ri documentation for cztop-0.13.0
Done installing documentation for ffi, czmq-ffi-gen, cztop after 3 seconds
3 gems installed
(base) C:\Users\julien>pry
[1] julien(main)> require 'cztop'
WARNING: ::CZMQ::FFI is not available without libczmq.
LoadError: Could not open library '/usr/local/lib/libzmq.dll': The specified module could not be found.
.
Could not open library '/opt/local/lib/libzmq.dll': The specified module could not be found.
.
Could not open library '/usr/lib64/libzmq.dll': The specified module could not be found.
.
Could not open library 'libzmq': The specified module could not be found.
.
Could not open library 'libzmq.dll': The specified module could not be found.
from C:/Ruby22-x64/lib/ruby/gems/2.2.0/gems/ffi-1.9.23-x64-mingw32/lib/ffi/library.rb:147:in `block in ffi_lib'
I ran into couple issues when using PUSH/PULL socket.
ISSUE 1:
cztop-0.11.4/lib/cztop/has_ffi_delegate.rb:26:in `attach_ffi_delegate': No such device (Errno::ENODEV)
Over TCP socket. I see the get above error.
Here how my Sender and Receive look.
## receiver.rb
require 'cztop'
@sock = CZTop::Socket::PULL.new("tcp://localhost:4545")
loop { msg = @sock.receive ; puts #{Process.pid} ---- #{msg.to_a}"; sleep 2}
## sender.rb
require 'cztop'
# bind
socket = CZTop::Socket::PUSH.new("tcp://*:4545")
i = 0
loop { socket << ["simple", "message", sprintf("%d", i)] ; i +=1 }
ISSUE 2:
When using IPC
: As per I understand, with PUSH socket one can attach multiple PULL sockets and zeroMQ do a fair scheduling and distribute messages to each PULL socket worker.
But I when I ran the above code over IPC
. I see only the one worker (all the time) receiving all the messages sent by server. (Not sure why is that so, may be I have missed some socket option
I believe )
## sender.rb
socket = CZTop::Socket::PUSH.new("ipc:///tmp/push-pull.sock")
i = 0
loop { socket << ["simple", "message", sprintf("%d", i)] ; i +=1 }
## receiver.rb
require 'cztop'
@sock = CZTop::Socket::PULL.new("ipc:///tmp/push-pull.sock")
loop { msg = @sock.receive ; puts #{Process.pid} ---- #{msg.to_a}"; sleep 2}
Just so, I have tested the above with ffi-rzmq
seem to work as expected. (both over TCP and IPC socket)
##sender
require 'ffi-rzmq'
context = ZMQ::Context.new(1)
puts "--"
# Socket to send messages on
sender = context.socket(ZMQ::PUSH)
sender.bind("tcp://*:4545")
t = 0
loop do
sender.send_string(t.to_s)
t += 1
sleep 2
end
## receiver.rb
require 'ffi-rzmq'
context = ZMQ::Context.new(1)
receiver = context.socket(ZMQ::PULL)
receiver.connect("tcp://localhost:4545")
while true
receiver.recv_string(msec = '')
puts "--- #{msec}"
end
Definitely missing something not sure what? Any Clue ?
https://github.com/zeromq/czmq#ztimerset---timer-set The bindings are being built for czmq, just missing the top level api in cztop for it.
Does Zmsg
use Ruby's internal IO operations?
destination.wait_writable
Because the internal design of the Ruby scheduler, if Zmsg
is invoking recv/send
or similar operations directly, it will prevent even Ruby threads from context switching (unless it also releases the GVL). I'll go take a look at Zmsg
to see what it does.
Hello,
I have a question regarding expected performance.
https://github.com/methodmissing/rbczmq reports to be able to reach ~200k messages per second.
While cztop
seems to reach 37k messages per second.
Do you have any idea of the reason?
irb(main):002:0> require 'cztop'
LoadError: no library specified
from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/ffi-1.9.10/lib/ffi/library.rb:162:in `ffi_libraries'
from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/ffi-1.9.10/lib/ffi/library.rb:240:in `attach_function'
from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/czmq-ffi-gen-0.8.3/lib/czmq-ffi-gen/signals.rb:4:in `<module:FFI>'
from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/czmq-ffi-gen-0.8.3/lib/czmq-ffi-gen/signals.rb:2:in `<module:CZMQ>'
from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/czmq-ffi-gen-0.8.3/lib/czmq-ffi-gen/signals.rb:1:in `<top (required)>'
from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/czmq-ffi-gen-0.8.3/lib/czmq-ffi-gen.rb:4:in `require_relative'
from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/czmq-ffi-gen-0.8.3/lib/czmq-ffi-gen.rb:4:in `<top (required)>'
from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/cztop-0.5.0/lib/cztop.rb:1:in `<top (required)>'
from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from (irb):2
from /Users/knishida/.rbenv/versions/2.3.1/bin/irb:11:in `<main>'
irb(main):003:0>
brew install rbenv zeromq czmq
rbenv install 2.3.1
rbenv global 2.3.1
gem install cztop
irb
It would be nice if, even though Cztop is new, it supported the latest stable version of ZMQ (or further back for whatever is available by default in common distro repositories). From a comment @paddor posted (celluloid/celluloid-zmq#56 (comment)), this was becoming difficult, and 4.2 was nearly ready to release; that said, I haven't seen much movement on that front (granted, I'm not sure where to look, but a google search didn't bring up much). I was planning to just wait for 4.2 to come out (since I run Manjaro, I would suspect a 4.2 package update wouldn't be far behind). Unfortunately it hasn't come out yet, and I'm itching to play with this again. I try to avoid manually compiling/installing libraries, especially those that are depended on by system components (Cinnamon apparently uses it on my desktop). That's what I have a package manager for anyways =).
I was wondering what the main technical hurdle was for supporting older versions of ZMQ. Would it be feasible to create a few separate branches off cztop to support a handful of common ZMQ versions? I'd be happy to work on making whatever changes are necessary to support at least 4.1, but would be interested in understanding what it would take to ensure other builds could be supported as well (so I don't end up doing a complete one-off if possible).
ruby script on windows 10 64-bit
require 'cztop'
puts ::CZMQ::FFI::CZMQ_VERSION
# => 4.2.1
puts ::CZMQ::FFI::LibZMQ::VERSION
# => 4.3.3
puts RbConfig::CONFIG["ruby_version"]
# => 2.6.0
puts RbConfig::CONFIG["arch"]
# => x64-msvcrt
at_exit do
puts "Exiting ..."
end
s = CZTop::Socket::PUSH.new
hangs after printing "Exiting ...".
Any ideas how to solve this issue ?
Hi there,
The taxi system example is a great use case to demonstrate the fanciness of this tool, however, it appears that it's not working or I'm doing something wrong.
Given that I've run generate_keys.sh
And I've started the the broker
And started the clients
When I run send_command("driver1", "PICKUP(8.541694,47.376887)")
I get Driver "driver1" has never connected
The broker never gets past line 43, and never says "socket is readable".
I got the same behavior on Ruby 2.3.0 and 2.2.3. OS X installed with --HEAD
flag with libsodium with homebrew.
Let me know if I can provide more details.
Thanks,
-joe
We can't use CZTop with czmq-ffi-gen 0.9x because of spec.add_runtime_dependency "czmq-ffi-gen", "~> 0.8.3"
: https://github.com/paddor/cztop/blob/master/cztop.gemspec#L21
Could you update the version?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.