paddor / cztop Goto Github PK

View Code? Open in Web Editor NEW

35.0 35.0 12.0 839 KB

CZMQ binding for Ruby

License: ISC License

Ruby 99.49% Shell 0.51%

async czmq czmq-binding ffi-bindings ruby zeromq zmq

cztop's People

Contributors

Stargazers

Watchers

Forkers

spacebra particle-iot rashidovich2 joegsn rfestag colstrom kozo2 ged mrkn washu piercelbrooks olleolleolle

cztop's Issues

dyld: lazy symbol binding failed: Symbol not found: _zsys_handler_reset

my env

macOS Sierra
$ ruby -v
ruby 2.3.3p222 (2016-11-21 revision 56859) [x86_64-darwin16]
$ brew info zmq
zeromq: stable 4.2.2 (bottled), HEAD
$ brew info czmq
czmq: stable 4.0.2 (bottled), HEAD
$ gem list
czmq-ffi-gen (0.13.0)
cztop (0.11.4)

what's this mean?

 $ ruby examples/cztop/simple_req_rep/rep.rb
dyld: lazy symbol binding failed: Symbol not found: _zsys_handler_reset
  Referenced from: /usr/local/lib/libczmq.dylib
  Expected in: flat namespace

dyld: Symbol not found: _zsys_handler_reset
  Referenced from: /usr/local/lib/libczmq.dylib
  Expected in: flat namespace

Abort trap: 6

test

Assertion failed: (!s_process_ctx), function zsys_init, file src/zsys.c, line 182. Abort trap: 6

I've been getting this failure about every 1 in 5 runs of a multi-threaded program I'm writing. It occurs whether I'm using CURVE_server/client or not. I'm using PUSH/PULL sockets. I'm wondering what would be most useful to provide to illustrate the issue or if you have any recommendations.

Thanks,

-joe

Collaboration?

This morning I started to replace my gem ffi-rzmq in the celluloid-zmq gem with rbczmq. Then @digitalextremist pointed me to your repository.

Interested in collaborating on this work?

If so, I think @digitalextremist could invite you into our chatrooms on slack where we are doing a lot of celluloid work. Let one of us know. Thanks!

Reimplement CZTop::Poller on top of zmq_poller_*()

Pros

it'll work with SERVER/CLIENT ("thread-safe") sockets as well

Cons

it'll only work on ZMQ >= 4.2
needs manual loop to get events from all registered sockets to build up an array of interesting sockets
- in Ruby this might be slow
- in ZMQ itself this might annoy other people

Solving this issue will help #1.

Install problem on Windows with 0.13.x

It seems that I cannot install correctly cztop 0.13.0 nor 0.13.1 on Windows with Ruby 2.2.6. It works when I install cztop 0.12.2 though.

I found that out via this issue I opened on iRuby: SciRuby/iruby#148
You can find my detailed system information in this linked issue.

When I require cztop, it seems to be looking for Unix paths instead of Windows.

Here's what I get for 0.13.0 (or 0.13.1 for that matter):

(base) C:\Users\julien>gem install cztop -v 0.13.0
Fetching: ffi-1.9.23-x64-mingw32.gem (100%)
Successfully installed ffi-1.9.23-x64-mingw32
Fetching: czmq-ffi-gen-0.15.0.gem (100%)
Successfully installed czmq-ffi-gen-0.15.0
Fetching: cztop-0.13.0.gem (100%)
Successfully installed cztop-0.13.0
Parsing documentation for ffi-1.9.23-x64-mingw32
Installing ri documentation for ffi-1.9.23-x64-mingw32
Parsing documentation for czmq-ffi-gen-0.15.0
Installing ri documentation for czmq-ffi-gen-0.15.0
Parsing documentation for cztop-0.13.0
Installing ri documentation for cztop-0.13.0
Done installing documentation for ffi, czmq-ffi-gen, cztop after 3 seconds
3 gems installed

(base) C:\Users\julien>pry
[1] julien(main)> require 'cztop'

WARNING: ::CZMQ::FFI is not available without libczmq.

LoadError: Could not open library '/usr/local/lib/libzmq.dll': The specified module could not be found.
.
Could not open library '/opt/local/lib/libzmq.dll': The specified module could not be found.
.
Could not open library '/usr/lib64/libzmq.dll': The specified module could not be found.
.
Could not open library 'libzmq': The specified module could not be found.
.
Could not open library 'libzmq.dll': The specified module could not be found.
from C:/Ruby22-x64/lib/ruby/gems/2.2.0/gems/ffi-1.9.23-x64-mingw32/lib/ffi/library.rb:147:in `block in ffi_lib'

Issue dealing with Push Pull Socket

I ran into couple issues when using PUSH/PULL socket.

ISSUE 1:

cztop-0.11.4/lib/cztop/has_ffi_delegate.rb:26:in `attach_ffi_delegate': No such device (Errno::ENODEV)

Over TCP socket. I see the get above error.

Here how my Sender and Receive look.

## receiver.rb
require 'cztop'
@sock = CZTop::Socket::PULL.new("tcp://localhost:4545")
loop { msg = @sock.receive ; puts #{Process.pid} ---- #{msg.to_a}"; sleep 2}

## sender.rb
require 'cztop'

# bind
socket = CZTop::Socket::PUSH.new("tcp://*:4545")
i = 0
loop {  socket << ["simple", "message", sprintf("%d", i)] ; i +=1 }

ISSUE 2:
When using IPC: As per I understand, with PUSH socket one can attach multiple PULL sockets and zeroMQ do a fair scheduling and distribute messages to each PULL socket worker.

But I when I ran the above code over IPC . I see only the one worker (all the time) receiving all the messages sent by server. (Not sure why is that so, may be I have missed some socket option I believe )

## sender.rb
socket = CZTop::Socket::PUSH.new("ipc:///tmp/push-pull.sock")
i = 0
loop {  socket << ["simple", "message", sprintf("%d", i)] ; i +=1 }

## receiver.rb
require 'cztop'
@sock = CZTop::Socket::PULL.new("ipc:///tmp/push-pull.sock")
loop { msg = @sock.receive ; puts #{Process.pid} ---- #{msg.to_a}"; sleep 2}

Just so, I have tested the above with ffi-rzmq seem to work as expected. (both over TCP and IPC socket)

##sender
require 'ffi-rzmq'

context = ZMQ::Context.new(1)
puts "--"
# Socket to send messages on
sender = context.socket(ZMQ::PUSH)
sender.bind("tcp://*:4545")
t = 0  
loop do 
  sender.send_string(t.to_s)
  t += 1
  sleep 2
end

## receiver.rb
require 'ffi-rzmq'

context = ZMQ::Context.new(1)

receiver = context.socket(ZMQ::PULL)
receiver.connect("tcp://localhost:4545")


while true
  receiver.recv_string(msec = '')
  puts "--- #{msec}"
end

Definitely missing something not sure what? Any Clue ?

Support ztimerset

https://github.com/zeromq/czmq#ztimerset---timer-set The bindings are being built for czmq, just missing the top level api in cztop for it.

Use of `wait_readable` and `wait_writable`.

Does Zmsg use Ruby's internal IO operations?

Otherwise, I'd recommend:

      destination.wait_writable

Because the internal design of the Ruby scheduler, if Zmsg is invoking recv/send or similar operations directly, it will prevent even Ruby threads from context switching (unless it also releases the GVL). I'll go take a look at Zmsg to see what it does.

Reported performance difference in throughput tests

Hello,
I have a question regarding expected performance.

https://github.com/methodmissing/rbczmq reports to be able to reach ~200k messages per second.
While cztop seems to reach 37k messages per second.

Do you have any idea of the reason?

FFI::NotFoundError: Function 'zlistx_head' not found on Mac

The error log

irb(main):002:0> require 'cztop'
LoadError: no library specified
    from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/ffi-1.9.10/lib/ffi/library.rb:162:in `ffi_libraries'
    from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/ffi-1.9.10/lib/ffi/library.rb:240:in `attach_function'
    from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/czmq-ffi-gen-0.8.3/lib/czmq-ffi-gen/signals.rb:4:in `<module:FFI>'
    from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/czmq-ffi-gen-0.8.3/lib/czmq-ffi-gen/signals.rb:2:in `<module:CZMQ>'
    from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/czmq-ffi-gen-0.8.3/lib/czmq-ffi-gen/signals.rb:1:in `<top (required)>'
    from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/czmq-ffi-gen-0.8.3/lib/czmq-ffi-gen.rb:4:in `require_relative'
    from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/czmq-ffi-gen-0.8.3/lib/czmq-ffi-gen.rb:4:in `<top (required)>'
    from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/cztop-0.5.0/lib/cztop.rb:1:in `<top (required)>'
    from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /Users/knishida/.rbenv/versions/2.3.1/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from (irb):2
    from /Users/knishida/.rbenv/versions/2.3.1/bin/irb:11:in `<main>'
irb(main):003:0>

How to reproduce the error

brew install rbenv zeromq czmq
rbenv install 2.3.1
rbenv global 2.3.1
gem install cztop
irb

Versions

OSX 10.11.6
rbenv 1.0.0
zeromq 4.1.5
czmq 3.0.2

Support ZMQ 4.1 (or older)

It would be nice if, even though Cztop is new, it supported the latest stable version of ZMQ (or further back for whatever is available by default in common distro repositories). From a comment @paddor posted (celluloid/celluloid-zmq#56 (comment)), this was becoming difficult, and 4.2 was nearly ready to release; that said, I haven't seen much movement on that front (granted, I'm not sure where to look, but a google search didn't bring up much). I was planning to just wait for 4.2 to come out (since I run Manjaro, I would suspect a 4.2 package update wouldn't be far behind). Unfortunately it hasn't come out yet, and I'm itching to play with this again. I try to avoid manually compiling/installing libraries, especially those that are depended on by system components (Cinnamon apparently uses it on my desktop). That's what I have a package manager for anyways =).

I was wondering what the main technical hurdle was for supporting older versions of ZMQ. Would it be feasible to create a few separate branches off cztop to support a handful of common ZMQ versions? I'd be happy to work on making whatever changes are necessary to support at least 4.1, but would be interested in understanding what it would take to ensure other builds could be supported as well (so I don't end up doing a complete one-off if possible).

Ruby hangs after exiting script when using CZTop socket

ruby script on windows 10 64-bit

require 'cztop'

puts ::CZMQ::FFI::CZMQ_VERSION

# => 4.2.1

puts ::CZMQ::FFI::LibZMQ::VERSION

# => 4.3.3

puts RbConfig::CONFIG["ruby_version"]

# => 2.6.0

puts RbConfig::CONFIG["arch"]

# => x64-msvcrt

at_exit do
  puts "Exiting ..."
end


s = CZTop::Socket::PUSH.new

hangs after printing "Exiting ...".

Any ideas how to solve this issue ?

taxi_system example not working

Hi there,

The taxi system example is a great use case to demonstrate the fanciness of this tool, however, it appears that it's not working or I'm doing something wrong.

Given that I've run generate_keys.sh
And I've started the the broker
And started the clients
When I run send_command("driver1", "PICKUP(8.541694,47.376887)")
I get Driver "driver1" has never connected

The broker never gets past line 43, and never says "socket is readable".
I got the same behavior on Ruby 2.3.0 and 2.2.3. OS X installed with --HEAD flag with libsodium with homebrew.

Let me know if I can provide more details.

Thanks,

-joe

Can't use with czmq-ffi-gen 0.9.x

We can't use CZTop with czmq-ffi-gen 0.9x because of spec.add_runtime_dependency "czmq-ffi-gen", "~> 0.8.3": https://github.com/paddor/cztop/blob/master/cztop.gemspec#L21

Could you update the version?