celluloid / celluloid-zmq Goto Github PK
View Code? Open in Web Editor NEWUNMAINTAINED: See celluloid/celluloid#779 - Celluloid actors that talk over the 0MQ protocol
Home Page: https://celluloid.io
License: MIT License
UNMAINTAINED: See celluloid/celluloid#779 - Celluloid actors that talk over the 0MQ protocol
Home Page: https://celluloid.io
License: MIT License
Not really sure if this is sensible at all, but it would be interesting. FFI tends to be a pain, so having a pure Java alternative would be very nice. The APIs seem similar enough, so it shouldn't be particularly difficult to fix. I can look into this, if you guys agree that this is a good idea?
JeroMQ: https://github.com/zeromq/jeromq
jrzmq: https://github.com/outcastgeek/jrzmq
Does the ! have a special meaning or is that a typo?
def run
loop { handle_message! @socket.read }
end
def handle_message(message)
...
The readme file says to "see LICENSE.txt for more information," but there is no License.txt in the project.
I propose to release 0.17 as at this point the master branch is quite stable
In early analysis of 0MQ security underlying DCell, it seems the best way to bring authentication and encryption to DCell is by implementing CurveZMQ:
https://github.com/zeromq/libcurve
Research the API; if possible and wise: integrate libcurve to provide security for DCell.
Coverage:
Probably a github newbie question but I need the Celluloid::ZMQ::Socket setsockopt function that tobert did:
I see it in his github clone, but not in the celluloid-zmq master.
By the way I've done my first little demo with celluloid-zmq and other than patching in setsockopt it was nice and easy.
It would be great to have an example which runs in multiple processes which communicate between each other, and not just separate threads which communicate within the same process.
Hi,
how to properly start and handle zmq_proxy or at least ZMQ::Poll?
For example for ROUTER and DEALER socket pair.
I keep getting segfaults from "assertion failed" error messages from programs involving ZMQ. Even just running rake spec
in this project brings those errors. I tried on OSX 10.10 and 10.11, same result. I tried with and without lib sodium, same result. Then I noticed that maybe it's because https://github.com/chuckremes/ffi-rzmq has been put into maintenance mode. Apparently https://github.com/methodmissing/rbczmq is the way to go. So I'm thinking about porting celluloid-zmq to that library. Is that a good idea? Or is the low-level approach of ffi-rzmq actually needed by celluloid-zmq?
$ rake spec
/usr/local/Cellar/ruby/2.4.1_1/bin/ruby -I/usr/local/lib/ruby/gems/2.4.0/gems/rspec-support-3.5.0/lib:/usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/lib /usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb
/opt/dev_opensource/celluloid-zmq/celluloid-zmq/culture/gems/loader.rb:7: warning: method redefined; discarding old []
/opt/dev_opensource/celluloid-zmq/culture/gems/loader.rb:7: warning: previous definition of [] was here
/opt/dev_opensource/celluloid-zmq/celluloid-zmq/culture/gems/loader.rb:16: warning: method redefined; discarding old []
/opt/dev_opensource/celluloid-zmq/culture/gems/loader.rb:16: warning: previous definition of [] was here
/opt/dev_opensource/celluloid-zmq/celluloid-zmq/culture/gems/loader.rb:93: warning: method redefined; discarding old loader
/opt/dev_opensource/celluloid-zmq/culture/gems/loader.rb:93: warning: previous definition of loader was here
/opt/dev_opensource/celluloid-zmq/celluloid-zmq/lib/celluloid/zmq/version.rb:3: warning: already initialized constant Celluloid::ZMQ::VERSION
/opt/dev_opensource/celluloid-zmq/lib/celluloid/zmq/version.rb:3: warning: previous definition of VERSION was here
[Coveralls] Set up the SimpleCov formatter.
[Coveralls] Using SimpleCov's default settings.
[Coveralls] Outside the CI environment, not sending data.
/usr/local/lib/ruby/gems/2.4.0/bundler/gems/celluloid-96fdf84c0d92/spec/support/configure_rspec.rb:7:in `block in <top (required)>': undefined method `verbose_retry=' for #<RSpec::Core::Configuration:0x007f92789055d0> (NoMethodError)
from /usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/lib/rspec/core.rb:98:in `configure'
from /usr/local/lib/ruby/gems/2.4.0/bundler/gems/celluloid-96fdf84c0d92/spec/support/configure_rspec.rb:1:in `<top (required)>'
from /usr/local/lib/ruby/gems/2.4.0/bundler/gems/celluloid-96fdf84c0d92/lib/celluloid/rspec.rb:54:in `require'
from /usr/local/lib/ruby/gems/2.4.0/bundler/gems/celluloid-96fdf84c0d92/lib/celluloid/rspec.rb:54:in `block in <top (required)>'
from /usr/local/lib/ruby/gems/2.4.0/bundler/gems/celluloid-96fdf84c0d92/lib/celluloid/rspec.rb:53:in `each'
from /usr/local/lib/ruby/gems/2.4.0/bundler/gems/celluloid-96fdf84c0d92/lib/celluloid/rspec.rb:53:in `<top (required)>'
from /opt/dev_opensource/celluloid-zmq/spec/spec_helper.rb:8:in `require'
from /opt/dev_opensource/celluloid-zmq/spec/spec_helper.rb:8:in `<top (required)>'
from /usr/local/Cellar/ruby/2.4.1_1/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /usr/local/Cellar/ruby/2.4.1_1/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/lib/rspec/core/configuration.rb:1394:in `block in requires='
from /usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/lib/rspec/core/configuration.rb:1394:in `each'
from /usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/lib/rspec/core/configuration.rb:1394:in `requires='
from /usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/lib/rspec/core/configuration_options.rb:112:in `block in process_options_into'
from /usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/lib/rspec/core/configuration_options.rb:111:in `each'
from /usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/lib/rspec/core/configuration_options.rb:111:in `process_options_into'
from /usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/lib/rspec/core/configuration_options.rb:21:in `configure'
from /usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/lib/rspec/core/runner.rb:99:in `setup'
from /usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/lib/rspec/core/runner.rb:86:in `run'
from /usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/lib/rspec/core/runner.rb:71:in `run'
from /usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/lib/rspec/core/runner.rb:45:in `invoke'
from /usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/exe/rspec:4:in `<main>'
/usr/local/Cellar/ruby/2.4.1_1/bin/ruby -I/usr/local/lib/ruby/gems/2.4.0/gems/rspec-support-3.5.0/lib:/usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/lib /usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb failed
Fail to find a google group or an irc channel had resisted me to post the query over here.
Looking through the README code. I'm trying to understand how does Celluloid-zmq handle Thread Safety when used with ffi-rzmq
.
Googling around I found this https://stackoverflow.com/questions/38773624/thread-issue-in-zeromq-ruby
But no concrete answer for it.
I see the Celluloid-zmq is moved to CZMQ but we are using the version that still uses ffi-rzmq
support.
getting runtime error:
.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/ffi-rzmq-0.9.6/lib/ffi-rzmq/libzmq.rb:273:in `module:ZMQ': The libzmq version 2.0.7 is incompatible with ffi-rzmq. (LoadError)
using rbenv with 1.9.3-p194 and gems :
$ gem list
*** LOCAL GEMS ***
addressable (2.3.2)
bigdecimal (1.1.0)
builder (3.0.0)
celluloid (0.12.1.pre)
celluloid-io (0.12.1)
celluloid-zmq (0.12.1)
certified (0.1.1)
cucumber (0.10.0)
dcell (0.11.0, 0.10.0)
diff-lcs (1.1.2)
dynect_rest (0.4.3)
expectations (2.0.0)
facter (1.6.13)
ffi (1.1.5)
ffi-rzmq (0.9.6)
gherkin (2.3.3)
hiredis (0.4.5)
http (0.3.0)
http_parser.rb (0.5.3)
io-console (0.3)
json (1.5.4, 1.4.6)
mime-types (1.19)
minitest (2.5.1)
mocha (0.9.11)
moneta (0.6.0)
netrc (0.7.7)
nio4r (0.4.0)
parallel (0.5.18)
rack (1.4.1)
rake (0.9.2.2, 0.8.7)
rdoc (3.9.4)
redis (3.0.1)
redis-namespace (1.2.1)
reel (0.0.2)
rest-client (1.6.7)
rspec (2.4.0)
rspec-core (2.4.0)
rspec-expectations (2.4.0)
rspec-mocks (2.4.0)
term-ansicolor (1.0.5)
timers (1.0.1)
websocket_parser (0.1.0)
tried standard gems as well as rebuilding all celluloid celluloid-io celluloid-zmq dcell ffi ffi-rzmq with same result.
running Ubuntu 10.04.4 LTS on x86_64
i tried with the standard ubuntu packages and without :
$ dpkg -l | egrep -i "zmq|zeromq|0mq"
rc libzmq0 2.0.10-1build1 The ZeroMQ messaging library
rc libzmq1 2.2.0-1chl1~lucid1 ZeroMQ lightweight messaging kernel (shared
Am I missing something, or does this project not have any tests?
I don't see any way to write data to a specific topic, although the SubSocket can subscribe. Am I missing something, or was this overlooked?
Thanks
If a dealer issues lots of async messages, the reactor gets deadlocked.
Here are the code snippets to reproduce the issue: router and dealer
Depending on your test box you might have to increase the number of asynchronous requests. In my case ~2000 was sufficient.
A call trace indicating the deadlock:
^CD, [2015-03-15T01:00:00.369225 #9950] DEBUG -- : Terminating 1 actor...
^CW, [2015-03-15T01:00:02.347057 #9950] WARN -- : Terminating task: type=:call, meta={:method_name=>:run}, status=:zmqwait
Celluloid::TaskFiber backtrace unavailable. Please try `Celluloid.task_class = Celluloid::TaskThread` if you need backtraces here.
/home/niam/.gem/ruby/2.2.0/bundler/gems/ffi-rzmq-eba1101e44f1/lib/ffi-rzmq/util.rb:45:in `zmq_strerror': Interrupt
from /home/niam/.gem/ruby/2.2.0/bundler/gems/ffi-rzmq-eba1101e44f1/lib/ffi-rzmq/util.rb:45:in `error_string'
from /projects/celluloid/celluloid-zmq/lib/celluloid/zmq/waker.rb:30:in `block in signal'
from /projects/celluloid/celluloid-zmq/lib/celluloid/zmq/waker.rb:28:in `synchronize'
from /projects/celluloid/celluloid-zmq/lib/celluloid/zmq/waker.rb:28:in `signal'
from /home/niam/.gem/ruby/2.2.0/bundler/gems/celluloid-21b8c5bd5e65/lib/celluloid/evented_mailbox.rb:29:in `<<'
from /home/niam/.gem/ruby/2.2.0/bundler/gems/celluloid-21b8c5bd5e65/lib/celluloid/proxies/actor_proxy.rb:35:in `terminate!'
from /home/niam/.gem/ruby/2.2.0/bundler/gems/celluloid-21b8c5bd5e65/lib/celluloid/proxies/cell_proxy.rb:65:in `terminate!'
from /home/niam/.gem/ruby/2.2.0/bundler/gems/celluloid-21b8c5bd5e65/lib/celluloid/actor_system.rb:72:in `block (2 levels) in shutdown'
from /home/niam/.gem/ruby/2.2.0/bundler/gems/celluloid-21b8c5bd5e65/lib/celluloid/actor_system.rb:70:in `each'
from /home/niam/.gem/ruby/2.2.0/bundler/gems/celluloid-21b8c5bd5e65/lib/celluloid/actor_system.rb:70:in `block in shutdown'
from /usr/lib/ruby/2.2.0/timeout.rb:89:in `block in timeout'
from /usr/lib/ruby/2.2.0/timeout.rb:34:in `block in catch'
from /usr/lib/ruby/2.2.0/timeout.rb:34:in `catch'
from /usr/lib/ruby/2.2.0/timeout.rb:34:in `catch'
from /usr/lib/ruby/2.2.0/timeout.rb:104:in `timeout'
from /home/niam/.gem/ruby/2.2.0/bundler/gems/celluloid-21b8c5bd5e65/lib/celluloid/actor_system.rb:66:in `shutdown'
from /home/niam/.gem/ruby/2.2.0/bundler/gems/celluloid-21b8c5bd5e65/lib/celluloid.rb:156:in `shutdown'
from /home/niam/.gem/ruby/2.2.0/bundler/gems/celluloid-21b8c5bd5e65/lib/celluloid.rb:145:in `block in register_shutdown'
/home/niam/.gem/ruby/2.2.0/bundler/gems/ffi-rzmq-eba1101e44f1/lib/ffi-rzmq/util.rb:38:in `zmq_errno': Interrupt
from /home/niam/.gem/ruby/2.2.0/bundler/gems/ffi-rzmq-eba1101e44f1/lib/ffi-rzmq/util.rb:38:in `errno'
from /home/niam/.gem/ruby/2.2.0/bundler/gems/ffi-rzmq-eba1101e44f1/lib/ffi-rzmq/util.rb:45:in `error_string'
from /projects/celluloid/celluloid-zmq/lib/celluloid/zmq/waker.rb:30:in `block in signal'
from /projects/celluloid/celluloid-zmq/lib/celluloid/zmq/waker.rb:28:in `synchronize'
from /projects/celluloid/celluloid-zmq/lib/celluloid/zmq/waker.rb:28:in `signal'
from /home/niam/.gem/ruby/2.2.0/bundler/gems/celluloid-21b8c5bd5e65/lib/celluloid/evented_mailbox.rb:29:in `<<'
from /home/niam/.gem/ruby/2.2.0/bundler/gems/celluloid-21b8c5bd5e65/lib/celluloid/proxies/future_proxy.rb:30:in `method_missing'
from ./zmq-test-dealer.rb:44:in `block in <main>'
from ./zmq-test-dealer.rb:43:in `times'
from ./zmq-test-dealer.rb:43:in `<main>'
I've been debugging this for a looong time today. Finally I found that it's class hierarchies deeper than one level that simply won't work on Celluloid::ZMQ 0.14.0 (perhaps this applies to "standard" Celluloid as well - I don't know).
Here's some simple example code that works on 0.13.0 but not on 0.14.0, if you have both installed just switch to 0.14 using the ENV var CELLULOID_14 (set to anything).
if ENV.key?('CELLULOID_14')
gem 'celluloid-zmq', '=0.14.0'
gem 'celluloid', '=0.14.0'
#gem 'celluloid-io', '=0.14.0'
else
gem 'celluloid-io', '=0.13.0'
gem 'celluloid-zmq', '=0.13.0'
gem 'celluloid', '=0.13.0'
end
require "celluloid/zmq"
puts Celluloid::VERSION
if (Celluloid::VERSION.split('.').map { |v| v.to_i })[1]<14
puts "Patching Celluloid Socket since Celluloid version is <0.14"
module Celluloid
module ZMQ
class Socket
def identity=(value)
@socket.identity = value
end
def identity
@socket.identity
end
end
end
end
end
class ZMQEntity
include Celluloid::ZMQ
def initialize(identity)
@identity = identity
end
def socket
@socket ||= Celluloid::ZMQ::RouterSocket.new.tap do |socket|
socket.identity = @identity
end
end
def read
puts "#{self.class} reading socket"
msg = [socket.read]
puts "#{self.class} read a part"
msg << socket.read while socket.more_parts?
puts "#{self.class} read the rest"
msg
end
def start
if @connect_to
socket.connect(@connect_to)
else
socket.bind(@bind_to)
end
async.run
end
def write(msg, to)
puts "write #{[to,msg].flatten.compact.inspect}"
socket.write([to,msg].flatten.compact)
end
def run
loop { async.handle_message read }
end
def handle_message(msg)
from, payload = msg[-2..-1]
receive_message(payload, from)
end
def receive_message(payload, from)
end
end
class Blah < ZMQEntity
end
class Client < Blah
def initialize(identity, destination, connect_to)
@identity = identity
@destination = destination
@connect_to = connect_to
end
def start
super
after(2) do
puts "initial writing stuff"
write("initial hello from client", @destination)
end
every(5) do
puts "writing stuff (5)"
write("hello from client (5)", @destination)
end
every(8) do
puts "writing stuff (8)"
write("hello from client (8)", @destination)
end
end
def receive_message(payload, from)
puts "client got message from #{from}: '#{payload}'"
end
end
class Server < Blah
def initialize(identity, bind_to)
@identity = identity
@bind_to = bind_to
end
def receive_message(payload, from)
puts "server got message '#{payload}' from #{from}"
write("hi there #{from}!", from)
end
end
client = Client.new("client", "server", "tcp://127.0.0.1:7788")
server = Server.new("server", "tcp://127.0.0.1:7788")
puts "starting client"
client.start
puts "starting server"
server.start
sleep
I am trying to get even the examples to work on my desktop (Manjaro - with zmq-4.1.2-3 installed). I am able to run Celluloid::ZMQ.init, and include Celluloid::ZMQ in a class. However, as soon as I attempt to instantiate the class, I get the above assertion and my process exits.
Is celluloid-zmq compatible with zmq-4.1.2-3? If not, which versions is it compatible with? Any thoughts on how I can troubleshoot this issue?
https://github.com/celluloid/celluloid-zmq/blob/master/lib/celluloid/zmq/mailbox.rb#L1-2
Because of this change:
Depend on Celluloid::EventedMailbox instead of Celluloid::IO::Mailbox
the require of celluloid-io is no longer needed.
Updated to master to try the new cztop version of celluloid-zmq. Getting this error:
/Users/tpitale/.gem/ruby/2.2.2/gems/ffi-1.9.10/lib/ffi/library.rb:261:in `attach_function': Function 'zlistx_head' not found in [/usr/local/lib/libczmq.dylib] (FFI::NotFoundError)
from /Users/tpitale/.gem/ruby/2.2.2/gems/czmq-ffi-gen-0.8.3/lib/czmq-ffi-gen/czmq/ffi.rb:410:in `<module:FFI>'
from /Users/tpitale/.gem/ruby/2.2.2/gems/czmq-ffi-gen-0.8.3/lib/czmq-ffi-gen/czmq/ffi.rb:10:in `<module:CZMQ>'
from /Users/tpitale/.gem/ruby/2.2.2/gems/czmq-ffi-gen-0.8.3/lib/czmq-ffi-gen/czmq/ffi.rb:9:in `<top (required)>'
from /Users/tpitale/.gem/ruby/2.2.2/gems/czmq-ffi-gen-0.8.3/lib/czmq-ffi-gen.rb:1:in `require_relative'
from /Users/tpitale/.gem/ruby/2.2.2/gems/czmq-ffi-gen-0.8.3/lib/czmq-ffi-gen.rb:1:in `<top (required)>'
from /Users/tpitale/.gem/ruby/2.2.2/gems/cztop-0.4.0/lib/cztop.rb:1:in `require'
from /Users/tpitale/.gem/ruby/2.2.2/gems/cztop-0.4.0/lib/cztop.rb:1:in `<top (required)>'
from /Users/tpitale/.gem/ruby/2.2.2/bundler/gems/celluloid-zmq-6e4b9eded922/lib/celluloid/zmq.rb:1:in `require'
from /Users/tpitale/.gem/ruby/2.2.2/bundler/gems/celluloid-zmq-6e4b9eded922/lib/celluloid/zmq.rb:1:in `<top (required)>'
from /Users/tpitale/.gem/ruby/2.2.2/bundler/gems/celluloid-zmq-6e4b9eded922/lib/celluloid/zmq/current.rb:2:in `require'
from /Users/tpitale/.gem/ruby/2.2.2/bundler/gems/celluloid-zmq-6e4b9eded922/lib/celluloid/zmq/current.rb:2:in `<top (required)>'
Running MRI 2.2.2, FFI 1.9.10 (appears to be the latest release).
I'm in the process of writing a few specs for this gem. I ran into a problem which I wanted to get some feedback on. In order to separate tests more cleanly from each other, I wrapped all tests in an around filter which calls Celluloid::ZMQ.init
before and Celluloid::ZMQ.terminate
afterwards, so it sets up a clean ZMQ context for each test. This seems to work nicely, except that zmq_term
hangs indefinitely until all sockets created within the context are explicitly closed. This means that every socket we create, we need to explicitly close.
The way I see it, there are two ways of handling this:
Simply update the documentation. Leave it up to the user to make sure they call socket.close
in a finalizer.
Keep track of all sockets created within an actor and add a finalizer to Celluloid::ZMQ which closes them automatically.
While (2) is more user friendly, it could also be problematic, since Celluloid can only have one finalizer per actor (why, btw?), if the user ever adds their own finalizer to the actor, they would overwrite the built in finalizer and mess up the sockets getting closed. So in a way, I think (1) is the better solution at this point.
What do you think?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.