GithubHelp home page GithubHelp logo

rzmq's Introduction

rzmq

R Bindings for 'ZeroMQ'

Project Status: Active – The project has reached a stable, usable state and is being actively developed. Package-License CRAN Downloads

Interface to the 'ZeroMQ' lightweight messaging kernel (see http://www.zeromq.org/ for more information).

Features

rzmq is a message queue for serialized R objects.

  • rzmq implements most the standard socket pairs that ZMQ offers.
  • ZMQ devices are not implemented yet, nor is zmq_poll.
  • Look for more features shortly.

Installation

Binary packages for OS-X or Windows can be installed directly from CRAN:

install.packages("rzmq")

Build from source

Installation from source requires ZeroMQ. On Debian or Ubuntu use libzmq3-dev:

sudo apt-get install -y libzmq3-dev

On Fedora we need zeromq-devel:

sudo yum install zeromq-devel

On CentOS / RHEL we install zeromq3-devel via EPEL:

sudo yum install epel-release
sudo yum install zeromq3-devel

On OS-X use zeromq from Homebrew:

brew install zeromq

Usage

A minimal example of remote execution.

execute this R script on the remote server:

#!/usr/bin/env Rscript
library(rzmq)
context = init.context()
socket = init.socket(context,"ZMQ_REP")
bind.socket(socket,"tcp://*:5555")
while(1) {
    msg = receive.socket(socket);
    fun <- msg$fun
    args <- msg$args
    print(args)
    ans <- do.call(fun,args)
    send.socket(socket,ans);
}

and execute this bit locally:

library(rzmq)

remote.exec <- function(socket,fun,...) {
    send.socket(socket,data=list(fun=fun,args=list(...)))
    receive.socket(socket)
}

substitute(expr)
context = init.context()
socket = init.socket(context,"ZMQ_REQ")
connect.socket(socket,"tcp://localhost:5555")

ans <- remote.exec(socket,sqrt,10000)

rzmq's People

Contributors

armstrtw avatar bicarlsen avatar brotchie avatar glycerine avatar jeroen avatar maelle avatar mschubert avatar takluyver avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rzmq's Issues

Uneven distribution of tasks with parallel pipeline

I am trying to implement the parallel pipeline pattern from http://zguide.zeromq.org/page:all#Divide-and-Conquer. Here my current implementation: master-worker.zip

With five workers and four tasks I would expect that all tasks are processed by different workers. However, I can see that the last task is always processed by the worker that did the first task (pid=7665 in this case):

bash$ for i in $(seq 5); do Rscript worker.R & done
[1] 7663
[2] 7664
[3] 7665
[4] 7666
[5] 7667
bash$ Rscript master.R
Total expected cost: 4 sec
1 : 7665 
2 : 7666 
3 : 7664 
4 : 7665 
[1] "Total elapsed time: 2.03767776489258"

Any idea what is going on here?

Unable to install on Ubuntu 14.04 (from CRAN or source)

I've been using the same libzmq set up for a while, installing it in a Docker container. Recently though, I've been getting this error:

> install_github('armstrtw/rzmq')
Downloading github repo armstrtw/rzmq@master
Installing rzmq
'/usr/lib/R/bin/R' --vanilla CMD INSTALL  \
  '/tmp/RtmpTby1uA/devtools84055feda/armstrtw-rzmq-320891d'  \
  --library='/home/jovyan/.R' --install-tests

* installing *source* package ‘rzmq’ ...
** libs
g++ -I/usr/share/R/include -DNDEBUG -I../inst/cppzmq     -fpic  -O3 -pipe  -g  -c interface.cpp -o interface.o
interface.cpp:23:14: error: expected constructor, destructor, or type conversion before ‘(’ token
 static_assert(ZMQ_VERSION_MAJOR >= 3,"The minimum required version of libzmq is 3.0.0.");
              ^
make: *** [interface.o] Error 1
ERROR: compilation failed for package ‘rzmq’
* removing ‘/home/jovyan/.R/rzmq’

Before I posted this issue, I thought it was complaining about my 0mq version (I have 3!!!). Now I see its not failing on ZMQ_VERSION_MAJOR. There's a syntax error in interface.cpp, which I assume is the vendorized version of the c++ binding.

Question about passing messages between remote machines

On an SGE cluster, I sent a message from a remote compute node:

library(rzmq)
context = init.context()
socket = init.socket(context,"ZMQ_REQ")
connect.socket(socket,"tcp://localhost:5555")
send.socket(socket, data = 123)

and tried to receive it from the login node:

library(rzmq)
context = init.context()
socket = init.socket(context,"ZMQ_REP")
bind.socket(socket,"tcp://*:5555")
receive.socket(socket)

Unsurprisingly, receive.socket() hung indefinitely (and also hung when I switched the roles of the two nodes) but returned 123 when I ran both code chunks in parallel processes on the same node. How do I come up with the socket addresses that will make the nodes talk to each other? I am having trouble finding beginner-friendly guidance in the ZeroMQ guide and references on TCP/IP.

subscribe example?

?subscribe does not show an example (as far as I could see).
experimentation failed:

context <- init.context()
socket <- init.socket(context, "ZMQ_REQ")
connect.socket(socket, "tcp://localhost:5432")
subscribe(socket, "") # following node.js example
--> Invalid argument

I'd be very grateful for a small working example!

Thanks,

  • Paul

Reusing messages in rzmq

I'm developing an rzmq-based R package for distributing function calls on HPC schedulers (clustermq). For this, I need to send the same message (the function itself and constant arguments; often 100s of MB) to all workers at different times (whenever they start up - so no PUB/SUB sockets).

ZeroMQ allows to reuse messages, i.e. not copying the memory to a new buffer like sendSocket() does.

I'm posting this issue here to see if there is interest in incorporating this functionality in the rzmq package (rather than my package; and to get feedback on my approach). The way I would do this is to:

  • Provide a function in rzmq that converts an R object to message_t object and return it via UNPROTECT
  • Let R manage the actual object (so the user is responsible not to lose the reference, as with the context object)
  • Modify the sendSocket() function to take a message_t object directly, call zmq_msg_copy() on it, and then send it
  • Potentially provide a function to zmq_msg_close() the message, but I don't think this is required (as the memory for the message is managed by R and we'd get a double free otherwise)

If you agree with this I will work on a PR (otherwise I'd add low-level code to my package).

ZMQ_HWM, ZMQ_MCAST_LOOP, ZMQ_RECOVERY_IVL_MSEC issues in latest zmq V3

Hello,

first of all thanks for making this rzmq avialable. Its very helpful.

However, the latest version of ZMQ has made some changes that are making the rzmq build break.

These, ZMQ_HWM, ZMQ_MCAST_LOOP, ZMQ_RECOVERY_IVL_MSEC have been removed or modified in the ZMQ.

It would be great if these can be adapted into the rzmq.

Thank you

compilation failed

hi not sure if this is an issue with my setup but when i try to do install.packages("rzmq") using R 3.0.2 on a centos 6 vm, I get:

Selection: 85
trying URL 'http://mirrors.nics.utk.edu/cran/src/contrib/rzmq_0.6.8.tar.gz'
Content type 'application/x-gzip' length 8163 bytes
opened URL
==================================================
downloaded 8163 bytes

* installing *source* package â?~rzmqâ?T ...
** package â?~rzmqâ?T successfully unpacked and MD5 sums checked
** libs
g++ -m64 -I/usr/include/R -DNDEBUG  -I/usr/local/include    -fpic  -O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic  -c interface.cpp -o interface.o
interface.cpp:22:19: error: zmq.hpp: No such file or directory
interface.cpp: In function â?~SEXPREC* get_zmq_version()â?T:
interface.cpp:29: error: â?~zmqâ?T has not been declared
interface.cpp: In function â?~int string_to_socket_type(std::string)â?T:
interface.cpp:39: error: â?~ZMQ_PAIRâ?T was not declared in this scope
interface.cpp:41: error: â?~ZMQ_PUBâ?T was not declared in this scope
interface.cpp:43: error: â?~ZMQ_SUBâ?T was not declared in this scope
interface.cpp:45: error: â?~ZMQ_REQâ?T was not declared in this scope
interface.cpp:47: error: â?~ZMQ_REPâ?T was not declared in this scope
interface.cpp:49: error: â?~ZMQ_DEALERâ?T was not declared in this scope
interface.cpp:51: error: â?~ZMQ_ROUTERâ?T was not declared in this scope
interface.cpp:53: error: â?~ZMQ_PULLâ?T was not declared in this scope
interface.cpp:55: error: â?~ZMQ_PUSHâ?T was not declared in this scope
interface.cpp:57: error: â?~ZMQ_XPUBâ?T was not declared in this scope
interface.cpp:59: error: â?~ZMQ_XSUBâ?T was not declared in this scope
interface.cpp: In function â?~void* checkExternalPointer(SEXPREC*, const char*)â?T:
interface.cpp:80: error: â?~strcmpâ?T was not declared in this scope
interface.cpp: In function â?~void contextFinalizer(SEXPREC*)â?T:
interface.cpp:90: error: â?~zmqâ?T has not been declared
interface.cpp:90: error: â?~contextâ?T was not declared in this scope
interface.cpp:90: error: expected type-specifier before â?~zmqâ?T
interface.cpp:90: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:90: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:90: error: â?~zmqâ?T has not been declared
interface.cpp:90: error: expected primary-expression before â?~>â?T token
interface.cpp:90: error: expected â?~)â?T before â?~;â?T token
interface.cpp:92: error: type â?~â?T argument given to â?~deleteâ?T, expected pointer
interface.cpp: In function â?~void socketFinalizer(SEXPREC*)â?T:
interface.cpp:98: error: â?~zmqâ?T has not been declared
interface.cpp:98: error: â?~socketâ?T was not declared in this scope
interface.cpp:98: error: expected type-specifier before â?~zmqâ?T
interface.cpp:98: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:98: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:98: error: â?~zmqâ?T has not been declared
interface.cpp:98: error: expected primary-expression before â?~>â?T token
interface.cpp:98: error: expected â?~)â?T before â?~;â?T token
interface.cpp:100: error: type â?~â?T argument given to â?~deleteâ?T, expected pointer
interface.cpp: In function â?~SEXPREC* initContext()â?T:
interface.cpp:107: error: â?~zmqâ?T has not been declared
interface.cpp:107: error: â?~contextâ?T was not declared in this scope
interface.cpp:107: error: expected type-specifier before â?~zmqâ?T
interface.cpp:107: error: expected â?~;â?T before â?~zmqâ?T
interface.cpp: In function â?~SEXPREC* initSocket(SEXPREC*, SEXPREC*)â?T:
interface.cpp:132: error: â?~zmqâ?T has not been declared
interface.cpp:132: error: â?~contextâ?T was not declared in this scope
interface.cpp:134: error: expected type-specifier before â?~zmqâ?T
interface.cpp:134: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:134: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:134: error: â?~zmqâ?T has not been declared
interface.cpp:134: error: expected primary-expression before â?~>â?T token
interface.cpp:134: error: expected â?~)â?T before â?~;â?T token
interface.cpp:140: error: â?~zmqâ?T has not been declared
interface.cpp:140: error: â?~socketâ?T was not declared in this scope
interface.cpp:140: error: expected type-specifier before â?~zmqâ?T
interface.cpp:140: error: expected â?~;â?T before â?~zmqâ?T
interface.cpp: In function â?~SEXPREC* bindSocket(SEXPREC*, SEXPREC*)â?T:
interface.cpp:162: error: â?~zmqâ?T has not been declared
interface.cpp:162: error: â?~socketâ?T was not declared in this scope
interface.cpp:162: error: expected type-specifier before â?~zmqâ?T
interface.cpp:162: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:162: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:162: error: â?~zmqâ?T has not been declared
interface.cpp:162: error: expected primary-expression before â?~>â?T token
interface.cpp:162: error: expected â?~)â?T before â?~;â?T token
interface.cpp: In function â?~SEXPREC* connectSocket(SEXPREC*, SEXPREC*)â?T:
interface.cpp:182: error: â?~zmqâ?T has not been declared
interface.cpp:182: error: â?~socketâ?T was not declared in this scope
interface.cpp:182: error: expected type-specifier before â?~zmqâ?T
interface.cpp:182: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:182: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:182: error: â?~zmqâ?T has not been declared
interface.cpp:182: error: expected primary-expression before â?~>â?T token
interface.cpp:182: error: expected â?~)â?T before â?~;â?T token
interface.cpp: In function â?~SEXPREC* sendSocket(SEXPREC*, SEXPREC*, SEXPREC*)â?T:
interface.cpp:208: error: â?~zmqâ?T has not been declared
interface.cpp:208: error: â?~socketâ?T was not declared in this scope
interface.cpp:208: error: expected type-specifier before â?~zmqâ?T
interface.cpp:208: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:208: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:208: error: â?~zmqâ?T has not been declared
interface.cpp:208: error: expected primary-expression before â?~>â?T token
interface.cpp:208: error: expected â?~)â?T before â?~;â?T token
interface.cpp:211: error: â?~zmqâ?T has not been declared
interface.cpp:211: error: expected â?~;â?T before â?~msgâ?T
interface.cpp:212: error: â?~msgâ?T was not declared in this scope
interface.cpp:212: error: â?~memcpyâ?T was not declared in this scope
interface.cpp:217: error: â?~ZMQ_SNDMOREâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* sendNullMsg(SEXPREC*, SEXPREC*)â?T:
interface.cpp:239: error: â?~zmqâ?T has not been declared
interface.cpp:239: error: â?~socketâ?T was not declared in this scope
interface.cpp:239: error: expected type-specifier before â?~zmqâ?T
interface.cpp:239: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:239: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:239: error: â?~zmqâ?T has not been declared
interface.cpp:239: error: expected primary-expression before â?~>â?T token
interface.cpp:239: error: expected â?~)â?T before â?~;â?T token
interface.cpp:241: error: â?~zmqâ?T has not been declared
interface.cpp:241: error: expected â?~;â?T before â?~msgâ?T
interface.cpp:246: error: â?~msgâ?T was not declared in this scope
interface.cpp:246: error: â?~ZMQ_SNDMOREâ?T was not declared in this scope
interface.cpp:248: error: â?~msgâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* receiveNullMsg(SEXPREC*)â?T:
interface.cpp:262: error: â?~zmqâ?T has not been declared
interface.cpp:262: error: â?~socketâ?T was not declared in this scope
interface.cpp:262: error: expected type-specifier before â?~zmqâ?T
interface.cpp:262: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:262: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:262: error: â?~zmqâ?T has not been declared
interface.cpp:262: error: expected primary-expression before â?~>â?T token
interface.cpp:262: error: expected â?~)â?T before â?~;â?T token
interface.cpp:264: error: â?~zmqâ?T has not been declared
interface.cpp:264: error: expected â?~;â?T before â?~msgâ?T
interface.cpp:266: error: â?~msgâ?T was not declared in this scope
interface.cpp:270: error: â?~msgâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* receiveSocket(SEXPREC*)â?T:
interface.cpp:278: error: â?~zmqâ?T has not been declared
interface.cpp:278: error: expected â?~;â?T before â?~msgâ?T
interface.cpp:279: error: â?~zmqâ?T has not been declared
interface.cpp:279: error: â?~socketâ?T was not declared in this scope
interface.cpp:279: error: expected type-specifier before â?~zmqâ?T
interface.cpp:279: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:279: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:279: error: â?~zmqâ?T has not been declared
interface.cpp:279: error: expected primary-expression before â?~>â?T token
interface.cpp:279: error: expected â?~)â?T before â?~;â?T token
interface.cpp:282: error: â?~msgâ?T was not declared in this scope
interface.cpp:287: error: â?~msgâ?T was not declared in this scope
interface.cpp:288: error: â?~memcpyâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* sendRawString(SEXPREC*, SEXPREC*, SEXPREC*)â?T:
interface.cpp:310: error: â?~zmqâ?T has not been declared
interface.cpp:310: error: â?~socketâ?T was not declared in this scope
interface.cpp:310: error: expected type-specifier before â?~zmqâ?T
interface.cpp:310: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:310: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:310: error: â?~zmqâ?T has not been declared
interface.cpp:310: error: expected primary-expression before â?~>â?T token
interface.cpp:310: error: expected â?~)â?T before â?~;â?T token
interface.cpp:317: error: â?~zmqâ?T has not been declared
interface.cpp:317: error: expected â?~;â?T before â?~msgâ?T
interface.cpp:318: error: â?~msgâ?T was not declared in this scope
interface.cpp:318: error: â?~strlenâ?T was not declared in this scope
interface.cpp:318: error: â?~memcpyâ?T was not declared in this scope
interface.cpp:323: error: â?~ZMQ_SNDMOREâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* receiveString(SEXPREC*)â?T:
interface.cpp:340: error: â?~zmqâ?T has not been declared
interface.cpp:340: error: expected â?~;â?T before â?~msgâ?T
interface.cpp:341: error: â?~zmqâ?T has not been declared
interface.cpp:341: error: â?~socketâ?T was not declared in this scope
interface.cpp:341: error: expected type-specifier before â?~zmqâ?T
interface.cpp:341: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:341: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:341: error: â?~zmqâ?T has not been declared
interface.cpp:341: error: expected primary-expression before â?~>â?T token
interface.cpp:341: error: expected â?~)â?T before â?~;â?T token
interface.cpp:344: error: â?~msgâ?T was not declared in this scope
interface.cpp:350: error: â?~msgâ?T was not declared in this scope
interface.cpp:355: error: â?~memcpyâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* receiveInt(SEXPREC*)â?T:
interface.cpp:367: error: â?~zmqâ?T has not been declared
interface.cpp:367: error: expected â?~;â?T before â?~msgâ?T
interface.cpp:369: error: â?~zmqâ?T has not been declared
interface.cpp:369: error: â?~socketâ?T was not declared in this scope
interface.cpp:369: error: expected type-specifier before â?~zmqâ?T
interface.cpp:369: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:369: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:369: error: â?~zmqâ?T has not been declared
interface.cpp:369: error: expected primary-expression before â?~>â?T token
interface.cpp:369: error: expected â?~)â?T before â?~;â?T token
interface.cpp:370: error: â?~msgâ?T was not declared in this scope
interface.cpp:375: error: â?~msgâ?T was not declared in this scope
interface.cpp:380: error: â?~msgâ?T was not declared in this scope
interface.cpp:380: error: â?~memcpyâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* receiveDouble(SEXPREC*)â?T:
interface.cpp:390: error: â?~zmqâ?T has not been declared
interface.cpp:390: error: expected â?~;â?T before â?~msgâ?T
interface.cpp:392: error: â?~zmqâ?T has not been declared
interface.cpp:392: error: â?~socketâ?T was not declared in this scope
interface.cpp:392: error: expected type-specifier before â?~zmqâ?T
interface.cpp:392: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:392: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:392: error: â?~zmqâ?T has not been declared
interface.cpp:392: error: expected primary-expression before â?~>â?T token
interface.cpp:392: error: expected â?~)â?T before â?~;â?T token
interface.cpp:393: error: â?~msgâ?T was not declared in this scope
interface.cpp:398: error: â?~msgâ?T was not declared in this scope
interface.cpp:403: error: â?~msgâ?T was not declared in this scope
interface.cpp:403: error: â?~memcpyâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* set_hwm(SEXPREC*, SEXPREC*)â?T:
interface.cpp:414: error: â?~zmqâ?T has not been declared
interface.cpp:414: error: â?~socketâ?T was not declared in this scope
interface.cpp:414: error: expected type-specifier before â?~zmqâ?T
interface.cpp:414: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:414: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:414: error: â?~zmqâ?T has not been declared
interface.cpp:414: error: expected primary-expression before â?~>â?T token
interface.cpp:414: error: expected â?~)â?T before â?~;â?T token
interface.cpp:421: error: â?~ZMQ_HWMâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* set_swap(SEXPREC*, SEXPREC*)â?T:
interface.cpp:436: error: â?~zmqâ?T has not been declared
interface.cpp:436: error: â?~socketâ?T was not declared in this scope
interface.cpp:436: error: expected type-specifier before â?~zmqâ?T
interface.cpp:436: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:436: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:436: error: â?~zmqâ?T has not been declared
interface.cpp:436: error: expected primary-expression before â?~>â?T token
interface.cpp:436: error: expected â?~)â?T before â?~;â?T token
interface.cpp:443: error: â?~ZMQ_SWAPâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* set_affinity(SEXPREC*, SEXPREC*)â?T:
interface.cpp:455: error: â?~zmqâ?T has not been declared
interface.cpp:455: error: â?~socketâ?T was not declared in this scope
interface.cpp:455: error: expected type-specifier before â?~zmqâ?T
interface.cpp:455: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:455: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:455: error: â?~zmqâ?T has not been declared
interface.cpp:455: error: expected primary-expression before â?~>â?T token
interface.cpp:455: error: expected â?~)â?T before â?~;â?T token
interface.cpp:462: error: â?~ZMQ_AFFINITYâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* set_identity(SEXPREC*, SEXPREC*)â?T:
interface.cpp:473: error: â?~zmqâ?T has not been declared
interface.cpp:473: error: â?~socketâ?T was not declared in this scope
interface.cpp:473: error: expected type-specifier before â?~zmqâ?T
interface.cpp:473: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:473: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:473: error: â?~zmqâ?T has not been declared
interface.cpp:473: error: expected primary-expression before â?~>â?T token
interface.cpp:473: error: expected â?~)â?T before â?~;â?T token
interface.cpp:479: error: â?~ZMQ_IDENTITYâ?T was not declared in this scope
interface.cpp:479: error: â?~strlenâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* subscribe(SEXPREC*, SEXPREC*)â?T:
interface.cpp:490: error: â?~zmqâ?T has not been declared
interface.cpp:490: error: â?~socketâ?T was not declared in this scope
interface.cpp:490: error: expected type-specifier before â?~zmqâ?T
interface.cpp:490: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:490: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:490: error: â?~zmqâ?T has not been declared
interface.cpp:490: error: expected primary-expression before â?~>â?T token
interface.cpp:490: error: expected â?~)â?T before â?~;â?T token
interface.cpp:496: error: â?~ZMQ_SUBSCRIBEâ?T was not declared in this scope
interface.cpp:496: error: â?~strlenâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* unsubscribe(SEXPREC*, SEXPREC*)â?T:
interface.cpp:507: error: â?~zmqâ?T has not been declared
interface.cpp:507: error: â?~socketâ?T was not declared in this scope
interface.cpp:507: error: expected type-specifier before â?~zmqâ?T
interface.cpp:507: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:507: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:507: error: â?~zmqâ?T has not been declared
interface.cpp:507: error: expected primary-expression before â?~>â?T token
interface.cpp:507: error: expected â?~)â?T before â?~;â?T token
interface.cpp:513: error: â?~ZMQ_UNSUBSCRIBEâ?T was not declared in this scope
interface.cpp:513: error: â?~strlenâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* set_rate(SEXPREC*, SEXPREC*)â?T:
interface.cpp:524: error: â?~zmqâ?T has not been declared
interface.cpp:524: error: â?~socketâ?T was not declared in this scope
interface.cpp:524: error: expected type-specifier before â?~zmqâ?T
interface.cpp:524: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:524: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:524: error: â?~zmqâ?T has not been declared
interface.cpp:524: error: expected primary-expression before â?~>â?T token
interface.cpp:524: error: expected â?~)â?T before â?~;â?T token
interface.cpp:531: error: â?~ZMQ_RATEâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* set_recovery_ivl(SEXPREC*, SEXPREC*)â?T:
interface.cpp:542: error: â?~zmqâ?T has not been declared
interface.cpp:542: error: â?~socketâ?T was not declared in this scope
interface.cpp:542: error: expected type-specifier before â?~zmqâ?T
interface.cpp:542: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:542: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:542: error: â?~zmqâ?T has not been declared
interface.cpp:542: error: expected primary-expression before â?~>â?T token
interface.cpp:542: error: expected â?~)â?T before â?~;â?T token
interface.cpp:549: error: â?~ZMQ_RECOVERY_IVLâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* set_recovery_ivl_msec(SEXPREC*, SEXPREC*)â?T:
interface.cpp:563: error: â?~zmqâ?T has not been declared
interface.cpp:563: error: â?~socketâ?T was not declared in this scope
interface.cpp:563: error: expected type-specifier before â?~zmqâ?T
interface.cpp:563: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:563: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:563: error: â?~zmqâ?T has not been declared
interface.cpp:563: error: expected primary-expression before â?~>â?T token
interface.cpp:563: error: expected â?~)â?T before â?~;â?T token
interface.cpp:570: error: â?~ZMQ_RECOVERY_IVL_MSECâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* set_mcast_loop(SEXPREC*, SEXPREC*)â?T:
interface.cpp:584: error: â?~zmqâ?T has not been declared
interface.cpp:584: error: â?~socketâ?T was not declared in this scope
interface.cpp:584: error: expected type-specifier before â?~zmqâ?T
interface.cpp:584: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:584: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:584: error: â?~zmqâ?T has not been declared
interface.cpp:584: error: expected primary-expression before â?~>â?T token
interface.cpp:584: error: expected â?~)â?T before â?~;â?T token
interface.cpp:591: error: â?~ZMQ_MCAST_LOOPâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* set_sndbuf(SEXPREC*, SEXPREC*)â?T:
interface.cpp:603: error: â?~zmqâ?T has not been declared
interface.cpp:603: error: â?~socketâ?T was not declared in this scope
interface.cpp:603: error: expected type-specifier before â?~zmqâ?T
interface.cpp:603: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:603: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:603: error: â?~zmqâ?T has not been declared
interface.cpp:603: error: expected primary-expression before â?~>â?T token
interface.cpp:603: error: expected â?~)â?T before â?~;â?T token
interface.cpp:610: error: â?~ZMQ_SNDBUFâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* set_rcvbuf(SEXPREC*, SEXPREC*)â?T:
interface.cpp:621: error: â?~zmqâ?T has not been declared
interface.cpp:621: error: â?~socketâ?T was not declared in this scope
interface.cpp:621: error: expected type-specifier before â?~zmqâ?T
interface.cpp:621: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:621: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:621: error: â?~zmqâ?T has not been declared
interface.cpp:621: error: expected primary-expression before â?~>â?T token
interface.cpp:621: error: expected â?~)â?T before â?~;â?T token
interface.cpp:628: error: â?~ZMQ_RCVBUFâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* set_linger(SEXPREC*, SEXPREC*)â?T:
interface.cpp:639: error: â?~zmqâ?T has not been declared
interface.cpp:639: error: â?~socketâ?T was not declared in this scope
interface.cpp:639: error: expected type-specifier before â?~zmqâ?T
interface.cpp:639: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:639: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:639: error: â?~zmqâ?T has not been declared
interface.cpp:639: error: expected primary-expression before â?~>â?T token
interface.cpp:639: error: expected â?~)â?T before â?~;â?T token
interface.cpp:646: error: â?~ZMQ_LINGERâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* set_reconnect_ivl(SEXPREC*, SEXPREC*)â?T:
interface.cpp:657: error: â?~zmqâ?T has not been declared
interface.cpp:657: error: â?~socketâ?T was not declared in this scope
interface.cpp:657: error: expected type-specifier before â?~zmqâ?T
interface.cpp:657: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:657: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:657: error: â?~zmqâ?T has not been declared
interface.cpp:657: error: expected primary-expression before â?~>â?T token
interface.cpp:657: error: expected â?~)â?T before â?~;â?T token
interface.cpp:664: error: â?~ZMQ_RECONNECT_IVLâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* set_zmq_backlog(SEXPREC*, SEXPREC*)â?T:
interface.cpp:675: error: â?~zmqâ?T has not been declared
interface.cpp:675: error: â?~socketâ?T was not declared in this scope
interface.cpp:675: error: expected type-specifier before â?~zmqâ?T
interface.cpp:675: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:675: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:675: error: â?~zmqâ?T has not been declared
interface.cpp:675: error: expected primary-expression before â?~>â?T token
interface.cpp:675: error: expected â?~)â?T before â?~;â?T token
interface.cpp:682: error: â?~ZMQ_BACKLOGâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* set_reconnect_ivl_max(SEXPREC*, SEXPREC*)â?T:
interface.cpp:693: error: â?~zmqâ?T has not been declared
interface.cpp:693: error: â?~socketâ?T was not declared in this scope
interface.cpp:693: error: expected type-specifier before â?~zmqâ?T
interface.cpp:693: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:693: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:693: error: â?~zmqâ?T has not been declared
interface.cpp:693: error: expected primary-expression before â?~>â?T token
interface.cpp:693: error: expected â?~)â?T before â?~;â?T token
interface.cpp:700: error: â?~ZMQ_RECONNECT_IVL_MAXâ?T was not declared in this scope
interface.cpp: In function â?~SEXPREC* get_rcvmore(SEXPREC*)â?T:
interface.cpp:712: error: â?~zmqâ?T has not been declared
interface.cpp:712: error: â?~socketâ?T was not declared in this scope
interface.cpp:712: error: expected type-specifier before â?~zmqâ?T
interface.cpp:712: error: expected â?~>â?T before â?~zmqâ?T
interface.cpp:712: error: expected â?~(â?T before â?~zmqâ?T
interface.cpp:712: error: â?~zmqâ?T has not been declared
interface.cpp:712: error: expected primary-expression before â?~>â?T token
interface.cpp:712: error: expected â?~)â?T before â?~;â?T token
interface.cpp:718: error: â?~ZMQ_RCVMOREâ?T was not declared in this scope
make: *** [interface.o] Error 1
ERROR: compilation failed for package â?~rzmqâ?T
* removing â?~/usr/lib64/R/library/rzmqâ?T

The downloaded source packages are in
        â?~/tmp/RtmpHRr337/downloaded_packagesâ?T
Updating HTML index of packages in '.Library'
Making 'packages.html' ... done
Warning message:
In install.packages("rzmq") :
  installation of package â?~rzmqâ?T had non-zero exit status

rzmq Client error with Python Server

I am trying to run the Request-reply example with a Python server and rzmq client.

The Python code I use is as in the zmq docs: http://zguide.zeromq.org/py:hwserver

The R code I am using is as in the docs, modified to send "Hello"

#!/usr/bin/env Rscript
library(rzmq)

remote.exec <- function(socket, fun, ...) {
  send.socket(socket, data = list(fun = fun, args = list(...)))
  receive.socket(socket)
}

substitute(expr)
context = init.context()
socket = init.socket(context, "ZMQ_REQ")
connect.socket(socket, "tcp://localhost:5555")

send.socket(socket, "Hello")
receive.socket(socket)

I see the server printing this:

Received request: b'B\n\x03\x00\x00\x00\x03\x06\x03\x00\x00\x05\x03\x00\x06\x00\x00\x00CP1252\x10\x00\x00\x00\x01\x00\x00\x00\t\x00\x04\x00\x05\x00\x00\x00Hello'

instead it should be:

Received request: b'Hello'

I tried running the R code as Rscript from command line, as well as from RStudio with same result. The client side (R) errors with following message:

expr
Error in unserialize(ans) : unknown input format
Calls: receive.socket -> unserialize
Execution halted

Any steps to debug this would be helpful. I am on Windows 10. I am running the Python server from WSL and the R code from RStudio and RStudio terminal. R version 3.6.3.

Appveyor builds and webhook

Hi @jeroen! 👋

It seems the builds for this repo are under your account. I see you tried to trigger Appveyor builds but the webhook has failed deliveries.

You should send an email to Appveyor support team at [email protected] They'll probably want to know this about the latest delivery

  • X-GitHub-Delivery: d7e95c42-552e-11e8-9fe8-2d2b1e247678

  • Time: 2018-05-11 17:20:31

Thanks!

enable polling

I'd like to be able to pass ZMQ_DONTWAIT to the socket->recv() call in receiveSocket, in order to be able to poll but not block if no message is availble.

Since R is not very thread friendly, non-blocking polling is very useful for implementing a server.

Would you consider adding the ability to pass flags to socket->recv() ? Is it is, the flags_ parameter is always defaulted to zero.

Thanks!

Jason

in interface.cpp inside rzmq.tgz :

SEXP receiveSocket(SEXP socket_) {
  SEXP ans;
  bool status(false);
  zmq::message_t msg;
  zmq::socket_t* socket = reinterpret_cast<zmq::socket_t*>(checkExternalPointer(socket_,"zmq::socket_t*"));
  if(!socket) { REprintf("bad socket object.\n");return R_NilValue; }
  try {
    status = socket->recv(&msg); // line 282   <<<<<<<<<<<<<<<<<< here
  } catch(std::exception& e) {
    REprintf("%s\n",e.what());
  }
  if(status) {
    PROTECT(ans = allocVector(RAWSXP,msg.size()));
    memcpy(RAW(ans),msg.data(),msg.size());
    UNPROTECT(1);
    return ans;
  }

  return R_NilValue;
}
-UU-:----F1  interface.cpp (rzmq.tgz)   35% L282   (C++/l Abbrev Isearch) ------------------------------------------------------------------------------------------------------------------------
in zmq.hpp:

    class socket_t
    {
 ...
        inline bool recv (message_t *msg_, int flags_ = 0)   // line 444 <<<<<<<<<<<<<<< here
        {
            int nbytes = zmq_msg_recv (&(msg_->msg), ptr, flags_);
            if (nbytes >= 0)
                return true;
            if (zmq_errno () == EAGAIN)
                return false;
            throw error_t ();
        }

    private:
        void *ptr;
        void *ctxptr;

        socket_t (const socket_t&) ZMQ_DELETED_FUNCTION;
        void operator = (const socket_t&) ZMQ_DELETED_FUNCTION;
    };

-UU-:----F1  zmq.hpp        66% L444   (C++/l Abbrev) ----------------------------------------

This package is failing an address sanitizer test.

Hi Jeroen!

I was running some address santizers over this package and saw a deadlock. There isn't much useful testing logs or traceback to check; the test just times out.

To reproduce the error, you'll need to compile R with sanitizers enabled. Rhub has them
https://cran.r-project.org/web/packages/rhub/vignettes/rhub.html

Or with one of the docker images here:
https://github.com/rocker-org/rocker#unversioned-images-builds-on-r-base

Would you mind taking a look? It's possible that it's something on my end.

Thanks!

Unable to install on SUSE Linux Enterprise Server 11 SP3

Hi, I install rzmq with R version 3.2.3 on SUSE Linux Enterprise Server 11 SP3, while failed:

vm3:# R CMD INSTALL rzmq
* installing to library ‘/usr/local/R/lib64/R/library’
* installing *source* package ‘rzmq’ ...
file ‘src/Makevars’ has the wrong MD5 checksum
** libs
I/usr/local/R/lib64/R/include -DNDEBUG -I../inst/cppzmq -I/usr/local/include       -c interface.cpp -o interface.o
make: I/usr/local/R/lib64/R/include: Command not found
make: [interface.o] Error 127 (ignored)
-shared -L/usr/local/R/lib64/R/lib -L/usr/local/lib64 -o rzmq.so interface.o -lzmq -L/usr/local/R/lib64/R/lib -lR
/bin/sh: line 2: -shared: command not found
make: *** [rzmq.so] Error 127
ERROR: compilation failed for package ‘rzmq’
* removing ‘/usr/local/R/lib64/R/library/rzmq

The etc/Makeconf setting is below:

CC = gcc -std=gnu99
CFLAGS = -g -O2 $(LTO)
CPICFLAGS = -fpic
CPPFLAGS = -I/usr/local/include
CXX = g++
CXXCPP = $(CXX) -E
CXXFLAGS = -g -O2 $(LTO)
CXXPICFLAGS = -fpic
CXX1X = 
CXX1XFLAGS = 
CXX1XPICFLAGS = 
CXX1XSTD =

I have installed stringi , dose something need to set for rzmq?

I have install zeromq:

vm3:/ # rpm -qa | grep mq
libzmq3-3.2.2-13.1
zeromq-devel-3.2.2-13.1
zeromq-3.2.2-13.1

Next CRAN release

I'm looking to release my package clustermq (using HPC schedulers with rzmq) on CRAN.

However, this requires XREQ/XREP sockets (#32) and the updated poll timeout (#34). Both have been merged here.

Is there a plan when to push these changes to CRAN, so my package can make use of them without needing a Github dependency?

how can I close an opened zmq socket in R?

  port = 1111
  zmq_context <- init.context()
  zmq_service.socket <- init.socket(zmq_context,"ZMQ_REQ")
  connect.socket(zmq_service.socket, paste("tcp://127.0.0.1:", port, sep = ""))

As I see it, I can just remove the object and expect garbage collector to waste it.

  rm(zmq_service.socket, zmq_context)

use vs pbdzmq

@jeroenooms its cool to see some new life breathed into this package! Out of curiosity, what was the choice on reviving this vs using/contributing to https://github.com/snoweye/pbdZMQ

nothing against this package at all, just given the IRKernel and other folks have switched to pbdZMQ it would seem to make sense to focus efforts in one place?

windows builds

In order to provide windows builds on CRAN, you need to compile static libraries of libzmq for win32 and win64 and mail them to the CRAN maintainers. They can then deploy them on the windows build server to link when building the windows binary for rzmq.

See here for a script that I used to cross compile libprotobuf.a for the RProtoBuf package. You can also create libzmq.a on an actual windows machine obviously.

Serializing on big vs. little endian systems

There is an option in R's serialize() function that determines whether data is serialized in big- or little endian format. Big endian is default, which doesn't make much sense, because most current systems use little endian. This has a performance penalty that could easily be avoided by setting the option xdr=FALSE in rzmq.R#L52 (but could be queried using .Platform$endian or combined to xdr=.Platform$endian=="big").

ll = lapply(1:1e6, function(x) as.list(setNames(runif(5), letters[1:5])))

fx = function(ll, xdr) { 
    serial = serialize(ll, NULL, xdr)
    unserial = unserialize(serial)
}                      

system.time(fx(ll, xdr=TRUE))
system.time(fx(ll, xdr=FALSE))

Which gives the following times on my institute's computing facilities:

> system.time(fx(ll, xdr=TRUE))
   user  system elapsed
 25.124   1.246  26.633
> system.time(fx(ll, xdr=FALSE))
   user  system elapsed
  2.873   0.315   3.279

No rzmq binary for R 3.3.* or 3.4.*

When following the installation instructions, here under the current R release, 3.3.0 or R-devel (which has version 3.4.0), no rzmq binary is found.

install.packages('rzmq', repos = 'http://irkernel.github.io')
## Warning in install.packages :
##   cannot open URL 'http://irkernel.github.io/bin/windows/contrib/3.3/PACKAGES.gz': HTTP status was '404 Not Found'

Installing from source fails for me, but installing the binary under R-3.2.5 works.

Related: IRkernel/IRkernel#298

Trying to set ZMQ_CONFLATE within rzmq

I'm trying to set option ZMQ_CONFLATE to 1 (http://api.zeromq.org/master:zmq-setsockopt) for a ZMQ_SUB socket.

I didn't see a built-in function to do that, so I tried to add my own. Here's what I did:

  1. Added function to R/zmq.R, following pattern of set.affinity:
set.conflate <- function(socket, option.value) {
    .Call("set_conflate",socket, option.value, PACKAGE="rzmq")
}
  1. Added set.conflate to export list in NAMESPACE file.

  2. Added set_conflate function in src/interface.cpp, following same pattern as
    set_affinity:

SEXP set_conflate(SEXP socket_, SEXP option_value_) {

  zmq::socket_t* socket = reinterpret_cast<zmq::socket_t*>(checkExternalPointer(socket_,"zmq::socket_t*"));
  if(!socket) { REprintf("bad socket object.\n");return R_NilValue; }
  if(TYPEOF(option_value_)!=INTSXP) { REprintf("option value must be an int.\n");return R_NilValue; }
  SEXP ans; PROTECT(ans = allocVector(LGLSXP,1)); LOGICAL(ans)[0] = 1;

  uint64_t option_value(INTEGER(option_value_)[0]);
  try {
    socket->setsockopt(ZMQ_CONFLATE, &option_value, sizeof(uint64_t));
  } catch(std::exception& e) {
    REprintf("%s\n",e.what());
    LOGICAL(ans)[0] = 0;
  }
  UNPROTECT(1);
  return ans;
}
  1. Added a line to src/interface.h in the extern "C" block:

SEXP set_conflate(SEXP socket_, SEXP option_value_);

  1. Deleted src/rzmq.so from my prior installation.

  2. Re-installed package from source, verified src/rzmq.so was re-built.

  3. When I tried to use it, it definitely finds the function and tries, but it says the argument is invalid:

> library(rzmq)
> sock = init.socket(init.context(), "ZMQ_SUB")
> connect.socket(sock, "tcp://host:port")
> subscribe(sock, "")
> set.conflate(sock, 1L)
Invalid argument
[1] FALSE

Two questions:

  1. Why would it say 1L is an invalid argument? The docs say it's expecting an integer.
  2. Is there a better way to accomplish setting ZMQ_CONFLATE to 1?

install_github fails with ropensci/rzmq

Unable to install rzmq from source using install_github because of invalid .gitmodules

Downloading GitHub repo ropensci/rzmq@master
Warning: Invalid submodule definition, skipping submodule installation
Error in info[!to_ignore, ] : incorrect number of dimensions
Calls: ... install_remotes -> vapply -> FUN -> update_submodules
Execution halted

This files because github has a .gitignore file without a url tag in it.

Compatibility issue with latest zmq from homebrew on OS X

$ brew update
Already up-to-date.
$ brew upgrade zmq
Error: zeromq 4.1.2 already installed
$ ls -l /usr/local/lib |grep libzmq
lrwxr-xr-x   1 ddauer admin      41 Jun 22 09:20 libzmq.5.dylib -> ../Cellar/zeromq/4.1.2/lib/libzmq.5.dylib
lrwxr-xr-x   1 ddauer admin      35 Jun 22 09:20 libzmq.a -> ../Cellar/zeromq/4.1.2/lib/libzmq.a
lrwxr-xr-x   1 ddauer admin      39 Jun 22 09:20 libzmq.dylib -> ../Cellar/zeromq/4.1.2/lib/libzmq.dylib
lrwxr-xr-x   1 ddauer admin      53 Jul 16 12:16 libzmqcollab.2.dylib -> ../Cellar/fontforge/20150612/lib/libzmqcollab.2.dylib
lrwxr-xr-x   1 ddauer admin      47 Jul 16 12:16 libzmqcollab.a -> ../Cellar/fontforge/20150612/lib/libzmqcollab.a
lrwxr-xr-x   1 ddauer admin      51 Jul 16 12:16 libzmqcollab.dylib -> ../Cellar/fontforge/20150612/lib/libzmqcollab.dylib

in R:

> library("rzmq", lib.loc="/Library/Frameworks/R.framework/Versions/3.2/Resources/library")
Error in dyn.load(file, DLLpath = DLLpath, ...) : 
  unable to load shared object '/Library/Frameworks/R.framework/Versions/3.2/Resources/library/rzmq/libs/rzmq.so':
  dlopen(/Library/Frameworks/R.framework/Versions/3.2/Resources/library/rzmq/libs/rzmq.so, 6): Library not loaded: /usr/local/lib/libzmq.4.dylib
  Referenced from: /Library/Frameworks/R.framework/Versions/3.2/Resources/library/rzmq/libs/rzmq.so
  Reason: image not found
Error: package or namespace load failed for ‘rzmq’

it seems to be looking for libzmq.4.dylib while only libzmq.5.dylib is available

$ otool -L /Library/Frameworks/R.framework/Versions/3.2/Resources/library/rzmq/libs/rzmq.so
/Library/Frameworks/R.framework/Versions/3.2/Resources/library/rzmq/libs/rzmq.so:
    rzmq.so (compatibility version 0.0.0, current version 0.0.0)
    /usr/local/lib/libzmq.4.dylib (compatibility version 5.0.0, current version 5.0.0)
        ...

quickfix:

install_name_tool -change /usr/local/lib/libzmq.4.dylib /usr/local/lib/libzmq.5.dylib /Library/Frameworks/R.framework/Versions/3.2/Resources/library/rzmq/libs/rzmq.so

General solution to handling system interrupts

A while ago, I sent a pull request to ignore the SIGWINCH signal for receive.socket and poll.socket by ignoring the signal. This worked reasonably well for me so far, but I have come to a point where I need to distinguish between interrupts where I should clean up (SIGINT, SIGTERM) vs. those that I can safely ignore (SIGWINCH, SIGCHLD, etc.).

Right now, rzmq does not allow me to distinguish between e.g. SIGINT (Ctrl+C) and SIGCHLD.

Consider the following example:

library(rzmq)
context = init.context()
socket = init.socket(context, "ZMQ_REP")
bind.socket(socket,"tcp://*:5555")
# p = parallel::mcparallel({Sys.sleep(2)}, detached=TRUE) # SIGCHLD
msg = poll.socket(list(socket), list("read"), timeout=5)

I would expect that

  1. Running this results in list(read=FALSE) as timeout is reached (works)
  2. Pressing Ctrl+C during poll.socket interrupts it (works)
  3. Changing the window size does not throw an error (works using SIG_IGN)
  4. Uncommenting the call to mcparallel, no error is thrown (does not work)
  5. A non-critical interrupt does not produce the unquenchable message Interrupted system call (does not work)
  6. Signals are properly propagated for R to handle (no idea yet)

A potential solution would be to link signal(SIGCHLD, SIG_IGN) as we did before for SIGWINCH. However, it may not be the best to just ignore all non-critical signals as those might be used by R to react to an event.

Another potential solution would be to handle the exception that poll.socket raises on an interrupt in compiled code and leave signals untouched. However, this causes R to crash on Ctrl+C.

Another potential solution would be to handle the exception that poll.socket raises on an interrupt in R code and leave signals untouched. However, one would have to try(...) as well as redirect stdout manually to quench the error message.

Instead, I would propose to add a signal handler to rzmq that listens to SIGTERM and SIGINT.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.