kxsystems / embedr

Embedding R inside q

embedR: Embedding R inside q

See https://code.kx.com/v2/interfaces/with-r/#calling-r-from-q

Installation

Download

Download the appropriate release archive from releases page.

Unpack and install content of the archive

environment	action
Linux	tar xzvf embedr_linux-v*.tar.gz -C $QHOME --strip 1
macOS	tar xzvf embedr_osx-v*.tar.gz -C $QHOME --strip 1
Windows	Open the archive and copy content of the embedr folder (embedr\*) to %QHOME% or c:\q Copy R_HOME/x64/.dll or R_HOME/i386/.dll to QHOME/w64 or QHOME/w32 respectively.

Calling R

When calling R, you need to set R_HOME. Required are:

# Linux/macOS
export R_HOME=`R RHOME`
# Windows
for /f "delims=" %a in ('R RHOME') do @set R_HOME=%a

The library has four main methods:

• Ropen: Initialise embedded R. Optional to call. Allows to set verbose mode as Ropen 1.
• Rcmd: run an R command, do not return a result
• Rget: run an R command, return the result to q
• Rset: set a variable in the R memory space

Interactive plotting

If using interactive plotting with lattice and/or ggplot2 you will need to call print on a chart object.

Examples

See examples folder.

Note: Examples are kdb+ 3.5 or higher.

Example 1

e4.q is a simple example of plot 'moving window volatility' of returns. Converted from http://www.mayin.org/ajayshah/KB/R/html/p4.html

Example 2

pcd.q is based on Corporate credit card transactions 2014-15.

Download CSV from the link above and save it in the same folder as pcd.q under name pcd2014v1.csv.

Example 3

http://data.london.gov.uk/datastore/package/tubenetwork-performance-data

Left for the reader :)

embedr's Issues

Investigate use of ALTREP for interop

Available from R 3.5 this could reduce number of copying to/from kdb and increase performance

release 1.0.0 for w32 is not working

release 1.0.0 for w32 is not working

MacOS release issue

Use of pre compiled Mac release of embedR causes the process to segfault using newer releases of R. This is seemingly as a result of modifications to the R C api.

• kdb+ 4.0 2020.07.15
• R 3.6.1 2019-07-05

Warning message:
In options(stringsAsFactors = TRUE) :
  'options(stringsAsFactors = TRUE)' is deprecated and will be disabled

 *** caught segfault ***
address 0x2e9e8, cause 'memory not mapped'

Traceback:
 1: initMethodDispatch(where)
 2: fun(libname, pkgname)
 3: doTryCatch(return(expr), name, parentenv, handler)
 4: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 5: tryCatchList(expr, classes, parentenv, handlers)
 6: tryCatch(fun(libname, pkgname), error = identity)
 7: runHook(".onLoad", env, package.lib, package)
 8: loadNamespace(package, lib.loc)
 9: doTryCatch(return(expr), name, parentenv, handler)
10: tryCatchOne(expr, names, parentenv, handlers[[1L]])
11: tryCatchList(expr, classes, parentenv, handlers)
12: tryCatch({    attr(package, "LibPath") <- which.lib.loc    ns <- loadNamespace(package, lib.loc)    env <- attachNamespace(ns, pos = pos, deps, exclude, include.only)}, error = function(e) {    P <- if (!is.null(cc <- conditionCall(e)))         paste(" in", deparse(cc)[1L])    else ""    msg <- gettextf("package or namespace load failed for %s%s:\n %s",         sQuote(package), P, conditionMessage(e))    if (logical.return)         message(paste("Error:", msg), domain = NA)    else stop(msg, call. = FALSE, domain = NA)})
13: library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE,     warn.conflicts = warn.conflicts, quietly = quietly, mask.ok = mask.ok,     exclude = exclude, include.only = include.only, attach.required = attach.required)
14: doTryCatch(return(expr), name, parentenv, handler)
15: tryCatchOne(expr, names, parentenv, handlers[[1L]])
16: tryCatchList(expr, classes, parentenv, handlers)
17: tryCatch(library(package, lib.loc = lib.loc, character.only = TRUE,     logical.return = TRUE, warn.conflicts = warn.conflicts, quietly = quietly,     mask.ok = mask.ok, exclude = exclude, include.only = include.only,     attach.required = attach.required), error = function(e) e)
18: require(pkg, quietly = TRUE, warn.conflicts = FALSE, character.only = TRUE)
19: .OptRequireMethods()

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace

This can be reconciled by building the interface from source which hasn't changed any of the code but will change the api to which it is compiled.

A new release of embedR should be drafted compiled against the newer R C api to rectify this issue

no up to date release for win32/w64

no up to date release for win32/w64

Windows compile issue with sockets

Havent compiled this under windows to test, but from viewing the code it doesn't look quite right...

src/rserver.c
has
static I spair[2];

This isnt used under windows by the socket code/etc. Should be something like
#ifdef _WIN32
static SOCKET spair[2];
#else
static int spair[2];
#endif

Missing 77 type

Type 77 causes mapping fault.

Reproduction:

q)Ropen()
0i
q)Rset["a";`sym?`a`b`c]
q)`:x set string 10?`4
`:x
q)Rset["a";get `:x]
Error: Value is not a valid kdb+ object; unknown type 77
Fatal error: unable to initialize the JIT

• R version 3.6.2
• kdb+ 4.0 2021.02.02
• embedR 1.2.1
• Source of issue:

  embedR/src/common.c

  Line 200 in 63640f5

  else if (KT<type && type < 77) {