NodeJS wrapper for GMT about building-wrappers

[joa-quim]

Hi, I never used Conda myself but to install GMT on MacOS I use Homebrew (for official release builds)

brew install homebrew/science/gmt

For the NodeJs thing, I know about nothing of it but it maybe worth investigating

https://github.com/davidanthoff/NodeJS.jl

because the GMT Julia wrapper already exists and works fine.

[davidedelerma]

Thanks @joa-quim , I wanted to build the version 6.0 of GMT (which is provided with conda-forge), the one provided with homebrew is the 5.4.2.
But thanks the installation with homebrew works fine.
Thank you for the link to that repo, but from the README doesn't seems to do exactly what I am trying to achieve.
I never used Julia (and I really want to start using it at a certain point), but it would be good if you can share more info about how you have built the wrapper for Julia as the concept should be the same.
Thanks,
Davide

[joa-quim]

Leonardo has rebuild the miniconda but he has not merged it yet. Maybe that solves your building issue. Otherwise you can just use the dependencies installed by brew and build the GMT trunk manually. It would be a mater of creating a minimal ConfigUser.cmake

Regarding the Julia wrapper. Well, Julia let us call a (C, not C++) shared library directly so what I did was to call the GMT API functions to interface Julia and the GMT lib. You can see how it was done in the gmt_main.jl program (it mimics what what we have done for the Matlab gmtmex).

[leouieda]

@davidedelerma I'm sorry to hear you're stuck in linked library hell. I often find myself there after updates.

Can you try uninstalling all of that and reinstalling? The latest trunk build on conda-forge is 6.0.0a9 an I notice you have a6 installed. As @joa-quim said, I updated this recently (it takes an annoyingly long time for the Mac CI builds). Maybe that fixes the problem.

I didn't have those problems when testing GMT/Python on OSX using TravisCI. I use the conda-package there.

For the wrapper, you might want to look at using a foreign function interface (ffi) to call the functions of the GMT shared library directly. This is what I'm doing with the Python wrapper. The Python package for FFI is ctypes. Googling around, I found a Node FFI package: https://github.com/node-ffi/node-ffi

The major challenge is wrapping some of the data structures so that you pass data in and out of the GMT functions. If you're fine with doing all IO through files, then you can avoid this all together.

[c1tt1]

Hi @leouieda I am actually working with @davidedelerma on this project, and Nodejs you need to write addons to interact with C/C++ code, like in this library https://github.com/node-ffi/node-ffi but the compiled C/C++ dynamic libraries are needed as you can see in the examples https://github.com/node-ffi/node-ffi/tree/master/example/factorial you need a .dylib for OSX, .so for Linux/Solaris/etc and .dll for Windows in order to call the C/C++ code from Javascript. The best case here would be compile the C code to dynamic libraries and call them from Javascript and was wondering how you compiled it when developing, I haven't been that lucky.

[leouieda]

@stern0 since version 5, GMT is now built as a dynamic library (libgmt.so etc) and the command line program makes calls to this library. So if you follow the instructions on the GMT website for building you will get a libgmt library. The conda packages also provide the gmt executable and the libgmt shared library.

I load this library in Python directly using ctypes: https://github.com/GenericMappingTools/gmt-python/blob/36f6b5a86dd20d10dadb17170568d083ffa04a4a/gmt/clib/core.py#L67

libgmt = ctypes.CDLL('.'.join([libname, clib_extension()]))

It actually looks very similar to what node-ffi does:

var libfactorial = ffi.Library('./libfactorial', {
  'factorial': [ 'uint64', [ 'int' ] ]
})

[c1tt1]

@leouieda thanks for the info it was helpful, we started working on it here https://github.com/stern0/gmt-node, unfortunately the node-ffi library has a lot of unfixed issues and hasn't been updated for over a year, which are critical to writing the library. @davidedelerma and I have decided with writing native C bindings approach rather than the node-ffi library.

[leouieda]

@stern0 I'm sorry to hear that. How painful is it to write node C bindings by hand? In Python I know it's a nightmare. Is there anything like Cython for Node?

[c1tt1]

Hi @leouieda sorry for the late reply, there is nothing like Cython or Cpython in Javascript land, the only thing similar was the node-ffi library which hasn't been maintained for over a year ago and has several issues. There is Node.js addons which are are dynamically-linked shared objects, written in C++, that can be loaded into Nodejs, this is the option we are trying to use.

My question is how up to date is the GMT_API documentation pdf file??

I have been having issues trying the API and here is an example inside the GMT_API pdf docs:

GMT_Close_VirtualFile (API, 0, input);

But inside the gmt.h header file that function signature is:

EXTERN_MSC int GMT_Close_VirtualFile  (void *API, const char *string);

This shows the function takes two arguments but the api docs lists 3 instead. I was wondering if that's a deprecated documentation, and if so it would be good to have an updated one. Please let me know your thoughts.

[joa-quim]

@stern0 Don't know how the PDF got that API documentation (a very old version?) because the HTML docs agree with the header file

int GMT_Close_VirtualFile (void *API, char *filename);

[c1tt1]

@joa-quim Thanks. It seems both pdf and html docs sample code docs have same mistakes.

GMT_Close_VirtualFile (API, 0, input);

But anyway I will work around that.

[leouieda]

@stern0 the docs can be a bit outdated so it's best to check the source as well to be sure. We'll be working hard to update and improve the docs for the 6.0 release. Any feedback on that would be very helpful!

There was a problem with the gmt server that updates the docs (fixed now) so you might have been looking at an older version.

[leouieda]

@stern0 the virtual file machinery is fully implemented in gmt-python now. If you want to have a look at what we're doing, here is a test for passing in a numpy 2d array: https://github.com/GenericMappingTools/gmt-python/blob/master/gmt/tests/test_clib.py#L395