zshipko / ocaml-rs

OCaml extensions in Rust

ocaml-rs - OCaml extensions in Rust

ocaml-rs allows for OCaml extensions to be written directly in Rust with no C stubs. It was originally forked from raml, but has been almost entirely re-written thanks to support from the OCaml Software Foundation.

Works with OCaml versions 4.10.0 and up

Please report any issues on github

NOTE: While ocaml-rs can be used safely, it does not prevent a wide range of potential errors or mistakes. It should be thought of as a Rust implementation of the existing C API. ocaml-interop can be used to perform safe OCaml/Rust interop.

Documentation

• ocaml-rs Book
• docs.rs

Getting started

ocaml-rust-starter is a basic example to help get started with ocaml-rs.

On the Rust side, you will need to add the following to your Cargo.toml:

ocaml = "*"

or

ocaml = {git = "https://github.com/zshipko/ocaml-rs"}

For macOS you will need also to add the following to your project's .cargo/config file:

[build]
rustflags = ["-C", "link-args=-Wl,-undefined,dynamic_lookup"]

This is because macOS doesn't allow undefined symbols in dynamic libraries by default.

Additionally, if you plan on releasing to opam, you will need to vendor your Rust dependencies to avoid making network requests during the build phase, since reaching out to crates.io/github will be blocked by the opam sandbox. To do this you should run:

cargo vendor

then follow the instructions for editing .cargo/config

Build options

By default, building ocaml-sys will invoke the ocamlopt command to figure out the version and location of the OCaml compiler. There are a few environment variables to control this.

• OCAMLOPT (default: ocamlopt) is the command that will invoke ocamlopt
• OCAML_VERSION (default: result of $OCAMLOPT -version) is the target runtime OCaml version.
• OCAML_WHERE_PATH (default: result of $OCAMLOPT -where) is the path of the OCaml standard library.
• OCAML_INTEROP_NO_CAML_STARTUP (default: unset) can be set when loading an ocaml-rs library into an OCaml bytecode runtime (such as utop) to avoid linking issues with caml_startup

If both OCAML_VERSION and OCAML_WHERE_PATH are present, their values are used without invoking ocamlopt. If any of those two env variables is undefined, then ocamlopt will be invoked to obtain both values.

Defining the OCAML_VERSION and OCAML_WHERE_PATH variables is useful for saving time in CI environments where an OCaml install is not really required (to run clippy for example).

Features

• derive
  • enabled by default, adds #[ocaml::func] and friends and derive implementations for FromValue and ToValue
• link
  • link the native OCaml runtime, this should only be used when no OCaml code will be linked statically
• no-std
  • Allows ocaml to be used in #![no_std] environments like MirageOS

ocaml-rs's Issues

wrapping problem on transition from 5 to 6 arguments

I have a 5 arg function that became 6:

pub fn neuro_model_fit(
        mut net: ONetwork,
        data: OTabularDataSet,
        batch_size: u64,
        epochs: u64,
        print_loss: Option<u64>,
        metrics..... new additional argument)...

which then yielded the error:

error: `mut` on a binding may not be repeated
   --> src/lib.rs:333:2
    |
333 |     mut net: ONetwork,
    |     ^^^ help: remove the additional `mut`s

which persisted after update to ocaml-0.13.2 and ocaml-derive-0.13.1

Please update the book with tips for linking in C libraries

Could you please update the book with tips for linking in C libraries?

E.g. I'm trying to wrap an existing OCaml library to make it callable form Rust, and I'm getting this error from Dune:

ld: cannot find -lrt: No such file or directory
ld: cannot find -lgmp: No such file or directory

I've tried a few attempts at getting Dune to find these libraries, but I'm not very familiar with how Dune works.

Since I'm guessing 1) you've already solved this problem, and 2) it's likely to be a common issue, could you please add a note to the book explaining the magic Dune stanzas to use?

Thanks!

`ocaml-build` slow because `dune` checks the `target` directories

Hi,

Recently I've been trying to bootstrap a project with quite a few transitive dependencies using ocaml-build. It works great, but a huge amount of time is spent doing fstat in the /target folder (a difference of around 800 ms when it's practically doing a no-op.

I'm not really sure whether this would be a dune feature, or something that would be better fixed in this library since I'm not too familiar with dune.

bytecode compatibility

I'm trying to run ocamldebug on some OCaml code that is linked to a Rust library (using ocaml-rs). I need to build to the bytecode format before I can do that with dune build some/path.bc but I get the following error:

Error: Error on dynamically loaded library: ./src/lib/crypto/kimchi_bindings/stubs/dllwires_15_stubs.so: Truncated file

I'm wondering if there's a way to make this work? Or if this is because FFI can't happen due to the bytecode format of OCaml

Missing ChangeLog and releases

I wrote a binding using v0.19.0, but it does not compile with v0.20.0 . The lack of ChangeLog between the 2 versions makes it a bit difficult to understand the source of the incompatibility.

How to return dynamic error?

To have nice errors, I converted my panics into functions that return Result<_, ocaml::Error>. But it seems like these errors have to be string literals:

ocaml::Error::Message("issue in ...")

so for now, I print out the error right before returning this, but it's not ideal and I'd like a way to actually return the error I want to return:

ocaml::Error::Message(format!("error: {}", e))

for example

`CamlRootsBlock` uses `usize` for `ntables` and `nitems` fields, but in OCaml `intnat` (`isize`) is used instead.

Hi @zshipko! I'm looking into using ocaml-sys as a dependency to ocaml-interop.

So far it is going well, but I noticed that in ocaml-sys the ntables and nitems fields are defined as usize, but in OCaml those are defined as intnat, which is just an int/long.

If done on purpose, why the difference?

NOTE: not a blocker for ocaml-interop because I'm using a different CamlRootsBlock representation (single local_roots pointer instead of an array of 5 pointers), so I still have to cast the pointers, just something I noticed.

Docs link for 0.5.0 is broken

I see that repr(transparent) is now stable. Docs.rs is now using rustc 1.34.0-nightly.

The 0.5.0 docs build failed, could you release a 0.5.1 so docs.rs builds and shows the documentation?

Example dune needs some changes for reliable/incremental builds

Hi there. Cool project!

Some rough spots to iron out in the example.

w.r.t. test/src/dune, I had to say not dot, but two dots (source_tree ..) for the launched cargo command to be able to find its .toml file after dune copies everything to the _build directory.

Additionally, to get incremental rebuilds to work, I had to change the ../rust path to something absolute, like /var/tmp/rust, so that dune wouldn't blow it away from the _build dir on subsequent runs. Without this, it would rebuild from scratch every time (a 1-2 minute long operation).

Does your dune do this differently? Curious if it can be told to leave the rust build artifacts alone

representing a `Box<T>` with a custom block

I have some bindings I'd like to upgrade, but they used some unsafe pointer mangling and Value::alloc_custom(ptr, finalizer) to build OCaml values. What's the new idiom for that, can I just say return Box<MyStruct> and have the #[ocaml::func] macro take care of the rest?

help with custom-ops...

.

when to call finalize?

Hey!

When it comes to custom types, there are two examples given. One in the README:

unsafe extern "C" fn mytype_finalizer(v: ocaml::Raw) {
    let ptr = v.as_pointer::<MyType>();
    ptr.drop_in_place()
}

ocaml::custom_finalize!(MyType, mytype_finalizer);

and one in the docs:

ocaml-rs/src/custom.rs

Line 190 in ff07e87

/// Derives `Custom` with the given finalizer for a type

which doesn't do anything for the finalizer. It is not clear to me if we have to drop the value or not in the finalizer.

In general it is not clear to me exactly when Rust is not in charge of freeing memory anymore.

Why is FromValue/IntoValue not implemented for u32?

It seems to be implemented for all numeric types excepted this one, is there a reason?

guidelines on Error vs panics and getting good stacktrace

Hey!

It looks like returning a Result in Rust throws an exception in OCaml if it's an error. Why not returning a result type in OCaml? Also, is it equivalent to panicking within the function then? What is the guidance there?

Another question: how to get good stacktraces when you have a panic in the Rust function?

Non-UTF8 string interop

FromValue is implementing for std::string::String, however while OCaml strings may contain arbitrary bytes Rust upholds several invariants on String, namely being valid UTF-8. For this reason I think FromValue should instead be implemented for std::ffi::OsString and the String implementations deprecated or comments adjusted to include a warning that panics can occur if the OCaml string passed is not UTF-8.

install error

hi

I installed on x86-64 system with no problem but then
switched to a armv7l and got this install error:

   Compiling ocaml v0.11.3
error[E0308]: mismatched types
  --> /home/dev/.cargo/registry/src/github.com-1ecc6299db9ec823/ocaml-0.11.3/src/value.rs:56:47
   |
56 |             let named = sys::caml_named_value(s.as_ptr());
   |                                               ^^^^^^^^^^ expected `i8`, found `u8`
   |
   = note: expected raw pointer `*const i8`
              found raw pointer `*const u8`

I don't know enough about rust at the moment to fix this but I note that the cargo.toml
for ocaml-sys says it's repository is the same as for ocaml-rs.
also its doc is docs.rs/ocaml-sys but ocaml-rs's is docs.rs/ocaml !
I also see that in earlier versions caml-named-value was declared in ocaml-rs with a u8
rather than i8.

custom-ops .identifier not set?

in using the custom! macro I noticed the ops.identifier field was not being set as per the distro source.
replacing with a pre-expanded version showed a test .name was not copied across and .identifier was null on inspection.
can you reproduce?

Linking two libraries

I'm building two libraries using exactly the same method as dune file from test/.

This causes multiple definition of certain symbols. Does no-std flag fix this issue? Is there any way to fix this issue in dune build rules?

services/ocaml/tectonic/src/libocxmr_tectonic.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o): In function `<&T as core::fmt::Debug>::fmt':
ocaml.5ccqhzx6-cgu.15:(.text._ZN42_$LT$$RF$T$u20$as$u20$core..fmt..Debug$GT$3fmt17h20a4c52cd7a5419eE+0x0): multiple definition of `<&T as core::fmt::Debug>::fmt'
services/ocaml/simulation-util/src/libocxmr_simulation_util.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o):ocaml.5ccqhzx6-cgu.15:(.text._ZN42_$LT$$RF$T$u20$as$u20$core..fmt..Debug$GT$3fmt17h20a4c52cd7a5419eE+0x0): first defined here
services/ocaml/tectonic/src/libocxmr_tectonic.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o): In function `core::any::TypeId::of':
ocaml.5ccqhzx6-cgu.15:(.text._ZN4core3any6TypeId2of17h3574351e52008d1bE+0x0): multiple definition of `core::any::TypeId::of'
services/ocaml/simulation-util/src/libocxmr_simulation_util.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o):ocaml.5ccqhzx6-cgu.15:(.text._ZN4core3any6TypeId2of17h3574351e52008d1bE+0x0): first defined here
services/ocaml/tectonic/src/libocxmr_tectonic.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o): In function `core::any::TypeId::of':
ocaml.5ccqhzx6-cgu.15:(.text._ZN4core3any6TypeId2of17hac6612b1b30d9d18E+0x0): multiple definition of `core::any::TypeId::of'
services/ocaml/simulation-util/src/libocxmr_simulation_util.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o):ocaml.5ccqhzx6-cgu.15:(.text._ZN4core3any6TypeId2of17hac6612b1b30d9d18E+0x0): first defined here
services/ocaml/tectonic/src/libocxmr_tectonic.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o): In function `core::alloc::layout::size_align':
ocaml.5ccqhzx6-cgu.15:(.text._ZN4core5alloc6layout10size_align17h54c428257d0607c8E+0x0): multiple definition of `core::alloc::layout::size_align'
services/ocaml/simulation-util/src/libocxmr_simulation_util.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o):ocaml.5ccqhzx6-cgu.15:(.text._ZN4core5alloc6layout10size_align17h54c428257d0607c8E+0x0): first defined here
services/ocaml/tectonic/src/libocxmr_tectonic.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o): In function `core::alloc::layout::size_align':
ocaml.5ccqhzx6-cgu.15:(.text._ZN4core5alloc6layout10size_align17ha326d03ba0179201E+0x0): multiple definition of `core::alloc::layout::size_align'
services/ocaml/simulation-util/src/libocxmr_simulation_util.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o):ocaml.5ccqhzx6-cgu.15:(.text._ZN4core5alloc6layout10size_align17ha326d03ba0179201E+0x0): first defined here
services/ocaml/tectonic/src/libocxmr_tectonic.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o): In function `core::alloc::layout::size_align':
ocaml.5ccqhzx6-cgu.15:(.text._ZN4core5alloc6layout10size_align17hd0b939910b139c55E+0x0): multiple definition of `core::alloc::layout::size_align'
services/ocaml/simulation-util/src/libocxmr_simulation_util.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o):ocaml.5ccqhzx6-cgu.15:(.text._ZN4core5alloc6layout10size_align17hd0b939910b139c55E+0x0): first defined here
services/ocaml/tectonic/src/libocxmr_tectonic.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o): In function `core::alloc::layout::size_align':
ocaml.5ccqhzx6-cgu.15:(.text._ZN4core5alloc6layout10size_align17ha326d03ba0179201E+0x0): multiple definition of `core::alloc::layout::size_align'
services/ocaml/simulation-util/src/libocxmr_simulation_util.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o):ocaml.5ccqhzx6-cgu.15:(.text._ZN4core5alloc6layout10size_align17ha326d03ba0179201E+0x0): first defined here
services/ocaml/tectonic/src/libocxmr_tectonic.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o): In function `<T as core::convert::From<T>>::from':
ocaml.5ccqhzx6-cgu.15:(.text._ZN50_$LT$T$u20$as$u20$core..convert..From$LT$T$GT$$GT$4from17h46e681782325ab55E+0x0): multiple definition of `<T as core::convert::From<T>>::from'
services/ocaml/simulation-util/src/libocxmr_simulation_util.a(ocaml-25a72f4d3a6f2ee8.ocaml.5ccqhzx6-cgu.15.rcgu.o):ocaml.5ccqhzx6-cgu.15:(.text._ZN50_$LT$T$u20$as$u20$core..convert..From$LT$T$GT$$GT$4from17h46e681782325ab55E+0x0): first defined here
collect2: error: ld returned 1 exit status
File "caml_startup", line 1:
Error: Error during linking

custom! with bounds

I'm trying to use the custom! macro on a generic struct with bounds:

ocaml::custom!(CamlRandomOracles<F> where F: Field  {
    finalize: CamlRandomOracles::caml_pointer_finalize
});

but it seems like custom! is unhappy. Is there a way to do this with custom!? Will I have to implement Custom manually?

missing lifetime specifier in pointer

I stumbled onto a bizarre error raised by the borrow checker if at least one of the function arguments is a reference and return type is ocaml::Pointer.

Example code that raised the error:

struct Keypair(schnorrkel::Keypair);
ocaml::custom!(Keypair);
struct Signature(schnorrkel::Signature);
ocaml::custom!(Signature);

#[ocaml::func]
pub unsafe fn keypair_sign(
    keypair: ocaml::Pointer<Keypair>,
    ctx: &[u8],
    bytes: &[u8],
) -> ocaml::Pointer<Signature> {
    let ctx = schnorrkel::signing_context(ctx);
    let signature = keypair.as_ref().0.sign(ctx.bytes(bytes));
    ocaml::Pointer::alloc_custom(Signature(signature))
}

It's not possible to make keypair_sign<'a> (a hack anyways) since OCaml functions may not contain generics but it is possible to hack using in_band_lifetimes feature:

#![feature(in_band_lifetimes)]

struct Keypair(schnorrkel::Keypair);
ocaml::custom!(Keypair);
struct Signature(schnorrkel::Signature);
ocaml::custom!(Signature);

#[ocaml::func]
pub unsafe fn keypair_sign(
    keypair: ocaml::Pointer<Keypair>,
    ctx: &'a [u8],
    bytes: &[u8],
) -> ocaml::Pointer<'a, Signature> {
    let ctx = schnorrkel::signing_context(ctx);
    let signature = keypair.as_ref().0.sign(ctx.bytes(bytes));
    ocaml::Pointer::alloc_custom(Signature(signature))
}

This error is not raised when arguments are Vec<u8> etc.

Issue with README?

Hey!

I've been wondering about this example in the README:

#[ocaml::func]
pub fn build_tuple(i: ocaml::Int) -> (ocaml::Int, ocaml::Int, ocaml::Int) {
    (i + 1, i + 2, i + 3)
}

where ocaml::Int seems to be defined as an isize in Rust. Isn't this going to pose issues when reaching values that need the full 64 bits? My understanding is that OCaml uses the last bit as tagging (unless you use int64).

value conversion

with the auto conversion for a slice:

unsafe fn as_slice<'a>(value: Value) -> &'a [Value] {
    ::core::slice::from_raw_parts(
        (value.0 as *const Value).offset(-1),
        crate::sys::wosize_val(value.0) + 1,
    )
}

what's the idea behind the offset and increasing the count?

Request for more complicated list examples

Firstly, thanks for working on this library - I've found it very useful!

I have a question about constructing lists. I'll be happy with just hearing your answer here, but I think this is also something worth including in examples. There is only one example, ml_new_list, which creates a static 5 element list. But what if we want to do something more complicated, where the list has a dynamic length? Here is my attempt:

caml!(make_list, |length|, <dest>, {
    let length = length.i32_val();
    let mut list = ocaml::List::new();
    for i in 1..length {
        list.push_hd(i.to_value())
    }
    dest = list.into()
} -> dest);

external make_list: int -> int list = "make_list"

let () = 
  Printf.printf "Calling \"make_list\"...\n";
  flush(stdout);
  let l = make_list 100000 in
  Printf.printf "...done\n";
  flush(stdout);
  Printf.printf "%d\n" (List.length l);
  ()

On my machine, this can

• print the expected output, if we reduce 100000 number to something much smaller
• segfault in above form
• infinite loop if I remove the printf statements
• print a senseless output (like 87376) if I add some other printf statements in-between

I think this is fair, because we are not obeying the rules of living in harmony with the garbage collector. In particular, our mut list is a local variable that is not specified using CAMLlocal macro (only dest and length are). So I think what happens, is that one of the allocations performed by push_hd triggers garbage collection run, which doesn't see anything pointing to our list and happily collects it, leaving the system in undefined state.

I can't just change mut list to a type defined with CAMLlocal, because they have incompatible types (ocaml::List vs ocaml::Value). How is ocaml::List intended to be used then?

Calling Lwt Ocaml code from Rust

If possible, please support calling Lwt Ocaml code from Rust.
If there's already a good workflow for this, please describe it in the book.
If this isn't something you plan to support, please note that in the book.

The best I can do right now is to run every Lwt function I wrap through Lwt_main.run, but I suspect there's substantial overhead when doing this for each and every function.

Thanks!

Undefined reference to "caml_startup" when trying to run `dune utop`

In an ocaml-rs project, for example ocaml-rust-starter, running dune utop gives the following error (while linking):

/usr/bin/ld: src/libocaml_rust_starter.a(ocaml_interop-4a398e80f3ca6ceb.ocaml_interop.8eq22uyz-cgu.12.rcgu.o): in function `_ZN3std4sync4once4Once9call_once28_$u7b$$u7b$closure$u7d$$u7d$17h7171c1b9a6fc7898E.llvm.3140953836603515050':
ocaml_interop.8eq22uyz-cgu.12:(.text._ZN3std4sync4once4Once9call_once28_$u7b$$u7b$closure$u7d$$u7d$17h7171c1b9a6fc7898E.llvm.3140953836603515050+0x3b): undefined reference to `caml_startup'
/usr/bin/ld: src/libocaml_rust_starter.a(ocaml_interop-4a398e80f3ca6ceb.ocaml_interop.8eq22uyz-cgu.12.rcgu.o): in function `_ZN4core3ops8function6FnOnce40call_once$u7b$$u7b$vtable.shim$u7d$$u7d$17h59471d095cabe6c5E.llvm.3140953836603515050':
ocaml_interop.8eq22uyz-cgu.12:(.text._ZN4core3ops8function6FnOnce40call_once$u7b$$u7b$vtable.shim$u7d$$u7d$17h59471d095cabe6c5E.llvm.3140953836603515050+0x3b): undefined reference to `caml_startup'
collect2: error: ld returned 1 exit status

Adding this function somewhere where the linker sees it, for example using the following code, "fixes" the issue and utop starts fine:

#[ocaml::func]
pub fn caml_startup() {}

OCaml version: 4.11.1
Rust version: nightly
GCC version: 10.2.0

More documentation on advantages of writing extensions in Rust over C

Could you explain in the README or blog post or somewhere the advantages of ocaml-rs over
writing the extensions in C. For example I could see a simple toy example where
it's very easy to make a mistake in the C extension, which leads to a segfault, but
when you port the C extension to Rust, then the Rust borrow checker finds statically
the bug you would have in the C code.

Love OCaml, love Rust, just want more doc, so I would have a bigger incentive
to port some C extensions I know of in Rust.

latest version: cannot find function `caml_format_exception` in crate `ocaml_sys`

I'm getting this error after going from 0.22.2 to 0.22.4

error[E0425]: cannot find function `caml_format_exception` in crate `ocaml_sys`
   --> /Users/davidwong/.cargo/registry/src/github.com-1ecc6299db9ec823/ocaml-0.22.4/src/value.rs:623:30
    |
623 |         let ptr = ocaml_sys::caml_format_exception(self.raw().0);
    |                              ^^^^^^^^^^^^^^^^^^^^^ not found in `ocaml_sys`

error[E0425]: cannot find function `caml_gc_minor` in crate `ocaml_sys`
  --> /Users/davidwong/.cargo/registry/src/github.com-1ecc6299db9ec823/ocaml-0.22.4/src/runtime.rs:17:20
   |
17 |         ocaml_sys::caml_gc_minor(ocaml_sys::UNIT);
   |                    ^^^^^^^^^^^^^ not found in `ocaml_sys`

error[E0425]: cannot find function `caml_gc_major` in crate `ocaml_sys`
  --> /Users/davidwong/.cargo/registry/src/github.com-1ecc6299db9ec823/ocaml-0.22.4/src/runtime.rs:23:16
   |
23 |     ocaml_sys::caml_gc_major(ocaml_sys::UNIT);
   |                ^^^^^^^^^^^^^ not found in `ocaml_sys`

error[E0425]: cannot find function `caml_gc_full_major` in crate `ocaml_sys`
  --> /Users/davidwong/.cargo/registry/src/github.com-1ecc6299db9ec823/ocaml-0.22.4/src/runtime.rs:28:16
   |
28 |     ocaml_sys::caml_gc_full_major(ocaml_sys::UNIT);
   |                ^^^^^^^^^^^^^^^^^^ not found in `ocaml_sys`

error[E0425]: cannot find function `caml_gc_compaction` in crate `ocaml_sys`
  --> /Users/davidwong/.cargo/registry/src/github.com-1ecc6299db9ec823/ocaml-0.22.4/src/runtime.rs:33:16
   |
33 |     ocaml_sys::caml_gc_compaction(ocaml_sys::UNIT);
   |                ^^^^^^^^^^^^^^^^^^ not found in `ocaml_sys`

has the callback api been tested?

I described a problem I have in ocaml's forum, at first glance related to global roots. But the bug persists without storing values in rust, and still manifests itself as an infinite loop in the GC. Looking at the gdb trace it looks like the gc is stuck when marking local roots. Is this part of the API tested by anyone?

Missing `links` manifest key for ocaml-sys

ocaml-sys should ensure that the runtime is only linked once. This is done by adding the links key to the Cargo.toml file. See:

https://doc.rust-lang.org/cargo/reference/build-scripts.html#the-links-manifest-key
https://doc.rust-lang.org/cargo/reference/resolver.html#links

(Otherwise bad things could happen, such as the resolver trying to include two different versions of ocaml-sys in the same crate, AFAIU.)

build failures on m1 mac because of wrong `c_char`

ocaml-sys uses cty to setup c_char and it uses an unsigned type (u8) on aarch64.
Cty: https://github.com/japaric/cty/blob/09bc8e384a7f5bd59f305a6f30c087a32633b690/src/lib.rs#L30

From what I can tell on apple's m1 chip this type needs to be signed (i8)
rust stdlib: https://doc.rust-lang.org/src/std/os/raw/mod.rs.html#101

The actual build error originates from within ocaml-interop when it tries to call caml_named_value or caml_startup as the string types don't match up.

Snippet from build error:

error[E0308]: mismatched types
  --> /Users/anuragsoni/.cargo/registry/src/github.com-1ecc6299db9ec823/ocaml-interop-0.5.3/src/closure.rs:25:30
   |
25 |             caml_named_value(s.as_ptr())
   |                              ^^^^^^^^^^ expected `u8`, found `i8`
   |
   = note: expected raw pointer `*const u8`
              found raw pointer `*const i8`

The Cty project has an issue open to track this japaric/cty#18 and a comment there points to another library (chlorine [1]) that provides c type definitions for no_std and it seems to match the behavior of the stdlib's raw char types.

[1] https://github.com/Lokathor/chlorine

Possibility for invalid memory address when allocating tuples

Just had this bug in ocaml-interop while writing some macros to get rid of the code duplication on my tuple conversions code, and decided to check ocaml-rs because it was a possible issue here too, and indeed there it was.

This code here:

ocaml-rs/src/value.rs

Line 318 in dc1de3c

sys::store_field(self.0, i, val.into_value(rt).0)

Which is called from the tuple macros:

ocaml-rs/src/conv.rs

Line 121 in dc1de3c

v.store_field(rt, $n, self.$n);

Reads self.0 before converting the value that is going to be stored in the field.

This is what I am doing on my macro to avoid the issue:

macro_rules! tuple_to_ocaml {
    ($($n:tt: $t:ident => $ot:ident),+) => {
        unsafe impl<$($t),+, $($ot: 'static),+> ToOCaml<($($ot),+)> for ($($t),+)
        where
            $($t: ToOCaml<$ot>),+
        {
            fn to_ocaml<'a>(&self, cr: &'a mut OCamlRuntime) -> OCaml<'a, ($($ot),+)> {
                let len = $crate::count_fields!($($t)*);

                unsafe {
                     // 1) root the tuple value
                    let ocaml_tuple: BoxRoot<($($ot),+)> = BoxRoot::new(alloc_tuple(cr, len));
                    $(
                        // Perform the field value conversion first (GC may run here)
                        let field_val = self.$n.to_ocaml(cr);
                        // Deref the tuple value here in case it moved
                        store_field(ocaml_tuple.get_raw(), $n, field_val.get_raw());
                    )+

                    cr.get(&ocaml_tuple)
                }
            }
        }
    };
}

An alternative (which is what my original written-by-hand code did) is to perform all the conversions before allocating the tuple block, and then perform the assignments. But this requires that you root all the values too, because they may move before you assign the tuple fields.

Boxroot assertion failing in multithreaded context despite using mutex

Using a lazily initialised global Runtime in a library is resulting in vendor/boxroot/boxroot.c:281: ring_push_back: Assertion '(*target)->hd.class == source->hd.class' failed errors.

With Mutex

Minimal test case with mutex: https://github.com/fmckeogh/ocaml-rust-minimal-failing-segfault/tree/f68ef75ace1d49599c9520e11fa1a4e05d3e94b7

(copied from repo Github Actions https://github.com/fmckeogh/ocaml-rust-minimal-failing-segfault/actions/runs/3260401056/jobs/5353935017)

Run cargo test --workspace --all-features --all-targets --no-fail-fast --release

    Finished release [optimized] target(s) in 0.07s
     Running unittests src/lib.rs (target/release/deps/sail-44467119e94355d2)

running 1 test
error: test failed, to rerun pass '--lib'

Caused by:
  process didn't exit successfully: `/home/runner/work/ocaml-rust-minimal-failing-segfault/ocaml-rust-minimal-failing-segfault/target/release/deps/sail-44467119e94355d2` (signal: 11, SIGSEGV: invalid memory reference)

Without Mutex (possibly different issue)

Minimal test case without: https://github.com/fmckeogh/ocaml-rust-minimal-failing-segfault/tree/d379707f6d35957ce93e28795bed39b858b29003

(Copied from local machine)

pc23430 ~/D/ocaml-rust-minimal-failing-segfault (main|✔) $ cargo test --release
   Compiling sail v0.1.0 (/home/fm208/Documents/ocaml-rust-minimal-failing-segfault)
    Finished release [optimized] target(s) in 0.34s
     Running unittests src/lib.rs (target/release/deps/sail-44467119e94355d2)

running 1 test
sail-44467119e94355d2: vendor/boxroot/boxroot.c:281: ring_push_back: Assertion `(*target)->hd.class == source->hd.class' failed.
sail-44467119e94355d2: vendor/boxroot/boxroot.c:281: ring_push_back: Assertion `(*target)->hd.class == source->hd.class' failed.
error: test failed, to rerun pass '--lib'

Caused by:
  process didn't exit successfully: `/home/fm208/Documents/ocaml-rust-minimal-failing-segfault/target/release/deps/sail-44467119e94355d2` (signal: 6, SIGABRT: process abort signal)

How to opam install a package that uses ocaml-rs

Hey!

I'm trying to publish an opam package that wraps a rust library. So far I've tried the following:

• run cargo vendor
• copy output of cargo vendor in .cargo/config
• add vendor in (dirs :standard \ "target" "vendor") of dune file to avoid errors if dune files are present there (which is my case)
• add all of that to the deps field in the dune rule

  deps
    (source_tree .)
    (source_tree .cargo)
    (source_tree vendor))
  

with all of that opam install . still attempts to make a connection to github. Any idea of what else I might be missing?

How do I use Rust bytes as OCaml bytes ?

I want to call a function that returns values of Vec<u8> from OCaml, but it will be an int array in OCaml.
What do I have to do to get a value of type bytes or string in OCaml?

Segfault when passing some large array of strings from rust to ocaml

Hello,
I came across some segfault while trying to send back a large amount of strings from a rust function called from ocaml (though I might jut be doing something silly here).
Here is a short repro written as an ocaml-rs test. This fails systematically on my box (ocaml 4.11.1, rustc 1.50). I haven't digged much into this but here is the backtrace that I get in gdb:

#0  caml_modify (fp=0x7ffff2de7c58, val=140737350119408) at memory.c:682
#1  0x0000555555641497 in ocaml::conv::<impl ocaml::value::IntoValue for alloc::vec::Vec<V>>::into_value ()
#2  0x000055555563eaec in string_array ()
#3  0x000055555560a115 in camlDune__exe__Foo__fun_440 () at src/foo.ml:19
#4  0x000055555560a030 in camlDune__exe__Foo__check_leaks_121 () at src/foo.ml:11
#5  0x000055555560a15c in camlDune__exe__Foo__entry () at src/foo.ml:18
#6  0x0000555555607dd9 in caml_program ()
#7  0x000055555573f3b0 in caml_start_program ()
#8  0x000055555571dbac in caml_startup_common (argv=0x7fffffffde98, pooling=<optimised out>, pooling@entry=0) at startup_nat.c:164
#9  0x000055555571dbff in caml_startup_exn (argv=<optimised out>) at startup_nat.c:174
#10 caml_startup (argv=<optimised out>) at startup_nat.c:174
#11 0x0000555555607b02 in main (argc=<optimised out>, argv=<optimised out>) at main.c:44

Handling of float arrays

A tricky bit when it comes to float arrays and the OCaml runtime is that they might be represented as a "flat" array of float or as an array of boxed floats depending on how the FLAT_FLOAT_ARRAY variable is set.

When writing C/C++ bindings, one can use the Double_field and Store_double_field macros to appropriately get/set values from such an array, the code of the macros depend on FLAT_FLOAT_ARRAY as can be seen here.

It would be nice to have access to Double_field and Store_double_field in ocaml-sys (as well as the value of FLAT_FLOAT_ARRAY) but as these are macros this is probably not straightforward to do. A C file could be added to make functions out of these macros and then expose them to Rust but I don't think there is any C file that gets compiled at the moment in ocaml-sys so that might be a big change.

Would you have any thoughts on how to handle float arrays with the current ocaml-sys crate? Maybe I'm missing some easy way around this?

Example of calling OCaml functions using `Value::named`

I'm trying to call an OCaml function from a Rust program. It is failing to obtain the registered function when using Value::named. I'm very new to Rust so I may be missing something, but I checked and the binary I produced contains the symbols from the object file generated by OCaml (it is being linked statically). Is there any initialization required that I am missing?

Rust code:

use ocaml::Value;
use ocaml::{ToValue, FromValue};

fn ocaml_twice(n: i32) -> i32 {
    let twice: Value = Value::named("twice").expect("Missing 'twice' function");
    let result = twice.call(n.to_value()).unwrap();
    return i32::from_value(result);
}

fn main() {
    ocaml::runtime::init();
    let result = ocaml_twice(10);
    println!("Result: {}", result);
    ocaml::runtime::shutdown();
}

OCaml code:

let twice x = 2 * x

let () = Callback.register "twice" twice

Dune file:

(executables
 (names callable)
 (modes object))

The cargo file contains:

[dependencies.ocaml]
version = "0.14.2"
features = ["link"]

build.rs:

use std::env;
use std::process::Command;

fn main() {
    let out_dir = env::var("OUT_DIR").unwrap();
    let dune_dir = "../_build/default/ocaml-code";
    Command::new("dune")
        .args(&["build", "../ocaml-code/callable.exe.o"])
        .status()
        .expect("Dune failed");
    Command::new("cp")
        .args(&[
            //&format!("{}/.callable.eobjs/native/dune__exe__Callable.o", dune_dir),
            &format!("{}/callable.exe.o", dune_dir),
            &format!("{}/libcallable.o", out_dir),
        ])
        .status()
        .expect("File copy failed.");
    Command::new("rm")
        .args(&["-f", &format!("{}/libcallable.a", out_dir)])
        .status()
        .expect("rm failed");
    Command::new("ar")
        .args(&[
            "qs",
            &format!("{}/libcallable.a", out_dir),
            &format!("{}/libcallable.o", out_dir),
        ])
        .status()
        .expect("ar failed");

    println!("cargo:rustc-link-search={}", out_dir);
    println!("cargo:rustc-link-lib=static=callable");
}

A potential way to automatically generate the corresponding .ml files?

Hey!

Sorry for using the issues to ask questions, but I wasn't sure what else to use ^^ And thanks for the answer to my other questions.
I found that writing the FFI code in OCaml was very error prone, and also it felt unnecessary, as it was redundant with what I wrote in Rust. For example, for C FFI in Rust you can automagically generate the bindings via https://github.com/rust-lang/rust-bindgen

I was wondering if there could be a way to automatically generate the corresponding *.ml files from Rust as well here. Food for thought, but maybe you have an opinion here?

doc link is broken

documentation url https://docs.rs/ocaml/0.4.3/ocaml/ doesn't lead anywhere

Recursive data structure interop

I would like to open a PR which would implement From/ToValue for Box<T> where T: From/ToValue. Would this be sufficient to support recursive data structures?

nightly only?

On stable rust (1.27.2) I cannot use ocaml-rs:

error[E0658]: the #[repr(transparent)] attribute is experimental (see issue #43036)
--> /home/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/ocaml-0.4.3/src/value.rs:13:1
|
13 | #[repr(transparent)]
| ^^^^^^^^^^^^^^^^^^^^

I suppose there's no way of avoiding it? or could it be feature-gated?

Passing int/int64/values to noalloc functions

Is it possible to pass values other than f64 to noalloc functions? In C, I bind a noalloc functions as follows:

external foo :
  (int, Bigarray.int8_unsigned_elt, Bigarray.c_layout) Bigarray.Array1.t 
  -> (int64[@unboxed]) -> (int[@untagged]) -> string -> unit
  = "foo_bc" "foo" [@@noalloc]

CAMLprim value foo(value a, int64_t b, intnat c, value d) {
    CAMLparam2(a, d);
    do_foo((Caml_ba_data_val(a), b, c, String_val(d));
    CAMLreturn(Val_unit);
}

Naively using ocaml::Value in a extern "C" function produces a warning (and seems to corrupt some parameters):

warning: `extern` fn uses type `Value`, which is not FFI-safe
  --> src/lib.rs:25:11
   |
25 |     data: ocaml::Value,
   |           ^^^^^^^^^^^^ not FFI-safe
   |
   = help: consider adding a `#[repr(C)]`, `#[repr(transparent)]`, or integer `#[repr(...)]` attribute to this enum
   = note: enum has no representation hint

Failure when building ocaml-sys docs on docs.rs

See: https://docs.rs/crate/ocaml-sys/0.20.0

Something changed in 0.20 that broke the build script when some of the env variables are not set (I noticed this elsewhere before seeing this in docs.rs).

I'm fixing something else now, but will let you know if I figure out what is wrong.

alloc_custom intention ?

I think Value::alloc_custom is broken, but not sure what was the intention of the method ?
It allocates a custom block pointer sized, but then copy the pointed value into it ? Should it use ptr::write instead of ptr::swap ?

either :

    /// Allocate a new value with a custom finalizer
    pub fn alloc_custom<T>(value: *mut T, finalizer: extern "C" fn(core::Value)) -> Value {
        let x = unsafe {
            let v = core::alloc::caml_alloc_final(mem::size_of::<*mut T>(), finalizer, 0, 1);
            let ptr = Value::new(v).custom_ptr_val_mut::<*mut T>();
            ptr::write(ptr, value);
            v
        };

        Value::new(x)
    }

    pub fn set_custom<T>(&mut self, value: *mut T) -> * mut T {
        let ptr = self.custom_ptr_val_mut::<*mut T>();
        unsafe {
            ptr::replace(ptr, value)
        }
    }

But i think it could just accept any value, though i'm not sure about potential alignment issue, if something needs larger alignment than word aligned.

    pub fn alloc_custom<T>(value: T, finalizer: extern "C" fn(core::Value)) -> Value {
        let x = unsafe {
            let v = core::alloc::caml_alloc_final(mem::size_of::<T>(), finalizer, 0, 1);
            let ptr = Value::new(v).custom_ptr_val_mut::<T>();
            ptr::write(ptr, value);
            v
        };

        Value::new(x)
    }

    pub fn set_custom<T>(&mut self, value: T) -> T {
        let ptr = self.custom_ptr_val_mut::<T>();
        unsafe {
            ptr::replace(ptr, value)
        }
    }

cc @c-cube

Misleading documentation of nativeint_val

The two functions Value::isize_val and Value::nativeint_val have exactly the same documentation, while being fundamentally different: The former converts from an Ocaml integer, the latter unwraps a native integer wrapped in a custom value.

This difference can lead to segfaults if one confuses them (as I did). Could you clarify the documentation so that others don't have to make the same mistake?

Please add more examples to the `rust-ocaml-starter` project

The rust-ocaml-starter project demonstrates how to wrap a simple "Hello World" function to make it callable in Rust.
Here are the contents of rust_ocaml_starter.ml:

let hello_world () = "Hello, world!"

let () = Callback.register "hello_world" hello_world

However, it doesn't specify (nor can I find in the documentation) how to wrap an example like this:

type t = A | B of int

let maybe_inc (t : t): t =
  match t with
    | A -> A
    | B x -> B (x + 1)

Could you explain how this could be done, and ideally update rust-ocaml-starter?
Or if you could just give me an idea of what to do I could send a pull request to rust-ocaml-starter.

Also, is it possible to wrap "opaque" OCaml types in Rust, e.g. network connections?
E.g. can I wrap code like this?

(* NetworkConnection.t is a complicated type that I don't want to fully wrap.
   But, I do want to construct it in OCaml and pass a reference to it back to Rust. *)
let make_network_connection (): NetworkConnection.t = ...

(* Later I'll invoke this function from Rust. *)
let use_network_connection (conn: NetworkConnection.t): unit = ...

Thanks!

Calling OCaml from Rust

I originally started looking at this project to see if I could use it to call OCaml functions from Rust. I have a static library compiled from an OCaml project and I was hoping that I would be able to call functions in it after having setup the runtime (presumably with runtime::init?). Can these crates be used for this purpose? What would that look like?

undefined symbols _caml_local_roots

Hey!

Any idea what could cause this error? Sorry, I'm grasping at straws here :(

Undefined symbols for architecture x86_64:
  "_caml_local_roots", referenced from:
      ocaml_sys::state::local_roots::h5f371de5d79ec31b in libmarlin_plonk_stubs.a(ocaml_sys-0ebbd1c370daa908.ocaml_sys.1za79sff-cgu.8.rcgu.o)
      ocaml_sys::state::set_local_roots::hc721dba7b42375f7 in libmarlin_plonk_stubs.a(ocaml_sys-0ebbd1c370daa908.ocaml_sys.1za79sff-cgu.8.rcgu.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
File "caml_startup", line 1:
Error: Error during linking (exit code 1)

feedback

I've been using this lib for months now, with a few opaque types behind Box<>, a lot of integer arguments, and the occasional creation of an OCaml array or tuple. It works great, so, thanks!

(yes, this is not an issue, just a thank you)

Call overhead

Hey, I had my code locked to revision 8bfcd27 and since I updated the overhead of a call went from 11ns up to 196ns. I guess some of the speed was sacrificed for security but this seems huge, is there anything I can do about it? I want to find the reason and fix it if possible but a lot of changes were made so you might be able to guide me where to look for the cause.