Esperanto makes OCaml a build-once run-anywhere language, like itself, except it doesn't need an interpreter or virtual machine. Indeed, OCaml is able to produce a native executable which requires few libraries:
- a standard C library
libasmrun.a(the caml-runtime)unix.cmxaandthreads.cmxa
Esperanto replaces the host's C library by Cosmopolitan. Then, it outputs a POSIX-approved polyglot format that runs on many platforms. For more details, please read the αcτµαlly pδrταblε εxεcµταblε.
The goal of esperanto is to provide a toolchain which is able to compile an
OCaml project to aarch64 and x86_64 and link these artifacts together to
produce an actually portable executable. Esperanto is primarily a distribution
tool rather than a development tool. We advise the user to have a finalized
project and then use Esperanto to produce a distribution.
The pipeline used to produce such a portable executable must respect certain constraints:
- not depend on dynamic objects (
*.so) - the C files required for compilation can also be compiled with the Cosmopolitan C library
Let's take the example of a simple program to understand the basic "pipeline"
required to produce a .com (for COsMpolitan) executable. We recommend using
Esperanto in a [Docker][docker] or "clean" system (like a special
[opam switch][switch]). Once again, the aim is to produce an artifact from a
finalized project, not to develop an application. So you need opam. Let's take
Docker as an example:
$ docker run -it --platform amd64 ocaml/opam:ubuntu-23-04-ocaml-4.14
$ opam pin add -y https://github.com/dinosaure/esperanto.gitEsperanto installs 4 packages:
conf-cosmopolitan, this package verifies that your system can, indeed, run ape programs.esperanto-cosmopolitan, this package is the Cosmopolitan toolchain as available from [cosmocc.zip][cosmo.zip] (v3.1.3) packaged foropam. It verifies the integrity of distribution files according to these [fingerprints][./toolchain/cosmocc.digest]
Finally, 2 versions of the OCaml compiler are available:
- To produce an
x86_64program (x86_64-esperanto) - To produce an
aarch64program (aarch64-esperanto)
Let's start to make a simple Hello World! example:
$ cat >main.ml <<EOF
let () = print_endline "Hello World!"
EOF
$ ocamlfind -toolchain x86_64_esperanto opt -o main.x86_64.elf main.ml
$ ocamlfind -toolchain aarch64_esperanto opt -o main.aarch64.elf main.ml
$ apelink -o main.com \
-l $(opam var bin)/ape-x86_64.elf \
-l $(opam var bin)/ape-aarch64.elf \
-M $(opam var bin)/ape-m1.c \
main.x86_64.elf main.aarch64.elf
$ sh -c "./main.com"
Hello World!Producing an executable from your OCaml project that is portable on all systems requires 2 precise points:
- project dependencies must be available to compile in a consistent context.
Such consistency can be achieved by
duneand a "monorepo" format for your project. - C dependencies must be able to compile with Cosmopolitan.
[cosmocc.zip][cosmocc.zip] offers a patched version of GCC and Cosmopolitan
exposes many functions. However, you may need to patch your C code (in
particular, you can use
__COSMOPOLITAN__).
If these 2 points are met (getting the dependency sources and having Cosmopolitan-compatible C code), we're ready to produce the final artifact using the Esperanto toolchain. So, 3 steps are necessary:
- compile project for x86_64 architecture
- compile project for aarch64 architecture
- link the 2
ELFfiles into a portable binary
To achieve our 2 objectives, we're going to use dune to provide us with the
famous contexts linked to our 2 toolchains (x86_64 and aarch64) and
opam monorepo to download locally the dependencies needed for our project (the
constraint implies that all dependencies use dune). Let's play with [hxd][hxd],
a small project.
You can take your favorite project and compile executables with the Esperanto toolchain. You need to specify two new contexts:
$ git clone https://github.com/dinosaure/hxd.git
$ cd hxd
$ opam repo add opam-overlays https://github.com/dune-universe/opam-overlays.git
$ sudo apt install pkg-config
$ cat >dune-workspace<<EOF
(lang dune 2.0)
(context (default))
(context
(default
(name x86_64-esperanto)
(toolchain x86_64_esperanto)
(host default)))
(context
(default
(name aarch64-esperanto)
(toolchain aarch64_esperanto)
(host default)))
EOFFinally, as a static program, you must fetch dependencies with opam monorepo and build your project with them:
$ opam monorepo lock --ocaml-version=4.14.1
$ opam monorepo pull
$ dune build bin/xxd.exe
$ apelink -o xxd.com \
-l $(opam var bin)/ape-x86_64.elf \
-l $(opam var bin)/ape-aarch64.elf \
-M $(opam var bin)/ape-m1.c \
_build/bin/x86_64-esperanto/xxd.exe.dbg \
_build/bin/aarch64-esperanto/xxd.exe.dbgNow, you have a portable/polyglot program! The user can assimilate the
program with the --assimilate option. Assimilation consists in "specializing"
the executable for the host system.
$ sh -c "./bin/xxd.com --assimilate"
$ uname -p
x86_64
$ file bin/xxd.com
bin/xxd.com: ELF 64-bit LSB executable, x86-64The --assimilate option modify the executable itself to become really
native to your platform - in other words, the executable no longer becomes
polyglot!
Such design comes from MirageOS and Solo5. For C stubs, the
cosmocc toolchain provides the __COSMOPOLITAN__ definition which permits to
orchestrate your compilation of C files according to the OCaml toolchain.
Currently, we support OCaml 4.14.
Currently, zsh < 5.9.0 does not support well Cosmopolitan/APE binaries.
However, the recent version fix the initial issue. In the case of you have
zsh < 5.9.0, you can use the --assimilate option to modify the executable
to a real native application:
$ zsh --version
zsh 5.8.1
$ zsh
$ ./a.out
zsh: exec format error: ./a.out
$ sh -c "./a.out --assimilate"
$ ./a.out
Hello World!If the issue persist with Python subprocess or old versions of fish, we also
advise an "assimilation" in these cases. However, as explained above, the binary
no longer becomes polyglot!
An other issue on Linux is about binfmt_misc which wants to interpret the
binary as a Windows executable (due to the header of the APE binary). A good
explanation is available here: jart/cosmopolitan#2
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent