Use Cython as numba alternative about eko

[felixhekhorn]

Cython and numba:

• https://stackoverflow.com/questions/43016076/calling-cython-functions-from-numba-jitted-code
• https://numba.pydata.org/numba-doc/latest/extending/high-level.html#importing-cython-functions

Cython submodules:

• https://stackoverflow.com/questions/62245550/using-and-importing-cimporting-nested-packages-in-cython

myfilea.pyx

# cython: language_level=3

from src2.myfile2 import f2

cpdef api double foo(double x):
    return x * f2(x)

src2/myfile2.pyx

# cython: language_level=3
def f2(x):
    return 2.*x*x

pkgcy/__init__.py

from scipy.integrate import quad
import os

# if os.environ.get("BUILD_WHEEL",False):
#     pass
# else:
#     import pyximport; pyximport.install()

# from myfilea import f3a as g
# # import myfile

import numba as nb
import ctypes
from numba.extending import get_cython_function_address

addr = get_cython_function_address("myfilea", "foo")
functype = ctypes.CFUNCTYPE(ctypes.c_double, ctypes.c_double)
myexp = functype(addr)

@nb.njit
def f(x):
    return x * myexp(x)

def run():
    # print(myfile.__pyx_capi__)
    # return f(10.)
    return quad(f, 0.,1.)

and

• empty src2/__init__.pyd
• $ cythonize -X language_level=3 -f -a -i myfilea.pyx src2/myfile2.pyx

[alecandido]

We can call more or less in the same way any C library from Numba:

https://numba.readthedocs.io/en/stable/user/cfunc.html#calling-c-code-from-numba

At that point, I'd write the code directly in C/C++, or in Rust and providing a C API.
Moreover, providing the C library for external users was the goal since the beginning. If EKO consumes the same one, at least during the transition, it's not bad (eventually, when the top-level infrastructure will move out of Numba, we could also consume a different kind of API). In any case, we aim to drop Numba.

[felixhekhorn]

yes, but the main advantage of using Cython is the minimal effort in rewriting, because I'm still afraid of doing all in one gigantic step ... this way we could, maybe, do step by step, though I'm not sure still ...

[alecandido]

this way we could, maybe, do step by step, though I'm not sure still...

This is exactly what I was pointing out: the same feature that allows Cython to be introduced one step at a time could be used with whatever C library (because it's based on ctypes).

[felixhekhorn]

Cython is the minimal effort in rewriting

well, we would still need to rewrite everything from scratch (using C/Rust syntax)

[alecandido]

well, we would still need to rewrite everything from scratch (using C/Rust syntax)

For as long as it's just math, the C/Rust syntax is mostly Python syntax. It was Fortran in the first place...
(and doing it incrementally should make this a fully negligible concern)

[felixhekhorn]

closed in favour of #189