jrenaud90 / cyrk

Runge-Kutta ODE Integrator Implemented in Cython and Numba

CyRK

---

Runge-Kutta ODE Integrator Implemented in Cython and Numba

CyRK provides fast integration tools to solve systems of ODEs using an adaptive time stepping scheme. CyRK can accept differential equations that are written in pure Python, njited numba, or cython-based cdef functions. These kinds of functions are generally easier to implement than pure c functions and can be used in existing Python software. Using CyRK can speed up development time while avoiding the slow performance that comes with using pure Python-based solvers like SciPy's solve_ivp.

The purpose of this package is to provide some functionality of scipy's solve_ivp with greatly improved performance.

Currently, CyRK's numba-based (njit-safe) implementation is 10-140x faster than scipy's solve_ivp function. The cython-based pysolve_ivp function that works with python (or njit'd) functions is 20-50x faster than scipy. The cython-based cysolver_ivp function that works with cython-based cdef functions is 50-700+x faster than scipy.

An additional benefit of the two cython implementations is that they are pre-compiled. This avoids most of the start-up performance hit experienced by just-in-time compilers like numba.

Installation

CyRK has been tested on Python 3.8--3.12; Windows, Ubuntu, and MacOS.

To install simply open a terminal and call:

pip install CyRK

If not installing from a wheel, CyRK will attempt to install Cython and Numpy in order to compile the source code. A "C++ 14" compatible compiler is required. Compiling CyRK has been tested on the latest versions of Windows, Ubuntu, and MacOS. Your milage may vary if you are using a older or different operating system. After everything has been compiled, cython will be uninstalled and CyRK's runtime dependencies (see the pyproject.toml file for the latest list) will be installed instead.

A new installation of CyRK can be tested quickly by running the following from a python console.

from CyRK import test_pysolver, test_cysolver, test_nbrk
test_pysolver()
# Should see "CyRK's PySolver was tested successfully."
test_cysolver()
# Should see "CyRK's CySolver was tested successfully."
test_nbrk()
# Should see "CyRK's nbrk_ode was tested successfully."

Troubleshooting Installation and Runtime Problems

Please report installation issues. We will work on a fix and/or add workaround information here.

• If you see a "Can not load module: CyRK.cy" or similar error then the cython extensions likely did not compile during installation. Try running pip install CyRK --no-binary="CyRK" to force python to recompile the cython extensions locally (rather than via a prebuilt wheel).
• On MacOS: If you run into problems installing CyRK then reinstall using the verbose flag (pip install -v .) to look at the installation log. If you see an error that looks like "clang: error: unsupported option '-fopenmp'" then you may have a problem with your llvm or libomp libraries. It is recommended that you install CyRK in an Anaconda environment with the following packages conda install numpy scipy cython llvm-openmp. See more discussion here and the steps taken here.
• CyRK has a number of runtime status codes which can be used to help determine what failed during integration. Learn more about these codes https://github.com/jrenaud90/CyRK/blob/main/Documentation/Status%20and%20Error%20Codes.md.

Development and Testing Dependencies

If you intend to work on CyRK's code base you will want to install the following dependencies in order to run CyRK's test suite and experimental notebooks.

conda install pytest scipy matplotlib jupyter

conda install can be replaced with pip install if you prefer.

Using CyRK

The following code can be found in a Jupyter Notebook called "Getting Started.ipynb" in the "Demos" folder.

Note: some older CyRK functions like cyrk_ode and CySolver class-based method have been deprecated. Read more in "Documentation/Deprecations.md". CyRK's API is similar to SciPy's solve_ivp function. A differential equation can be defined in python such as:

# For even more speed up you can use numba's njit to compile the diffeq
from numba import njit
@njit
def diffeq_nb(t, y):
    dy = np.empty_like(y)
    dy[0] = (1. - 0.01 * y[1]) * y[0]
    dy[1] = (0.02 * y[0] - 1.) * y[1]
    return dy

Numba-based nbsolve_ivp

Future Development Note: The numba-based solver is currently in a feature-locked state and will not receive new features (as of CyRK v0.9.0). The reason for this is because it uses a different backend than the rest of CyRK and is not as flexible or easy to expand without significant code duplication. Please see GitHub Issue: TBD to see the status of this new numba-based solver or share your interest in continued development.

The system of ODEs can then be solved using CyRK's numba solver by,

import numpy as np
from CyRK import nbsolve_ivp

initial_conds = np.asarray((20., 20.), dtype=np.float64, order='C')
time_span = (0., 50.)
rtol = 1.0e-7
atol = 1.0e-8

result = \
    nbsolve_ivp(diffeq_nb, time_span, initial_conds, rk_method=1, rtol=rtol, atol=atol)

print("Was Integration was successful?", result.success)
print(result.message)
print("Size of solution: ", result.size)
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.plot(result.t, result.y[0], c='r')
ax.plot(result.t, result.y[1], c='b')

nbsolve_ivp Arguments

nbsolve_ivp(
    diffeq: callable,                  # Differential equation defined as a numba.njit'd python function
    t_span: Tuple[float, float],       # Python tuple of floats defining the start and stop points for integration
    y0: np.ndarray,                    # Numpy array defining initial y0 conditions.
    args: tuple = tuple(),             # Python Tuple of additional args passed to the differential equation. These can be any njit-safe object.
    rtol: float = 1.e-3,               # Relative tolerance used to control integration error.
    atol: float = 1.e-6,               # Absolute tolerance (near 0) used to control integration error.
    rtols: np.ndarray = EMPTY_ARR,     # Overrides rtol if provided. Array of floats of rtols if you'd like a different rtol for each y.
    atols: np.ndarray = EMPTY_ARR,     # Overrides atol if provided. Array of floats of atols if you'd like a different atol for each y.
    max_step: float = np.inf,          # Maximum allowed step size.
    first_step: float = None,          # Initial step size. If set to 0.0 then CyRK will guess a good step size.
    rk_method: int = 1,                # Integration method. Current options: 0 == RK23, 1 == RK45, 2 == DOP853
    t_eval: np.ndarray = EMPTY_ARR,    # `nbsolve_ivp` uses an adaptive time stepping protocol based on the recent error at each step. This results in a final non-uniform time domain of variable size. If the user would like the results at specific time steps then they can provide a np.ndarray array at the desired steps via `t_eval`. The solver will then interpolate the results to fit this 
    capture_extra: bool = False,       # Set to True if the diffeq is designed to provide extra outputs.
    interpolate_extra: bool = False,   # See "Documentation/Extra Output.md" for details.
    max_num_steps: int = 0             # Maximum number of steps allowed. If exceeded then integration will fail. 0 (the default) turns this off.
    )

Python wrapped pysolve_ivp

CyRK's main integration functions utilize a C++ backend system which is then wrapped and accessible to Python via Cython. The easiest way to interface with this system is through CyRK's pysolve_ivp function. It follows a very similar format to both nbsolve_ivp and Scipy's solve_ivp. First you must build a function in Python. This could look the same as the function described above for nbsolve_ivp (see diffeq_nb). However, there are a few advantages that pysolve_ivp provides over nbsolve_ivp:

1. It accepts both functions that use numba's njit wrapper (as diffeq_nb did above) or pure Python functions (nbsolve_ivp only accepts njit'd functions).
2. You can provide the resultant dy/dt as an argument which can provide a significant performance boost.

Utilizing point 2, we can re-write the differential equation function as,

# Note if using this format, `dy` must be the first argument. Additionally, a special flag must be set to True when calling pysolve_ivp, see below.
def cy_diffeq(dy, t, y):
    dy[0] = (1. - 0.01 * y[1]) * y[0]
    dy[1] = (0.02 * y[0] - 1.) * y[1]

Since this function is not using any special functions we could easily wrap it with njit for additional performance boost: cy_diffeq = njit(cy_diffeq).

Once you have built your function the procedure to solve it is:

import numpy as np
from CyRK import pysolve_ivp

initial_conds = np.asarray((20., 20.), dtype=np.complex128, order='C')
time_span = (0., 50.)
rtol = 1.0e-7
atol = 1.0e-8

result = \
    pysolve_ivp(cy_diffeq, time_span, initial_conds, method="RK45", rtol=rtol, atol=atol,
                # Note if you did build a differential equation that has `dy` as the first argument then you must pass the following flag as `True`.
                # You could easily pass the `diffeq_nb` example which returns dy. You would just set this flag to False (and experience a hit to your performance).
                pass_dy_as_arg=True)

print("Was Integration was successful?", result.success)
print(result.message)
print("Size of solution: ", result.size)
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.plot(result.t, result.y[0], c='r')
ax.plot(result.t, result.y[1], c='b')

pysolve_ivp Arguments

def pysolve_ivp(
        object py_diffeq,            # Differential equation defined as a python function
        tuple time_span,             # Python tuple of floats defining the start and stop points for integration
        double[::1] y0,              # Numpy array defining initial y0 conditions.
        str method = 'RK45',         # Integration method. Current options are: RK23, RK45, DOP853
        double[::1] t_eval = None,   # Array of time steps at which to save data. If not provided then all adaptive time steps will be saved. There is a slight performance hit using this feature.
        bint dense_output = False,   # If True, then dense interpolators will be saved to the solution. This allows a user to call solution as if a function (in time).
        tuple args = None,           # Python Tuple of additional args passed to the differential equation. These can be any python object.
        size_t expected_size = 0,    # Expected size of the solution. There is a slight performance improvement if selecting the the exact or slightly more time steps than the adaptive stepper will require (if you happen to know this ahead of time).
        unsigned int num_extra = 0,  # Number of extra outputs you want to capture during integration. There is a performance hit if this is used in conjunction with t_eval or dense_output.
        double first_step = 0.0,     # Initial step size. If set to 0.0 then CyRK will guess a good step size.
        double max_step = INF,       # Maximum allowed step size.
        rtol = 1.0e-3,               # Relative tolerance used to control integration error. This can be provided as a numpy array if you'd like a different rtol for each y.
        atol = 1.0e-6,               # Absolute tolerance (near 0) used to control integration error. This can be provided as a numpy array if you'd like a different atol for each y.
        size_t max_num_steps = 0,    # Maximum number of steps allowed. If exceeded then integration will fail. 0 (the default) turns this off.
        size_t max_ram_MB = 2000,    # Maximum amount of system memory the integrator is allowed to use. If this is exceeded then integration will fail.
        bint pass_dy_as_arg = False  # Flag if differential equation returns dy (False) or is passed dy as the first argument (True).
        ):

Pure Cython cysolve_ivp

A final method is provided to users in the form of cysolve_ivp. This function can only be accessed and used by code written in Cython. Details about how to setup and use Cython can be found on the project's website. The below code examples assume you are running the code in a Jupyter Notebook.

cysolve_ivp has a slightly different interface than nbsolve_ivp and pysolve_ivp as it only accepts C types. For that reason, python functions will not work with cysolve_ivp. While developing in Cython is more challenging than Python, there is a huge performance advantage (cysolve_ivp is roughly 5x faster than pysolve_ivp and 700x faster than scipy's solve_ivp). Below is a demonstration of how it can be used.

First a pure Cython file (written as a Jupyter notebook).

%%cython --force 
# distutils: language = c++
# cython: boundscheck=False, wraparound=False, nonecheck=False, cdivision=True, initializedcheck=False

import numpy as np
cimport numpy as np
np.import_array()

# Note the "distutils" and "cython" headers above are functional. They tell cython how to compile the code. In this case we want to use C++ and to turn off several safety checks (which improve performance).

# The cython diffeq is much less flexible than the others described above. It must follow this format, including the type information. 
# Currently, CyRK only allows additional arguments to be passed in as a double array pointer (they all must be of type double). Mixed type args will be explored in the future if there is demand for it (make a GitHub issue if you'd like to see this feature!).
# The "noexcept nogil" tells cython that the Python Global Interpretor Lock is not required, and that no exceptions should be raised by the code within this function (both improve performance).
# If you do need the gil for your differential equation then you must use the `cysolve_ivp_gil` function instead of `cysolve_ivp`

# Import the required functions from CyRK
from CyRK cimport cysolve_ivp, DiffeqFuncType, WrapCySolverResult, CySolveOutput, PreEvalFunc

# Note that currently you must provide the "const void* args, PreEvalFunc pre_eval_func" as inputs even if they are unused.
# See "Advanced CySolver.md" in the documentation for information about these parameters.
cdef void cython_diffeq(double* dy, double t, double* y, const void* args, PreEvalFunc pre_eval_func) noexcept nogil:
    
    # Unpack args
    # CySolver assumes an arbitrary data type for additional arguments. So we must cast them to the array of 
    # doubles that we want to use for this equation
    cdef double* args_as_dbls = <double*>args
    cdef double a = args_as_dbls[0]
    cdef double b = args_as_dbls[1]
    
    # Build Coeffs
    cdef double coeff_1 = (1. - a * y[1])
    cdef double coeff_2 = (b * y[0] - 1.)
    
    # Store results
    dy[0] = coeff_1 * y[0]
    dy[1] = coeff_2 * y[1]
    # We can also capture additional output with cysolve_ivp.
    dy[2] = coeff_1
    dy[3] = coeff_2

# Import the required functions from CyRK
from CyRK cimport cysolve_ivp, DiffeqFuncType, WrapCySolverResult, CySolveOutput

# Let's get the integration number for the RK45 method
from CyRK cimport RK45_METHOD_INT

# Now let's import cysolve_ivp and build a function that runs it. We will not make this function `cdef` like the diffeq was. That way we can run it from python (this is not a requirement. If you want you can do everything within Cython).
# Since this function is not `cdef` we can use Python types for its input. We just need to clean them up and convert them to pure C before we call cysolve_ivp.
def run_cysolver(tuple t_span, double[::1] y0):
    
    # Cast our diffeq to the accepted format
    cdef DiffeqFuncType diffeq = cython_diffeq
    
    # Convert the python user input to pure C types
    cdef double* y0_ptr       = &y0[0]
    cdef unsigned int num_y   = len(y0)
    cdef double[2] t_span_arr = [t_span[0], t_span[1]]
    cdef double* t_span_ptr   = &t_span_arr[0]

    # Assume constant args
    cdef double[2] args   = [0.01, 0.02]
    cdef double* args_dbl_ptr = &args[0]
    # Need to cast the arg double pointer to void
    cdef void* args_ptr = <void*>args_dbl_ptr

    # Run the integrator!
    cdef CySolveOutput result = cysolve_ivp(
        diffeq,
        t_span_ptr,
        y0_ptr,
        num_y,
        method = RK45_METHOD_INT, # Integration method
        rtol = 1.0e-7,
        atol = 1.0e-8,
        args_ptr = args_ptr,
        num_extra = 2
    )

    # The CySolveOutput is not accesible via Python. We need to wrap it first
    cdef WrapCySolverResult pysafe_result = WrapCySolverResult()
    pysafe_result.set_cyresult_pointer(result)

    return pysafe_result

Now we can make a python script that calls our new cythonized wrapper function. Everything below is in pure Python.

# Assume we are working in a Jupyter notebook so we don't need to import `run_cysolver` if it was defined in an earlier cell.
# from my_cython_code import run_cysolver

import numpy as np
initial_conds = np.asarray((20., 20.), dtype=np.float64, order='C')
time_span = (0., 50.)

result = run_cysolver(time_span, initial_conds)

print("Was Integration was successful?", result.success)
print(result.message)
print("Size of solution: ", result.size)
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.plot(result.t, result.y[0], c='r')
ax.plot(result.t, result.y[1], c='b')

# Can also plot the extra output. They are small for this example so scaling them up by 100
ax.plot(result.t, 100*result.y[2], c='green', ls=':')
ax.plot(result.t, 100*result.y[3], c='purple', ls=':')

There is a lot more you can do to interface with CyRK's C++ backend and fully optimize the integrators to your needs. These details will be documented in "Documentation/Advanced CySolver.md".

cysolve_ivp and cysolve_ivp_gil Arguments

cdef shared_ptr[CySolverResult] cysolve_ivp(
    DiffeqFuncType diffeq_ptr,        # Differential equation defined as a cython function
    double* t_span_ptr,               # Pointer to array (size 2) of floats defining the start and stop points for integration
    double* y0_ptr,                   # Pointer to array defining initial y0 conditions.
    unsigned int num_y,               # Size of y0_ptr array.
    unsigned int method = 1,          # Integration method. Current options: 0 == RK23, 1 == RK45, 2 == DOP853
    double rtol = 1.0e-3,             # Relative tolerance used to control integration error.
    double atol = 1.0e-6,             # Absolute tolerance (near 0) used to control integration error.
    void* args_ptr = NULL,            # Pointer to array of additional arguments passed to the diffeq. See "Advanced CySolver.md" for more details.
    unsigned int num_extra = 0,       # Number of extra outputs you want to capture during integration. There is a performance hit if this is used in conjunction with t_eval or dense_output.
    size_t max_num_steps = 0,         # Maximum number of steps allowed. If exceeded then integration will fail. 0 (the default) turns this off.
    size_t max_ram_MB = 2000,         # Maximum amount of system memory the integrator is allowed to use. If this is exceeded then integration will fail.
    bint dense_output = False,        # If True, then dense interpolators will be saved to the solution. This allows a user to call solution as if a function (in time).
    double* t_eval = NULL,            # Pointer to an array of time steps at which to save data. If not provided then all adaptive time steps will be saved. There is a slight performance hit using this feature.
    size_t len_t_eval = 0,            # Size of t_eval.
    PreEvalFunc pre_eval_func = NULL  # Optional additional function that is called within `diffeq_ptr` using current `t` and `y`. See "Advanced CySolver.md" for more details.
    double* rtols_ptr = NULL,         # Overrides rtol if provided. Pointer to array of floats of rtols if you'd like a different rtol for each y.
    double* atols_ptr = NULL,         # Overrides atol if provided. Pointer to array of floats of atols if you'd like a different atol for each y.
    double max_step = MAX_STEP,       # Maximum allowed step size.
    double first_step = 0.0           # Initial step size. If set to 0.0 then CyRK will guess a good step size.
    size_t expected_size = 0,         # Expected size of the solution. There is a slight performance improvement if selecting the the exact or slightly more time steps than the adaptive stepper will require (if you happen to know this ahead of time).
    )

Limitations and Known Issues

• Issue 30: CyRK's cysolve_ivp and pysolve_ivp does not allow for complex-valued dependent variables.

Citing CyRK

It is great to see CyRK used in other software or in scientific studies. We ask that you cite back to CyRK's GitHub website so interested parties can learn about this package. It would also be great to hear about the work being done with CyRK, so get in touch!

Renaud, Joe P. (2022). CyRK - ODE Integrator Implemented in Cython and Numba. Zenodo. https://doi.org/10.5281/zenodo.7093266

In addition to citing CyRK, please consider citing SciPy and its references for the specific Runge-Kutta model that was used in your work. CyRK is largely an adaptation of SciPy's functionality. Find more details here.

Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, CJ Carey, İlhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E.A. Quintero, Charles R Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. (2020) SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17(3), 261-272.

Contribute to CyRK

Please look here for an up-to-date list of contributors to the CyRK package.

CyRK is open-source and is distributed under the Creative Commons Attribution-ShareAlike 4.0 International license. You are welcome to fork this repository and make any edits with attribution back to this project (please see the Citing CyRK section).

• We encourage users to report bugs or feature requests using GitHub Issues.
• If you would like to contribute but don't know where to start, check out the good first issue tag on GitHub.
• Users are welcome to submit pull requests and should feel free to create them before the final code is completed so that feedback and suggestions can be given early on.

cyrk's Issues

Unable to import cyrk_ode

I'm excited to try out the cython compiled ODE solver but having difficulty with what appears to be binary compatibility with numpy.

Error:

ValueError                                Traceback (most recent call last)
Input In [111], in <cell line: 2>()
      1 from numba import njit
----> 2 from CyRK import cyrk_ode

File ~\miniconda3\envs\arc\lib\site-packages\CyRK\__init__.py:9, in <module>
      6 from .nb.nbrk import nbrk_ode
      8 # Import cython solver
----> 9 from CyRK.cy.cyrk import cyrk_ode

File ~\miniconda3\envs\arc\lib\site-packages\CyRK\cy\_cyrk.pyx:1, in init CyRK.cy.cyrk()

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

Relevant installed package versions:

• python=3.9.12
• numba=0.55.1
• numpy=1.21.5
• cython=0.29.32
• cyrk=0.1.1

Everything but cyrk is installed from conda standard channel.

Ubuntu tests failing - Integration hangs

CyRK's tests are currently failing on ubuntu systems. Digging into it, it appears only some of the tests are failing. For example if you run a diffeq using the DOP853 (rk_method=2) these appear to work fine. The test using RK23 or RK45 fails unless you bring the timespan down to (0, 10) or lower. Otherwise the integrator just sits there and hangs. It does not appear to be crashing but the test will eventually timeout and fail.

CyRK's CySolver does not allow for complex-valued dependent variables.

CyRK's CySolver only allows np.float64 y0 and dydt. This is due to limitations between fused types (used in cyrk_ode) and cython cdef classes.

A workaround is to convert your N-dimensional complex system of ODEs into a 2N-dimensional floating point system of ODEs. Pass this onto CySolver, and then convert the output back into complex values.

Min Step Size Bug

During the original implementation of CyRK I misunderstood how the minimum step size check was supposed to work. Scipy defines the minimum step size based on the actual value of time,

import numpy as np
min_step = 10 * np.abs(np.nextafter(t, direction * np.inf) - t)

whereas CyRK originally had a constant step size based off of the floating point error,
defines the minimum step size based on the actual value of time,

import numpy as np
min_step = 10 * np.finfo(np.float64).eps

This latter approach works for when t is close to zero, but is not accurate when t is far from zero since the number of floats between numbers decreases at higher values.

Since min_step is used to set the step_size from becoming too small during integration - having an incorrect value could lead to incorrect or unexpected behavior.

This bug affects both the cyrk_ode and nbrk_ode.

Add in a dependent variable catcher

Right now there is no way (for either the Numba or Cython solver) to capture other variables besides the y-variables and time domain. However, there may be secondary variables that are calculated which the user may want to record without having to rerun functions.

Solution Idea:
If a differential equation is designed as the following where z is a container for independent variables (much like dy),

def diffeq(t, y, dy, z, arg_0, arg_1, ...):
    id_0 = ...
    id_1 = ...
    dy[0] = id_0 ...
    dy[1] = id_1 ....
    z[0] = id_0
    z[1] = id_1

Then the solvers (at least the cython one) should be able to pass in that z argument and track it. The trick will be how to handle the output of the solver to keep it consistent for when the user does not want to track any independent variables.

Import cython RK constants into `nbrk_ode`

As of version 0.5.0, the cyrk_ode function utilizes constants defined in a separate module CyRK.rk. These same constants are used in the nbrk_ode solver (however, the numba version expects np.ndarrays and this file defines things in terms of carrays). It would be good to have nbrk_ode import these Runge-Kutta constants so that they are not defined in two different locations.

Create cython based cdef class integrator

Create a cython-based cdef class integrator to improve performance by allowing users to define cython-based diffeq.

The form could look like:

def BaseClass:
   # ... Integration functionality.
   
   def diffeq(self):
       # ... Placeholder diffeq.

def ProblemSolver(BaseClass):
   
   def diffeq(self):
       # ... Specific diffeq.

Add in type checker for Cython solver

As of v0.2.0, CyRK's cython integrator only allows for complex numbers. Non-complex differential equations have to be passed into the integrator by setting all of the imaginary portions to 0. This works fine but roughly doubles the memory usage and computation steps for various parts of the calculation. To improve performance, the integrator should allow for pure floats if complex calculations are not required.

This could either be done by:

• Have the cython integrator dynamically determine the type of y0 and use that type throughout the calculation
• Have a wrapper that is called before cyrk_ode is called. This wrapper (which could be pure python or numba code) would then check the type of y0 and then pass the arguments the appropriate cyrk_ode.
  • There would need to be two different cyrk_odes: cyrk_ode_complex and cyrk_ode_float.

The first method would be more ideal as the second would require maintaining two very similar functions. It would also introduce a (small?) overhead in that initial python wrapper function call.

Modify `cyrk_ode` to use numpy arrays as storage variables rather than python lists

Currently the cyrk_ode function uses python lists to store values found during integration. However, python lists lead to a big performance hit.

It was found during the creation of the CySolver class that if numpy arrays are allocated at some large size then data can be stored in them instead. If the amount of data exceeds the size of these arrays then they can be expanded with a much lower performance hit than dealing with python lists.

This has already been coded up in the CySolver class. It just needs to be ported over to cyrk_ode and tested. I expect there will be a good amount of performance gained. Good key words to search for in CySolver to see how it was implemented are "expected_size" and "time_domain_array_view".

Allow for `atol` to be an array

For systems of ODEs, each y could have a different absolute tolerance compared to one another. Right now only a constant atol is allowed and is applied equally to all y-variables.

building wheels for CyRK does not package cython's binaries for PyPI distribution

See details (and hopefully a future solution) about this issue here.

Current workaround is to not build or upload wheels during CI. Users will have to recompile the cython code locally as of CyRK v0.2.0.

Could not find the package using either pip or Conda

Hey. I am trying to install CyRK on an HPC environment. I have pip version 21.3.1.
I tried
pip3 install --user CyRK
and got
Collecting CyRK Could not find a version that satisfies the requirement CyRK (from versions: ) No matching distribution found for CyRK

Then I tried to use Conda to install the package on my local machine, and got
`PackagesNotFoundError: The following packages are not available from current channels:

• cyrk`

Can someone please let me know why this package cannot be found? Thanks!

`t_eval` of `CyRK` behaves different to `solve_ivp`

I just figured out that t_eval isn't behaving in the same way as scipy.integrate.solve_ivp:

solve_ivp solves the ode for the points in t_eval

Times at which to store the computed solution, must be sorted and lie within t_span. If None (default), use points selected by the solver.

CyRK interpolates the solution t_eval

Both solvers uses an adaptive time stepping protocol based on the recent error at each step. This results in a final non-uniform time domain of variable size. If the user would like the results at specific time steps then they can provide a np.ndarray array at the desired steps via t_eval. The solver will then interpolate the results to fit this array.

In my kind of special case CyRK is actually slower than solve_ivp.

Numba solver is worse than scipy at large integration times

As can be seen in the README.md figure, when the time span is large (in that example, when the end step is >~ 1e4) the numba solver becomes equal to if not worse than the scipy solver. This also occurs when the relative integration tolerance is very small. So it appears to get worse with longer integration times in general.

This is not an issue for the cython solver.

CyRK's cython integrator is setup to only work with complex numbers

As of v0.2.0, CyRK's cython integrator only allows for complex numbers. Non-complex differential equations have to be passed into the integrator by setting all of the imaginary portions to 0. This works fine but roughly doubles the memory usage and computation steps for various parts of the calculation. To improve performance, the integrator should allow for pure floats if complex calculations are not required.

This could either be done by:

• Have the cython integrator dynamically determine the type of y0 and use that type throughout the calculation
• Have a wrapper that is called before cyrk_ode is called. This wrapper (which could be pure python or numba code) would then check the type of y0 and then pass the arguments the appropriate cyrk_ode.
  • There would need to be two different cyrk_odes: cyrk_ode_complex and cyrk_ode_float.

The first method would be more ideal as the second would require maintaining two very similar functions. It would also introduce a (small?) overhead in that initial python wrapper function call.

Some Backwards Integrations Fail when using the C++ Backend.

See the pytest.skip in the accuracy test in "test_a_pysolve_ivp.py" test file. Most (all?) of these are failing at the np.allclose check. I have manually checked some other cases with backward integration and it worked fine. Not sure what the trigger is.

This is likely to do with the backend's messy indexing when it comes to forward and backward integration. Code could really use a second pair of eyes to help clean it up and squash those bugs.

Will need to check that any index fixes still work for backward integration with t_eval and dense_output are set.

CySolver and cyrk_ode optional arguments and extra output must be float64s numbers

For performance and simplicity reasons, it was decided that the optional arguments (args) passed to both the CySolver and cyrk_ode solvers must be given as floating point numbers (no complex values, strings, booleans, etc). It would be nice to offer the ability to pass other types of values via the args parameter.

For the same reasons, the optional capture of additional outputs for both solvers is also limited to certain types (floats for CySolver; floats or complex for cyrk_ode).

Allow for intermediate results to be stored during integration

In the current implementation of CyRK the only variables that are stored during integration are the dependent-y variables. For example,

# y = dy(t, y0)
t, y, success, message = nbrk_ode(dy, time_span, y0)

However, it is often useful to a user for additional parameters to be stored. For instance, a and b below.

def dy(t, y):
    dy_ = np.empty(2, dtype=np.complex128)
    a = g(t, y)  # Some function of time and y
    b = h(t, y)  # Some function of time and y
    dy_[0] = a * y[0]
    dy_[1] = b * y[1]
    return dy_

The values of a and b are lost after the integration is completed. The user would then need to make separate calls to the g and h function to retrieve their values. This is duplicating calculations and forcing the user to write additional code.

Propose CyRK stores these intermediate results during integration and returns them to the user.

One way this could be done is by having the user define the differential equation like,

def dy(t, y):
    dy_ = np.empty(2, dtype=np.complex128)
    a = g(t, y)  # Some function of time and y
    b = h(t, y)  # Some function of time and y
    dy_[0] = a * y[0]
    dy_[1] = b * y[1]
    return dy_, (a, b)

Then both cyrk_ode and nbrk_ode would store the result (probably best as a np.array) and return the result to the user.

To-Do

• Implement intermediate saving for cyrk_ode
  • Implement basic saving
  • Implement interpolation for t_eval
  • Create tests
• Implement intermediate saving for nbrk_ode
  • Implement basic saving
  • Implement interpolation for t_eval
  • Create tests

PyPI macos wheels changed architecture with 0.8.7

The macos workflow for building wheels is defaulting to whatever the macos-latest runner architecture is (which changed recently from x64-86 to arm64).

I think you could easily distribute both architectures by running the macos workflow on a macos-latest and a macos-13 runner (they have arm64 and x64 architectures natively).

Add unit tests

Add unit tests and add link to GitHub workflow

cache_njit flag causing long run times on Windows w/ Python 3.11

Had to set cache_njit=False in the test suite for the helper.py conversion functions (see this commit) due to very long run times for Windows when running Python 3.11. This behavior was new with the development of CyRK v0.8.3 and PR #39.

Very hard to debug the issue as it was not occurring on my local machine (also Win 10 & Py 3.11). Perhaps is is a different numba version or something like that. Did not look into the specific versions of dependencies. #a675494

Had to turn off `fastmath` for `nbrk_ode`

Prior to v0.5.0, numba's fastmath variable was turned on for nbrk_ode. With the fix to issue #20 in PR #23 we noticed that the standard set of tests would fail on MacOS and Ubuntu systems. It was very random, even rerunning a test without a change in code could cause different results.
Some consistent findings:

• Tests never failed on Windows.
• More tests would fail on MacOS vs. Ubuntu.
• Setting fastmath=False (in CyRK.nb.nbrk at the njit decorator) allowed tests to pass.

PR #23 changed the way minimum step size was calculated. Prior to this change it was set to:

10 * np.finfo(np.float64).eps

now it is set, correctly, to

10. * abs(np.nextafter(t, direction * np.inf) - t)

For small t this value is quite small (5e-323 vs. a EPS of ~1e-16). I believe there is something not playing well between this small value floating around (pun) while fastmath is on. At least on some operating systems.

It is more important that the step size is set correctly so, as of 0.5.0, fastmath is turned off for nbrk_ode there is a slight performance hit with this change.