python-for-hpc / ramba Goto Github PK

It should be possible to convert function/lambda-based initialization and map skeletons to use deferred operations. All are per-element operations. We will need to add mechanisms to allow custom JIT functions to be called from within a deferred op, and mechanisms for providing global index arguments (tuple or otherwise) for initialization functions.

Currently, data communication in deferred ops sends multiple copies of the same data if overlapping ranges are referred to by multiple slices. For the default Parallel Research Kernels stencil (a radius=2 "star" or "plus" pattern), this results in 1.5x times the minimum data volume to be transmitted. For a general 5x5 stencil, this would result in 7.5x the necessary data volume.

The axis option to sum can be a tuple of ints, i.e., can specify multiple axes. Currently. ramba only supports signle axis (axis option as an int).

On line 5319 copy is a boolean arg, not callable:

Unclear to me what the right fix is, libcopy.deepcopy(self) seems to work.

In particular: module parser is not available in Python 3.10+
Ray 2.0 marks placement_group parameter as deprecated

At very least, README should indicate how to install and run in simple cases.

Broadcast, broadcast_to do not create DAG entries. Operations with broadcasted arrays cause errors.

For doing node-aware partitioning, we want the Ramba workers/RemoteStates to be grouped so that all the workers on a given node are laid out consecutively. In MPI mode, there are some configurations where the ranks are not natively laid out consecutively on the nodes. Thus, we need to have some kind of a mapping between MPI ranks and workers so that from Ramba's perspective the invariant is maintained that chunks of the worker array of size num_workers/num_nodes are laid out consecutively.

Currently, system does not detect when different ranges of same arrays are used for reads and writes when fusing deferred loops. This can have correctness issues when same-sized but different ranges are written / read in consecutive operations. Work-around for now is to manually insert a deferred_op.do_ops() to prevent fusion of such operations.

example:
a = ramba.array(7)
a.asarray()

returns garbage value. However, after sync or a[()], it works. This is when using DAG.

This seems to be triggered by use of numba functions. It may be related to import issues. No problem in Ray or MPI in SPMD mode.