python-for-hpc / numbawithopenmp

Numba

A Just-In-Time Compiler for Numerical Functions in Python

Numba is an open source, NumPy-aware optimizing compiler for Python sponsored by Anaconda, Inc. It uses the LLVM compiler project to generate machine code from Python syntax.

Numba can compile a large subset of numerically-focused Python, including many NumPy functions. Additionally, Numba has support for automatic parallelization of loops, generation of GPU-accelerated code, and creation of ufuncs and C callbacks.

For more information about Numba, see the Numba homepage: https://numba.pydata.org

Supported Platforms

• Operating systems and CPUs:
  • Linux: x86 (32-bit), x86_64, ppc64le (POWER8 and 9), ARMv7 (32-bit), ARMv8 (64-bit).
  • Windows: x86, x86_64.
  • macOS: x86_64, (M1/Arm64, unofficial support only).
  • *BSD: (unofficial support only).
• (Optional) Accelerators and GPUs:
  • NVIDIA GPUs (Kepler architecture or later) via CUDA driver on Linux, Windows, macOS (< 10.14).

Dependencies

• Python versions: 3.7-3.9
• llvmlite 0.38.*
• NumPy >=1.17 (can build with 1.11 for ABI compatibility).

Optionally:

• SciPy >=1.0.0 (for numpy.linalg support).

Installing

The easiest way to install Numba and get updates is by using the Anaconda Distribution: https://www.anaconda.com/download

$ conda install numba

For more options, see the Installation Guide: https://numba.readthedocs.io/en/stable/user/installing.html

Documentation

https://numba.readthedocs.io/en/stable/index.html

Contact

Numba has a discourse forum for discussions:

• https://numba.discourse.group

Continuous Integration

numbawithopenmp's Issues

Target region with print execution on device(0) host fails with undefined reference for numba_gil_ensure

Reporting a bug

• I have tried using the latest released version of Numba (most recent is
  visible in the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
• I have included a self contained code sample to reproduce the problem.
  i.e. it's possible to run as 'python bug.py'.

Running An OpenMP target region that uses print and executes on device(0), that this using the host as a target, fails with undefined reference errors on the compled shared library, missing the symbol numba_gil_ensure

Slice setitem requires NRT in GPU target regions.

local_forcefield_rhs[i, :] = forcefield_rhe[i, :]

The above line will try to generate two code copies....one where it is proven there is no overlap and another where there could be overlap. If there could be overlap then an intermediate array must be created which requires NRT at the moment. So, even if your particular usage doesn't need to allocate it, the analysis to determine that isn't done at compile time but handled at runtime and so will still fail.

Lowering parallel in a target region in LLVM fails due to missing data values

Reporting a bug

• I have tried using the latest released version of Numba (most recent is
  visible in the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
• I have included a self contained code sample to reproduce the problem.
  i.e. it's possible to run as 'python bug.py'.

LLVM lowering fails with instruction domination errors when the parallel region is outlined. I'm attaching the error log and the original LLVM IR before lowering. Check values %.42 and %.67. They are allocas in the target region, used in the parallel region but do not appear in the data-sharing qualifiers, hence the error. I'm attaching files to help debug the issue: the python code, the error log, and the device image IR before lowering.

hello-target-parallel.py.txt
error_log.txt
device0f1.ll.txt

test_target_map_from_array fails

Output

test_target_map_from_array (numba.tests.test_openmp.TestOpenmpTarget) ... Bad usedef tag: QUAL.OMP.MAP.FROM.STRUCT a*data

Looks like a frontend parsing error

Calling runtime calls in GPU target device region fails

Reporting a bug

• I have tried using the latest released version of Numba (most recent is
  visible in the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
• I have included a self contained code sample to reproduce the problem.
  i.e. it's possible to run as 'python bug.py'.

We import OpenMP runtime calls using CFFI. This works for host code where the OpenMP runtime library is dynamically loaded. However, on GPU device the OpenMP runtime library is a statically linked bitcode. Using the same import mechanism fails with:

.egg/numba/core/base.py", line 1114, in add_dynamic_addr
    assert self.allow_dynamic_globals, "dyn globals disabled in this target"
AssertionError: Failed in nopython mode pipeline (step: Handle with contexts)

If we don't import, it fails with a typing error:

.egg/numba/core/dispatcher.py", line 423, in error_rewrite
    raise e.with_traceback(None)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: Handle with contexts)

I'm attaching examples for both cases:
target-runtime-call.py.txt
target-runtime-call-noimport.py.txt

Unary negative in GPU target regions.

a = 7
b = -a

The above won't work in GPU target regions.

Lazy load the OpenMP runtime library

Lazy loading the OpenMP runtime library may solve problem with testing

Fix openmp target tests for GPU testing

• Make device(0) testing optional/configurable through env var
• Fix target context definitions to always include device clause specifications
• Simplify tests

Create static libraries for nrt, helpelib etc. with for target offloading on host

fluidSim fails

Fluidsim-pyomp from hecbench/pyomp fails with the following:

Libomptarget message: explicit extension not allowed: host address specified is 0x0000000100e2aca0 (64 bytes), but device allocation maps to host at 0x0000000100e2aca8 (64 bytes)
Libomptarget error: Call to getTargetPointer returned null pointer (device failure or illegal mapping).
Libomptarget error: Call to targetDataBegin failed, abort target.
Libomptarget error: Failed to process data before launching the kernel.
Libomptarget error: Consult https://openmp.llvm.org/design/Runtimes.html for debugging options.
Libomptarget error: Source location information not present. Compile with -g or -gline-tables-only.
Libomptarget fatal error 1: failure of target construct while offloading is mandatory

minibude fails

Minibude-pyomp from hec-bench/pyomp fails as follows:

Libomptarget message: explicit extension not allowed: host address specified is 0x0000200084fe0014 (262144 bytes), but device allocation maps to host at 0x0000200084fe0010 (262144 bytes)
Libomptarget error: Call to getTargetPointer returned null pointer (device failure or illegal mapping).
Libomptarget error: Consult https://openmp.llvm.org/design/Runtimes.html for debugging options.
Libomptarget error: Source location information not present. Compile with -g or -gline-tables-only.
Libomptarget fatal error 1: failure of target construct while offloading is mandatory

test_target_teams_distribute_nested fails

Output

======================================================================
FAIL: test_target_teams_distribute_nested (numba.tests.test_openmp.TestOpenmpTarget) (device=1)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/p/vast1/ggeorgak/projects/compilers/numba/NumbaWithOpenmp/numba/tests/test_openmp.py", line 2977, in func_with_subtest
    func(self, device)
  File "/p/vast1/ggeorgak/projects/compilers/numba/NumbaWithOpenmp/numba/tests/test_openmp.py", line 2795, in target_teams_distribute_nest
ed
    np.testing.assert_equal(len(u), n)
  File "/p/vast1/ggeorgak/projects/compilers/numba/lassen/venv-3.8/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 425,
 in assert_equal
    raise AssertionError(msg)
AssertionError: 
Items are not equal:
 ACTUAL: 1
 DESIRED: 100

Looking at LLVM IR it looks like QUAL.OMP.NUM_TEAMS, QUAL.OMP.THREAD_LIMIT are set both to 1, so sequentialize execution

  %.105 = call token @llvm.directive.region.entry() [ "DIR.OMP.TARGET"(i32 0), "QUAL.OMP.NUM_TEAMS"(i32 1), "QUAL.OMP.THREAD_LIMIT"(i32 1),  ...