pymumps / pymumps

PyMUMPS: A parallel sparse direct solver

PyMUMPS: A parallel sparse direct solver

Requirements

• MUMPS
• mpi4py

Installation

PyMUMPS can be installed from PyPI using pip:

pip install pymumps

Custom build flags, e.g. to specify the MUMPS installation location, can be specified using --global-option:

pip install pymumps --global-option="build_ext" \
    --global-option="-I$MUMPS_PREFIX/include" \
    --global-option="-L$MUMPS_PREFIX/lib" \

Use python setup.py build_ext --help to get a list of all allowed options.

There is also conda recipe:

conda install -c conda-forge pymumps

Examples

Centralized input & output. The sparse matrix and right hand side are input only on the rank 0 process. The system is solved using all available processes and the result is available on the rank 0 process.

from mumps import DMumpsContext
ctx = DMumpsContext()
if ctx.myid == 0:
    ctx.set_centralized_sparse(A)
    x = b.copy()
    ctx.set_rhs(x) # Modified in place
ctx.run(job=6) # Analysis + Factorization + Solve
ctx.destroy() # Cleanup

Re-use symbolic or numeric factorizations.

from mumps import DMumpsContext
ctx = DMumpsContext()
if ctx.myid == 0:
    ctx.set_centralized_assembled_rows_cols(A.row+1, A.col+1) # 1-based
ctx.run(job=1) # Analysis

if ctx.myid == 0:
    ctx.set_centralized_assembled_values(A.data)
ctx.run(job=2) # Factorization

if ctx.myid == 0:
    x = b1.copy()
    ctx.set_rhs(x)
ctx.run(job=3) # Solve

# Reuse factorizations by running `job=3` with new right hand sides
# or analyses by supplying new values and running `job=2` to repeat
# the factorization process.

pymumps's Issues

Add support for complex arithmetic

First, I tried to add

   elif dtype == 'complex':
        return arr.__array_interface__['data'][0]

in

pymumps/mumps/_dmumps.pyx

Line 338 in db8abc9

else:

and then

import numpy as np
import mumps
import scipy.sparse.linalg
import scipy.sparse

A and b are real

row = np.array([0, 0, 1, 2, 2], dtype="i")
col = np.array([0, 1, 2, 0, 2], dtype="i")
val = np.array([1, 2, 3, 4, 5], dtype="d")

A = scipy.sparse.csr_matrix((val, (row, col)))
b = np.array([1,1,1], dtype='d')
xsol = scipy.sparse.linalg.spsolve(A,b) 

ctx = mumps.DMumpsContext(sym=0, par=1)
if ctx.myid == 0:
    ctx.set_centralized_sparse(A)
    x = b.copy()
    ctx.set_rhs(x)

ctx.set_silent()
ctx.run(job=6) 

if ctx.myid == 0:
    print("PyMumps solution is %s." % (x,))
    print("Scipy solution is %s." % (xsol,))
ctx.destroy()
#Scipy solution is [-0.16666667  0.58333333  0.33333333].
#PyMumps solution is [-0.16666667  0.58333333  0.33333333].

A is real and b is complex

row = np.array([0, 0, 1, 2, 2], dtype="i")
col = np.array([0, 1, 2, 0, 2], dtype="i")
val = np.array([1, 2, 3, 4, 5], dtype="d")
A = scipy.sparse.csr_matrix((val, (row, col)))
b = np.array([1,1,1], dtype='complex')
xsol = scipy.sparse.linalg.spsolve(A,b) 

ctx = mumps.DMumpsContext(sym=0, par=1)
if ctx.myid == 0:
    ctx.set_centralized_sparse(A)
    x = b.copy()
    ctx.set_rhs(x)

ctx.set_silent()
ctx.run(job=6) 

if ctx.myid == 0:
    print("PyMumps solution is %s." % (x,))
    print("Scipy solution is %s." % (xsol,))

ctx.destroy()
#Scipy solution is [-0.16666667+0.j  0.58333333+0.j  0.33333333+0.j].
#PyMumps solution is [0.25+0.375j 0.  +0.j    1.  +0.j   ].

A is complex and b is real

row = np.array([0, 0, 1, 2, 2], dtype="i")
col = np.array([0, 1, 2, 0, 2], dtype="i")
val = np.array([1, 2, 3, 4, 5], dtype="complex")
A = scipy.sparse.csr_matrix((val, (row, col)))
b = np.array([1,1,1], dtype='d')
xsol = scipy.sparse.linalg.spsolve(A,b) 

ctx = mumps.DMumpsContext(sym=0, par=1)
if ctx.myid == 0:
    ctx.set_centralized_sparse(A)
    x = b.copy()
    ctx.set_rhs(x)

ctx.set_silent()
ctx.run(job=6) 

if ctx.myid == 0:
    print("PyMumps solution is %s." % (x,))
    print("Scipy solution is %s." % (xsol,))

ctx.destroy()
#RuntimeError: MUMPS error: -6

Why pymumps is faster than mumps on a small scale

Hello developers, I am using pymumps to solve my problems, and I also use the C++interface of mumps to write programs. I found a phenomenon that when the unknown quantity exceeds 400000, the speed of pymumps is about the same as that of mumps, but when the unknown quantity is less than 400000, the speed of pymumps is twice that of mumps! Why is this? Did you make any optimization?

Set up CI

We should be running tests on all supported plarforms.

undefined symbol error

I am trying to build PyMumps version 0.3.2 for python 3.7.7 on an RHEL8 system.
(Indeed, I am trying to write a Spack (https://spack.io) recipe to do so). I have Mumps 5.2.0 installed on the system.

I am able to build PyMumps, but when I try to run a simple python test code (to solve Ax=b for x given A,b), I receive an undefined symbol error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/software/spack-software/2020.05.14/linux-rhel8-ivybridge/gcc-8.4.0/py-pymumps-0.3.2-dieeka4jysowuodru5ugkrgendaughju/lib/python3.7/site-packages/mumps/__init__.py", line 2, in <module>
    import mumps._dmumps
ImportError: /software/spack-software/2020.05.14/linux-rhel8-ivybridge/gcc-8.4.0/mumps-5.2.0-2vqesppnxfqdnx5xc3s262onaa6oyqfg/lib/libdmumps.so: undefined symbol: __mumps_ooc_common_MOD_low_level_strat_io

The nm command does indeed show that libdmumps.so has an undefined variable reference to __mumps_ooc_common_MOD_low_level_strat_io, which is defined as a global (external) BSS symbol in libmumps_common.so. (Both libraries are part of the installed Mumps package). As a test, I tried setting LD_PRELOAD to preload the libmumps_common.so, and with that the python test works as expected. But that is not a real solution.

I am able to construct and successfully run a similar Fortran test code for Mumps (using the same Mumps installation)(again solving Ax=b for x), and this works as expected (without needing to mess with LD_PRELOAD). Based on this, it looks to me like the issue is with PyMumps and not Mumps.

I have tried the PyMumps build again, after making a small modification to setup.py:

diff -Naur spack-src/setup.py spack-src.patched/setup.py
--- spack-src/setup.py  2018-11-05 16:57:21.000000000 -0500
+++ spack-src.patched/setup.py  2020-10-26 11:23:41.363767947 -0400
@@ -35,7 +35,7 @@
         Extension(
             'mumps._dmumps',
             sources=['mumps/_dmumps.pyx'],
-            libraries=['dmumps'],
+            libraries=['dmumps', 'mumps_common'],
         ),
     ],
     install_requires=['mpi4py'],

I.e: adding a dependency on the 'mumps_common' library.

When I do this, the Python module again builds, and when I run the same test code it behaves as expected --- the python code runs w/out the undefined symbol load error, and produces the expected result.

I confess I am not very familiar with Mumps/PyMumps, but it appears to me that the setup.py should be adding this additional library dependency.

Non-deterministic behavior with multiple solves

I ran into an issue when solving multiple linear systems sequentially with a single instance of DMumpsContext. I do not know if this is an issue with Mumps or PyMumps, so I apologize if this is not an issue with PyMumps.

I was able to distill my issue down to a relatively small script:

from scipy.sparse import coo_matrix, tril
import mumps

mats = list()

row = [0, 0, 0, 0, 1, 2, 3, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 5, 6, 7, 5, 8, 6, 8, 7, 8, 7]
col = [0, 1, 2, 3, 0, 0, 0, 5, 6, 7, 5, 8, 6, 8, 7, 8, 7, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4]
data = [2.e+00, 1.e-16, 1.e-16, 1.e-16, 1.e-16, 1.e-16, 1.e-16, 0.e+00, 0.e+00,
        0.e+00, 0.e+00, 1.e+00, 0.e+00, 1.e+00, 0.e+00, 1.e+00, -1.e+00, 0.e+00,
        0.e+00, 0.e+00, 0.e+00, 1.e+00, 0.e+00, 1.e+00, 0.e+00, 1.e+00, -1.e+00]
mat = tril(coo_matrix((data, (row, col)), shape=(9, 9)))
mats.append(mat)

row = [0, 0, 0, 0, 1, 2, 3, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 5, 6, 7, 5, 8, 6, 8, 7, 8, 7]
col = [0, 1, 2, 3, 0, 0, 0, 5, 6, 7, 5, 8, 6, 8, 7, 8, 7, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4]
data = [2.e+00, 1.e-16, 1.e-16, 1.e-16, 1.e-16, 1.e-16, 1.e-16, 0.e+00, 0.e+00,
        0.e+00, 0.e+00, 1.e+00, 0.e+00, 1.e+00, 0.e+00, 1.e+00, -1.e+00, 0.e+00,
        0.e+00, 0.e+00, 0.e+00, 1.e+00, 0.e+00, 1.e+00, 0.e+00, 1.e+00, -1.e+00]
mat = tril(coo_matrix((data, (row, col)), shape=(9, 9)))
mats.append(mat)

row = [0, 0, 0, 0, 1, 2, 3, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 5, 6, 7, 5, 8, 6, 8, 7, 8, 7, 5, 6, 7, 8]
col =[0, 1, 2, 3, 0, 0, 0, 5, 6, 7, 5, 8, 6, 8, 7, 8, 7, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 5, 6, 7, 8]
data = [2.00000000e+00, 1.00000000e-16, 1.00000000e-16, 1.00000000e-16,
        1.00000000e-16, 1.00000000e-16, 1.00000000e-16, 0.00000000e+00,
        0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.00000000e+00,
        0.00000000e+00, 1.00000000e+00, 0.00000000e+00, 1.00000000e+00,
        -1.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
        0.00000000e+00, 1.00000000e+00, 0.00000000e+00, 1.00000000e+00,
        0.00000000e+00, 1.00000000e+00, -1.00000000e+00, -5.62341325e-09,
        -5.62341325e-09, -5.62341325e-09, -5.62341325e-09]
mat = tril(coo_matrix((data, (row, col)), shape=(9, 9)))
mats.append(mat)

row = [0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 5, 6, 7, 5, 8, 6, 8, 7, 8, 7, 5, 6, 7, 8]
col = [0, 1, 2, 3, 0, 1, 0, 2, 0, 3, 4, 5, 6, 7, 5, 8, 6, 8, 7, 8, 7, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 5, 6, 7, 8]
data = [2.00010000e+00, 1.00000000e-16, 1.00000000e-16, 1.00000000e-16,
        1.00000000e-16, 1.00000000e-04, 1.00000000e-16, 1.00000000e-04,
        1.00000000e-16, 1.00000000e-04, 1.00000000e-04, 0.00000000e+00,
        0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.00000000e+00,
        0.00000000e+00, 1.00000000e+00, 0.00000000e+00, 1.00000000e+00,
        -1.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
        0.00000000e+00, 1.00000000e+00, 0.00000000e+00, 1.00000000e+00,
        0.00000000e+00, 1.00000000e+00, -1.00000000e+00, -5.62341325e-09,
        -5.62341325e-09, -5.62341325e-09, -5.62341325e-09]
mat = tril(coo_matrix((data, (row, col)), shape=(9, 9)))
mats.append(mat)

solver = mumps.DMumpsContext(sym=2, par=1, comm=None)
solver.set_silent()
solver.set_icntl(13, 1)
solver.set_icntl(24, 0)
final_status = None
for i, mat in enumerate(mats):
    solver.set_shape(9)
    solver.set_centralized_assembled_rows_cols(mat.row + 1, mat.col + 1)
    try:
        solver.run(job=1)
    except RuntimeError:
        pass
    sym_status = solver.id.infog[0]
    if sym_status == 0:
        solver.set_centralized_assembled_values(mat.data)
        try:
            solver.run(job=2)
        except RuntimeError:
            pass
        num_status = solver.id.infog[0]
    else:
        num_status = None
    final_status = num_status
solver.destroy()
print(final_status)

If I run the above script multiple times, I get different results. Sometimes the status for the last factorization is 0 and sometimes it is 10. Sometimes the analysis phase is not successful for the last matrix. Note that if I include solver.run(job=-1) inside the for loop, I get deterministic behavior. Additionally, if I only factorize the last matrix (and exclude the first three), the results are deterministic and the factorization status is always 0.

I could simply run job=-1 every time I run an analysis, but then I lose any options set in ICNTL.

Any guidance is appreciated.

Transplant Mumpy code

Take the code from here: https://gitlab.kwant-project.org/kwant/mumpy and transplant it into this repo.

Most of the files are orthogonal, but we will need to take extra care with:

• setup.py: mumpy has more advanced setup.py machinery that can be taken over. We should remember to include the mpi4py dependency.

• versioning: existing pymumps hardcodes its version string in the setup.py

conda recipe

@sdrave I would like to use pymumps within an optimization project that we have with Pyomo (Sandia National Labs). To facilitate the installation of pymumps I created a conda-forge recipe that makes it easier to installl pymumps. to do that I have forked your repo, added a couple of things to setup.py and written a meta.yml file for the recipe. I would like to get your permission to do this before the pull request to conda-forge is merged into feedstock. I am currently listing myself as the maintainer but if you are interested in the recipe too I could list you as a maintainer as well. Also the source is obtained from my fork but if it is of you interest we could pull that from your repo instead (we will need to merge my changes).

Thanks

Install of 0.3.2 fails due to missing cython

Creating a fresh virtualenv and trying to install pymumps fails.

python -m venv /tmp/mumps
. /tmp/mumps/bin/activate
python -m pip install pymumps

Collecting pymumps
  Using cached PyMUMPS-0.3.2.tar.gz (7.9 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [19 lines of output]
      Traceback (most recent call last):
        File "/tmp/pip-install-adykwjrf/pymumps_a229b08a713041b2b0a464a7e87330c6/setup.py", line 6, in <module>
          import Cython
      ModuleNotFoundError: No module named 'Cython'
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-adykwjrf/pymumps_a229b08a713041b2b0a464a7e87330c6/setup.py", line 8, in <module>
          raise ImportError('''
      ImportError:
      Cython is required for building this package. Please install using
      
          pip install cython
      
      or upgrade to a recent PIP release.
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Add PyMumps code

We need to get the latest state from https://github.com/sdrave/pymumps, given that we're not making this repo a fork of that one.

Install with statically linked libraries

I'm installing with MUMPS built from source, which by default builds static libraries. I can link these libraries with Pymumps if I compile them with -fPIC, but there is currently no way to specify libraries to link statically, which I believe should be done via the extra_objects argument in Extension. I think this could also be accomplished by prepending the location of my static MUMPS libraries to PYMUMPS_SETUP_LIBRARY_DIRS, but this would not be reliable if I had static and dynamic libraries in the same location.

Installing pymumps with mumps from conda-forge

Hi,

I tried installing the package and even though the installation is successful I get an error when running the example. I install the dependencies with conda-forge (MUMPS5.1.2). The error seems to be happening when linking with the share library:

dlopen(_dmumps.cpython-36m-darwin.so, 2)
Symbol not found: ___mumps_fac_descband_data_m_MOD_inode_waited_for

Is there an easy fix? Does pymumps work with other all version

Thanks

Set up documentation

Just API documentation for the moment, but this still involves running sphinx.

We can use readthedocs, as it's just a short configuration file to set up and they have Github integration.

Accessing mumps par%INFO mumps par%RINFO

@sdrave is there a way to access elements from INFO and RINFO from pymumps? Thanks