reference-scalapack / scalapack Goto Github PK

I have backtracked the exact commits for the release tars on ScaLAPACK homepage.

It would be nice, for clarity and consistency, to have the tags listed on github as well.

I only took the two most recent ones.

v2.0.2 tar.gz was created from hash: 68165d7
v2.0.1 tar.gz was created from hash: d979994

Sadly, I cannot create a PR since there are no changes, only tags for commits.
All I did was:

git tag -a -m "Release 2.0.1" v2.0.1 d979994d53c204437317e3e9fa9ba69b082d5b4e
git tag -a -m "Release 2.0.2" v2.0.2 68165d785dad5f8bd2b87abd091da6cfb86f08d4
git push --tags

Thanks!

scalapack 2.1.0 fails to build when implicit declaration of functions is an error. (This is the default behavior of the clang included with Apple's Xcode 12 or later and can also be achieved on other compilers by adding -Werror=implicit-function-declaration to CFLAGS.) grepping the build log for these errors, they are:

BLACS/SRC/igsum2d_.c:153:7: error: implicit declaration of function 'BI_imvcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/igsum2d_.c:166:6: error: implicit declaration of function 'BI_ivmcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/sgsum2d_.c:154:7: error: implicit declaration of function 'BI_smvcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/sgsum2d_.c:167:6: error: implicit declaration of function 'BI_svmcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/dgsum2d_.c:154:7: error: implicit declaration of function 'BI_dmvcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/dgsum2d_.c:167:6: error: implicit declaration of function 'BI_dvmcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/cgsum2d_.c:154:7: error: implicit declaration of function 'BI_smvcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/cgsum2d_.c:169:6: error: implicit declaration of function 'BI_svmcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/zgsum2d_.c:156:7: error: implicit declaration of function 'BI_dmvcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/zgsum2d_.c:171:6: error: implicit declaration of function 'BI_dvmcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/igamx2d_.c:199:7: error: implicit declaration of function 'BI_imvcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/igamx2d_.c:278:6: error: implicit declaration of function 'BI_ivmcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/igamx2d_.c:280:16: error: implicit declaration of function 'BI_TransDist' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/sgamx2d_.c:202:7: error: implicit declaration of function 'BI_smvcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/sgamx2d_.c:281:6: error: implicit declaration of function 'BI_svmcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/sgamx2d_.c:283:16: error: implicit declaration of function 'BI_TransDist' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/dgamx2d_.c:202:7: error: implicit declaration of function 'BI_dmvcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/dgamx2d_.c:281:6: error: implicit declaration of function 'BI_dvmcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BLACS/SRC/dgamx2d_.c:283:16: error: implicit declaration of function 'BI_TransDist' is invalid in C99 [-Werror,-Wimplicit-function-declaration]

This was reported to MacPorts here: https://trac.macports.org/ticket/62567

@ scalapack-2.2.0
I am on ARM64 now the first problem is that using mpich configure step passes but make step fails at 68%. My command cmake .. -DCMAKE_INSTALL_PREFIX=$PREFIX/local -DCMAKE_Fortran_COMPILER=gfortran-11 -DCMAKE_Fortran_FLAGS="-fallow-argument-mismatch" -DCMAKE_C_COMPILER=clang

uk_config.log.txt
uk_make.log.txt

Additionally cmake fails to identify openmpi (after uninstalling mpich). I have tried this stuff but to no avail! Is it incompatible with openmpi ?

Punmlq, punmrq functions take matrix A with K householder vectors, which placed in matrix horizontally. So tau should have dimension LOCr(IA+K-1) instead of LOCc(IA+K-1)

GCC10 is the default on Ubuntu 22.04.

During the build step there are many compilation errors of the form:

scalapack/SRC/pstrord.f:3264:54:

  490 |      $         CALL IGAMX2D( ICTXT, 'All', TOP, 1, 1, MMAX, 1, -1,
      |                                                      2
......
 3264 |                CALL IGAMX2D( ICTXT, 'All', TOP, 1, 1, INFO, 1, -1,
      |                                                      1
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (rank-1 and scalar)

See this link for the explanation and the fix.
https://gcc.gnu.org/gcc-10/porting_to.html

Until the code is fixed the workaround is to configure cmake with the following flag: -DCMAKE_Fortran_FLAGS="-fallow-argument-mismatch" which converts the error into a warning.

e.g.

mkdir -p build && cd build && cmake -DCMAKE_Fortran_FLAGS="-fallow-argument-mismatch" .. && make scalapack

Currently there seems to be no way to avoid building all tests when building with cmake.
This takes considerable time, and allowing users to only compile the actual library would be easy by including an option like

option(SCALAPACK_BUILD_TESTS "Build all tests of the ScaLAPACK library" ON)
and later writing

if (${SCALAPACK_BUILD_TESTS})
  add_subdirectory(TESTING)
endif()

If this is acceptable, I can make a small PR for this.

I ran into an issue while executing pdgetrf on a submatrix A(2:M, 2:M). Therefore I used the following call:

CALL PDGETRF(M-1, N-1, MATA, 2, 2, DESCA, IPIV, INFO)

which caused the following error:

{    0,    0}:  On entry to PDGETRF parameter number    4 had an illegal value
{    1,    1}:  On entry to PDGETRF parameter number    4 had an illegal value
....

but in the documentation there is no word about the contraint, that IA and JA must be a multiple of the block size.

More general, why is this restriction? I currently searching in P?GETRF, P?GETF2, .... why this is required? If there is no reason, then the checks should be removed.

From #19 (comment):

Furthermore, the MPI C interface is newly assumed to expect the type MpiInt, which is a macro again, by default evaluates to int, but can be (in hypothetical and very unlikely circumstances) overridden by the user as well. However, I know of no MPI C library providing other than LP64 interface, so this freedom will be most likely never used.

Sorry, but this is just wrong. The MPI standard uses int and no other type is conforming. There is no scenario involving a valid implementation of MPI where MpiInt should ever be used.

The MPI standard has MPI_Fint specifically to ensure compatibility with the MPI Fortran library, and only this type should be used for this purpose.

I don't have strong feelings about Int but MpiInt should be removed. Its existence is dangerous and unnecessary.

There are some building blocks (like reduction to Hessenberg and Schur) making it possible to solve this problem in ScaLAPACK. But at the same time, there is no driver.

It would be a feature improving ScaLAPACK user experience.

This is a gap in ScaLAPACK API: it makes problematic to compute the eigenvalues of a real unsymmetrical matrix.

It seems http://www.netlib.org/scalapack/ does not advertise the new repository place?

I think many communities using ScaLAPACK have really wanted a good, common place to put/report bugs/request.

This would be very welcome! :)

PS. Great job on moving it here!!! Much appreciated!

OS: FreeBSD 12.1
Version 2.0.2

Error: /usr/local/lib/libscalapack.so.2 is linked to /usr/local/lib/libmpi.so.12 from net/mpich but it is not declared as a dependency
Warning: you need LIB_DEPENDS+=libmpi.so:net/mpich

Recently (#56), I added a workflow to test the building system on Ubuntu. It will be good to have at least one configuration to Windows, and another to MacOS.

Here was my first attempt, that do not work:

diff --git a/.github/workflows/cmake.yml b/.github/workflows/cmake.yml
index 4d080e2..2fec569 100644
--- a/.github/workflows/cmake.yml
+++ b/.github/workflows/cmake.yml
@@ -33,7 +33,12 @@ defaults:
 jobs:
 
   build-all:
-    runs-on: ubuntu-latest
+    runs-on: ${{ matrix.os }}
+
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ macos-latest, windows-latest, ubuntu-latest ]
 
     steps:
 
@@ -43,19 +48,42 @@ jobs:
     - name: Install ninja-build tool
       uses: seanmiddleditch/gha-setup-ninja@v3
 
-    - name: Setup MPI
+    - name: Setup MPI on Ubuntu
+      if: ${{ matrix.os == 'ubuntu-latest' }}
       # uses: mpi4py/setup-mpi@v1
       run: |
         sudo apt -y update
         sudo apt -y install openmpi-bin libopenmpi-dev
 
-    - name: Install BLAS and LAPACK
-      run: sudo apt -y install libblas-dev liblapack-dev
+    - name: Setup MPI and gfortran on MacOS
+      if: ${{ matrix.os == 'macos-latest' }}
+      # uses: mpi4py/setup-mpi@v1
+      run: |
+        brew install open-mpi
+
+    - name: Setup MPI on Windows
+      if: ${{ matrix.os == 'windows-latest' }}
+      uses: mpi4py/setup-mpi@v1
+
+    # - name: Install BLAS and LAPACK
+    #   run: sudo apt -y install libblas-dev liblapack-dev
+
+    - name: Specific configurations for CMake on MacOS
+      if: ${{ matrix.os == 'macos-latest' }}
+      run: >
+        cmake -B build -G Ninja
+        -DCMAKE_C_COMPILER="mpicc"
+        -DCMAKE_Fortran_COMPILER="mpifort"
+
+    - name: Specific configurations for CMake on Windows
+      if: ${{ matrix.os == 'windows-latest' }}
+      run: >
+        cmake -B build -G Ninja
+        -D MPI_BASE_DIR="$MSMPI_BIN"
         
     - name: CMake configuration
       run: >
-        cmake -B build
-        -G Ninja
+        cmake -B build -G Ninja
         -D CMAKE_BUILD_TYPE=${{env.BUILD_TYPE}}
         -D CMAKE_INSTALL_PREFIX=${{github.workspace}}/scalapack_install
         -D BUILD_SHARED_LIBS=ON

Error on Windows:

 CMake Error at CMakeLists.txt:71 (message):
  --> MPI Library NOT FOUND -- please set MPI_BASE_DIR accordingly --

Error on MacOS:

CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_Fortran_PREPROCESS_SOURCE
CMake Error at /usr/local/Cellar/cmake/3.22.3/share/cmake/Modules/CMakeDetermineCompilerABI.cmake:49 (try_compile):
  Failed to generate test project build system.

First of all, thanks for the work on scalapack and sorry if the issue is not relevant.

I am trying to compile scalapack 2.2.0 with gcc 10, and I am getting error of type:

[ 45%] Building Fortran object CMakeFiles/scalapack.dir/SRC/pstrsen.f.o
/home/conda/feedstock_root/build_artifacts/scalapack_1646690481794/work/SRC/pstrsen.f:608:48:

  529 |      $           CALL IGAMX2D( ICTXT, 'All', TOP, 1, 1, MMAX, 1, -1,
      |                                                      2
......
 3253 |             CALL IGAMX2D( ICTXT, 'All', TOP, 1, 1, IERR, 1, -1,
      |                                                   1
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (rank-1 and scalar)
/home/conda/feedstock_root/build_artifacts/scalapack_1646690481794/work/SRC/pstrord.f:3264:54:

  490 |      $         CALL IGAMX2D( ICTXT, 'All', TOP, 1, 1, MMAX, 1, -1,
      |                                                      2
......
 3264 |                CALL IGAMX2D( ICTXT, 'All', TOP, 1, 1, INFO, 1, -1,
      |                                                      1
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (rank-1 and scalar)
make[2]: *** [CMakeFiles/scalapack.dir/build.make:12812: CMakeFiles/scalapack.dir/SRC/pstrord.f.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1234: CMakeFiles/scalapack.dir/all] Error 2
make: *** [Makefile:166: all] Error 2
Traceback (most recent call last):

In theory all the problems of this kind should have been fixed by #23, but on that specific file we are still getting them. Notice that the same error was also reported by the Homebrew CI, see https://github.com/Homebrew/homebrew-core/pull/94420/files#diff-e45b9bfcc61ceafda0d6200c3c06dec48c0da5a6259d8346659d3fe822b0ae0bL35 .

Hello,

I am attempting to compile scalapack with intel oneapi 2022 compiler, with gcc headers from gcc 10.2

When compiling, I keep getting errors along the lines of:

error: call to undeclared function 'BI_ivmcopy'; ISO C99 and later do not support implicit function declarations

My initial solution is to add function definitions to Bdef.h . This is tedious however because similar errors occur for other functions.

How would you suggest that I fix this? If you do not have any other ideas I can continue to add more function definitions to Bdef.h and I think that it will compile eventually, but I appreciate your thoughts as to a different solution.

In developing SLATE, we noticed issues in trapezoid matrix norm (lantr) for one and inf norms, where the loop bounds were incorrect, generally off-by-one so it either missed a row or read an extra row. Also, in some cases for unit diagonal, the unit diagonal was added on every rank, rather than only on ranks containing the diagonal block. Here is SLATE's tester output (abbreviated) showing which cases fail:

input: ./test trnorm --p 2 --q 2 --uplo lower,upper --diag nonunit,unit --dim 100,100x200,200x100 --type s,d,c,z --norm one
                                                                                    
type    uplo     diag   norm     m     n   nb   p   q      error  status
   s   lower  nonunit      1   100   100   50   2   2   0.00e+00  pass
   s   lower  nonunit      1   200   100   50   2   2   1.04e-08  pass
   s   lower     unit      1   100   100   50   2   2   0.00e+00  pass
   s   lower     unit      1   200   100   50   2   2   1.04e-08  pass
   s   upper  nonunit      1   100   100   50   2   2   0.00e+00  pass
   s   upper  nonunit      1   100   200   50   2   2   0.00e+00  pass
   s   upper     unit      1   100   100   50   2   2   0.00e+00  pass
   s   upper     unit      1   100   200   50   2   2   1.76e-03  FAILED
                                                                
   d   lower  nonunit      1   100   100   50   2   2   0.00e+00  pass
   d   lower  nonunit      1   200   100   50   2   2   0.00e+00  pass
   d   lower     unit      1   100   100   50   2   2   0.00e+00  pass
   d   lower     unit      1   200   100   50   2   2   0.00e+00  pass
   d   upper  nonunit      1   100   100   50   2   2   0.00e+00  pass
   d   upper  nonunit      1   100   200   50   2   2   0.00e+00  pass
   d   upper     unit      1   100   100   50   2   2   0.00e+00  pass
   d   upper     unit      1   100   200   50   2   2   1.76e-03  FAILED
                                                                
   c   lower  nonunit      1   100   100   50   2   2   0.00e+00  pass
   c   lower  nonunit      1   200   100   50   2   2   1.37e-08  pass
   c   lower     unit      1   100   100   50   2   2   0.00e+00  pass
   c   lower     unit      1   200   100   50   2   2   6.86e-09  pass
   c   upper  nonunit      1   100   100   50   2   2   0.00e+00  pass
   c   upper  nonunit      1   100   200   50   2   2   0.00e+00  pass
   c   upper     unit      1   100   100   50   2   2   0.00e+00  pass
   c   upper     unit      1   100   200   50   2   2   1.19e-03  FAILED
                                                                
   z   lower  nonunit      1   100   100   50   2   2   0.00e+00  pass
   z   lower  nonunit      1   200   100   50   2   2   3.84e-17  pass
   z   lower     unit      1   100   100   50   2   2   0.00e+00  pass
   z   lower     unit      1   200   100   50   2   2   2.56e-17  pass
   z   upper  nonunit      1   100   100   50   2   2   0.00e+00  pass
   z   upper  nonunit      1   100   200   50   2   2   0.00e+00  pass
   z   upper     unit      1   100   100   50   2   2   0.00e+00  pass
   z   upper     unit      1   100   200   50   2   2   1.19e-03  FAILED
4 tests FAILED.

input: ./test trnorm --p 2 --q 2 --uplo lower,upper --diag nonunit,unit --dim 100,100x200,200x100 --type s,d,c,z --norm inf
                                                                                    
type    uplo     diag   norm     m     n   nb   p   q      error  status
   s   lower  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   s   lower  nonunit    inf   200   100   50   2   2   0.00e+00  pass
   s   lower     unit    inf   100   100   50   2   2   1.87e-03  FAILED
   s   lower     unit    inf   200   100   50   2   2   0.00e+00  pass
   s   upper  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   s   upper  nonunit    inf   100   200   50   2   2   0.00e+00  pass
   s   upper     unit    inf   100   100   50   2   2   1.89e-03  FAILED
   s   upper     unit    inf   100   200   50   2   2   6.69e-04  FAILED
                                                                
   d   lower  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   d   lower  nonunit    inf   200   100   50   2   2   0.00e+00  pass
   d   lower     unit    inf   100   100   50   2   2   1.87e-03  FAILED
   d   lower     unit    inf   200   100   50   2   2   0.00e+00  pass
   d   upper  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   d   upper  nonunit    inf   100   200   50   2   2   0.00e+00  pass
   d   upper     unit    inf   100   100   50   2   2   1.89e-03  FAILED
   d   upper     unit    inf   100   200   50   2   2   6.69e-04  FAILED
                                                                
   c   lower  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   c   lower  nonunit    inf   200   100   50   2   2   0.00e+00  pass
   c   lower     unit    inf   100   100   50   2   2   1.27e-03  FAILED
   c   lower     unit    inf   200   100   50   2   2   0.00e+00  pass
   c   upper  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   c   upper  nonunit    inf   100   200   50   2   2   6.95e-09  pass
   c   upper     unit    inf   100   100   50   2   2   1.29e-03  FAILED
   c   upper     unit    inf   100   200   50   2   2   4.52e-04  FAILED
                                                                
   z   lower  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   z   lower  nonunit    inf   200   100   50   2   2   0.00e+00  pass
   z   lower     unit    inf   100   100   50   2   2   1.27e-03  FAILED
   z   lower     unit    inf   200   100   50   2   2   0.00e+00  pass
   z   upper  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   z   upper  nonunit    inf   100   200   50   2   2   1.29e-17  pass
   z   upper     unit    inf   100   100   50   2   2   1.29e-03  FAILED
   z   upper     unit    inf   100   200   50   2   2   4.52e-04  FAILED
12 tests FAILED.

After fixes, everything passes:

input: ./test trnorm --p 2 --q 2 --uplo lower,upper --diag nonunit,unit --dim 100,100x200,200x100 --type s,d,c,z --norm one
                                                                                    
type    uplo     diag   norm     m     n   nb   p   q      error  status
   s   lower  nonunit      1   100   100   50   2   2   0.00e+00  pass
   s   lower  nonunit      1   200   100   50   2   2   1.04e-08  pass
   s   lower     unit      1   100   100   50   2   2   0.00e+00  pass
   s   lower     unit      1   200   100   50   2   2   1.04e-08  pass
   s   upper  nonunit      1   100   100   50   2   2   0.00e+00  pass
   s   upper  nonunit      1   100   200   50   2   2   0.00e+00  pass
   s   upper     unit      1   100   100   50   2   2   0.00e+00  pass
   s   upper     unit      1   100   200   50   2   2   0.00e+00  pass
                                                                
   d   lower  nonunit      1   100   100   50   2   2   0.00e+00  pass
   d   lower  nonunit      1   200   100   50   2   2   0.00e+00  pass
   d   lower     unit      1   100   100   50   2   2   0.00e+00  pass
   d   lower     unit      1   200   100   50   2   2   0.00e+00  pass
   d   upper  nonunit      1   100   100   50   2   2   0.00e+00  pass
   d   upper  nonunit      1   100   200   50   2   2   0.00e+00  pass
   d   upper     unit      1   100   100   50   2   2   0.00e+00  pass
   d   upper     unit      1   100   200   50   2   2   0.00e+00  pass
                                                                
   c   lower  nonunit      1   100   100   50   2   2   0.00e+00  pass
   c   lower  nonunit      1   200   100   50   2   2   1.37e-08  pass
   c   lower     unit      1   100   100   50   2   2   0.00e+00  pass
   c   lower     unit      1   200   100   50   2   2   6.86e-09  pass
   c   upper  nonunit      1   100   100   50   2   2   0.00e+00  pass
   c   upper  nonunit      1   100   200   50   2   2   0.00e+00  pass
   c   upper     unit      1   100   100   50   2   2   0.00e+00  pass
   c   upper     unit      1   100   200   50   2   2   0.00e+00  pass
                                                                
   z   lower  nonunit      1   100   100   50   2   2   0.00e+00  pass
   z   lower  nonunit      1   200   100   50   2   2   3.84e-17  pass
   z   lower     unit      1   100   100   50   2   2   0.00e+00  pass
   z   lower     unit      1   200   100   50   2   2   2.56e-17  pass
   z   upper  nonunit      1   100   100   50   2   2   0.00e+00  pass
   z   upper  nonunit      1   100   200   50   2   2   0.00e+00  pass
   z   upper     unit      1   100   100   50   2   2   1.84e-17  pass
   z   upper     unit      1   100   200   50   2   2   0.00e+00  pass
All tests passed.

input: ./test trnorm --p 2 --q 2 --uplo lower,upper --diag nonunit,unit --dim 100,100x200,200x100 --type s,d,c,z --norm inf
                                                                            
type    uplo     diag   norm     m     n   nb   p   q      error  status
   s   lower  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   s   lower  nonunit    inf   200   100   50   2   2   0.00e+00  pass
   s   lower     unit    inf   100   100   50   2   2   0.00e+00  pass
   s   lower     unit    inf   200   100   50   2   2   0.00e+00  pass
   s   upper  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   s   upper  nonunit    inf   100   200   50   2   2   0.00e+00  pass
   s   upper     unit    inf   100   100   50   2   2   0.00e+00  pass
   s   upper     unit    inf   100   200   50   2   2   0.00e+00  pass
                                                                
   d   lower  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   d   lower  nonunit    inf   200   100   50   2   2   0.00e+00  pass
   d   lower     unit    inf   100   100   50   2   2   0.00e+00  pass
   d   lower     unit    inf   200   100   50   2   2   0.00e+00  pass
   d   upper  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   d   upper  nonunit    inf   100   200   50   2   2   0.00e+00  pass
   d   upper     unit    inf   100   100   50   2   2   0.00e+00  pass
   d   upper     unit    inf   100   200   50   2   2   0.00e+00  pass
                                                                
   c   lower  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   c   lower  nonunit    inf   200   100   50   2   2   0.00e+00  pass
   c   lower     unit    inf   100   100   50   2   2   0.00e+00  pass
   c   lower     unit    inf   200   100   50   2   2   0.00e+00  pass
   c   upper  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   c   upper  nonunit    inf   100   200   50   2   2   6.95e-09  pass
   c   upper     unit    inf   100   100   50   2   2   0.00e+00  pass
   c   upper     unit    inf   100   200   50   2   2   1.39e-08  pass
                                                                
   z   lower  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   z   lower  nonunit    inf   200   100   50   2   2   0.00e+00  pass
   z   lower     unit    inf   100   100   50   2   2   0.00e+00  pass
   z   lower     unit    inf   200   100   50   2   2   0.00e+00  pass
   z   upper  nonunit    inf   100   100   50   2   2   0.00e+00  pass
   z   upper  nonunit    inf   100   200   50   2   2   1.29e-17  pass
   z   upper     unit    inf   100   100   50   2   2   0.00e+00  pass
   z   upper     unit    inf   100   200   50   2   2   0.00e+00  pass
All tests passed.

Building Scalapack with OpenBLAS as the optimized BLAS/LAPACK library fails, because OpenBLAS builds demand that pthread is linked, unless OpenBLAS has been built without its default shared-memory threading enabled. See below for abridged build log.

This is probably something CMake could detect, based on the build information that OpenBLAS makes available about itself, or just detect OpenBLAS and always tack on -lpthread.

~/nfs_zpool/scalapack$ cmake . -D LAPACK_LIBRARIES=$OPENBLAS_LIB -D BLAS_LIBRARIES=$OPENBLAS_LIB
-- The C compiler identification is GNU 9.4.0
-- The Fortran compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /usr/bin/gfortran - skipped
-- Found MPI_C: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so (found version "3.1")
-- Found MPI_Fortran: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_usempif08.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- Found MPI_LIBRARY : TRUE
-- --> MPI C Compiler : /usr/bin/mpicc
-- --> C Compiler : /usr/bin/cc
-- --> MPI Fortran Compiler : /usr/bin/mpif90
-- --> Fortran Compiler : /usr/bin/gfortran
-- Reducing RELEASE optimization level to O2
-- =========
-- Compiling and Building BLACS INSTALL Testing to set correct variables
-- Configure in the INSTALL directory successful
-- Build in the BLACS INSTALL directory successful
-- =========
-- Testing FORTRAN_MANGLING
-- CDEFS set to Add_
-- =========
-- CHECKING BLAS AND LAPACK LIBRARIES
-- --> LAPACK supplied by user is /mnt/nfsroot/libs/OpenBLAS_latest_zen/lib/libopenblas.a.
-- Looking for Fortran dgesv
-- Looking for Fortran dgesv - not found
-- --> LAPACK routine dgesv is found: .
-- --> Searching for optimized LAPACK and BLAS libraries on your machine.
-- Found BLAS: /mnt/nfsroot/libs/OpenBLAS_latest_zen/lib/libopenblas.a
-- Found LAPACK: /mnt/nfsroot/libs/OpenBLAS_latest_zen/lib/libopenblas.a
-- BLAS library: /mnt/nfsroot/libs/OpenBLAS_latest_zen/lib/libopenblas.a
-- LAPACK library: /mnt/nfsroot/libs/OpenBLAS_latest_zen/lib/libopenblas.a
-- =========
-- Configuring done
-- Generating done
-- Build files have been written to: /home/gytibor/nfs_zpool/scalapack
...
[ 68%] Built target scalapack
Scanning dependencies of target xslu
[ 68%] Building Fortran object TESTING/LIN/CMakeFiles/xslu.dir/psludriver.f.o
[ 68%] Building Fortran object TESTING/LIN/CMakeFiles/xslu.dir/psluinfo.f.o
[ 69%] Building Fortran object TESTING/LIN/CMakeFiles/xslu.dir/psgetrrv.f.o
[ 69%] Building Fortran object TESTING/LIN/CMakeFiles/xslu.dir/psmatgen.f.o
[ 69%] Building Fortran object TESTING/LIN/CMakeFiles/xslu.dir/pmatgeninc.f.o
[ 69%] Building Fortran object TESTING/LIN/CMakeFiles/xslu.dir/pslaschk.f.o
[ 69%] Building Fortran object TESTING/LIN/CMakeFiles/xslu.dir/pslafchk.f.o
[ 69%] Linking Fortran executable ../xslu
/usr/bin/ld: /mnt/nfsroot/libs/OpenBLAS_latest_zen/lib/libopenblas.a(blas_server.o): undefined reference to symbol 'pthread_setaffinity_np@@GLIBC_2.3.4'
/usr/bin/ld: /lib/x86_64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
make[2]: *** [TESTING/LIN/CMakeFiles/xslu.dir/build.make:193: TESTING/xslu] Error 1
make[1]: *** [CMakeFiles/Makefile2:1260: TESTING/LIN/CMakeFiles/xslu.dir/all] Error 2
make: *** [Makefile:166: all] Error 2

There are problem with using a uninitialized variable in P?TRORD that leads to undefined behavior. I'm talking about INFODUM variable in pdtrord.f and pstrord.f.

Example from pdtrord.f:525 :

CALL IGAMX2D( ICTXT, 'All', TOP, 1, 1, INFODUM, 1, -1, -1, -1,
     $        -1, -1 )
INFO = INFODUM( 1 )

Here we find maximum of uninitialized INFODUM from each process and put this maximum into INFO. So we ignore INFO value and replace it with some undefined value from INFODUM (that often is zero and that's why tests can be passed without warnings or errors).

You can find same problem with IERR in same file on 3256 and 3268 lines.

CALL IGAMX2D( ICTXT, 'All', TOP, 1, 1, INFODUM, 1, -1,
     $           -1, -1, -1, -1 )
IERR = INFODUM( 1 )

CALL IGAMX2D( ICTXT, 'All', TOP, 1, 1, INFODUM, 1, -1,
     $           -1, -1, -1, -1 )
IERR = INFODUM( 1 )

My suggestion is fix INFO distribution on 525 line (and other places) like this:

CALL IGAMX2D( ICTXT, 'All', TOP, 1, 1, INFO, 1, -1,
     $           -1, -1, -1, -1 )

What do you think? If it's ok then I can make PR with those fixes.

Input parameters checks in p?potri don't work due to incorrect branch condition.
Please have a look at the small fix: #79

I encountered the following error while installing the development version of scalapack with openmpi 4.0.5/gcc 10.1.0:

Warning: Fortran 2018 deleted feature: Shared DO termination label 10 at (1)
blacstest.f:8492:39:

8489 | CALL BTRECV(3, 1, NERR2, K, IBTMSGID()+50)
| 2
......
8492 | CALL BTRECV(3, NERR2*6, IERR, K, IBTMSGID()+51)
| 1
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (scalar and rank-1)
blacstest.f:8493:39:

8489 | CALL BTRECV(3, 1, NERR2, K, IBTMSGID()+50)
| 2

Trying to compile scalapack with GCC 11.

Same problem as with GCC 10 (#21). This was closed a year a half ago but there has been no release since then so the incompatible version is still present everywhere.

Could you consider doing at least a point release, even if just for compatibility?

#48 adopted a versioning policy for the shared library, to use libscalapack.so.MAJOR.MINOR, implying a (possible) ABI break with each minor ScaLAPACK release. Since this is yet not part of a release, I suggest considering a more conservative approach that avoids needless breakage of user binaries on library updates.

Personally, I use semantic versioning for projects, incrementing the minor version for feature additions. The project version is distinct from the version of the libraries built with the project, following the libtool convention

https://www.gnu.org/software/libtool/manual/html_node/Libtool-versioning.html

of having specific versions for the libraries that basically just count the number of ABI changes and how many generations of the library are backwards compatible, with CURRENT.REVISION.AGE instead of MAJOR.MINOR.PATCH. In practice, this means that the library stays at version 0 for a long time, just gets revision increases, reminding of the epic shift from libc.so.5 to libc.so.6 for glibc (been some time now;-). You might automate the library revision number, calculating from the main release version.

So … you could start with libscalapack.so.2.2.0, with 2.2.1 release then with 2.2.2 release libscalapack.so.2.3.0. On release 2.3.0, if it doesn't break ABI, just adds things, it would be libscalapack.so.2.4.0. The actual SONAME would be libscalapack.so.2 for all of those. The jump to libscalapack.so.3 should happen if ABI compatibility has to be broken.

Hi,
I am using the cmake to configure the scalapack 2.1.0,
However, I encountered an error like below

CMake Error at BLACS/TESTING/CMakeLists.txt:4 (if):
  if given arguments:

    "GNU" "STREQUAL" "GNU" "AND" "CMAKE_Fortran_COMPILER_VERSION" "VERSION_GREATER_EQUAL" "10"

  Unknown arguments specified

and Following are the contents of BLACS/TESTING/CMakeLists.txt

set(FTestObj
   blacstest.f btprim.f tools.f)

if ("${CMAKE_Fortran_COMPILER_ID}" STREQUAL "GNU" AND CMAKE_Fortran_COMPILER_VERSION VERSION_GREATER_EQUAL 10)
    set_source_files_properties(blacstest.f PROPERTIES COMPILE_FLAGS "-std=legacy")
endif()

Is this a bug? thank you very much
Best,
Geng

The system environment is as follows：
Rocky Linux release 8.5
cmake version 3.21.0
scalapack-2.2.0
openmpi-4.1.2

cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local/scalapack
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_EXE_LINKER_FLAGS="-fopenmp"
-DBLAS_LIBRARIES=/usr/local/openblas/lib/libopenblas.a
-DLAPACK_LIBRARIES=/usr/local/openblas/lib/libopenblas.a

hi，The build has the following error：

[ 68%] Built target scalapack
[ 68%] Linking Fortran executable ../xzls
../../lib/libscalapack.a(igebs2d_.c.o): In function igebs2d_': igebs2d_.c:(.text+0xa8): undefined reference to MPI_Type_match_size'
igebs2d_.c:(.text+0x16e): undefined reference to MPI_Type_free' igebs2d_.c:(.text+0x2cd): undefined reference to MPI_Bcast'
../../lib/libscalapack.a(sgebs2d_.c.o): In function sgebs2d_': sgebs2d_.c:(.text+0xb6): undefined reference to ompi_mpi_float'
sgebs2d_.c:(.text+0x146): undefined reference to MPI_Type_free' sgebs2d_.c:(.text+0x2aa): undefined reference to MPI_Bcast'
../../lib/libscalapack.a(igebr2d_.c.o): In function igebr2d_': igebr2d_.c:(.text+0xae): undefined reference to MPI_Type_match_size'
igebr2d_.c:(.text+0x192): undefined reference to MPI_Type_free' igebr2d_.c:(.text+0x2bc): undefined reference to MPI_Bcast'
../../lib/libscalapack.a(sgebr2d_.c.o): In function sgebr2d_': sgebr2d_.c:(.text+0xc2): undefined reference to ompi_mpi_float'
sgebr2d_.c:(.text+0x182): undefined reference to MPI_Type_free' sgebr2d_.c:(.text+0x2aa): undefined reference to MPI_Bcast'
../../lib/libscalapack.a(igsum2d_.c.o): In function igsum2d_': igsum2d_.c:(.text+0x166): undefined reference to MPI_Type_match_size'
igsum2d_.c:(.text+0x3dd): undefined reference to ompi_mpi_op_sum' igsum2d_.c:(.text+0x3ea): undefined reference to MPI_Reduce'
igsum2d_.c:(.text+0x4ea): undefined reference to ompi_mpi_op_sum' igsum2d_.c:(.text+0x4f2): undefined reference to MPI_Allreduce'
../../lib/libscalapack.a(dgamx2d_.c.o): In function dgamx2d_': dgamx2d_.c:(.text+0x45): undefined reference to MPI_Type_match_size'
dgamx2d_.c:(.text+0x29a): undefined reference to ompi_mpi_double' dgamx2d_.c:(.text+0x2d1): undefined reference to MPI_Type_create_struct'
dgamx2d_.c:(.text+0x2db): undefined reference to MPI_Type_commit' dgamx2d_.c:(.text+0x43c): undefined reference to ompi_mpi_double'
dgamx2d_.c:(.text+0x445): undefined reference to ompi_mpi_double' dgamx2d_.c:(.text+0x4fe): undefined reference to MPI_Type_free'
dgamx2d_.c:(.text+0x5c1): undefined reference to MPI_Op_create' dgamx2d_.c:(.text+0x60a): undefined reference to MPI_Reduce'
dgamx2d_.c:(.text+0x62b): undefined reference to MPI_Op_free' dgamx2d_.c:(.text+0x642): undefined reference to MPI_Type_free'
dgamx2d_.c:(.text+0x7b5): undefined reference to MPI_Allreduce' dgamx2d_.c:(.text+0x845): undefined reference to MPI_Op_create'
../../lib/libscalapack.a(dgamx2d_.c.o): In function .LC3': dgamx2d_.c:(.rodata.cst8+0x0): undefined reference to ompi_mpi_unsigned_short'
../../lib/libscalapack.a(blacs_get_.c.o): In function blacs_get_': blacs_get_.c:(.text+0x7f): undefined reference to ompi_mpi_comm_world'
blacs_get_.c:(.text+0x89): undefined reference to MPI_Comm_get_attr' blacs_get_.c:(.text+0xc3): undefined reference to MPI_Comm_c2f'
../../lib/libscalapack.a(blacs_exit_.c.o): In function blacs_exit_': blacs_exit_.c:(.text+0x140): undefined reference to MPI_Finalize'
../../lib/libscalapack.a(blacs_pinfo_.c.o): In function blacs_pinfo_': blacs_pinfo_.c:(.text+0x43): undefined reference to ompi_mpi_comm_world'
blacs_pinfo_.c:(.text+0x48): undefined reference to MPI_Comm_size' blacs_pinfo_.c:(.text+0x52): undefined reference to ompi_mpi_comm_world'
blacs_pinfo_.c:(.text+0x57): undefined reference to MPI_Comm_rank' blacs_pinfo_.c:(.text+0x86): undefined reference to MPI_Initialized'
blacs_pinfo_.c:(.text+0x9d): undefined reference to ompi_mpi_comm_world' blacs_pinfo_.c:(.text+0xac): undefined reference to MPI_Comm_c2f'
blacs_pinfo_.c:(.text+0xcb): undefined reference to MPI_Init' ../../lib/libscalapack.a(blacs_map_.c.o): In function blacs_gridmap_':
blacs_map_.c:(.text+0x1a7): undefined reference to ompi_mpi_comm_null' blacs_map_.c:(.text+0x31d): undefined reference to MPI_Comm_dup'
blacs_map_.c:(.text+0x32e): undefined reference to MPI_Comm_rank' blacs_map_.c:(.text+0x349): undefined reference to MPI_Comm_split'
blacs_map_.c:(.text+0x35d): undefined reference to MPI_Comm_split' ../../lib/libscalapack.a(blacs_grid_.c.o): In function blacs_gridexit_':
blacs_grid_.c:(.text+0x2f): undefined reference to MPI_Comm_free' blacs_grid_.c:(.text+0x38): undefined reference to MPI_Comm_free'
blacs_grid_.c:(.text+0x40): undefined reference to MPI_Comm_free' blacs_grid_.c:(.text+0x49): undefined reference to MPI_Comm_free'
../../lib/libscalapack.a(blacs_barr_.c.o): In function blacs_barrier_': blacs_barr_.c:(.text+0x35): undefined reference to MPI_Barrier'
blacs_barr_.c:(.text+0x45): undefined reference to MPI_Barrier' blacs_barr_.c:(.text+0x54): undefined reference to MPI_Barrier'
../../lib/libscalapack.a(blacs_get_-C.c.o): In function Cblacs_get': blacs_get_-C.c:(.text+0x4f): undefined reference to ompi_mpi_comm_world'
blacs_get_-C.c:(.text+0x7f): undefined reference to ompi_mpi_comm_world' blacs_get_-C.c:(.text+0x89): undefined reference to MPI_Comm_get_attr'
../../lib/libscalapack.a(blacs_pinfo_-C.c.o): In function Cblacs_pinfo': blacs_pinfo_-C.c:(.text+0x43): undefined reference to ompi_mpi_comm_world'
blacs_pinfo_-C.c:(.text+0x48): undefined reference to MPI_Comm_size' blacs_pinfo_-C.c:(.text+0x52): undefined reference to ompi_mpi_comm_world'
blacs_pinfo_-C.c:(.text+0x57): undefined reference to MPI_Comm_rank' blacs_pinfo_-C.c:(.text+0x86): undefined reference to MPI_Initialized'
blacs_pinfo_-C.c:(.text+0x9d): undefined reference to ompi_mpi_comm_world' blacs_pinfo_-C.c:(.text+0xac): undefined reference to MPI_Comm_c2f'
blacs_pinfo_-C.c:(.text+0xcb): undefined reference to MPI_Init' ../../lib/libscalapack.a(blacs_grid_-C.c.o): In function Cblacs_gridexit':
blacs_grid_-C.c:(.text+0x2a): undefined reference to MPI_Comm_free' blacs_grid_-C.c:(.text+0x33): undefined reference to MPI_Comm_free'
blacs_grid_-C.c:(.text+0x3b): undefined reference to MPI_Comm_free' blacs_grid_-C.c:(.text+0x44): undefined reference to MPI_Comm_free'
../../lib/libscalapack.a(sys2blacs_-C.c.o): In function Csys2blacs_handle': sys2blacs_-C.c:(.text+0x1e): undefined reference to ompi_mpi_comm_null'
sys2blacs_-C.c:(.text+0x85): undefined reference to ompi_mpi_comm_null' sys2blacs_-C.c:(.text+0xae): undefined reference to ompi_mpi_comm_null'
sys2blacs_-C.c:(.text+0xd9): undefined reference to ompi_mpi_comm_world' sys2blacs_-C.c:(.text+0x1ac): undefined reference to ompi_mpi_comm_null'
sys2blacs_-C.c:(.text+0x1ef): undefined reference to ompi_mpi_comm_null' sys2blacs_-C.c:(.text+0x234): undefined reference to ompi_mpi_comm_null'
sys2blacs_-C.c:(.text+0x273): undefined reference to ompi_mpi_comm_world' ../../lib/libscalapack.a(BI_Ssend.c.o): In function BI_Ssend':
BI_Ssend.c:(.text+0x1d): undefined reference to MPI_Send' ../../lib/libscalapack.a(BI_Srecv.c.o): In function BI_Srecv':
BI_Srecv.c:(.text+0x27): undefined reference to MPI_Recv' ../../lib/libscalapack.a(BI_Arecv.c.o): In function BI_Arecv':
BI_Arecv.c:(.text+0x40): undefined reference to MPI_Irecv' BI_Arecv.c:(.text+0x58): undefined reference to MPI_Error_class'
BI_Arecv.c:(.text+0xbd): undefined reference to MPI_Irecv' ../../lib/libscalapack.a(BI_BeComb.c.o): In function BI_BeComb':
BI_BeComb.c:(.text+0x116): undefined reference to MPI_Sendrecv' ../../lib/libscalapack.a(BI_BlacsAbort.c.o): In function BI_BlacsAbort':
BI_BlacsAbort.c:(.text+0x1e): undefined reference to ompi_mpi_comm_world' BI_BlacsAbort.c:(.text+0x24): undefined reference to MPI_Abort'
../../lib/libscalapack.a(BI_BuffIsFree.c.o): In function BI_BuffIsFree': BI_BuffIsFree.c:(.text+0x26): undefined reference to MPI_Testall'
BI_BuffIsFree.c:(.text+0x4c): undefined reference to MPI_Waitall' ../../lib/libscalapack.a(BI_GetMpiGeType.c.o): In function BI_GetMpiGeType':
BI_GetMpiGeType.c:(.text+0x18): undefined reference to MPI_Type_vector' BI_GetMpiGeType.c:(.text+0x22): undefined reference to MPI_Type_commit'
../../lib/libscalapack.a(BI_TransUserComm.c.o): In function BI_TransUserComm': BI_TransUserComm.c:(.text+0x40): undefined reference to MPI_Comm_f2c'
BI_TransUserComm.c:(.text+0x50): undefined reference to MPI_Comm_group' BI_TransUserComm.c:(.text+0x64): undefined reference to MPI_Group_incl'
BI_TransUserComm.c:(.text+0x76): undefined reference to MPI_Comm_create' BI_TransUserComm.c:(.text+0x80): undefined reference to MPI_Group_free'
BI_TransUserComm.c:(.text+0x8a): undefined reference to MPI_Group_free' ../../lib/libscalapack.a(dgebs2d_.c.o): In function dgebs2d_':
dgebs2d_.c:(.text+0xb6): undefined reference to ompi_mpi_double' dgebs2d_.c:(.text+0x146): undefined reference to MPI_Type_free'
dgebs2d_.c:(.text+0x2aa): undefined reference to MPI_Bcast' ../../lib/libscalapack.a(zgebs2d_.c.o): In function zgebs2d_':
zgebs2d_.c:(.text+0xb6): undefined reference to ompi_mpi_dblcplex' zgebs2d_.c:(.text+0x146): undefined reference to MPI_Type_free'
zgebs2d_.c:(.text+0x2aa): undefined reference to MPI_Bcast' ../../lib/libscalapack.a(dgebr2d_.c.o): In function dgebr2d_':
dgebr2d_.c:(.text+0xc2): undefined reference to ompi_mpi_double' dgebr2d_.c:(.text+0x182): undefined reference to MPI_Type_free'
dgebr2d_.c:(.text+0x2aa): undefined reference to MPI_Bcast' ../../lib/libscalapack.a(zgebr2d_.c.o): In function zgebr2d_':
zgebr2d_.c:(.text+0xc2): undefined reference to ompi_mpi_dblcplex' zgebr2d_.c:(.text+0x182): undefined reference to MPI_Type_free'
zgebr2d_.c:(.text+0x2aa): undefined reference to MPI_Bcast' ../../lib/libscalapack.a(ztrbs2d_.c.o): In function ztrbs2d_':
ztrbs2d_.c:(.text+0xcd): undefined reference to ompi_mpi_dblcplex' ztrbs2d_.c:(.text+0x176): undefined reference to MPI_Type_free'
ztrbs2d_.c:(.text+0x2dc): undefined reference to MPI_Bcast' ../../lib/libscalapack.a(ztrbr2d_.c.o): In function ztrbr2d_':
ztrbr2d_.c:(.text+0xe0): undefined reference to ompi_mpi_dblcplex' ztrbr2d_.c:(.text+0x1c1): undefined reference to MPI_Type_free'
ztrbr2d_.c:(.text+0x2db): undefined reference to MPI_Bcast' ../../lib/libscalapack.a(dgsum2d_.c.o): In function dgsum2d_':
dgsum2d_.c:(.text+0x15e): undefined reference to ompi_mpi_double' dgsum2d_.c:(.text+0x166): undefined reference to ompi_mpi_double'
dgsum2d_.c:(.text+0x404): undefined reference to ompi_mpi_op_sum' dgsum2d_.c:(.text+0x409): undefined reference to ompi_mpi_double'
dgsum2d_.c:(.text+0x416): undefined reference to MPI_Reduce' dgsum2d_.c:(.text+0x512): undefined reference to ompi_mpi_op_sum'
dgsum2d_.c:(.text+0x517): undefined reference to ompi_mpi_double' dgsum2d_.c:(.text+0x51f): undefined reference to MPI_Allreduce'
../../lib/libscalapack.a(zgsum2d_.c.o): In function zgsum2d_': zgsum2d_.c:(.text+0x167): undefined reference to ompi_mpi_dblcplex'
zgsum2d_.c:(.text+0x16f): undefined reference to ompi_mpi_dblcplex' zgsum2d_.c:(.text+0x3f5): undefined reference to MPI_Op_create'
zgsum2d_.c:(.text+0x42f): undefined reference to MPI_Reduce' zgsum2d_.c:(.text+0x452): undefined reference to MPI_Op_free'
zgsum2d_.c:(.text+0x542): undefined reference to MPI_Allreduce' ../../lib/libscalapack.a(igamx2d_.c.o): In function igamx2d_':
igamx2d_.c:(.text+0x45): undefined reference to MPI_Type_match_size' igamx2d_.c:(.text+0x2c4): undefined reference to ompi_mpi_unsigned_short'
igamx2d_.c:(.text+0x2c9): undefined reference to MPI_Type_create_struct' igamx2d_.c:(.text+0x2d3): undefined reference to MPI_Type_commit'
igamx2d_.c:(.text+0x4ce): undefined reference to MPI_Type_free' igamx2d_.c:(.text+0x591): undefined reference to MPI_Op_create'
igamx2d_.c:(.text+0x5da): undefined reference to MPI_Reduce' igamx2d_.c:(.text+0x5fb): undefined reference to MPI_Op_free'
igamx2d_.c:(.text+0x612): undefined reference to MPI_Type_free' igamx2d_.c:(.text+0x79d): undefined reference to MPI_Allreduce'
igamx2d_.c:(.text+0x82d): undefined reference to MPI_Op_create' ../../lib/libscalapack.a(zgamx2d_.c.o): In function zgamx2d_':
zgamx2d_.c:(.text+0x43): undefined reference to MPI_Type_match_size' zgamx2d_.c:(.text+0x2aa): undefined reference to ompi_mpi_dblcplex'
zgamx2d_.c:(.text+0x2e6): undefined reference to MPI_Type_create_struct' zgamx2d_.c:(.text+0x2f0): undefined reference to MPI_Type_commit'
zgamx2d_.c:(.text+0x46c): undefined reference to ompi_mpi_dblcplex' zgamx2d_.c:(.text+0x475): undefined reference to ompi_mpi_dblcplex'
zgamx2d_.c:(.text+0x52e): undefined reference to MPI_Type_free' zgamx2d_.c:(.text+0x5f1): undefined reference to MPI_Op_create'
zgamx2d_.c:(.text+0x63a): undefined reference to MPI_Reduce' zgamx2d_.c:(.text+0x65b): undefined reference to MPI_Op_free'
zgamx2d_.c:(.text+0x672): undefined reference to MPI_Type_free' zgamx2d_.c:(.text+0x7dd): undefined reference to MPI_Allreduce'
zgamx2d_.c:(.text+0x872): undefined reference to MPI_Op_create' ../../lib/libscalapack.a(zgamx2d_.c.o): In function .LC3':
zgamx2d_.c:(.rodata.cst8+0x0): undefined reference to ompi_mpi_unsigned_short' ../../lib/libscalapack.a(dgamn2d_.c.o): In function dgamn2d_':
dgamn2d_.c:(.text+0x45): undefined reference to MPI_Type_match_size' dgamn2d_.c:(.text+0x29a): undefined reference to ompi_mpi_double'
dgamn2d_.c:(.text+0x2d1): undefined reference to MPI_Type_create_struct' dgamn2d_.c:(.text+0x2db): undefined reference to MPI_Type_commit'
dgamn2d_.c:(.text+0x43c): undefined reference to ompi_mpi_double' dgamn2d_.c:(.text+0x445): undefined reference to ompi_mpi_double'
dgamn2d_.c:(.text+0x4fe): undefined reference to MPI_Type_free' dgamn2d_.c:(.text+0x5c1): undefined reference to MPI_Op_create'
dgamn2d_.c:(.text+0x60a): undefined reference to MPI_Reduce' dgamn2d_.c:(.text+0x62b): undefined reference to MPI_Op_free'
dgamn2d_.c:(.text+0x642): undefined reference to MPI_Type_free' dgamn2d_.c:(.text+0x7b5): undefined reference to MPI_Allreduce'
dgamn2d_.c:(.text+0x845): undefined reference to MPI_Op_create' ../../lib/libscalapack.a(dgamn2d_.c.o): In function .LC3':
dgamn2d_.c:(.rodata.cst8+0x0): undefined reference to ompi_mpi_unsigned_short' ../../lib/libscalapack.a(dwalltime00_.c.o): In function dwalltime00_':
dwalltime00_.c:(.text+0x1): undefined reference to MPI_Wtime' ../../lib/libscalapack.a(dgesd2d_-C.c.o): In function Cdgesd2d':
dgesd2d_-C.c:(.text+0x26): undefined reference to ompi_mpi_double' dgesd2d_-C.c:(.text+0x7c): undefined reference to MPI_Type_free'
../../lib/libscalapack.a(zgesd2d_-C.c.o): In function Czgesd2d': zgesd2d_-C.c:(.text+0x26): undefined reference to ompi_mpi_dblcplex'
zgesd2d_-C.c:(.text+0x7c): undefined reference to MPI_Type_free' ../../lib/libscalapack.a(dgerv2d_-C.c.o): In function Cdgerv2d':
dgerv2d_-C.c:(.text+0x26): undefined reference to ompi_mpi_double' dgerv2d_-C.c:(.text+0x79): undefined reference to MPI_Type_free'
../../lib/libscalapack.a(zgerv2d_-C.c.o): In function Czgerv2d': zgerv2d_-C.c:(.text+0x26): undefined reference to ompi_mpi_dblcplex'
zgerv2d_-C.c:(.text+0x79): undefined reference to MPI_Type_free' ../../lib/libscalapack.a(dgebs2d_-C.c.o): In function Cdgebs2d':
dgebs2d_-C.c:(.text+0xaa): undefined reference to ompi_mpi_double' dgebs2d_-C.c:(.text+0x14e): undefined reference to MPI_Type_free'
dgebs2d_-C.c:(.text+0x2ab): undefined reference to MPI_Bcast' ../../lib/libscalapack.a(zgebs2d_-C.c.o): In function Czgebs2d':
zgebs2d_-C.c:(.text+0xaa): undefined reference to ompi_mpi_dblcplex' zgebs2d_-C.c:(.text+0x14e): undefined reference to MPI_Type_free'
zgebs2d_-C.c:(.text+0x2ab): undefined reference to MPI_Bcast' ../../lib/libscalapack.a(dgebr2d_-C.c.o): In function Cdgebr2d':
dgebr2d_-C.c:(.text+0xa7): undefined reference to ompi_mpi_double' dgebr2d_-C.c:(.text+0x163): undefined reference to MPI_Type_free'
dgebr2d_-C.c:(.text+0x28b): undefined reference to MPI_Bcast' ../../lib/libscalapack.a(zgebr2d_-C.c.o): In function Czgebr2d':
zgebr2d_-C.c:(.text+0xa7): undefined reference to ompi_mpi_dblcplex' zgebr2d_-C.c:(.text+0x163): undefined reference to MPI_Type_free'
zgebr2d_-C.c:(.text+0x28b): undefined reference to MPI_Bcast' ../../lib/libscalapack.a(dgsum2d_-C.c.o): In function Cdgsum2d':
dgsum2d_-C.c:(.text+0x149): undefined reference to ompi_mpi_double' dgsum2d_-C.c:(.text+0x151): undefined reference to ompi_mpi_double'
dgsum2d_-C.c:(.text+0x29d): undefined reference to ompi_mpi_op_sum' dgsum2d_-C.c:(.text+0x2a2): undefined reference to ompi_mpi_double'
dgsum2d_-C.c:(.text+0x2af): undefined reference to MPI_Reduce' dgsum2d_-C.c:(.text+0x4e2): undefined reference to ompi_mpi_op_sum'
dgsum2d_-C.c:(.text+0x4e7): undefined reference to ompi_mpi_double' dgsum2d_-C.c:(.text+0x4ef): undefined reference to MPI_Allreduce'
../../lib/libscalapack.a(zgsum2d_-C.c.o): In function Czgsum2d': zgsum2d_-C.c:(.text+0x142): undefined reference to ompi_mpi_dblcplex'
zgsum2d_-C.c:(.text+0x14a): undefined reference to ompi_mpi_dblcplex' zgsum2d_-C.c:(.text+0x295): undefined reference to MPI_Op_create'
zgsum2d_-C.c:(.text+0x2d0): undefined reference to MPI_Reduce' zgsum2d_-C.c:(.text+0x2f4): undefined reference to MPI_Op_free'
zgsum2d_-C.c:(.text+0x542): undefined reference to MPI_Allreduce' ../../lib/libscalapack.a(BI_Rsend.c.o): In function BI_Rsend':
BI_Rsend.c:(.text+0x1d): undefined reference to MPI_Rsend' ../../lib/libscalapack.a(BI_Asend.c.o): In function BI_Asend':
BI_Asend.c:(.text+0x40): undefined reference to MPI_Isend' BI_Asend.c:(.text+0x58): undefined reference to MPI_Error_class'
BI_Asend.c:(.text+0xbd): undefined reference to MPI_Isend' ../../lib/libscalapack.a(BI_Pack.c.o): In function BI_Pack':
BI_Pack.c:(.text+0x47): undefined reference to MPI_Pack' BI_Pack.c:(.text+0x53): undefined reference to ompi_mpi_packed'
BI_Pack.c:(.text+0x81): undefined reference to MPI_Pack_size' ../../lib/libscalapack.a(BI_GetMpiTrType.c.o): In function BI_GetMpiTrType':
BI_GetMpiTrType.c:(.text+0x16d): undefined reference to MPI_Type_indexed' BI_GetMpiTrType.c:(.text+0x177): undefined reference to MPI_Type_commit'
../../lib/libscalapack.a(dgesd2d_.c.o): In function dgesd2d_': dgesd2d_.c:(.text+0x31): undefined reference to ompi_mpi_double'
dgesd2d_.c:(.text+0x85): undefined reference to MPI_Type_free' ../../lib/libscalapack.a(zgesd2d_.c.o): In function zgesd2d_':
zgesd2d_.c:(.text+0x31): undefined reference to ompi_mpi_dblcplex' zgesd2d_.c:(.text+0x85): undefined reference to MPI_Type_free'
../../lib/libscalapack.a(dgerv2d_.c.o): In function dgerv2d_': dgerv2d_.c:(.text+0x31): undefined reference to ompi_mpi_double'
dgerv2d_.c:(.text+0x81): undefined reference to MPI_Type_free' ../../lib/libscalapack.a(zgerv2d_.c.o): In function zgerv2d_':
zgerv2d_.c:(.text+0x31): undefined reference to ompi_mpi_dblcplex' zgerv2d_.c:(.text+0x81): undefined reference to MPI_Type_free'
../../lib/libscalapack.a(igamn2d_.c.o): In function igamn2d_': igamn2d_.c:(.text+0x45): undefined reference to MPI_Type_match_size'
igamn2d_.c:(.text+0x2c4): undefined reference to ompi_mpi_unsigned_short' igamn2d_.c:(.text+0x2c9): undefined reference to MPI_Type_create_struct'
igamn2d_.c:(.text+0x2d3): undefined reference to MPI_Type_commit' igamn2d_.c:(.text+0x4ce): undefined reference to MPI_Type_free'
igamn2d_.c:(.text+0x591): undefined reference to MPI_Op_create' igamn2d_.c:(.text+0x5da): undefined reference to MPI_Reduce'
igamn2d_.c:(.text+0x5fb): undefined reference to MPI_Op_free' igamn2d_.c:(.text+0x612): undefined reference to MPI_Type_free'
igamn2d_.c:(.text+0x79d): undefined reference to MPI_Allreduce' igamn2d_.c:(.text+0x82d): undefined reference to MPI_Op_create'
collect2: error: ld returned 1 exit status
gmake[2]: *** [TESTING/LIN/CMakeFiles/xzls.dir/build.make:204: TESTING/xzls] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:1260: TESTING/LIN/CMakeFiles/xzls.dir/all] Error 2
gmake: *** [Makefile:166: all] Error 2

hello
there are two questions I want to consult.

1. how to buid the scalapack on the windows 10 with cmake?

• I have tried to build by cmake with msmpi and mingw(or VS2019), but all the results failed.
• I have down the Prebuilt libraries from http://icl.cs.utk.edu/lapack-for-windows/scalapack/index.html, but cannot run the example with the libraries.

2. where is the user guide about how to use the scalapack on windows?
   thanks

While solving a linear least squares problem of a skinny matrix using PDGEQRF, PDORMQR, and PDTRTRS, the latter returned info = -904, which denotes that the 4th entry of the 9th argument had an illegal value.

Excerpts from pdtrtrs.f:

!        1     2      3     4  5     6  7   8   9      10 11  12  13     14
PDTRTRS( UPLO, TRANS, DIAG, N, NRHS, A, IA, JA, DESCA, B, IB, JB, DESCB, INFO )

      PARAMETER          ( BLOCK_CYCLIC_2D = 1, DLEN_ = 9, DTYPE_ = 1,
     $                     CTXT_ = 2, M_ = 3, N_ = 4, MB_ = 5, NB_ = 6,
     $                     RSRC_ = 7, CSRC_ = 8, LLD_ = 9 )

*  M_A    (global) DESCA( M_ )    The number of rows in the global
*                                 array A.
*  N_A    (global) DESCA( N_ )    The number of columns in the global
*                                 array A.
*  MB_A   (global) DESCA( MB_ )   The blocking factor used to distribute
*                                 the rows of the array.
*  NB_A   (global) DESCA( NB_ )   The blocking factor used to distribute
*                                 the columns of the array.

But the check for this error involves the 5th (MB_) and 6th (NB_) entry of the the 9th (DESCA) argument instead of the 4th (N_).

            ELSE IF( DESCA( MB_ ).NE.DESCA( NB_ ) ) THEN
               INFO = -904

• Is this on purpose?
• Why do the blocking factors for rows and columns have to be equal? Is this documented?

scalapack 2.2 was recently released.

CMakeLists.txt however still declares

set(SCALAPACK_VERSION 2.1.0)
set(CPACK_PACKAGE_VERSION_MAJOR 2)
set(CPACK_PACKAGE_VERSION_MINOR 1)
set(CPACK_PACKAGE_VERSION_PATCH 0)

at

As a separate suggestion, less characters would need to be updated if the subversions are used:

set(CPACK_PACKAGE_VERSION_MAJOR 2)
set(CPACK_PACKAGE_VERSION_MINOR 1)
set(CPACK_PACKAGE_VERSION_PATCH 0)
set(SCALAPACK_VERSION ${CPACK_PACKAGE_VERSION_MAJOR}.${CPACK_PACKAGE_VERSION_MINOR}.${CPACK_PACKAGE_VERSION_PATCH})

This is a gap in ScaLAPACK API: it makes using P?GEBAL routine problematic.

This is a gap in ScaLAPACK API.

I'm using ScaLAPACK in my code, and in my CMakeLists.txt, I try to find ScaLAPACK using just:
find_package(SCALAPACK)
On both ubuntu 20.04 and Fedora 32, this finds the exported .cmake file.
However, it looks like there is something wrong in the installed scalapack-targets.cmake file.
On ubuntu I get:

CMake Error at /usr/lib/cmake/scalapack-2.1.0.openmpi/scalapack-targets.cmake:79 (message):
  The imported target "scalapack" references the file

     "/usr/lib/libscalapack-openmpi.so.2.1.0"

  but this file does not exist.  Possible reasons include:

  * The file was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and contained

     "/usr/lib/cmake/scalapack-2.1.0.openmpi/scalapack-targets.cmake"

  but not all the files it references.

Call Stack (most recent call first):
  /usr/lib/cmake/scalapack-2.1.0.openmpi/scalapack-config.cmake:2 (include)
  CMakeLists.txt:296 (find_package)

On Fedora 32, I get a similar error:

CMake Error at /usr/lib64/cmake/scalapack-2.1.0/scalapack-targets.cmake:79 (message):
  The imported target "scalapack" references the file

     "/usr/lib64/libscalapack.so.2.1.0"

  but this file does not exist.  Possible reasons include:

  * The file was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and contained

     "/usr/lib64/cmake/scalapack-2.1.0/scalapack-targets.cmake"

  but not all the files it references.

Call Stack (most recent call first):
  /usr/lib64/cmake/scalapack-2.1.0/scalapack-config.cmake:2 (include)
  CMakeLists.txt:296 (find_package)

For the new v2.2.0, the handling of the fortran compiler was changed in commit #61c60f5 to keep the identity of the fortran compiler (CMAKE_Fortran_COMPILER) separate from the MPI fortran compiler (MPI_Fortran_COMPILER). Previously the mpi wrapper was also used as the plain fortran compiler.

That change makes sense. The serial and mpi variants of the fortran compiler should be handled separately. But it has a consequence on the tests in TESTING. These tests link against -lscalapack, but libscalapack requires libmpi. It's not automatically pulled in, the -lmpi need to be added when compiling the tests.

The tests are currently configured to use the standard compiler (implicitly, CMAKE_Fortran_COMPILER). Before commit #61c60f5, the identification with MPI_Fortran_COMPILER meant that mpifort added the required -lmpi. Now that CMAKE_Fortran_COMPILER is kept separate, gfortran is used without -lmpi, so tests fail to link, giving errors like (in the case of xclu from TESTING/LIN)

/usr/bin/gfortran -Wl,-z,relro -Wl,--as-needed -g -O2 -ffile-prefix-map=/build/scalapack-2.2.0=. -fstack-protector-strong -std=legacy -std=legacy CMakeFiles/xclu.dir/pcludriver.f.o CMakeFiles/xclu.dir/pcluinfo.f.o CMakeFiles/xclu.dir/pcgetrrv.f.o CMakeFiles/xclu.dir/pcmatgen.f.o CMakeFiles/xclu.dir/pmatgeninc.f.o CMakeFiles/xclu.dir/pclaschk.f.o CMakeFiles/xclu.dir/pclafchk.f.o -o ../xclu  -Wl,-rpath,/build/scalapack-2.2.0/build-mpich/lib ../../lib/libscalapack-mpich.so.2.2.0 /usr/lib/x86_64-linux-gnu/liblapack.so /usr/lib/x86_64-linux-gnu/libblas.so /usr/lib/x86_64-linux-gnu/libblas.so 
/usr/bin/ld: ../../lib/libscalapack-mpich.so.2.2.0: undefined reference to `MPI_Barrier'
/usr/bin/ld: ../../lib/libscalapack-mpich.so.2.2.0: undefined reference to `MPI_Waitall'
/usr/bin/ld: ../../lib/libscalapack-mpich.so.2.2.0: undefined reference to `MPI_Send'
/usr/bin/ld: ../../lib/libscalapack-mpich.so.2.2.0: undefined reference to `MPI_Comm_dup'
/usr/bin/ld: ../../lib/libscalapack-mpich.so.2.2.0: undefined reference to `MPI_Irecv'
/usr/bin/ld: ../../lib/libscalapack-mpich.so.2.2.0: undefined reference to `MPI_Isend'
...
collect2: error: ld returned 1 exit status
make[4]: *** [TESTING/LIN/CMakeFiles/xclu.dir/build.make:193: TESTING/xclu] Error 1

I guess the test configuration should be updated so that the tests get compiled using MPI_Fortran_COMPILER

The scalapack library gets installed in /usr/lib (or /usr/lib64, if LIB_SUFFIX=64), defined in the SCALAPACK_install_library macro at

Some systems provide a multiarch library installation, for instance installing amd64 libraries into /usr/lib/x86_64-linux-gnu rather than /usr/lib. This allows the same filesystem to be shared with multiple systems. Debian and Ubuntu, for instance, provide multiarch installations.

Cmake provides flexibility to manage both types of installation via the CMAKE_INSTALL_LIBDIR variable. This can be easily done for scalapack by replacing

    LIBRARY DESTINATION lib${LIB_SUFFIX}

with

    LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}${LIB_SUFFIX}

In the "default" case CMAKE_INSTALL_LIBDIR is set to lib, so nothing would change, see https://cmake.org/cmake/help/latest/command/install.html#directory

Systems that need the multiarch location can get it by adding GNUInstallDirs

include(GNUInstallDirs)

Scalapack could perhaps add a configuration variable to choose whether to include it or not.

CMAKE_INSTALL_LIBDIR could then also be used instead of lib for the pkg-config and cmake file installations at ll.124,330,334 (perhaps the libdir variable at l.124 is redundant). It would also want to be applied to ARCHIVE at l.82.

If the multiarch (GNUInstallDirs) destination is activated then probably LIB_SUFFIX should be left empty. But that's a matter for the build configuration, LIB_SUFFIX handling in CMakeLists.txt would not need to be specifically changed.

An example of this kind of patch has been applied to the debian builds (2.2.1-3), build logs at https://buildd.debian.org/status/package.php?p=scalapack

scalapack 2.1 has just been released.

But CMakeLists.txt line 254 (and following) still sets the version at 2.0.2:

set(SCALAPACK_VERSION 2.0.2)
set(CPACK_PACKAGE_VERSION_MAJOR 2)
set(CPACK_PACKAGE_VERSION_MINOR 0)
set(CPACK_PACKAGE_VERSION_PATCH 2)

Should these variables have been updated to 2.1?

I am building the ScaLAPACK library using the provided CMake script. After installation, the installed library DSO maintains the RUNPATH present in the output of the build process. However, the default behavior of CMake is to remove the RUNPATH during the installation of the library according to the community wiki. I couldn't find in the CMake files any obvious command setting the RUNPATH of the installed binary (e.g. CMAKE_INSTALL_RPATH_USE_LINK_PATH). Is this behavior a bug, or a deliberate design choice?

As a work around, the build can be configured with

-D CMAKE_INSTALL_REMOVE_ENVIRONMENT_RPATH:BOOL=true

and the installed library binary will have no RUNPATH entry. However, it would be nice to know the cause of the issue.

To reproduce the issue, configure the commit 7e4e07070a489686287c36ab473d21cf29a54bdd with settings from the CMakeUserPresets.json file (cmake -S . --preset default-config):

{
  "version": 2,
  "cmakeMinimumRequired": {
    "major": 3,
    "minor": 20,
    "patch": 0
  },
  "configurePresets": [
    {
      "name": "default-config",
      "binaryDir": "${sourceDir}/build/Release",
      "generator": "Unix Makefiles",
      "cacheVariables": {
        "CMAKE_Fortran_FLAGS": "-fallow-argument-mismatch",
        "BUILD_SHARED_LIBS": true,
        "CMAKE_BUILD_TYPE": "Release",
        "CMAKE_PREFIX_PATH" : "$env{HOME}/opt/openblas",
        "CMAKE_INSTALL_PREFIX": "$env{HOME}/opt/scalapack"
      }
    }
  ],
  "buildPresets": [
    {
      "name": "default-build",
      "configurePreset": "default-config"
    }
  ]
}

The OpenBLAS library is used, and the path to the installation directory with OpenBLAS is contained in the CMAKE_PREFIX_PATH entry of the settings files. The output of the build command contains in the RUNPATH entry the paths for the MPI and OpenBLAS libraries. After installation, the paths to the OpenBLAS library is removed but not the path to the MPI library.

• Before installation:

$ readelf -d <path to build output> | grep RUNPATH
 0x000000000000001d (RUNPATH)            Library runpath: [/opt/apps/resif/aion/2020b/epyc/software/UCX/1.9.0-GCCcore-10.2.0/lib64:/usr/lib64:/opt/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/lib:/home/users/gkafanas/opt/openblas/lib64:]

• After installation:

$ readelf -d <path to installed output> | grep RUNPATH
0x000000000000001d (RUNPATH)            Library runpath: [/opt/apps/resif/aion/2020b/epyc/software/UCX/1.9.0-GCCcore-10.2.0/lib64:/usr/lib64:/opt/apps/resif/aion/2020b/epyc/software/OpenMPI/4.0.5-GCC-10.2.0/lib]

I would expect the installed ScaLAPACK DSO to have no RUNPATH entry.

I would understand it if it was a design choice to maintain the RUNPATH. However, none of the CMake variables required to install the DSO with the RUNPATH is set, so there may be some unintended side effect.

They seem to compile and work fine, so don't know if this was just an oversight?

See attached log.

At the moment the minimum required version of CMake is set to 2.8, which is quite old.

CMake 3.20.2 raises the following warning:

CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

In current master, two tests fail if run in parallel:

69/70 Testing: xshseqr
69/70 Test: xshseqr
Command: "/sw/env/gcc-10.3.0/openmpi/4.1.1/bin/mpiexec" "-n" "2" "./xshseqr"
Directory: /home/rrztest/src/scalapack/TESTING
"xshseqr" start time: Jul 25 20:04 CEST
Output:
----------------------------------------------------------

 ScaLAPACK Test for PSHSEQR

 epsilon   =    5.96046448E-08
 threshold =    30.0000000    

 Residual and Orthogonality Residual computed by:

 Residual      =  || T - Q^T*A*Q ||_F / ( ||A||_F * eps * sqrt(N) )

 Orthogonality =  MAX( || I - Q^T*Q ||_F, || I - Q*Q^T ||_F ) /  (eps * N)

 Test passes if both residuals are less then threshold

    N  NB    P    Q  QR Time  CHECK
----- --- ---- ---- -------- ------

Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation.

Backtrace for this error:
#0  0x151fa27c93ff in ???
#1  0x151fa455124f in pstrord_
        at /home/rrztest/src/scalapack/SRC/pstrord.f:1087
#2  0x151fa457a300 in pslaqr3_
        at /home/rrztest/src/scalapack/SRC/pslaqr3.f:880
#3  0x151fa4565178 in pslaqr0_
        at /home/rrztest/src/scalapack/SRC/pslaqr0.f:598
#4  0x151fa456209d in pshseqr_
        at /home/rrztest/src/scalapack/SRC/pshseqr.f:441
#5  0x4036cf in pshseqrdriver
        at /home/rrztest/src/scalapack/TESTING/EIG/pshseqrdriver.f:413
#6  0x404427 in main
        at /home/rrztest/src/scalapack/TESTING/EIG/pshseqrdriver.f:565
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpiexec noticed that process rank 0 with PID 0 on node node002 exited on signal 8 (Floating point exception).
--------------------------------------------------------------------------
<end of output>
Test time =   2.91 sec
----------------------------------------------------------
Test Failed.
"xshseqr" end time: Jul 25 20:04 CEST
"xshseqr" time elapsed: 00:00:02
----------------------------------------------------------

70/70 Testing: xdhseqr
70/70 Test: xdhseqr
Command: "/sw/env/gcc-10.3.0/openmpi/4.1.1/bin/mpiexec" "-n" "2" "./xdhseqr"
Directory: /home/rrztest/src/scalapack/TESTING
"xdhseqr" start time: Jul 25 20:04 CEST
Output:
----------------------------------------------------------

 ScaLAPACK Test for PDHSEQR

 epsilon   =    1.1102230246251565E-016
 threshold =    30.000000000000000     

 Residual and Orthogonality Residual computed by:

 Residual      =  || T - Q^T*A*Q ||_F / ( ||A||_F * eps * sqrt(N) )

 Orthogonality =  MAX( || I - Q^T*Q ||_F, || I - Q*Q^T ||_F ) /  (eps * N)

 Test passes if both residuals are less then threshold

    N  NB    P    Q  QR Time  CHECK
----- --- ---- ---- -------- ------

Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation.

Backtrace for this error:
#0  0x1488be0113ff in ???
#1  0x1488bff4ebae in pdtrord_
        at /home/rrztest/src/scalapack/SRC/pdtrord.f:1087
#2  0x1488bff77f2f in pdlaqr3_
        at /home/rrztest/src/scalapack/SRC/pdlaqr3.f:878
#3  0x1488bff62d2b in pdlaqr0_
        at /home/rrztest/src/scalapack/SRC/pdlaqr0.f:598
#4  0x1488bff5fc1d in pdhseqr_
        at /home/rrztest/src/scalapack/SRC/pdhseqr.f:441
#5  0x4036e2 in pdhseqrdriver
        at /home/rrztest/src/scalapack/TESTING/EIG/pdhseqrdriver.f:412
#6  0x404445 in main
        at /home/rrztest/src/scalapack/TESTING/EIG/pdhseqrdriver.f:564
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpiexec noticed that process rank 0 with PID 0 on node node002 exited on signal 8 (Floating point exception).
--------------------------------------------------------------------------
<end of output>
Test time =   2.70 sec
----------------------------------------------------------
Test Failed.
"xdhseqr" end time: Jul 25 20:04 CEST
"xdhseqr" time elapsed: 00:00:02
----------------------------------------------------------

End testing: Jul 25 20:04 CEST

Both tests pass fine with -n 1. I tested on two machines with differing compilers and MPI versions (4.1.1 and 1.10.7).

I observe weirdly long runtimes (hundreds of seconds) for some 2.2.0 tests when run inside the pkgsrc build framework, but they do succeed eventually. These FPEs are more definite.

I suggest adding topics such as scalapack, lapack, linear-algebra in the About section at https://github.com/Reference-ScaLAPACK/scalapack .

The main CMakeLists script contains a check

if ( "${CMAKE_Fortran_COMPILER}" MATCHES "ifort" )

where an attempt is made to detect the Intel Fortran compiler. However, this particular form of the condition results in a false positive when compiling ScaLAPACK with "mpifort", which can wrap any Fortran compiler.

The above condition should use the variable CMAKE_Fortran_COMPILER_ID instead, which is set to "Intel" with the Intel Fortran compiler (or, for instance, to "GNU" with gfortran).

This condition occurs in two places.

PR #53 fixed the dependencies of the ScaLAPACK libraries on BLACS. It should now be safe to use make -j lib, i.e., to use multiple jobs to build. However, make -j all may still fail as related in PR #53.

Inside pdstedc function pdlasrt is called with passing of lwork. But: lwmin for the 1st one is

6*n + 2*np*nq

while lwmin for the 2nd function is

lwmin = max( n, np*( nb+nq ) )

and sometimes this can lead to error in pdlasrt, despite lwork in pdsted equal to lwmin. For example, nq is small and np $\approx$ nb:

$6n + 2np*nq < np*nb < nb*(nb + nq)$

Code for reproduction (I can't load file) and typical output is below.

Typical output for mpirun -np 4 ./a.out 128 127:

(0x1): m = n = 128 mb = nb = 70
(1x0): m = n = 128 mb = nb = 70
(1x1): m = n = 128 mb = nb = 70
(1x1): requested work = 7496 and iwork = 914
(0x0): m = n = 128 mb = nb = 70
(0x0): requested work = 10568 and iwork = 914
(1x0): requested work = 8888 and iwork = 914
(0x1): requested work = 8888 and iwork = 914
{    0,    1}:  On entry to PDLASRT parameter number    9 had an illegal value
error in pdstedc call, code = -9

C++ code below request minimal workspaces for pdstedc and call it with corresponding lwork and liwork.

#include <mpi.h>
#include <iostream>

/*
  To compile:
  mpic++ test.cpp -L PATH_TO_LIBS -lscalapack -llapack -lrefblas -lgfortran
  To run:
  mpirun -np 4 ./a.out n nb

*/

extern "C" {
    void Cblacs_get(int, int, int*);
    void Cblacs_gridinit(int*, const char*, int, int);
    void Cblacs_gridinfo(int, int*, int*, int*,int*);
    void Cblacs_gridexit(int);

    int numroc_(const int* n, int* nb, const  int* iproc, const int* isrc, const int* nprocs);
    void descinit_(int* desc, const int* m, const int* n, const int* mb, const int* nb,
                   const int* irsrc, const int* icsrc, const int* ictxt, const int* ld, int* info);
    void pdstedc_( const char* comz, const int* n, double* d, double* e, double* q, const int* iq,
                   const int* jq, const int* descq, double* work, const int* lwork, int* iwork, const int* liwork,
                   int* info);

}

int main(int argc, char **argv) {

    // some constatns
    constexpr int bignum = 1'000'000;
    const int izero = 0;
    const int ione = 1;
    const char compz = 'I';

    // init grid 2x2
    int ictxt, myrow, mycol;
    int p = 2;
    int q = 2;

    MPI_Init(&argc, &argv);
    int myrank;
    int nprocs;
    MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
    MPI_Comm_size(MPI_COMM_WORLD, &nprocs);


    Cblacs_get(0, 0, &ictxt);
    Cblacs_gridinit(&ictxt, "Row-major", p, q);
    Cblacs_gridinfo(ictxt, &p, &q, &myrow, &mycol);

    if (argc != 3)
    {
        std::cerr<<"wrong argC"<<std::endl;
        return 1;
    }

    const int mn = std::stoi(argv[1]);
    const int mnb = std::stoi(argv[2]);

    std::cout<<"("<<myrow<<"x"<<mycol<<"): "<<"m = n = "<<mn<<" mb = nb = "<<mnb<<std::endl;

    int info;

    // init input arguments for pdstedc
    int descq[9];
    descinit_(descq, &mn, &mn, &mnb, &mnb, &izero, &izero, &ictxt, &mn, &info);

    if (info != 0)
    {
        std::cerr<<"error in descinit, code = "<<info<<std::endl;
        return 0;
    }

    // allocate memory (I hope, that requested size < bignum)
    double* d = new double[bignum];
    double* e = new double[bignum];
    double* qmat = new double[bignum];
    double* work = new double[bignum];
    int* iwork = new int[bignum];

    double workReq = 0.0/0.0;
    int iworkReq = -1000;

    const int& imone = -1;

    // request minimal workspaces in pdstedc
    pdstedc_(&compz, &mn, d, e, qmat, &ione, &ione, descq, &workReq, &imone, &iworkReq, &imone, &info);
    if (info != 0)
    {
        std::cerr<<"error in workspaces request, code = "<<info<<std::endl;
        return 0;
    }

    std::cout<<"("<<myrow<<"x"<<mycol<<"): "<<"requested work = "<<workReq<<" and iwork = "<<iworkReq<<std::endl;

    const int lwork = workReq;
    const int liwork = iworkReq;

    // run pdstedc with minimal workspaces
    pdstedc_(&compz, &mn, d, e, qmat, &ione, &ione, descq, work, &lwork, iwork, &liwork, &info);
    if (info != 0)
    {
        std::cerr<<"error in pdstedc call, code = "<<info<<std::endl;
        return 0;
    }


    delete[] d;
    delete[] e;
    delete[] qmat;
    delete[] work;
    delete[] iwork;

    Cblacs_gridexit(ictxt);
    MPI_Finalize();
}

Note: the same problem is actual for single precision versions.

Hello,

I tracked down a compilation issue that was causing unexpected results. In the file SRC/Makefile, I think this line:

$(CC) -c $(CFLAGS) $(CDEFS) $(@:.o=.c) -o $@

should maybe be:

$(CC) -c $(CCFLAGS) $(CDEFS) $(@:.o=.c) -o $@

Running this: $ ./setup.py --prefix=/home/gpaz/scalapack/2.1.0 --fcflags=" -fallow-argument-mismatch" --downall
I have obtained this:

Any solution?

This issue covers apparently very basic build trouble on the path to package ScaLAPACK for pkgsrc, so I hope an issue is warranted. I do have changes to suggest, too, convering handling of dependencies (.pc file), but first the current version should at least build …

I might be too obvious, but I have trouble building scalapack 2.2.0. CMake lokates my compiler/MPI setup just fine:

$ mkdir b
$ cd b
$ cmake -DMPI_BASE_DIR=$(dirname $(dirname $(which mpicxx))) -DBUILD_SHARED_LIBS=on ..
[…]
-- Found MPI_C: /stuff/sw/env/gcc-10.3.0/pkgsrc/cvs-20210222/lib/libmpi.so (found version "3.0") 
-- Found MPI_Fortran: /stuff/sw/env/gcc-10.3.0/pkgsrc/cvs-20210222/lib/libmpi_usempif08.so (found version "3.0") 
-- Found MPI: TRUE (found version "3.0")  
-- Found MPI_LIBRARY : TRUE 
-- --> MPI C Compiler : /stuff/sw/env/gcc-10.3.0/pkgsrc/cvs-20210222/bin/mpicc
-- --> C Compiler : /stuff/sw/env/gcc-10.3.0/pkgsrc/cvs-20210222/bin/mpicc
-- --> MPI Fortran Compiler : /stuff/sw/env/gcc-10.3.0/pkgsrc/cvs-20210222/bin/mpif90
-- --> Fortran Compiler : /usr/bin/gfortran

But it doesn't even try to link to MPI, apparently:

[ 69%] Building Fortran object TESTING/LIN/CMakeFiles/xslu.dir/pslafchk.f.o
[ 69%] Linking Fortran executable ../xslu
/usr/bin/ld: ../../lib/libscalapack.so: undefined reference to `ompi_mpi_op_sum'
/usr/bin/ld: ../../lib/libscalapack.so: undefined reference to `MPI_Bcast'
/usr/bin/ld: ../../lib/libscalapack.so: undefined reference to `MPI_Op_create'
/usr/bin/ld: ../../lib/libscalapack.so: undefined reference to `MPI_Pack_size'
/usr/bin/ld: ../../lib/libscalapack.so: undefined reference to `MPI_Send'
/usr/bin/ld: ../../lib/libscalapack.so: undefined reference to `ompi_mpi_packed'

and, obviously

$ ldd lib/libscalapack.so 
        linux-vdso.so.1 (0x00007ffe647b2000)
        libopenblas.so.0 => /stuff/sw/env/gcc-10.3.0/pkgsrc/cvs-20210222/lib/libopenblas.so.0 (0x00007f4f73090000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x00007f4f73057000)
        libgfortran.so.5 => /usr/lib/libgfortran.so.5 (0x00007f4f72ea0000)
        libm.so.6 => /lib/libm.so.6 (0x00007f4f72ddc000)
        libdl.so.2 => /lib/libdl.so.2 (0x00007f4f72dd7000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f4f72dbf000)
        libquadmath.so.0 => /usr/lib/libquadmath.so.0 (0x00007f4f72d81000)
        libc.so.6 => /lib/libc.so.6 (0x00007f4f72bf6000)
        /lib/ld-linux-x86-64.so.2 (0x00007f4f75245000)

Shouldn't there be a reference to libmpi?

I also don't see e.g. MPI_LIBRARIES, MPI_LINK_FLAGS, or anything like that used in CMakeLists.txt. I guess it is supposed to make more use of the MPI compiler wrappers? How is this supposed to work? (Note: Don't be fooled by the cvs-20210222 name, it is a current checkout of the pkgsrc tree, which I am trying to add scalapack to.)

When I tried to hack in use of MPI_LIBRARIES, the main MPI symbols were resolved, but I still had some missing ones at some stage. So, before I dive too deep, I hoped to get some clarification that the build should normally just work or that I'm doing something wrong. I did have a working draft installation of scalapack-2.1.0 before. I might have to revisit why that worked.

Hello,

While compiling ScaLAPACK 2.1.0 + patches from #26 with GCC 10.3.0 (for GNU Guix on GNU/Linux, x86_64), the xshseqr test does not complete:

90/96 Test #90: xzevc ............................   Passed    1.63 sec
      Start 91: xssyevr
91/96 Test #91: xssyevr ..........................   Passed   10.31 sec
      Start 92: xdsyevr
92/96 Test #92: xdsyevr ..........................   Passed   11.23 sec
      Start 93: xcheevr
93/96 Test #93: xcheevr ..........................   Passed   17.69 sec
      Start 94: xzheevr
94/96 Test #94: xzheevr ..........................   Passed   19.26 sec
      Start 95: xshseqr
95/96 Test #95: xshseqr ..........................***Timeout 1500.00 sec

 ScaLAPACK Test for PSHSEQR

 epsilon   =    5.96046448E-08
 threshold =    30.0000000    

 Residual and Orthogonality Residual computed by:

 Residual      =  || T - Q^T*A*Q ||_F / ( ||A||_F * eps * sqrt(N) )

 Orthogonality =  MAX( || I - Q^T*Q ||_F, || I - Q*Q^T ||_F ) /  (eps * N)

 Test passes if both residuals are less then threshold

    N  NB    P    Q  QR Time  CHECK
----- --- ---- ---- -------- ------
{    0,    1}:  On entry to PSTRORD parameter number **** had an illegal value
{    1,    0}:  On entry to PSTRORD parameter number **** had an illegal value
{    1,    1}:  On entry to PSTRORD parameter number **** had an illegal value
{    0,    0}:  On entry to PSTRORD parameter number **** had an illegal value
{    1,    1}:  On entry to PSTRORD parameter number **** had an illegal value
{    0,    1}:  On entry to PSTRORD parameter number **** had an illegal value
{    1,    0}:  On entry to PSTRORD parameter number **** had an illegal value

Those last lines are repeated ad infinitum.

Any idea what could be wrong?

Thanks in advance.

This is a gap in ScaLAPACK API. The link below shows that HSEQR performs much faster than LAHQR. It means that ScaLAPACK implements an inefficient algorithm for complex matrices.

http://netlib.org/scalapack/scalapack-2.0.0.html

By default, the about to be released GCC 10 is more strict about Fortran argument types.

When attempting to build ScaLAPACK with GCC 10, compiling BLACS code produced many errors of the type

Rank mismatch between actual argument at (1) and actual argument at (2)

The errors were reduced to warning by adding -fallow-argument-mismatch to gfortran.

I get the following error when building with CMake 3.13.3:

       CMake Error at CMAKE/FortranMangling.cmake:27 (MESSAGE):
       Configure in the BLACS INSTALL directory FAILED
       Call Stack (most recent call first):
       CMakeLists.txt:122 (COMPILE)

This patch fixes it for both CMake 3.13.3 and 2.8.10.2 (and has better diagnostic output):

--- old/CMAKE/FortranMangling.cmake	2019-01-25 21:50:50.619858036 -0500
+++ new/CMAKE/FortranMangling.cmake	2019-01-25 22:01:47.716985334 -0500
@@ -19,25 +19,27 @@
          "-DCMAKE_Fortran_COMPILER=${CMAKE_Fortran_COMPILER}"
-         "-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}"
+         "-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}" "${PROJECT_SOURCE_DIR}//BLACS/INSTALL"
         WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}/BLACS/INSTALL/        
-        RESULT_VARIABLE RESVAR OUTPUT_VARIABLE LOG1 ERROR_VARIABLE LOG1
+        RESULT_VARIABLE RESVAR OUTPUT_VARIABLE LOG1_OUT ERROR_VARIABLE LOG1_ERR
     )
     if(RESVAR EQUAL 0)
     MESSAGE(STATUS "Configure in the INSTALL directory successful")
     else()
+    MESSAGE(STATUS " Build Output:\n ${LOG1_OUT}")
+    MESSAGE(STATUS " Error Output:\n ${LOG1_ERR}")
     MESSAGE(FATAL_ERROR " Configure in the BLACS INSTALL directory FAILED")
-    MESSAGE(FATAL_ERROR " Output Build:\n ${LOG1}")
     endif()
 
     # Build:
     EXECUTE_PROCESS(COMMAND ${CMAKE_COMMAND} --build
         ${PROJECT_SOURCE_DIR}/BLACS/INSTALL/ 
-        RESULT_VARIABLE RESVAR OUTPUT_VARIABLE LOG2 ERROR_VARIABLE LOG2
+        RESULT_VARIABLE RESVAR OUTPUT_VARIABLE LOG2_OUT ERROR_VARIABLE LOG2_ERR
     )
     if(RESVAR  EQUAL 0)
     MESSAGE(STATUS "Build in the BLACS INSTALL directory successful")
     else()
+    MESSAGE(STATUS " Build Output:\n ${LOG2_OUT}")
+    MESSAGE(STATUS " Error Output:\n ${LOG2_ERR}")
     MESSAGE(FATAL_ERROR " Build in the BLACS INSTALL directory FAILED")
-    MESSAGE(FATAL_ERROR " Output Build:\n ${LOG2}")
     endif()
     # Clean up:
     FILE(REMOVE_RECURSE ${PROJECT_SOURCE_DIR}/BLACS/INSTALL/CMakeCache.txt)
@@ -62,6 +64,7 @@
           MESSAGE(STATUS "CDEFS set to ${xintface_OUT}")
           SET(CDEFS ${xintface_OUT} CACHE STRING "Fortran Mangling" FORCE)
       else()
+          MESSAGE(STATUS " xintface Output:\n ${xintface_OUT}")
           MESSAGE(FATAL_ERROR "FORTRAN_MANGLING:ERROR ${xintface_ERR}")
       endif() 
 endmacro(FORTRAN_MANGLING)

Hello guys,

I know there is the CMake already in place, but I still want to implement support for the meson builder. That way I can wrap this github project as a subproject dependency on my repositories. That way, if the used doenst have it already installed on their machine, it will trigger the compilation. Since it uses ninja, it also may shorten the compilation time considerably.

Thx!