flame / libflame

High-performance object-based library for DLA computations

libflame library

libflame is a portable library for dense matrix computations, providing much of the functionality present in LAPACK, developed by current and former members of the Science of High-Performance Computing (SHPC) group in the Institute for Computational Engineering and Sciences at The University of Texas at Austin. libflame includes a compatibility layer, lapack2flame, which includes a complete LAPACK implementation. The source code for libflame is hosted on github:

https://github.com/flame/libflame

For detailed instructions on how to configure, build, install, and link against libflame, please refer to the libflame reference manual. This manual contains all the information you need to get started with libflame, including installation directions, usage examples, and a complete API reference. If you have LaTeX installed on your system, you may simply change into the 'docs/libflame' subdirectory of the top-level directory of the libflame source tree and build the document from its source. You may also find a copy of the document here on github.

You can keep in touch with developers and other users of the project by joining the following mailing list:

• libflame-discuss Please join and post to this mailing list if you have general questions or feedback regarding libflame.
• NOTE: We do not yet have a developer's mailing list. For now, developers and those with developer-related questions can simply post to libflame-discuss.

If you encounter any problems or wish to submit a bug report, please consider opening an issue here on github.

Also, please read the LICENSE file for information on copying and distributing this software.

Thanks for your interest in libflame!

Regards,

Field G. Van Zee (for the SHPC team)

libflame's Issues

FLA_lapack2flame_return_defs.h is missing when built out of tree

if I build with --enable-lapack2flame out of tree

dynamic linking failure

This looks similar to the BLIS issue I had reported there and had been fixed. A bunch of these happen:

`/usr/bin/ld: error: can't create dynamic relocation R_X86_64_PC32 against symbol: FLA_Obj_attach_buffer in readonly segment; recompile object files with -fPIC

defined in obj/amd64-portbld-freebsd12.0/base/flamec/main/FLA_Obj.o
referenced by FLA_gebrd.c
obj/amd64-portbld-freebsd12.0/map/lapack2flamec/FLA_gebrd.o:(sgebrd_)`

upon linking with lld 6.0.0 on FreeBSD 12-HEAD

clang: argument list too long

On OSX with clang, I'm enabling --enable-max-arg-list-hack and that fixes the archiving of the static library. But clang still fails to build the dynamic library and errors out with the message "argument list too long". What's the way to circumvent that one?

Thanks.

LAPACKE symbols in library

I was wondering if the constructed library libflame.so also provides LAPACKE symbols in addition to LAPACK symbols?

Given that this is a C library, it makes sense to have both.

Thanks,
Aisha

`collect2: fatal error: execvp: Argument list too long` error when building dynamic library

It seems that collect2 still calls execvp with many arguments, even when I use --enable-max-arg-list-hack and supply the list of files in a file to the linker. I get the following error message when I try to build libflame on macOS in Homebrew's superenv:

kyungminlee@MBP libflame-20191206-16245-13p44gs % make -n
echo "Dynamically linking lib/x86_64-apple-darwin19.0.0/libflame.dylib"
gcc-9 -dynamiclib -Wl,-install_name,libflame.1.dylib -L/Users/kyungminlee/.brew/opt/blis/lib  -L/Users/kyungminlee/.brew/opt/blis/lib -L/Users/kyungminlee/.brew/Cellar/gcc/9.2.0_2/lib/gcc/9/gcc/x86_64-apple-darwin19/9.2.0 -L/Users/kyungminlee/.brew/Cellar/gcc/9.2.0_2/lib/gcc/9/gcc/x86_64-apple-darwin19/9.2.0/../../.. -lblis -lgfortran -lquadmath -lm -lpthread -o lib/x86_64-apple-darwin19.0.0/libflame.dylib @ar_obj_list
echo "Creating symlink lib/x86_64-apple-darwin19.0.0/libflame.1.dylib"
ln -sf libflame.dylib libflame.1.dylib.loc
mv libflame.1.dylib.loc ./lib/x86_64-apple-darwin19.0.0/libflame.1.dylib
kyungminlee@MBP libflame-20191206-16245-13p44gs % make
Dynamically linking lib/x86_64-apple-darwin19.0.0/libflame.dylib
collect2: fatal error: execvp: Argument list too long
compilation terminated.
make: *** [lib/x86_64-apple-darwin19.0.0/libflame.dylib] Error 1
kyungminlee@MBP libflame-20191206-16245-13p44gs % gcc-9 -dynamiclib -Wl,-install_name,libflame.1.dylib -L/Users/kyungminlee/.brew/opt/blis/lib  -L/Users/kyungminlee/.brew/opt/blis/lib -L/Users/kyungminlee/.brew/Cellar/gcc/9.2.0_2/lib/gcc/9/gcc/x86_64-apple-darwin19/9.2.0 -L/Users/kyungminlee/.brew/Cellar/gcc/9.2.0_2/lib/gcc/9/gcc/x86_64-apple-darwin19/9.2.0/../../.. -lblis -lgfortran -lquadmath -lm -lpthread -o lib/x86_64-apple-darwin19.0.0/libflame.dylib @ar_obj_list
collect2: fatal error: execvp: Argument list too long
compilation terminated.
kyungminlee@MBP libflame-20191206-16245-13p44gs % 

The configure flags are --disable-debug --enable-verbose-make-output --enable-dynamic-build --enable-lapack2flame --enable-max-arg-list-hack --enable-supermatrix --enable-multithreading=pthreads.

How about using the same hack as the static library, and then passing the static library to the linker? Something like the following may work

$(CAT) $(AR_OBJ_LIST_FILE) | xargs -n$(AR_CHUNK_SIZE) $(AR) $(ARFLAGS) $(LIBFLAME_A)
$(LINKER) $(SOFLAGS) $(LDFLAGS) -o $@ -Wl,-force_load,$(LIBFLAME_A)

out of source build is not supported?

if I build out of source

../libflame/configure --prefix=/usr ...

I get

cat: version: No such file or directory
...
cat: ./so_version: No such file or directory
cat: ./so_version: No such file or directory
...
../libflame/configure: line8496: build/update-version-file.sh: No such file or directory
...

etc.

Can I safely ignore these messages?

Macro Definition Conflict

here are errors:
In file included from TestLibflame.cpp:8:
/usr/include/c++/8/bits/sstream.tcc: In member function ‘virtual std::__cxx11::basic_stringbuf<_CharT, _Traits, _Alloc>::int_type std::__cxx11::basic_stringbuf<_CharT, _Traits, _Alloc>::overflow(std::__cxx11::basic_stringbuf<_CharT, _Traits, _Alloc>::int_type)’:
/libflame-master/include/FLAME.h:840:21: error: expected unqualified-id before ‘(’ token
#define max( x, y ) ( (x) > (y) ? (x) : (y) )
^
/libflame-master/include/FLAME.h:837:21: error: expected unqualified-id before ‘(’ token
#define min( x, y ) ( (x) < (y) ? (x) : (y) )
^
/usr/include/c++/8/complex: In function ‘_Tp std::__complex_abs(const std::complex<_Tp>&)’:
/libflame-master/include/FLAME.h:840:21: error: expected unqualified-id before ‘(’ token
#define max( x, y ) ( (x) > (y) ? (x) : (y) )
^
The error messages you've encountered indicate a macro definition conflict within the code you're trying to compile. This conflict arises when the same identifier is defined with different meanings in different parts of the code or in different libraries. In your case, the max and min macros defined in the FLAME.h file from the libflame library are conflicting with the standard C++ library functions std::max and std::min.
How can this issue be resolved?

Test any LAPACK compliant APIs

libFLAME has a test suite under "libflame/test" directory to validate libFLAME APIs. But this test suite is very libFLAME specific as it uses mainly FLA_* APIs.

I wanted to know if there is test suite in libFLAME that can work with any LAPACK compliant library apart from the netlib test which is in FORTRAN. I was looking something similar to what is in BLIS "test" that supports linking to any BLAS library.

Configure errors with OpenMP

Dear libflame developers,

I tried to compile libflame either master branch or v.5.2.0.
For compiling from the source code, I did:

./configure --prefix=/home/software/libflame --enable-multithreading=openmp

For installation with spack, I did:

spack install libflame threads=openmp

Both gave the same error:

checking whether user requested support for multithreading... yes
checking user-requested multithreading model... openmp
checking for (guessing) OpenMP flags for /home/software/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/gcc-9.2.0-wqdecm4rkyyhejagxwmnabt6lscgm45d/bin/gcc... unknown
configure: error: configure doesn't know what flag to give /home/software/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/gcc-9.2.0-wqdecm4rkyyhejagxwmnabt6lscgm45d/bin/gcc in order to enable OpenMP support. Please submit a bug report to the FLAME developers.

Do you have any suggestions?

Puzzles about using FLASH_LU_piv

Hi, I want to utilize the function 'FLASH_lu_piv' to perform a LU factorization. But I have met a NULL pointer error. This error could be my mistake for misunderstanding the usage of the function.

• Configuration

./configure --enable-dynamic-build --enable-external-lapack-for-subproblems --enable-external-lapack-interfaces --enable-multithreading=openmp --enable-supermatrix --enable-cblas-interfaces --enable-portable-timer 

• Code samples

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
#include <unistd.h>
#include <FLAME.h>

int main( void )
{
    // matrix parameters: A, column major
    int N;
    // test parameters
    int N_START, N_END, INC, REPEAT;

    printf("[INPUT]: input N_START, N_END, INC, REPEAT\n");
    if(scanf("%d %d %d %d", &N_START, &N_END, &INC, &REPEAT) == 4){
        printf("[TRUE]: true parameters for scanf\n");
    }
    else{
        fprintf(stderr, "[ERROR]: false parameters for scanf\n");
        exit(EXIT_FAILURE);
    }

    // Initialize libflame.
    FLA_Init();

    // N, REPEAT
    N = N_START;
    while (N <= N_END){
        for(int re_count = 0; re_count < REPEAT; ++ re_count){
                dim_t blocksize = 40, depth = 1;

                FLA_Obj IPIV;
                FLA_Obj_create( FLA_INT, N, 1, 0, 0, &IPIV);

                FLA_Obj A, A_FLASH;
                FLA_Obj_create( FLA_DOUBLE, N, N, 0, 0, &A );
                FLA_Random_matrix( A );

                int info;
                FLASH_Obj_create_hier_copy_of_flat(A, depth, &blocksize, &A_FLASH);
                info = FLASH_LU_piv(A_FLASH, IPIV);

                FLASH_Obj_free( &A_FLASH ); FLA_Obj_free( &IPIV );
            }
        N += INC;
    }
    
    FLA_Finalize();

    return 0;
}

• build

gcc -o flame.x flame.c -I /home/xx/lib/libflame/include/ /home/xx/project/libflame/lib/x86_64-unknown-linux-gnu/libflame.a /home/xx/lib/openblas/lib/libopenblas.a -fopenmp -lpthread -lm

• Error Information
  

Lapack dsyevr

Hello I would like to use the lapack method: dsyevr
In the flame implementation there is a method dsyevr_ which takes all arguments as pointers (unlike the original lapack method).
Is there a better C interface to use for EVD, QR, LU decomposition?
Many thanks

Configuration with upstream Fortran LAPACK ignores `F77` environmental variable

I am trying to compile libflame with clang/flang. I specify the compilers via environmental variables like this:

CC=mpicc F77=mpifort CFLAGS="-march=znver2 -O3 -DNDEBUG" \
./configure --enable-lapack2flame --enable-dynamic-build --disable-warnings --prefix=${BIN_DIR}/libflame-5.2.0

Compilation used to work just fine, but now I get the error that the command f77 was not found:

$ make
Generating monolithic include/x86_64-unknown-linux-gnu/FLAME.h......................................................................................................................................................................................................................................................................................................................
Generated monolithic include/x86_64-unknown-linux-gnu/FLAME.h
Generating monolithic include/x86_64-unknown-linux-gnu/blis1.h..................
Generated monolithic include/x86_64-unknown-linux-gnu/blis1.h
Generating monolithic include/x86_64-unknown-linux-gnu/FLA_f2c.h...
Generated monolithic include/x86_64-unknown-linux-gnu/FLA_f2c.h
Building src/map/common/lapacksrc/fortran/la_constants.f90
bash: f77: command not found
make: *** [Makefile:610: obj/x86_64-unknown-linux-gnu/src/la_constants.mod] Error 127

I ran a git bisect session and identified 74b56a2 (#89) as the first bad commit.

It seems like the F77 compiler specified via environmental variables is ignored when building the upstream versions of the LAPACK library. Can you confirm that?

In the meantime, adding the configuration option --enable-legacy-lapack works as a workaround.

Any up-to-date examples on basic operations such as GEMM?

The existing examples are so old and contains some un-expected library dependencies like in examples/Gemm folder.

Generate pkgconfig file at build time

It would be very useful to have a .pc file be generated when libflame is built, as was done for blis. Unfortunately, I'm not knowledgeable enough about pkgconfig to write one that could be considered reliable myself.

Unknown OpenMP flags for clang

Hey guys

I am trying to build libFLAME on our AMD Naples machines using the AOCC v.2.2.0 compilers (clang, clang++, flang). We run CentOS 7.8 on our compute nodes.

Here is the configure options that I pass:

# CFLAGS are set identical to OpenBLAS
cflags='-O3 -march=native -fopenmp=libiomp5 -mavx2 -fvectorize -mfma -m3dnow -floop-unswitch-aggressive'
fflags='-O3 -march=native -fopenmp=libiomp5 -mavx -frecursive'

./configure --prefix=$dir_install \
            --enable-max-arg-list-hack \
            --enable-static-build \
            --enable-dynamic-build \
            --enable-lapack2flame \
            --disable-external-lapack-for-subproblems \
            --disable-external-lapack-interfaces \
            --enable-multithreading=openmp \
            --enable-supermatrix \
            --enable-vector-intrinsics=sse \
            --enable-cblas-interfaces \
            --with-cc=clang \
            EXTRA_CFLAGS="$cflags" \
            F77=flang \
            FFLAGS="$fflags"

However, I get the following complaint over OpenMP flags:

checking user-requested multithreading model... openmp
checking for (guessing) OpenMP flags for clang... unknown
configure: error: configure doesn't know what flag to give clang in order to enable OpenMP support. Please submit a bug report to the FLAME developers.

I have played with the cflags and fflags variables above, e.g. with/without -fopenmp, or -fopenmp=libomp or -fopenmp=libiomp5; however, all have failed. Seems like the configure is not listening to the EXTRA_CLFAGS argument passed to it.

Do you have any idea how to fix this?

Regards
Ehsan

Please make official release

Having a release with an actual version number is 1. confidence inspiring 2. good for people who need to keep track of whether an installed version is up to date.

Crash in SVD computation

Hi,

There is a crash observed when libflame API DGESDD is invoked for the attached dataset. Size 143x143. The computation is resulting in denormal numbers and finally crash occurs.

dgesvd_143t.txt

The crash is in function FLA_Bsvd_ext_opd_var1() at the line FLA_Abort();

Also attached test code. It is stripped portion of a larger application code. Shows how inputs are passed
example_dgesddc.txt

make fail: argument list too long

Compiling src/lapack/util/app/qutinc/front/flamec/FLASH_Apply_Q_UT_inc.c
Compiling src/lapack/util/app/qutinc/front/flamec/FLASH_Apply_Q_UT_inc_create_workspace.c
Compiling src/lapack/util/app/qutinc/front/flamec/FLA_Apply_Q_UT_inc_internal.c
Compiling src/lapack/util/app/qutinc/front/flamec/FLA_Apply_Q_UT_inc_lhfc.c
Compiling src/lapack/util/app/qutinc/lnfc/flamec/FLA_Apply_Q_UT_inc_lnfc_blk_var1.c
Compiling src/lapack/util/app/qutinc/lhfc/flamec/FLA_Apply_Q_UT_inc_lhfc_blk_var1.c
Compiling src/lapack/util/app/qutinc/front/flamec/FLA_Apply_Q_UT_inc_lnfc.c
Archiving lib/x86_64-unknown-linux-gnu/libflame.a
make: execvp: /usr/bin/env: Argument list too long
make: *** [Makefile:542: lib/x86_64-unknown-linux-gnu/libflame.a] Error 127

what I did:

./configure
make -j12

misc info

libflame: master
Debian sid/unstable

Reference guide 403 forbidden error

I tried to access the reference guide and I got an error about not having sufficient permissions.

I was at this website: https://shpc.oden.utexas.edu/libFLAME.html

And tried to click on the link: https://www.cs.utexas.edu/~flame/web/libflame.pdf

And I got a 403 forbidden error.

Netlib Testing Failures

Testing libFLAME library on Netlib's LAPACK test suite (version 3.4.2 & 3.5.0) results in failures in QR decomposition tests.
The failures are observed in only single and double precision QR test cases and not in complex/double complex tests.
Single and double precision variant of QR has libFLAME's C implementation while the complex variants are taking the f2c converted C files.
Attached are the test logs with summary of different tests (libFLAME tested on LAPACK test suites and LAPACK tested on LAPACK test suite). The failure messages in the log files can be reached by searching for SQR or GQR which are the tags for those test cases.
Edit note: BLAS library used for the tests was the library obtained from building Netlib's LAPACK package.

libflame_testing_results_on_lapack3.4.2.txt
libflame_testing_results_on_lapack3.5.0.txt
lapack3.5.0_testing_results_on_lapack3.5.0.txt

Function or conditional branch/case not yet implemented.

I recently compiled nwchem using libflame in place of LAPACK. When trying to run a simple calculation, execution fails with

libflame: /home/amd/harish/rome2.2/aocl-rel/aocl22-rc4/final-packages/libflame/src/lapack/red/tridiagut/front/flamec/FLA_Tridiag_UT_internal.c (line 26):
libflame: Function or conditional branch/case not yet implemented.
libflame: Aborting.

Program received signal SIGABRT: Process abort signal.

Clearly the program is trying to do something not yet implemented in libflame. But my understanding is that libflame fully supports the LAPACK specification as of v2.1, so I'm a bit confused about how to interpret this message.

I compiled nwchem using the precompiled flame library downloaded from https://developer.amd.com/amd-aocl/. Is that library compiled with the lapack2flame compatibility option? If it isn't, could that explain why I am seeing this error?

Thanks for any assistance!

Unable to build on macOS

I've tried building both the 5.2.0 release as well as the master branch, but nothing will build for me on macOS 10.14.6 with Clang 10.0.1. Every time, I get the following error:

Archiving lib/x86_64-apple-darwin18.7.0/libflame.a
Dynamically linking lib/x86_64-apple-darwin18.7.0/libflame.so
ld: unknown option: -soname
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [lib/x86_64-apple-darwin18.7.0/libflame.so] Error 1

I'm building with the following configure options:

--enable-lapack2flame --enable-static-build --enable-dynamic-build --disable-debug --enable-multithreading=no --disable-supermatrix --enable-max-arg-list-hack

For more information, see:

• build log
• build env

I also saw #7 but this appears to be an unrelated issue.

Building on MacOS

I'm building on MacOS with gcc instead of clang. This requires forcing use of the clang integrated assembler.

CFLAGS += -Wa,-q

Missing power9 configuration

checking build system type... Invalid configuration `power9': machine `power9' not recognized
configure: error: /bin/sh build/ac-utils/config.sub power9 failed

missing symbols in static library

Hello,

I'm building libflame with -enable-max-arg-list-hack --enable-lapack2flame and it seems to be fine but then when I try to build scalapack with it, linking with the static libflame, building the tests/examples fail with

.../lib/libflame.a(FLA_hetrd.o):FLA_hetrd.c:function ssytrd_: error: undefined reference to 'ssytrd_check'
.../lib/libflame.a(FLA_hetrd.o):FLA_hetrd.c:function dsytrd_: error: undefined reference to 'dsytrd_check'
.../lib/libflame.a(FLA_hetrd.o):FLA_hetrd.c:function ssytd2_: error: undefined reference to 'ssytd2_check'
.../lib/libflame.a(FLA_hetrd.o):FLA_hetrd.c:function dsytd2_: error: undefined reference to 'dsytd2_check'

(I suppose this is simply the first case where it fails, all the _check symbols seem to be missing form the static library, from the output of nm)

If I build libflame with --enable-dynamic-build and build scalapack linking with the dynamic libflame, everything seems to work.

spack install fails while linking

Hi
I was trying to install liflame using spack on centos 7 using: spack install [email protected] %[email protected]. This failed while linking with the error:

3886 ld.gold: --push-state: unknown option
3887 ld.gold: use the --help option for usage information
3888 collect2: error: ld returned 1 exit status
3889 make[1]: *** [Makefile:12: generate] Error 1
3890 make[1]: Leaving directory '/scratch/gubmor00/spack-stage/spack-stage-amdlibflame-3.1blxdx37qerueegbcdtxr5l3qt3cyvwbg/spack-src/src/aocl_dtl'
3891 make: *** [Makefile:579: src/aocl_dtl/libaocldtl.a] Error 2
3892 make: *** Waiting for unfinished jobs....

I used GNU ld version 2.27-44.base.el7_9.1.
After upgrading to GNU ld 2.36.1 the problem went away. Is the problem caused by my initial linker being too old?

libflame and R

I am trying to replace standard lapack library in R by libflame to see if any gain can be obtained. Unfortunately, libflames.so fails to load with the following error message:

/usr/local/R-3.4.3_gcc/lib64/R/lib/libRlapack.so: undefined symbol: blas_zhemv2_x_

Don't be confused, here, libRlapack.so is just a symlink to libflame.so which was compiled with

./configure EXTRA_CFLAGS="-O3 -mtune=native -march=native" LDFLAGS=-L/opt/OpenBLAS/lib LIBS=-lopenblas --enable-dynamic-build --disable-static-build --enable-lapack2flame --enable-supermatrix --enable-memory-alignment=8 --enable-max-arg-list-hack

The lacking symbol blas_zhemv2_x_ is used in calls found in map/lapack2flamec/f2c/c/zla_herfsx_extended.c
and map/lapack2flamec/f2c/c/zla_porfsx_extended.c
which seems to correspond to xblas.
If xblas is not used, shouldn't these files be excluded from lapack mapping?

Accuracy issue in SVD API "SGESDD "

Hi,
We are observing few failures in one of our customer applications using libFLAME and BLIS for SVD API “SGESDD”. The outputs of singular values S and the Orthogonal matrix U are differing from expected output. The tests pass with OpenBLAS and MKL libraries' outputs for the same API.

Input Matrix A Size : 9 x 100
Input values: {1} -> All 1s
Parameters:
JobZ : ‘O’
M : 9
N : 100
LDA : 9
LDU : M
LDVT : 1

Outputs from libflame+BLIS

Singular values (S)
3.0000e+01
4.6447e-06
5.2638e-13
1.2358e-19
0.0000e+00
-0.0000e+00
-0.0000e+00
-0.0000e+00
-0.0000e+00

Orthogonal Matrix(U)
3.3333e-01 -9.4281e-01 -0.0000e+00 -0.0000e+00 0.0000e+00 0.0000e+00 0.0000e+00 0.0000e+00 0.0000e+00
3.3333e-01 1.1785e-01 -9.3541e-01 -1.2102e-07 7.0755e-15 0.0000e+00 0.0000e+00 0.0000e+00 0.0000e+00
3.3333e-01 1.1785e-01 1.3363e-01 -9.2582e-01 1.3064e-08 0.0000e+00 0.0000e+00 0.0000e+00 0.0000e+00
3.3333e-01 1.1785e-01 1.3363e-01 1.5430e-01 9.1287e-01 0.0000e+00 0.0000e+00 0.0000e+00 0.0000e+00
3.3333e-01 1.1785e-01 1.3363e-01 1.5430e-01 -1.8257e-01 4.4721e-01 4.4721e-01 4.4721e-01 4.4721e-01
3.3333e-01 1.1785e-01 1.3363e-01 1.5430e-01 -1.8257e-01 -8.6180e-01 1.3820e-01 1.3820e-01 1.3820e-01
3.3333e-01 1.1785e-01 1.3363e-01 1.5430e-01 -1.8257e-01 1.3820e-01 -8.6180e-01 1.3820e-01 1.3820e-01
3.3333e-01 1.1785e-01 1.3363e-01 1.5430e-01 -1.8257e-01 1.3820e-01 1.3820e-01 -8.6180e-01 1.3820e-01
3.3333e-01 1.1785e-01 1.3363e-01 1.5430e-01 -1.8257e-01 1.3820e-01 1.3820e-01 1.3820e-01 -8.6180e-01

Expected output:

Singular values(S)
3.0000e+01
4.4731e-06
2.8951e-12
3.2130e-18
2.3120e-24
1.2895e-30
1.3683e-36
1.6802e-42
2.9427e-44

Orthogonal Matrix(U)
-3.3333e-01 9.4281e-01 6.4572e-07 9.9341e-09 9.9341e-09 -1.9868e-08 -1.9868e-08 -1.9868e-08 0.0000e+00
-3.3333e-01 -1.1785e-01 9.3541e-01 -1.0867e-06 -7.6012e-09 -1.5052e-08 1.5202e-08 5.1177e-09 0.0000e+00
-3.3333e-01 -1.1785e-01 -1.3363e-01 9.2582e-01 7.9038e-07 -1.7868e-08 3.0130e-10 -2.3329e-09 0.0000e+00
-3.3333e-01 -1.1785e-01 -1.3363e-01 -1.5430e-01 9.1287e-01 -5.9798e-07 2.1286e-08 -1.2825e-08 0.0000e+00
-3.3333e-01 -1.1785e-01 -1.3363e-01 -1.5430e-01 -1.8257e-01 8.9443e-01 -1.0503e-06 -1.6157e-08 0.0000e+00
-3.3333e-01 -1.1785e-01 -1.3363e-01 -1.5430e-01 -1.8257e-01 -2.2361e-01 8.6603e-01 1.3324e-06 -2.2352e-08
-3.3333e-01 -1.1785e-01 -1.3363e-01 -1.5430e-01 -1.8257e-01 -2.2361e-01 -2.8868e-01 8.1635e-01 -1.5403e-02
-3.3333e-01 -1.1785e-01 -1.3363e-01 -1.5430e-01 -1.8257e-01 -2.2361e-01 -2.8867e-01 -4.2152e-01 -6.9928e-01
-3.3333e-01 -1.1785e-01 -1.3363e-01 -1.5430e-01 -1.8257e-01 -2.2361e-01 -2.8867e-01 -3.9484e-01 7.1468e-01

Any analysis or help regarding this will be highly appreciated.

Is there a progress function, e.g. for factorization functions?

It would be nice, especially for big problems, if there was a callback progress routine for the LAPACK functions (comparable to the mkl_progress function of the Intel MKL) to be able to inform the user.
If there is already such a function in the "depths of the code", I would be very grateful for an info.

undefined symbol: dpotrf_

Hi, I've ran the install of libflame and when I switch the lapack to libflame and run R, I get this message.

Error: package or namespace load failed for ‘stats’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/usr/lib/R/library/stats/libs/stats.so':
  /usr/lib/R/library/stats/libs/stats.so: undefined symbol: dpotrf_
During startup - Warning message:
package ‘stats’ in options("defaultPackages") was not found 

I've just downloaded the latest version from github. I also run into this error when I try to make with the standard config ./run-conf/run-configure.sh:
make: execvp: /usr/bin/env: Argument list too long

But when I adjust the run-configure.sh to include:
--enable-max-arg-list-hack \
I receive this error, when running configure
./run-conf/run-configure.sh: line 42: --enable-max-arg-list-hack: command not found

Invalid Makefile generated by configure

After a vanilla ./configure call with the current dev branch on a recent version of Ubuntu (15.04), running make immediately fails with the following output:
Makefile:394: *** recipe commences before first target. Stop.

The relevant line is:

       @echo "Archiving $@"

and it is surrounded by:

# --- Static library archiver rules for libflame ---
$(MK_ALL_FLAMEC_LIB): $(MK_ALL_FLAMEC_OBJS)
define EOL


endef
ifeq ($(FLA_ENABLE_VERBOSE_MAKE_OUTPUT),yes)
ifeq ($(FLA_ENABLE_MAX_ARG_LIST_HACK),yes)
        @$(eval ar_args:=)
        $(foreach obj,$?,$(eval ar_args+=$(obj))$(if $(word 2048,$(ar_args)),$(AR) $(ARFLAGS) $@ $(ar_args)$(EOL)$(eval ar_args:=)))
        $(AR) $(ARFLAGS) $@ $(ar_args)
#       echo $(ARFLAGS) $@ > $(AR_ARG_LIST_FILE)
#       $(CAT) $(AR_OBJ_LIST_FILE) >> $(AR_ARG_LIST_FILE)
#       $(AR) @$(AR_ARG_LIST_FILE)
        $(RANLIB) $@
else
        $(AR) $(ARFLAGS) $@ $?
        $(RANLIB) $@
endif
        mkdir -p include_local
        cp -f $(MK_HEADER_FILES) include_local
else
        @echo "Archiving $@"
ifeq ($(FLA_ENABLE_MAX_ARG_LIST_HACK),yes)
        @$(eval ar_args:=)
        @$(foreach obj,$?,$(eval ar_args+=$(obj))$(if $(word 2048,$(ar_args)),@$(AR) $(ARFLAGS) $@ $(ar_args)$(EOL)$(eval ar_args:=)))
        @$(AR) $(ARFLAGS) $@ $(ar_args)
#       @echo $(ARFLAGS) $@ > $(AR_ARG_LIST_FILE)
#       @$(CAT) $(AR_OBJ_LIST_FILE) >> $(AR_ARG_LIST_FILE)
#       @$(AR) @$(AR_ARG_LIST_FILE)
        @$(RANLIB) $@
else
        @$(AR) $(ARFLAGS) $@ $?
        @$(RANLIB) $@
endif
        @mkdir -p include_local
        @cp -f $(MK_HEADER_FILES) include_local
endif

My guess is that the define EOL portion might be causing Make to think that the command for the rule has ended.

linkage problem due to flag order in Makefile

For proper linkage of the dynamic libraries, LDFLAGS should be placed at the end when linking,
e.g.

libflame/Makefile

Line 588 in fcdb12b

@$(LINKER) $(SOFLAGS) $(LDFLAGS) -o $@ $^

should write

 @$(LINKER) $(SOFLAGS) -o $@ $^ $(LDFLAGS) 

and other lines whenever a linker is invoked.

Will `amd/libflame` merge to master?

As the only LAPACK implemention offered by AMD, will amd/libflame merge to master? Is there any plan?

Segmentation fault - invalid memory reference when running Netlib LAPACK 3.9.0 xlintsts

Hi!

I know that this is somewhat one the wrong side of the bleeding edge for libFLAME, but in the process of investigating BLIS+libFLAME as the main blas/lapack libs for EasyBuilds foss toolchain I got hit by this.

Building libFLAME through EasyBuild using the gobff/2020b toolchain.
I.e., GCC is 10.2.0

Building Netlib lapack tmglib (and ref lapack lib) and testing files with:
-O0 -frecursive -std=legacy -mieee-fp -fno-trapping-math -fno-math-errno -march=native
(so as to not introduce errors form the compiler in the test suite)

Linking with -lrefblas -lflame -lreflapack (since libflame doesn't contain all functions needed from lapack 3.9.0 yet)
Running xlintsts I get:

easybuild-kvm [TESTING]$ ./LIN/xlintsts < stest.in
 Tests of the REAL LAPACK routines 
 LAPACK VERSION 3.5.0

 The following parameter values will be used:
    M   :       0     1     2     3     5    10    50
    N   :       0     1     2     3     5    10    50
    NRHS:       1     2    15
    NB  :       1     3     3     3    20
    NX  :       1     0     5     9     1
    RANK:      30    50    90

 Routines pass computational tests if test ratio is less than   30.00

 Relative machine underflow is taken to be    0.117549E-37
 Relative machine overflow  is taken to be    0.340282E+39
 Relative machine precision is taken to be    0.596046E-07



Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0  0x7f0da67ec20f in ???
#1  0x420137 in ???
#2  0x7f0da6ed8a94 in ???
#3  0x7f0da6eb4d57 in ???
#4  0x4b3248 in ???
#5  0x42ac5c in ???
#6  0x422ab2 in ???
#7  0x4267b7 in ???
#8  0x7f0da67cd0b2 in ???
#9  0x40620d in ???
#10  0xffffffffffffffff in ???
Segmentation fault

confiugre/make installation directories directive and no effect?

should something like:
--prefix=/local.CPUnative --includedir=/local.CPUnative/include/libflame --libdir=/local.CPUnative/lib/libflame
be expected to make install to respective dirs? Because it has not effect.

make install also fails symlinking if PREFIX/inlude dir already exists.

System: centos 7.2 + devtools 4

Revision string unkown and problems compiling

First issue is that

> .\configure.cmd  x64 release cl
.\configure.cmd: Checking/updating revision file.
gen-check-rev-file.py: Found export. Checking for revision file...
gen-check-rev-file.py: Revision file found containing revision string "unknown". Export is valid snapshot!
.\configure.cmd: Gathering source files into local flat directories.
.\configure.cmd: Creating configure definitions file.
.\configure.cmd: Configuration and setup complete. You may now run nmake.

Ignore the revision string. I then tried to build, which errors....

> nmake

Microsoft (R) Program Maintenance Utility Version 14.00.24210.0
Copyright (C) Microsoft Corporation.  All rights reserved.

nmake: Creating .\obj\flamec\x64\debug directory
nmake: Compiling .\src\flamec\bl1_abort.c
nmake: Compiling .\src\flamec\bl1_allocm.c
nmake: Compiling .\src\flamec\bl1_allocv.c
nmake: Compiling .\src\flamec\bl1_amax.c
nmake: Compiling .\src\flamec\bl1_apdiagmv.c
nmake: Compiling .\src\flamec\bl1_asum.c
nmake: Compiling .\src\flamec\bl1_axmyv2.c
nmake: Compiling .\src\flamec\bl1_axpy.c
nmake: Compiling .\src\flamec\bl1_axpymrt.c
nmake: Compiling .\src\flamec\bl1_axpymt.c
nmake: Compiling .\src\flamec\bl1_axpysmt.c
nmake: Compiling .\src\flamec\bl1_axpysv.c
nmake: Compiling .\src\flamec\bl1_axpyv.c
nmake: Compiling .\src\flamec\bl1_axpyv2b.c
nmake: Compiling .\src\flamec\bl1_axpyv2bdotaxpy.c
NMAKE : fatal error U1077: 'cl.exe' : return code '0x2'
Stop.

Unable to configure vector intrinsic when using clang

configure doesn't know how to set sse flag for clang

checking whether user requested use of vector intrinsics... yes
checking user-requested vector intrinsic type... sse
checking for (guessing) SSE flags for /home/ryutas/local/bin/clang... unknown
configure: error: configure doesn't know what flag to give /home/ryutas/local/bin/clang in order to enable SSE. Please submit a bug report to the FLAME developers.

Seg Fault in test_libflame.x when using libblis as BLAS library with libflame

I am using fortran interfaces, I am checking Cholesky factorization performance. one thing I noticed when debugging seg fault in test_libflame.x application. The pointer to int (4bytes) is type-casted to f77_int* (int64_t* - 8 bytes) in blis library - This eventually results in crash. For more details:
check the function bl1_sscal(int n, float* alpha, float* x, int incx ) in bl1_scal.c file. Here the blis interface will be called through F77_sscal( &n, alpha, x, &incx). This function in blis library defined as:

void sscal( f77_int* n, float* alpha, float* x, ff7_int* incx) in file bla_scal.c Here f77_int is defined as int64_t. As a result the value of n will be a wrong value. This will result in a crash at times.

Please note BLIS_BLAS2BLIS_INT_TYPE_SIZE = 64.
Am I right here ?

Enabling build scripts for Windows

I've been working on "porting" the Windows specific logic in the build scripts from BLIS to the configure and Makefile to their equivalent in libflame (diffs are here: https://github.com/JuliaPackaging/Yggdrasil/pull/8671/files)

It's a work in progress (still requires testing the binaries), but was wondering if this would be of interest as a PR to upstream here? Would someone be willing to review it?

As an extra, there is also a fix for #100 when linking while building for macOS.

Please create a new official release

Hello, I spend a bit of time trying to figure out why libflame 5.2.0 did not build for me, only to realize I hit the bug where using absolute paths in CC=/absolute/path/to/gcc instead of CC=gcc would fail to guess the openmp flags.

This bug was fixed in 2019 1188aa4

Can you please create a new release so that no more developer time is wasted on this? It'd be great to work with tagged releases for reproducibility, etc.

Macros with general names in Flame.h break compilation

With a configuration like
./configure --enable-arg-max-hack --enable-lapack2flame
we currently get a lot of non-namespaced defines when including Flame.h in a source code

Specifically, the min define breaks compilation with GCC 10.2

 /sw/csi/gcc/10.2.0/include/c++/10.2.0/bits/istream.tcc: In member function ‘std::streamsize std::basic_istream<_CharT, _Traits>::readsome(std::basic_istream<_CharT, _Traits>::char_type*, std::streamsize)’:
/home/zampins/amd/src/local/include/FLAME.h:830:21: error: expected unqualified-id before ‘(’ token
  830 | #define min( x, y ) ( (x) < (y) ? (x) : (y) )

Here is a list of the most interesting ones. Is it possible to namespace them?

#undef  NULL
#define NULL 0

#undef  TRUE
#undef  FALSE
#define TRUE  1
#define FALSE 0

#undef min
#define min( x, y ) ( (x) < (y) ? (x) : (y) )

#undef max
#define max( x, y ) ( (x) > (y) ? (x) : (y) )

#undef signof
#define signof( a, b ) ( (b) >= 0 ? (a) : -(a) )

#undef exchange
#define exchange( a, b, temp ) { temp = a; a = b; b = temp; }
#define TRUE_ (1)
#define FALSE_ (0)

#ifndef f2c_abs
  #define f2c_abs(x) ((x) >= 0 ? (x) : -(x))
#endif
#ifndef f2c_dabs
  #define f2c_dabs(x) (doublereal)f2c_abs(x)
#endif
#ifndef min
  #define min(a,b) ((a) <= (b) ? (a) : (b))
#endif
#ifndef max
  #define max(a,b) ((a) >= (b) ? (a) : (b))
#endif
#ifndef dmin
  #define dmin(a,b) (doublereal)min(a,b)
#endif
#ifndef dmax
  #define dmax(a,b) (doublereal)max(a,b)
#endif

replacing cblas and clapack, need your suggestions!

Dear Field G. Van Zee:
What's the relationship between libflame and blis?
Does libflame covers all the operations in blis?
If I want to replace CBLAS and CLAPACK with libflame, what's your suggestions?

Thank you.
Best!
Feiteng

libflame.a.in: No such file or directory

CentOS-7.6
GCC-7.3
libflame-3.0.1 https://github.com/amd/libflame
./configure --prefix=/usr/local/libflame --enable-lapack2flame --enable-cblas-interfaces --enable-max-arg-list-hack
make -j
The error is as follows，What is the reason：
Compiling src/lapack/util/app/qutinc/lhfc/flamec/FLA_Apply_Q_UT_inc_lhfc_blk_var1.c
Compiling src/lapack/util/app/qutinc/lnfc/flamec/FLA_Apply_Q_UT_inc_lnfc_blk_var1.c
Archiving lib/x86_64-unknown-linux-gnu/libflame.a
ar: @lib/x86_64-unknown-linux-gnu/libflame.a.in: No such file or directory
make: *** [lib/x86_64-unknown-linux-gnu/libflame.a] Error 1

libflame LAPACK interface: multiple calls within a OMP do loop

Dear libflame developer,

I aim to compute a series of LU factorisation and solves of dense matricies (basically using dgetrf and dgetrs calls from LAPACK). A minimal example with a small matrix follows

program test
  
   use iso_fortran_env, only: dp => real64
   use iso_c_binding

   implicit none

   real(dp), allocatable :: a(:,:)
   integer(dp), allocatable :: piv(:)

   integer :: info, n, i, j

   n = 256
   allocate( a(n,n), piv(n))

   !-- Fill a matrix
   do i=1,n
      a(i,i)  = 2.0_dp
      if ( i < n ) a(i,i+1)= -1.0_dp
      if ( i > 1 ) a(i,i-1)= -1.0_dp
   end do


   !$omp parallel do
   do j=1, 1000
      !-- Lu factorisations
      call dgetrf(n,n, a(1:n,1:n), n, piv(1:n), info)
   end do
   !$omp end parallel do

   deallocate(a, piv)

end program test

This piece of code runs fine with LAPACK or the MKL. I also manage to run it fine with LIBFLAME but only when OMP_NUM_THREADS=1. As soon as OMP_NUM_THREADS > 1, the code crashes and yields

libflame: src/base/flamec/check/base/main/FLA_Obj_datatype_check.c (line 18):
libflame: Encountered NULL pointer.
libflame: Aborting.

I tried it on several platforms with sandybridge, haswell or zen2 processors and using the GNU or the Intel compilers but I couldn't make it work. Any idea what is causing this problem?

Best regards.

Build support for Cray programming environment

Hi, in order to build libflame with Cray's compiler wrappers (CC/cc/ftn or crayCC,craycc,crayftn) I need to apply the following patch to configure:

diff --git a/configure b/configure
index 33a67b14..43662526 100755
--- a/configure
+++ b/configure
@@ -3481,7 +3481,7 @@ ac_compiler_gnu=$ac_cv_c_compiler_gnu
 
 
 
-               CC_VENDOR=$(echo "$CC" | egrep -o 'icc|gcc|clang|emcc|pnacl|IBM' | { read first rest ; echo $first ; })
+               CC_VENDOR=$(echo "$CC" | egrep -o 'icc|gcc|clang|emcc|pnacl|IBM|cc|craycc' | { read first rest ; echo $first ; })
 
        if test "${CC_VENDOR}" = "" ; then
                as_fn_error $? "configure can't determine the compiler vendor for $CC. Please submit a bug report to the FLAME developers." "$LINENO" 5
@@ -6525,6 +6525,9 @@ $as_echo_n "checking for (guessing) OpenMP flags for ${CC_VENDOR}... " >&6; }
                                *xlc*)
                        fla_c_openmp_flags='-qsmp=omp'
                ;;
+                                cc)
+                        fla_c_openmp_flags='-fopenmp'
+                ;;
                                *)
                        fla_c_openmp_flags='unknown'
                ;;

would it be possible to make these changes upstream?

Multiple definition of lsame during compiling with shared version and internal blas

My system: Arch Linux 64 bit, latest updates, GCC 9.1.0. Configure line:

./configure --prefix=/usr --enable-static-build --enable-dynamic-build --enable-multithreading=openmp --enable-supermatrix --enable-max-arg-list-hack --enable-builtin-blas

Compiling with these settings (two processes for make step) gives me the following error at the very end:

Dynamically linking lib/x86_64-unknown-linux-gnu/libflame.so
/usr/bin/ld: obj/x86_64-unknown-linux-gnu/src/flablas/f2c/lsame.o: in function `lsame_':
lsame.c:(.text+0x0): multiple definition of `lsame_'; obj/x86_64-unknown-linux-gnu/src/map/lapack2flamec/f2c/install/static/lsame.o:lsame.c:(.text+0x0): first defined here

If I configure libflame without --enable-builtin-blas option then building works just fine and I get both static and dynamic libraries.

How it can be fixed?

Build error for master branch

Hi libflame developers,

I've made a spack package recipe for libflame : [1]. I've patched the makefile as described for the release version and I can build it.

But when I try to build from master branch I get the following error :

(ipy3) [sajid@xrmlite ~]$ spack install libflame@master%[email protected]
==> Installing libflame
==> Searching for binary cache of libflame
==> Warning: No Spack mirrors are currently configured
==> No binary for libflame found: installing from source
==> Cloning git repository: https://github.com/flame/libflame.git on branch master
==> No checksum needed when fetching with git
==> Already staged libflame-master-itdlxoywywuqko47ktvrhvmqmw3eoffc in /home/sajid/packages/spack/var/spack/stage/libflame-master-itdlxoywywuqko47ktvrhvmqmw3eoffc
==> No patches needed for libflame
==> Building libflame [AutotoolsPackage]
==> Executing phase: 'autoreconf'
==> Executing phase: 'configure'
==> Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
    'make' '-j16'

2 errors found in build log:
     123     configure: creating config.dist_path
     124     configure: determining default version string.
     125     configure: found '.git' directory; assuming git clone.
     126     configure: executing: git describe --tags.
     127     configure: git returned an error: 'fatal: No names found, cannot describe anything.'.
     128     configure: using string from unmodified version file.
  >> 129     cat: /home/sajid/packages/spack/var/spack/stage/libflame-master-itdlxoywywuqko47ktvrhvmqmw3eoffc/spack-src/version: No such file or directory
     130     configure: so major version: 1
     131     configure: so minor.build version: 0.0
     132     configure: creating ./config.status
     133     config.status: creating build/config.mk
     134     config.status: creating build/post-configure.sh
     135     config.status: creating build/FLA_config.h

     ...

     4867    Compiling /home/sajid/packages/spack/var/spack/stage/libflame-master-itdlxoywywuqko47ktvrhvmqmw3eoffc/spack-src/src/lapack/util/app/qutinc/front/flamec/FLA_Appl
             y_Q_UT_inc_lhfc.c
     4868    Compiling /home/sajid/packages/spack/var/spack/stage/libflame-master-itdlxoywywuqko47ktvrhvmqmw3eoffc/spack-src/src/lapack/util/app/qutinc/front/flamec/FLA_Appl
             y_Q_UT_inc_lnfc.c
     4869    Compiling /home/sajid/packages/spack/var/spack/stage/libflame-master-itdlxoywywuqko47ktvrhvmqmw3eoffc/spack-src/src/lapack/util/app/qutinc/lnfc/flamec/FLA_Apply
             _Q_UT_inc_lnfc_blk_var1.c
     4870    Compiling /home/sajid/packages/spack/var/spack/stage/libflame-master-itdlxoywywuqko47ktvrhvmqmw3eoffc/spack-src/src/lapack/util/app/qutinc/lhfc/flamec/FLA_Apply
             _Q_UT_inc_lhfc_blk_var1.c
     4871    Archiving lib/x86_64-unknown-linux-gnu/libflame.a
     4872    make: execvp: /usr/bin/env: Argument list too long
  >> 4873    make: *** [lib/x86_64-unknown-linux-gnu/libflame.a] Error 127

See build log for details:
  /home/sajid/packages/spack/var/spack/stage/libflame-master-itdlxoywywuqko47ktvrhvmqmw3eoffc/spack-build-out.txt

Relevant logs are available here : [2]

Undefined symbols in shared library

When I try to build either the 5.2.0 release or the master branch on CentOS 7 with either GCC 7.2.0 or Intel 18.0.1, the installation completes, but the resulting library is missing dozens of symbols. The library is also not executable.

$ ldd -r /home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so
ldd: warning: you do not have execution permission for `/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so'
	linux-vdso.so.1 =>  (0x00007ffce2388000)
	libgfortran.so.4 => /usr/local/gcc/7.2.0/lib64/libgfortran.so.4 (0x00002b75731b1000)
	libm.so.6 => /lib64/libm.so.6 (0x00002b7573583000)
	libquadmath.so.0 => /usr/local/gcc/7.2.0/lib64/libquadmath.so.0 (0x00002b7573885000)
	libc.so.6 => /lib64/libc.so.6 (0x00002b7573ac4000)
	libgcc_s.so.1 => /usr/local/gcc/7.2.0/lib/../lib64/libgcc_s.so.1 (0x00002b7573e91000)
	/lib64/ld-linux-x86-64.so.2 (0x00002b7572088000)
undefined symbol: cgemv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: cscal_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: sdot_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dtrmm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zhpr_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: snrm2_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zherk_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: sspmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: cswap_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dswap_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dgemm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ztrmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zgbmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ctbsv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dger_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dtpsv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: sspr_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: cher_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: caxpy_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ztpsv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zdrot_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zscal_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: sswap_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ctpsv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dzasum_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: daxpy_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ctrsv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: izamax_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: sasum_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dsymm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zhpr2_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: stpsv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ssbmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: cgbmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zhbmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: cgemm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dtpmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: chemv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ssyr2_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ctbmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: saxpy_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: drotm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dnrm2_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ctrmm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ztbsv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dtrsv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zdotu_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dsyr_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: srot_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ccopy_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zgeru_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: csrot_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zhpmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: csyr2k_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ctrsm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: sscal_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: cgerc_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dcopy_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: scnrm2_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zgemm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: sgbmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: sspr2_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ztrsv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: cgeru_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dgbmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dscal_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: chpr_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zdscal_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: strmm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ssymm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zgemv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ddot_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: isamax_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zsyrk_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: stbsv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: strsv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: scasum_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: sgemm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: scopy_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zdotc_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dsyr2k_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dsyrk_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: csscal_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dasum_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ssyr_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ssyr2k_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dspr2_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zher2_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dtrsm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: idamax_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dtbsv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ctrmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: strmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zaxpy_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zher_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ssymv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zhemm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dtrmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zgerc_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: chemm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zher2k_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ztpmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zswap_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dspmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zsymm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ztrmm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dtbmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dgemv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: sgemv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: chpr2_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: cdotu_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ctpmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: cher2k_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: csymm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ztrsm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: cherk_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: cdotc_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dsbmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: csyrk_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: stpmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: chpmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: sger_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: cher2_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zsyr2k_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: strsm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dznrm2_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: stbmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: chbmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dsyr2_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zcopy_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dspr_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: icamax_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: dsymv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: zhemv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ssyrk_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: srotm_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: drot_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)
undefined symbol: ztbmv_	(/home/adamjs5/spack/opt/spack/linux-centos7-x86_64/gcc-7.2.0/libflame-master-jnhuuve5add5y7ion655bizjmr3s6jui/lib/libflame.so)

I'm passing the following options to configure:

--enable-lapack2flame --enable-static-build --enable-dynamic-build --disable-debug --enable-multithreading=no --disable-supermatrix --enable-max-arg-list-hack

For additional info, see:

• build log
• build env

Build error for release with default gcc on CentOS

Hi libflame developers,

I'm seeing the following error when I attempt to build the 5.1.0 release with gcc-4.8.5 on centOS 7.5.

[sajid@xrmlite spack]$ spack install [email protected] %[email protected]                                                                                        
==> Installing libflame                                                                                                                               
==> Searching for binary cache of libflame                                                                                                            
==> Warning: No Spack mirrors are currently configured                                                                                                
==> No binary for libflame found: installing from source                                                                                              
==> Fetching https://github.com/flame/libflame/archive/5.1.0.tar.gz                                                                                   
######################################################################## 100.0%                                                                       
==> Staging archive: /home/sajid/packages/spack/var/spack/stage/libflame-5.1.0-d7l3mqr355nxmvshee5ngl5g3jvi2jc7/5.1.0.tar.gz                          
==> Created stage in /home/sajid/packages/spack/var/spack/stage/libflame-5.1.0-d7l3mqr355nxmvshee5ngl5g3jvi2jc7                                       
==> Applied patch /home/sajid/packages/spack/var/spack/repos/builtin/packages/libflame/Makefile_5.1.0.patch                                           
==> Building libflame [AutotoolsPackage]                                                                                                              
==> Executing phase: 'autoreconf'                                                                                                                     
==> Executing phase: 'configure'                                                                                                                      
==> Executing phase: 'build'                                                                                                                          
==> Error: ProcessError: Command exited with status 2:                                                                                                
    'make' '-j16'                                                                                                                                     
                                                                                                                                                      
3 errors found in build log:                                                                                                                          
  >> 4       cat: revision: No such file or directory                                                                                                 
     5       checking build system type... x86_64-unknown-linux-gnu                                                                                   
     6       checking host system type... x86_64-unknown-linux-gnu                                                                                    
     7       checking for GNU make... make                                                                                                            
     8       checking for GNU bash... bash                                                                                                            
     9       checking whether user requested a specific C compiler... no                                                                              
     10      configure: CC environment variable is set to /home/sajid/packages/                                                                       
             spack/lib/spack/env/gcc/gcc, which will override --with-cc option                                                                        
             and default search list for C compiler.                                                                                                  
                                                                                                                                                      
     ...                                                                                                                                              
                                                                                                                                                      
     5599    Compiling src/lapack/util/app/qutinc/front/flamec/FLA_Apply_Q_UT_i                                                                       
             nc_lhfc.c                                                                                                                                
     5600    Compiling src/lapack/util/app/qutinc/front/flamec/FLA_Apply_Q_UT_i                                                                       
             nc_lnfc.c                                                                                                                                
     5601    Compiling src/lapack/util/app/qutinc/lhfc/flamec/FLA_Apply_Q_UT_in                                                                       
             c_lhfc_blk_var1.c                                                                                                                        
     5602    Compiling src/lapack/util/app/qutinc/lnfc/flamec/FLA_Apply_Q_UT_in                                                                       
             c_lnfc_blk_var1.c                                                                                                                        
     5603    Archiving lib/x86_64-unknown-linux-gnu/libflame.a                                                                                        
     5604    Dynamically linking lib/x86_64-unknown-linux-gnu/libflame.so                                                                             
  >> 5605    ar: @@ar: No such file or directory                                                                                                      
  >> 5606    make: *** [lib/x86_64-unknown-linux-gnu/libflame.a] Error 1                                                                              
     5607    make: *** Waiting for unfinished jobs....                                                                                                
                                                                                                                                                      
See build log for details:                                                                                                                            
  /home/sajid/packages/spack/var/spack/stage/libflame-5.1.0-d7l3mqr355nxmvshee5ngl5g3jvi2jc7/spack-build-out.txt                                      

The build recipe I'm using is here 1.

Linking error on macOS 14 / arm64

Archiving lib/arm-apple-darwin23.2.0/libflame.a
Dynamically linking lib/arm-apple-darwin23.2.0/libflame.dylib
ld: Undefined symbols:
  _caxpy_, referenced from:
      _bl1_caxpy in libflame.a[4](bl1_axpy.o)
  _ccopy_, referenced from:
      _bl1_ccopy in libflame.a[13](bl1_copy.o)
  _cgemm_, referenced from:
      _bl1_cgemm_blas in libflame.a[44](bl1_gemm.o)
  _cgemv_, referenced from:
      _bl1_cgemv_blas in libflame.a[32](bl1_gemv.o)
  _cgerc_, referenced from:
      _bl1_cger in libflame.a[33](bl1_ger.o)
      _bl1_cgerc_blas in libflame.a[33](bl1_ger.o)
  _cgeru_, referenced from:
      _bl1_cger in libflame.a[33](bl1_ger.o)
      _bl1_cgeru_blas in libflame.a[33](bl1_ger.o)
  _chemm_, referenced from:
      _bl1_chemm_blas in libflame.a[45](bl1_hemm.o)
  _chemv_, referenced from:
      _bl1_chemv_blas in libflame.a[34](bl1_hemv.o)
  _cher2_, referenced from:
      _bl1_cher2_blas in libflame.a[36](bl1_her2.o)
  _cher2k_, referenced from:
      _bl1_cher2k_blas in libflame.a[46](bl1_her2k.o)
  _cher_, referenced from:
      _bl1_cher_blas in libflame.a[35](bl1_her.o)
  _cherk_, referenced from:
      _bl1_cherk_blas in libflame.a[47](bl1_herk.o)
  _cscal_, referenced from:
      _bl1_cscal in libflame.a[25](bl1_scal.o)
  _csscal_, referenced from:
      _bl1_csscal in libflame.a[25](bl1_scal.o)
  _cswap_, referenced from:
      _bl1_cswap in libflame.a[29](bl1_swap.o)
  _csymm_, referenced from:
      _bl1_csymv_blas in libflame.a[37](bl1_symv.o)
      _bl1_csymm_blas in libflame.a[48](bl1_symm.o)
  _csyr2k_, referenced from:
      _bl1_csyr2_blas in libflame.a[39](bl1_syr2.o)
      _bl1_csyr2k_blas in libflame.a[49](bl1_syr2k.o)
  _csyrk_, referenced from:
      _bl1_csyr_blas in libflame.a[38](bl1_syr.o)
      _bl1_csyrk_blas in libflame.a[50](bl1_syrk.o)
  _ctrmm_, referenced from:
      _bl1_ctrmm_blas in libflame.a[51](bl1_trmm.o)
  _ctrmv_, referenced from:
      _bl1_ctrmv_blas in libflame.a[40](bl1_trmv.o)
  _ctrsm_, referenced from:
      _bl1_ctrsm_blas in libflame.a[53](bl1_trsm.o)
  _ctrsv_, referenced from:
      _bl1_ctrsv_blas in libflame.a[42](bl1_trsv.o)
  _dasum_, referenced from:
      _bl1_dasum in libflame.a[3](bl1_asum.o)
  _daxpy_, referenced from:
      _bl1_daxpy in libflame.a[4](bl1_axpy.o)
  _dcopy_, referenced from:
      _bl1_dcopy in libflame.a[13](bl1_copy.o)
  _ddot_, referenced from:
      _bl1_ddot in libflame.a[18](bl1_dot.o)
  _dgemm_, referenced from:
      _bl1_dgemm_blas in libflame.a[44](bl1_gemm.o)
  _dgemv_, referenced from:
      _bl1_dgemv_blas in libflame.a[32](bl1_gemv.o)
  _dger_, referenced from:
      _bl1_dger in libflame.a[33](bl1_ger.o)
      _bl1_dger_blas in libflame.a[33](bl1_ger.o)
  _dnrm2_, referenced from:
      _bl1_dnrm2 in libflame.a[24](bl1_nrm2.o)
  _dscal_, referenced from:
      _bl1_dscal in libflame.a[25](bl1_scal.o)
  _dswap_, referenced from:
      _bl1_dswap in libflame.a[29](bl1_swap.o)
  _dsymm_, referenced from:
      _bl1_dsymm_blas in libflame.a[48](bl1_symm.o)
  _dsymv_, referenced from:
      _bl1_dsymv_blas in libflame.a[37](bl1_symv.o)
  _dsyr2_, referenced from:
      _bl1_dsyr2_blas in libflame.a[39](bl1_syr2.o)
  _dsyr2k_, referenced from:
      _bl1_dsyr2k_blas in libflame.a[49](bl1_syr2k.o)
  _dsyr_, referenced from:
      _bl1_dsyr_blas in libflame.a[38](bl1_syr.o)
  _dsyrk_, referenced from:
      _bl1_dsyrk_blas in libflame.a[50](bl1_syrk.o)
  _dtrmm_, referenced from:
      _bl1_dtrmm_blas in libflame.a[51](bl1_trmm.o)
  _dtrmv_, referenced from:
      _bl1_dtrmv_blas in libflame.a[40](bl1_trmv.o)
  _dtrsm_, referenced from:
      _bl1_dtrsm_blas in libflame.a[53](bl1_trsm.o)
  _dtrsv_, referenced from:
      _bl1_dtrsv_blas in libflame.a[42](bl1_trsv.o)
  _dzasum_, referenced from:
      _bl1_zasum in libflame.a[3](bl1_asum.o)
  _dznrm2_, referenced from:
      _bl1_znrm2 in libflame.a[24](bl1_nrm2.o)
  _icamax_, referenced from:
      _bl1_camax in libflame.a[2](bl1_amax.o)
  _idamax_, referenced from:
      _bl1_damax in libflame.a[2](bl1_amax.o)
  _isamax_, referenced from:
      _bl1_samax in libflame.a[2](bl1_amax.o)
  _izamax_, referenced from:
      _bl1_zamax in libflame.a[2](bl1_amax.o)
  _sasum_, referenced from:
      _bl1_sasum in libflame.a[3](bl1_asum.o)
  _saxpy_, referenced from:
      _bl1_saxpy in libflame.a[4](bl1_axpy.o)
  _scasum_, referenced from:
      _bl1_casum in libflame.a[3](bl1_asum.o)
  _scnrm2_, referenced from:
      _bl1_cnrm2 in libflame.a[24](bl1_nrm2.o)
  _scopy_, referenced from:
      _bl1_scopy in libflame.a[13](bl1_copy.o)
  _sdot_, referenced from:
      _bl1_sdot in libflame.a[18](bl1_dot.o)
  _sgemm_, referenced from:
      _bl1_sgemm_blas in libflame.a[44](bl1_gemm.o)
  _sgemv_, referenced from:
      _bl1_sgemv_blas in libflame.a[32](bl1_gemv.o)
  _sger_, referenced from:
      _bl1_sger in libflame.a[33](bl1_ger.o)
      _bl1_sger_blas in libflame.a[33](bl1_ger.o)
  _snrm2_, referenced from:
      _bl1_snrm2 in libflame.a[24](bl1_nrm2.o)
  _sscal_, referenced from:
      _bl1_sscal in libflame.a[25](bl1_scal.o)
  _sswap_, referenced from:
      _bl1_sswap in libflame.a[29](bl1_swap.o)
  _ssymm_, referenced from:
      _bl1_ssymm_blas in libflame.a[48](bl1_symm.o)
  _ssymv_, referenced from:
      _bl1_ssymv_blas in libflame.a[37](bl1_symv.o)
  _ssyr2_, referenced from:
      _bl1_ssyr2_blas in libflame.a[39](bl1_syr2.o)
  _ssyr2k_, referenced from:
      _bl1_ssyr2k_blas in libflame.a[49](bl1_syr2k.o)
  _ssyr_, referenced from:
      _bl1_ssyr_blas in libflame.a[38](bl1_syr.o)
  _ssyrk_, referenced from:
      _bl1_ssyrk_blas in libflame.a[50](bl1_syrk.o)
  _strmm_, referenced from:
      _bl1_strmm_blas in libflame.a[51](bl1_trmm.o)
  _strmv_, referenced from:
      _bl1_strmv_blas in libflame.a[40](bl1_trmv.o)
  _strsm_, referenced from:
      _bl1_strsm_blas in libflame.a[53](bl1_trsm.o)
  _strsv_, referenced from:
      _bl1_strsv_blas in libflame.a[42](bl1_trsv.o)
  _zaxpy_, referenced from:
      _bl1_zaxpy in libflame.a[4](bl1_axpy.o)
  _zcopy_, referenced from:
      _bl1_zcopy in libflame.a[13](bl1_copy.o)
  _zdscal_, referenced from:
      _bl1_zdscal in libflame.a[25](bl1_scal.o)
  _zgemm_, referenced from:
      _bl1_zgemm_blas in libflame.a[44](bl1_gemm.o)
  _zgemv_, referenced from:
      _bl1_zgemv_blas in libflame.a[32](bl1_gemv.o)
  _zgerc_, referenced from:
      _bl1_zger in libflame.a[33](bl1_ger.o)
      _bl1_zgerc_blas in libflame.a[33](bl1_ger.o)
  _zgeru_, referenced from:
      _bl1_zger in libflame.a[33](bl1_ger.o)
      _bl1_zgeru_blas in libflame.a[33](bl1_ger.o)
  _zhemm_, referenced from:
      _bl1_zhemm_blas in libflame.a[45](bl1_hemm.o)
  _zhemv_, referenced from:
      _bl1_zhemv_blas in libflame.a[34](bl1_hemv.o)
  _zher2_, referenced from:
      _bl1_zher2_blas in libflame.a[36](bl1_her2.o)
  _zher2k_, referenced from:
      _bl1_zher2k_blas in libflame.a[46](bl1_her2k.o)
  _zher_, referenced from:
      _bl1_zher_blas in libflame.a[35](bl1_her.o)
  _zherk_, referenced from:
      _bl1_zherk_blas in libflame.a[47](bl1_herk.o)
  _zscal_, referenced from:
      _bl1_zscal in libflame.a[25](bl1_scal.o)
  _zswap_, referenced from:
      _bl1_zswap in libflame.a[29](bl1_swap.o)
  _zsymm_, referenced from:
      _bl1_zsymv_blas in libflame.a[37](bl1_symv.o)
      _bl1_zsymm_blas in libflame.a[48](bl1_symm.o)
  _zsyr2k_, referenced from:
      _bl1_zsyr2_blas in libflame.a[39](bl1_syr2.o)
      _bl1_zsyr2k_blas in libflame.a[49](bl1_syr2k.o)
  _zsyrk_, referenced from:
      _bl1_zsyr_blas in libflame.a[38](bl1_syr.o)
      _bl1_zsyrk_blas in libflame.a[50](bl1_syrk.o)
  _ztrmm_, referenced from:
      _bl1_ztrmm_blas in libflame.a[51](bl1_trmm.o)
  _ztrmv_, referenced from:
      _bl1_ztrmv_blas in libflame.a[40](bl1_trmv.o)
  _ztrsm_, referenced from:
      _bl1_ztrsm_blas in libflame.a[53](bl1_trsm.o)
  _ztrsv_, referenced from:
      _bl1_ztrsv_blas in libflame.a[42](bl1_trsv.o)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [lib/arm-apple-darwin23.2.0/libflame.dylib] Error 1

Some linker flags are missing?

P. S. Here is the config:

libflame configuration summary

distribution path............................... : .

library version................................. : 5.2.0

shared object (so) version (major).............. : 1
shared object (so) version (minor.build)........ : 0.0

build system type............................... : arm-apple-darwin23.2.0
host system type................................ : arm-apple-darwin23.2.0

OS name......................................... : Darwin
OS version...................................... : 23.2.0

Enable verbose make output...................... : no

Enable maximum argument list hack............... : yes

C compiler...................................... : /usr/bin/clang
C compiler vendor............................... : clang
Library archiver................................ : ar
Library archive indexer......................... : ranlib
Python interpreter.............................. : python3

Enable Windows build (experimental)............. : no

Create static library........................... : yes
Create shared (dynamically-linked) library...... : yes

Enable non-critical code........................ : yes

Enable built-in BLAS implementation............. : no

Enable lapack2flame............................. : no
Enable lapack2flash (experimental).............. : no
Enable legacy lapack............................ : no
Enable external LAPACK for subproblems.......... : no
Enable external LAPACK interfaces............... : no

Enable multithreading support................... : yes
   Threading implementation..................... : POSIX threads

Enable SuperMatrix support...................... : no

Enable GPU support.............................. : no

Enable HIP support.............................. : no

Enable SCC support.............................. : no

Enable support for Texas Instruments' DSP....... : no

Enable vector intrinsics........................ : no

Enable memory alignment......................... : no

C compiler language flags....................... : 

Enable compiler optimizations................... : yes
   C compiler optimization flags................ : 
Enable compiler warnings........................ : yes
   C compiler warning flags..................... : 
Enable compiler debugging symbols............... : no
   C compiler debug flags....................... : 
Enable compiler profiling symbols............... : no
   C compiler profiling flags................... : 
Enable LTO...................................... : no

User-specified CFLAGS (prepended)............... : -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -arch arm64

Enable internal error checking.................. : yes
   Internal error checking level................ : full

Enable memory leak counter...................... : no

Enable level-3 BLAS front-end control trees..... : no

Enable BLIS use of FLA_malloc()................. : yes

Enable interfaces to internal libgoto symbols... : no

Enable interfaces to CBLAS...................... : no

Enable user-defined default m blocksize......... : no
Enable user-defined default k blocksize......... : no
Enable user-defined default n blocksize......... : no

Enable portable timer........................... : yes
   Portable timer function...................... : clock_gettime()

Autodetect Fortran linker flags................. : yes
   Fortran linker flags......................... :  -L/opt/local/lib -L/opt/local/lib/gcc13/gcc/arm64-apple-darwin23/13.2.0 -L/opt/local/lib/gcc13/gcc/arm64-apple-darwin23/13.2.0/../../.. -lgfortran -lemutls_w -lquadmath
Autodetect Fortran name-mangling................ : yes
   Unmangled name............................... : foobar
   Mangled name................................. : foobar_

Installation directories
   prefix....................................... : /opt/local
   exec_prefix.................................. : ${prefix}

   libraries will be installed in............... : ${exec_prefix}/lib
   header files will be installed in............ : ${prefix}/include

Configuration complete!