perazz / fortran-lapack

Modularized Fortran LAPACK implementation

Combustion researcher, Software Developer and Consultant at WERC

With 15+ years experience working with legendary professor Rolf Reitz on:

• Accurate, fast combustion models for DI, SI and Multi-Fuel engines
• The fastest chemical kinetics simulations for CFD
• Tailored, state-of-the-art CFD with combustion, sprays, turbulence, real-gas/multiphase EoS
• fast parallel codes with MPI/OpenCL
• Working on FRESCO, my Combustion Fluid Dynamics code

Help me out speeding up the clean energy transition, get in touch!

fortran-lapack's Issues

API style definition

See https://fortran-lang.discourse.group/t/linear-algebra-api-call-for-feedback/7455/3

Ideally, we would like for each API operation:

• a subroutine version, that performs no internal allocations, and intent(inout) arguments where necessary
• a function version, that has all intent(in) arguments except for the error handler
• an overloaded operator(.xyz.), where applicable, that has only intent(in) arguments

The function and operator versions should be made pure where possible

Add comments explaining what BLAS routines do

For example add a comment

! dasum takes the sum of the absolute values.

taken from https://netlib.org/lapack/explore-html/de/d05/dasum_8f_source.html

to

     real(dp) function stdlib_dasum(n,dx,incx)
     
        ! -- reference blas level1 routine --
        ! -- reference blas is a software package provided by univ. of tennessee,    --
        ! -- univ. of california berkeley, univ. of colorado denver and nag ltd..--
     
           ! .. scalar arguments ..
           integer(int32) incx,n
           ! ..
           ! .. array arguments ..
           real(dp) dx(*)
           ! ..
     
        ! =====================================================================
     
           ! .. local scalars ..
           real(dp) dtemp
           integer(int32) i,m,mp1,nincx
           ! ..
           ! .. intrinsic functions ..
           intrinsic dabs,mod

Assumed shape vs contiguous arrays

We already discussed that on Discourse.

BLAS/LAPACK routines mostly use assumed size arrays, thus contiguous arrays, but can handle non contiguous data by using the inc*/ld* arguments.

In the modern wrapper (e.g. solve) you are developing, you are defining assumed-shape arguments, which is a good thing.

The problem is that compilers may/will generate copy-in/copy-out in the underlying calls to the LAPACK routines.

subroutine brand_new(A)
   real, intent(inout) :: A(:,:)
   interface
      subroutine old_timer(n,A,lda)
         integer n, lda
         real A(lda,*)
      end subroutine old_timer
   end interface
   n = size(A,1)
   call old_timer(n,A,n)   ! possible copy-in/copy-out at this point
end subroutine brand_new

How can we avoid that? We talked about inquiring the strides, using the C interoperability and the C array descriptors. So it could look like:

subroutine brand_new(A)
   real, intent(inout) :: A(:,:)
   interface
      subroutine old_timer(n,A,lda)
         integer n, lda
         real A(lda,*)
      end subroutine old_timer
   end interface
   n = size(A,1)
   inc = stride(A,1)   ! C routine behind
   lda = stride(A,2)   ! C routine behind
   if (inc == 1) then
      call old_timer(n,A(1,1),lda)   ! not nice, but supposed to work and avoid copy-in/copy-out
   else
      call old_timer(n,A,n)   ! copy-in/copy-out cannot be avoided in this case
   end if
end subroutine brand_new

What do you think ?

String issues

1. Strings in the f77 source code are all capitalized.
   In most of the code, string comparisons are done with lsame which is case insensitive.
   However, in some places (e.g., ilaenv), capitalized string literals are compared.

So it is necessary that the uppercase/lowercase state of string constants in the original code is not altered by the modernization script.

2. Function names in calls to ilaenv and other environment routines should not be prefixed with stdlib_*:

mnthr = stdlib_ilaenv(6,'stdlib_dgelsd',' ',m,n,nrhs,-1)

should become

mnthr = stdlib_ilaenv(6,'DGELSD',' ',m,n,nrhs,-1)

Matrix inverse (general matrix)

Implement inverse matrix interface

• 3 real and 3 complex kinds
• in-place inversion via call invert(a)
• functional interface via aa = inv(a)
• implement real tests with eye
• complex tests: invert diagonal matrix
• operator .inv.

Diagonal matrix inputs

Provide functions for diagonal and identity matrices.

References:

• Numpy: eye(N, M=None, k=0, dtype=<class 'float'>, order='C', *, device=None, like=None)
• Numpy: identity(n, dtype=None, *, like=None) --> square matrices only
• Scipy: eye(m, n=None, k=0, dtype=<class 'float'>, format=None) --> sparse only
• IMSL: EYE(N)

Error: Procedure 'select' called with an implicit interface

Running fpm build I eventually get

94466 |                  bwork( i ) = select( wr( i ), wi( i ) )
      |                              1
Error: Procedure 'select' called with an implicit interface at (1) [-Werror=implicit-interface]
compilation terminated due to -fmax-errors=1.
<ERROR> Compilation failed for object " src_stdlib_linalg_lapack_s.f90.o "
<ERROR> stopping due to failed compilation
STOP 1

Condition number and rank of a matrix

Compute the condition number of a matrix

`solve` non-square matrices with least squares

solve should also work for non-square matrices (e.g. Matlab).
It should return the least-squares solution.

Define constants at module level

At https://github.com/perazz/fortran-lapack/blob/main/src/stdlib_linalg_blas_d.f90 the lines of code

           real(dp) :: one, zero
           parameter(one=1.0_dp, zero=0.0_dp)

appear in many procedures. Instead define private named constants

real(dp), parameter :: one=1.0_dp, zero=0.0_dp

before the contains statement. In a few places

real(dp), parameter :: dzero = 0.0_dp

appears. I think dzero can be replaced with zero once it is made a module-level constant.

High-level API: provide status / error handling

Define a derived type for linalg state/error handling that:

• fast object creation -> fixed size (no allocatables or derived types contained)
• Conditional flow control (halt on error vs. return a state object)
• Easy setup (accepts variable arguments)

Provide KIND-agnostic LAPACK interface

• Similar to #17
• Special care to mixed-precision LAPACK routines (decide an interface name; ensure correctness)

Fix LAPACK interface

Some routines of the LAPACK interface did not capture array arguments properly.

E.g., cbbcsd:

fortran-lapack/src/stdlib_linalg_lapack.f90

Line 45 in 5cdb311

complex(sp) :: u1,ldu1,u2,ldu2,v1t,ldv1t,v2t,ldv2t

Remove gotos when possible

I think stdlib_scsum1 in stdlib_linalg_lapack_s.f90 can be rewritten without gotos as

     pure real(sp) function stdlib_scsum1(n,cx,incx)
        ! -- lapack auxiliary routine --
        ! -- lapack is a software package provided by univ. of tennessee,    --
        ! -- univ. of california berkeley, univ. of colorado denver and nag ltd..--
           ! Scalar Arguments
           integer(ilp),intent(in) :: incx,n
           ! Array Arguments
           complex(sp),intent(in) :: cx(*)
        ! =====================================================================
           ! Local Scalars
           integer(ilp) :: i,nincx
           real(sp) :: stemp
           ! Intrinsic Functions
           intrinsic :: abs
           ! Executable Statements
           stdlib_scsum1 = zero
           stemp = zero
           if (n <= 0) return
           if (incx /= 1) then
               ! code for increment not equal to 1
               nincx = n*incx
               do i = 1,nincx,incx
                  ! next line modified.
                  stemp = stemp + abs(cx(i))
               end do
               stdlib_scsum1 = stemp
           else
               ! code for increment equal to 1
               do i = 1,n
                  ! next line modified.
                  stemp = stemp + abs(cx(i))
               end do
               stdlib_scsum1 = stemp
           end if
           return
     end function stdlib_scsum1

The original is

     pure real(sp) function stdlib_scsum1(n,cx,incx)
        ! -- lapack auxiliary routine --
        ! -- lapack is a software package provided by univ. of tennessee,    --
        ! -- univ. of california berkeley, univ. of colorado denver and nag ltd..--
           ! Scalar Arguments
           integer(ilp),intent(in) :: incx,n
           ! Array Arguments
           complex(sp),intent(in) :: cx(*)
        ! =====================================================================
           ! Local Scalars
           integer(ilp) :: i,nincx
           real(sp) :: stemp
           ! Intrinsic Functions
           intrinsic :: abs
           ! Executable Statements
           stdlib_scsum1 = zero
           stemp = zero
           if (n <= 0) return
           if (incx == 1) go to 20
           ! code for increment not equal to 1
           nincx = n*incx
           do i = 1,nincx,incx
              ! next line modified.
              stemp = stemp + abs(cx(i))
           end do
           stdlib_scsum1 = stemp
           return
           ! code for increment equal to 1
           20 continue
           do i = 1,n
              ! next line modified.
              stemp = stemp + abs(cx(i))
           end do
           stdlib_scsum1 = stemp
           return
     end function stdlib_scsum1

A general question is whether doing the work to eliminate as many gotos as possible is worth it. If it has to be done manually, perhaps not.

Avoid the use of double precision function dabs and dble

Instead of dabs(x) write real(x, kind=dp) and instead of dble(i) write real(i, kind=dp). Ideally the code should be written so that it still works if dp is set to a quadruple precision kind.

Add explicit `intent`

It would be useful if the modernized libraries include intent specifications for all variables.
This can likely be gathered from the BLAS/LAPACK documentation.

Export `fypp` version for `stdlib`

• Should include guards for quad-precision support via #:if WITH_QP

Ensure consistent indentation

In stdlib_linalg_lapack_s.f90 in stdlib_slartg there is an else if that is not indented consistently with the rest of the block. It's a minor thing, but ultimately there should be an auto-formatting step that ensures consistent indentation.

     pure subroutine stdlib_slartg(f,g,c,s,r)
        ! -- lapack auxiliary routine --
        ! -- lapack is a software package provided by univ. of tennessee,    --
        ! -- univ. of california berkeley, univ. of colorado denver and nag ltd..--
           ! february 2021
        ! Scalar Arguments
        real(sp),intent(out) :: c,r,s
        real(sp),intent(in) :: f,g
        ! Local Scalars
        real(sp) :: d,f1,fs,g1,gs,p,u,uu
        ! Intrinsic Functions
        intrinsic :: abs,sign,sqrt
        ! Executable Statements
        f1 = abs(f)
        g1 = abs(g)
        if (g == zero) then
           c = one
           s = zero
           r = f
        else if (f == zero) then
           c = zero
           s = sign(one,g)
           r = g1
     else if (f1 > rtmin .and. f1 < rtmax .and. g1 > rtmin .and. g1 < rtmax) &
               then
           d = sqrt(f*f + g*g)
           p = one/d
           c = f1*p
           s = g*sign(p,f)
           r = sign(d,f)
        else
           u = min(safmax,max(safmin,f1,g1))
           uu = one/u
           fs = f*uu
           gs = g*uu
           d = sqrt(fs*fs + gs*gs)
           p = one/d
           c = abs(fs)*p
           s = gs*sign(p,f)
           r = sign(d,f)*u
        end if
        return
     end subroutine stdlib_slartg

`svd` for singular value decomposition

Don't label loops unless there is a cycle or exit

In https://github.com/perazz/fortran-lapack/blob/main/src/stdlib_linalg_blas_d.f90 there are many labeled loops such as

                       loop_10: do i = 1, leny
                           y(i) = zero
                       end do loop_10

without a cycle or exit and which are not deeply nested. In that case the labeling of the loop just creates clutter and should be removed IMO.

Formatting: remove multiple consecutive blank lines

I think code is more readable when there are not too many blank lines, so that you can see more of it on your screen. I suggest that multiple consecutive blank lines, for example found at https://github.com/perazz/fortran-lapack/blob/main/src/stdlib_linalg_blas_d.f90 , be replaced by a single blank line.

Documentation

• Document linear algebra functions

Remove double-dot styled comments

Just remove the double dot markers ..

Misc about basic utilities (eye, diag,...)

Some random thoughts:

eye is just a special case of diag with a scalar, and has not a simpler interface (because most of time the mold parameter will be coded I think): is it really worth having it?

I propose adding set_diag / get_diag procedures, in order to manipulate arbitrary diagonals (not only the central one) of already existing matrices.

Implementations detail that is not important at this point: what are the performance expectations for these utilities? I'm asking because the use of do concurrent constructs with branches or with merge are likely not well optimized by most of compilers.

Pseudo-inverse function

To complement the inv function, a pseudo-inverse function named pinv as in Matlab would be useful. It can be computed using Lapack routines https://fortran-lang.discourse.group/t/what-is-the-best-pseudo-inverse-moore-penrose-inverse-subroutine/3936

Eigenvalues

Provide functions for eigenvalues and eigenvectors

`solve`: subroutine version should allocate nothing

Implement least squares operator

• NumPy | lstsq(a, b, rcond='warn')
• Scipy | lstsq(a, b, cond=None, overwrite_a=False, overwrite_b=False, check_finite=True, lapack_driver=None)
• IMSL | Result = IMSL_QRSOL(B, [A] [, AUXQR] [, BASIS] [, /DOUBLE] [, QR] [, PIVOT] [, RESIDUAL] [, TOLERANCE])

Quadruple precision BLAS and LAPACK

Eventually a quadruple precision BLAS and LAPACK should be created whose only difference from the double precision versions is that dp is replaced by qp.

Replace assumed-size arguments with explicit-shape

According to https://fortran-lang.org/learn/best_practices/arrays/ assumed-size arrays are the least preferred form of array argument. Also real(dp) :: dx(n) is more self-documenting than real(dp) :: dx(*).

Remove "end of" comments when the end statement has the procedure name, and remove return statements just before procedure ends.

The end of the function stdlib_dasum in https://github.com/perazz/fortran-lapack/blob/main/src/stdlib_linalg_blas_d.f90 looks like

           return
           ! end of stdlib_dasum
     end function stdlib_dasum

with similar lines for other procedures. If the last line of the function were just end the comment would inform the reader which procedure is being terminated, but since the procedure name is listed in end function stdlib_dasum the comment is redundant and can be removed. Also a return that is just before the end of a procedure is redundant and can be removed.

Rename project?

Since the reference Lapack is in Fortran, maybe rename this project to modern-fortran-lapack to distinguish it from the reference Lapack?

Remove data statements where possible

They're often used for parameter variables:

fortran-lapack/src/stdlib_linalg_blas_d.f90

Lines 1072 to 1073 in f8b0ed1

	! .. data statements ..
	data zero, two/0.d0, 2.d0/

`solve` linear system (general matrix)

• add real implementation
• add complex implementation
• add real test
• add complex test