camfort / fortran-src

Fortran parsing and static analysis infrastructure

fortran-src

Provides lexing/parsing and early static analyses of Fortran code. The following Fortran standards are covered:

• FORTRAN 66 (ANSI X3.9-1966)
• FORTRAN 77 (ANSI X3.9-1978 / ISO 1539:1980)
• Fortran 90 (ISO/IEC 1539:1991)
• Fortran 95 (ISO/IEC 1539-1:1997
• Fortran 2003 (partial)

Parsing is configurable, and you can select the Fortran standard to target, including special extended modes for nonstandard FORTRAN 77.

Includes data flow and basic block analysis, a renamer, and type analysis.

This package primarily exports a Haskell library, but also builds an executable that can be used for testing and debugging. For example usage, see the CamFort project, which uses fortran-src as its front end.

Obtaining

We provide prebuilt binaries for Windows, Mac and Linux.

Usage

Add fortran-src as a dependency in your Haskell project. We're on Hackage and also on Stackage.

Command-line tool

You can also invoke fortran-src on the command line.

For features that output graphs, the intended usage is to pipe it into the command dot -Tpdf and redirect that into a PDF file. The dot command is part of the GraphViz project, please see their manual for the many other options that can be explored for visualisation purposes.

Usage: fortran-src [OPTION...] <file>
  -v VERSION, -F VERSION  --fortranVersion=VERSION         Fortran version to use, format: Fortran[66/77/77Legacy/77Extended/90]
  -a ACTION               --action=ACTION                  choose the action, possible values: lex|parse
  -t                      --typecheck                      parse and run typechecker
  -R                      --rename                         parse and rename variables
  -B                      --bblocks                        analyse basic blocks
  -S                      --supergraph                     analyse super graph of basic blocks
  -r                      --reprint                        Parse and output using pretty printer
                          --dot                            output graphs in GraphViz DOT format
                          --dump-mod-file                  dump the information contained within mod files
  -I DIR                  --include-dir=DIR                directory to search for precompiled 'mod files'
  -c                      --compile                        compile an .fsmod file from the input
                          --show-block-numbers[=LINE-NUM]  Show the corresponding AST-block identifier number next to every line of code.
                          --show-flows-to=AST-BLOCK-ID     dump a graph showing flows-to information from the given AST-block ID; prefix with 's' for supergraph
                          --show-flows-from=AST-BLOCK-ID   dump a graph showing flows-from information from the given AST-block ID; prefix with 's' for supergraph

If you do not pass a --fortranVersion flag, the version will be guessed from the file name:

• Files ending in *.f are parsed with extended FORTRAN 77 syntax.
• Files ending in *.f90 are parsed with Fortran 90 syntax (and respectively for *.f2003/*.f03, *.f2008/*.f08).
• Unknown extensions are parsed like *.f files.

Building

You will need the GMP library plus header files: on many platforms, this will be via the package libgmp-dev.

Haskell library dependencies are listed in package.yaml. fortran-src supports building with Stack or Cabal.

fortran-src supports GHC 9.0 through GHC 9.2. We regularly test at least the minimum and maximum supported GHCs. Releases prior to/newer than those may have issues. We welcome fixes that would let us support a wider range of compilers.

You will likely need at least 3 GiBs of memory to build fortran-src.

For installing GHC and build tools, we strongly recommend ghcup.

When latest recommended is used, it means the latest version of the tool that ghcup tags with recommended. This sometimes lags behind the latest-tagged version. With ghcup installed, run ghcup list for a better understanding.

Following are general guides for any OS that provides the relevant tools. If you have trouble, consider checking the CI workflow files in .github/workflows.

Stack

We support the latest recommended version of Stack (as of 2021-09-17, Stack 2.7). Generally, any Stack 2.x should work. (Stack 1.x may work with minor alternations -- you may have to download the resolver manually.)

stack build

For an interactive shell:

stack build
stack ghci

Note that running stack ghci before running stack build won't work properly, due to stack ghci not running build tools like Alex and Happy. So parser modules will not be built, and you'll receive an error after building the other modules. You can cheat a bit and run stack build until you see Building library for [...] (= preprocessing has finished), then hit <Ctrl-C> to stop the build and run stack ghci as usual.

Cabal

We support the latest recommended version of Cabal (as of 2021-09-17, Cabal 3.4)

cabal build

Testing

Unit tests are stored in test. Run with stack test or cabal test.

Usage

As a dependency

fortran-src is available on Hackage and Stackage, so for Cabal or Stack projects you should only need to add fortran-src to your project dependencies.

If you need a specific version of fortran-src in a Stack setup, you can stuff a Hackage reference into stack.yaml using extra-deps, like:

resolver: ...
...

extra-deps:
- ...
- fortran-src-$VERSION

As a CLI tool

If you have Cabal properly configured, you should be able install fortran-src from Hackage:

cabal install fortran-src

We provide prebuilt binaries for some platforms: see the Releases tab.

Otherwise, you can build from source and use convenience commands like cabal run, stack run. See #Building for details.

Contributing

We welcome bug reports, fixes and feature proposals. Add an issue or create a pull request on the GitHub repository.

Support

You may be able to find maintainers on the Libera.Chat IRC network. Check in #fortran-src and #camfort . Otherwise, you could get into contact with one of the team on the CamFort team page -- or create an issue describing your problem and we'll have a look.

For maintainers

See doc/maintainers.md in camfort/camfort.

fortran-src's Issues

Support scalefactor on variable name

      program scale
      character a*8
      end

We currently don't support this, but it is in the Fortran 77 standard.

Cannot parse program units that do not have a named "end".

Bug in Fortran90.y. Small example:

module simple
  implicit none
end module

This raises pattern matching error in unitNameCheck. I think the fix should allow more than just TId to be matched, with some cases for other kinds of "end" which are not named.

Grouping does not complain if there is an orphan 'end do'

For example:

program simple
  implicit none

  end do
end program simple

Running this through grouping causes no problem (e.g. try fortran-src simple.f90 -S, when I think we should report this as bad Fortran.

CASE statements on multiple values fail to parse (Fortran 90)

Example program:

program foo
  implicit none
  integer :: i = 0
  select case(i)
  case(0,1)
     WRITE (*,*) "0 or 1"
  case default
     WRITE (*,*) "anything else"
  end select
end program

Looking at the parser this does seem to be accounted for, so I'm not sure why it is failing.
Error is:
ProgramFileforpar: 5:10: Parsing failed.Last parsed token: TComma (5:9,5:9).

Problem with analyseBBlocks and intrinsics / function call disambiguation and flows graph

Consider the following example from camfort:

program bug
  implicit none
  integer :: i
  real, dimension(2) :: a, b
  real :: x
  do i = 1, 2
    x = abs(a(i))
    b(i) = x
  end do
end program

We should infer a specification of != stencil readOnce, pointed(dim=1) :: a here but we don't because there are no subscripts flowing to b(i). I've narrowed this down to a problem with the basic blocks / flows graph I think. Basically, the transformations carried out by processFunctions of BBlocks don't seem to be interacting well with the flows graph here. Perhaps the newly created statements aren't getting picked up in the flows graph?

`fromJust` exception for subroutines without their definition in scope

This could relate to issue #14. I am getting a fromJust exception coming out of the following piece of code called with fortran-src -S fun.f90

program fun
  implicit none
  integer :: x

  call someSUB(x)

  if (.true.) then
    x = someFUN(x)
  end if

end program fun

Flow analysis seems incorrect when inside modules.

There appears to be some problems with building flows graphs (and other related information) when working inside subroutines/functions inside a Fortran 90 module. I have extended DataFlowSpec.hs
with a module-subroutine-Fortran90 version of the loop4 test. I don't expect the block numbers to be the same, but the flows graph is clearly now a different shape. Also there are some fromJust exceptions getting thrown up in some cases.

Fortran 90 Doesn't Parse Recursive Functions Without A Type Spec

Example:

program test                                                                                                               
implicit none                                                                                                              
contains                                                                                                                   
  recursive function f() result(i)                                                                                                   
      integer :: i                                                                                                         
      i = 0                                                                                                                
  end function                                                                                                             
end program test

This fails to parse, but is valid according to 12.5.2.2 of the spec.

Working on it as part of #37.

Cannot parse one-armed-if with assignment to array.

program simple
  implicit none
  integer, dimension(1) :: x
  if (.true.) x(1) = 1
end program simple

fails with:
ProgramFileforpar: 4:15: Lexing failed. Last parsed token: TRightPar (4:13,4:13).

Can't track down why this is. The parsing for an if with just the "true" branch seems fine. A possible shift/reduce problem? Doesn't appear for assignments to non-arrays or assignments to arrays with multi-armed if.

STOP should permit a character string

program simple
  implicit none
  STOP 'Computer says no.'
end program simple

Breaks at the string. Section 8.4 of Fortran 90 spec allows for a scalar-char-constant here.

Nested labeled DO loops can't be grouped when sharing final statement

      SUBROUTINE A
        DO 42 I = 1,10
          DO 42 J = 1,10
            STOP
 42     CONTINUE
      END

Grouped array subscript

I've noticed some code using an array subscript a bit like the following:

program arra
  integer, dimension(5) :: x
  integer :: i
  x([1,2,5]) = 42
end program

This assigns to 1, 2 and 5 indices at the same time. I haven't found which standard (if any) introduces this. But I suspect it's Fortran 95.

Empty module "blocks" unsupported

This is accepted by NAG but not fortran-src:

module bug
contains
! foo
subroutine a()
integer :: x
end subroutine a
end module bug

Lexing hexadecimal numbers

program a
  print *, z'0'
end

A/I/L field descriptors clash with identifiers in fixed form lexer

If an identifier starts with a, i, or l and are immediately followed by an integer, then the lexer mistakenly parses a field descriptor regardless of the context.

haddock failure

src/Language/Fortran/Analysis/Types.hs:87:3: error:
    parse error on input ‘-- | any (not . isIxSingle) (aStrip ixAList) = recordCType CTArray (varName v)  -- it's an array (or a string?) FIXME’

ping @phadej

Support .CMN files

This is not in the standard as far as I can see, but most compilers support naked common block definitions inside of '.CMN' files, which can then be be included via an include statement. I think we need this support to make common-block elimination more useful. Probably not a very big change.

Problem parsing `case` (Fortran 90)

program simplecase
  implicit none
  integer :: d = 0
  select case (d)
   case (0)
     d = 1
  end select
end program simplecase

Gives error:
ProgramFileforpar: 5:12: Parsing failed.Last parsed token: TRightPar (5:11,5:11).

I believe the problem here is that RANGE in Fortran90.y does not allow singleton ranges.
I suggest extending the non-terminal with a production for EXPRESSION.

parses variable named "double" incorrectly

I put this line in a function:

double = x + x

and it seems to be parsing it as a Do statement

IVMAP only getting partially built

Example program:

program foo
  implicit none
  integer :: i, j
  real, dimension(100) :: a, b
  do i=1,100
     do j=1,100
        a(i) = b(i) + b(1)
     end do
  end do
end program

At some point in the last month (I think), a bug has been introduced which means that the induction-variable analysis is wrong. See fortran-src -S foo.90:

...
ivMap: fromList [(5,fromList ["foo_j2"])]

i is missing.

Empty 'if' block causes exception

For example

program emptyIf
  implicit none
  if (.true.) then
  end if

end program emptyIf

throws up:

$ fortran-src EmptyIf.f90 
ProgramFilefortran-src: Trying to find how long an empty list spans for.
CallStack (from HasCallStack):
  error, called at src/Language/Fortran/AST.hs:500:17 in main:Language.Fortran.AST

INCLUDE statements not parsed

      program foo
      implicit none
      include 'bar.cmn'
      end  

This is part of Fortran 90 (see p.43 of the standard, ftp://ftp.nag.co.uk/sc22wg5/N001-N1100/N692.pdf)
However, it seems fairly common to use this in Fortran 77 too.... although it is not standard. Perhaps we can allow it in Fortran 77 extended mode?

Add Parse<->Print isomorphism tests

@dorchard As discussed, this is a suggestion to add tests to help ensure consistency of the parser/printer.

Gen->Print->Parse

We could use QuickCheck to generate syntax trees, then 'pretty' show them in a canonical way and parse them. We would expect the parsed syntax trees to be the same (modulo some information such as position) as the original syntax tree.

File->Parse->Print

Assuming we had example programs in a 'pretty' canonical form, we would expect parsing the files, then printing them, to give the same text as the original file.

function call disambiguation stops at first level

I am testing a function call of the following form:

y = dbl(dbl(x))

but only the outer dbl is recognised as a function call. The inner one remains interpreted as an array.

Separating statements with ';' seems incomplete (e.g. call statements) (Fortran 90)

program simple
  implicit none
  call foo(1); call bar(1)
end program simple

I believe this should be allowed as call is a statement, and all statements can be delimited by semicolon or newline according to section 1.5.3 of the spec (ftp://ftp.nag.co.uk/sc22wg5/N001-N1100/N692.pdf).
I have noticed this problem also with allocate.

More permissive INCLUDE?

I note in the Fortran 90 standard (ftp://ftp.nag.co.uk/sc22wg5/N001-N1100/N692.pdf, section 3.4) that "An INCLUDE line is not a Fortran statement. An INCLUDE line must appear on a single source line where a statement may appear;” We only allow it to appear in the place of a NONEXECUTABLE_STATEMENT and I just saw some code where it appears between end program and subroutine…. From the standard it kinda sounds like you can do an include where any kind of statement can appear (include a PROGRM statement), so maybe we need to be more flexible than we are at the moment - without creating a shift/reduce :s

Error messages for badly grouped DO blocks are not informative.

For the following program:

      program goto
      implicit none
      integer i, j

      do 10 i = 0, 10
      do 20 j = 0, 20
 10   continue
 20   continue
      end

Compare the output of fortran-src with gfortran:

174:tmp dorchard$ fortran-src goto-bad.f
ProgramFilefortran-src: src/Language/Fortran/Transformation/Grouping.hs:(186,3)-(189,89): Non-exhaustive patterns in case

vs.

174:tmp dorchard$ gfortran goto-bad.f
goto-bad.f:8:72: Error: End of nonblock DO statement at (1) is interwoven with another DO loop
goto-bad.f:10:9:

       end
         1
Error: END DO statement expected at (1)
f951: Error: Unexpected end of file in ‘goto-bad.f’

Off-by-one error in `Util.Position.getSpan`?

Util.Position.getSpan seems to have an off-by-one error, which gets revealed in the source spans that are spat out by units-check as part of the constraints.

They are all short by one, e.g. (12:9)-(12:19) should really be (12:9)-(12:20).

This is assuming that we start both lines and columns at 1 (from my observation Vim and Atom do this, but Emacs starts lines at 1 and columns at 0).

Comments following `contains` fail.

program simple
  implicit none
  contains
  ! Comments
end program simple

raises error:
ProgramFileforpar: 4:13: Parsing failed.Last parsed token: TComment (4:3,4:12) "! comments".

Note also that the comment is presented here in lowercase. Is there "too much" lowercase conversion happening?

Integer labels in Fortran 90 cause parser error

Small example:

program example
  implicit none

  integer :: i = 12
  10  if (i .gt. 10) continue
end program

ProgramFilefortran-src: 6:1: Parsing failed.Last parsed token: TNewline (5:30,5:30).

I can't figure out why this is happening as it looks all fine in the grammar. Lexing issue? Strangely the parse error points to the newline at the end of the line, but the issue seems to do with the integer label.

DATA statements in F77 can contain "symbolic name"

For example,
DATA X/a,b,c/
is allowed if a, b, and c are variables that contain constants (parameter I think). We can't parse this at the moment.

Comment between continuation lines in F77 breaks lexer

Example:

      SUBROUTINE lex
      IMPLICIT NONE
      INTEGER X, Y,
c break
     &   ZZ
      
      END

Remove the comment and the problem goes away.

Recognise and represent intrinsics in the AST explicitly

Inconsistent AST node for label

Fortran 90 parser uses ValInteger while Fortran 66/77 parsers use ValLabel. I'd like to eliminate ValLabel altogether and at some point it should become an annotation.

I can't do this at this point because there are five instances in BBlocks that I don't know if they are safe to remove or what. If they can be changed to ValInteger please let me know.

BBlocks analysis produces refuted pattern for some kinds of nested

module test
  implicit none
  contains

   subroutine foo 
     integer:: i, j

     do i = 1, 2
        do j = 1, 2
        end do
     end do

   end subroutine

end module

Via camfort's stencil analysis, this throw up the error:

$ camfort stencils-infer test.f90 out
...
camfort: src/Language/Fortran/Analysis/BBlocks.hs:39:47-82: Irrefutable pattern failed for pattern Just gr
``

Comment on the same line as program unit statement breaks parser

subroutine a ! this comment breaks
  print *, 42
end

Snippet above illustrates the issue.

Eliminate UNIT node altogether

UNIT node is only used in backspace, rewind, and endfile statements. It seems this can be eliminated and merged with generic control information lists.

Double array access is not lexed

program example
  arr(1)(1:1) = x
end program example

Need to check if this is even valid Fortran.

Source files starting with new line fail to parse

e.g.

      program foo
      implicit none
      end

gives:
ProgramFileforpar: 2:1: Parsing failed.Last parsed token: TNewline (1:1,1:1).

(for all parsers).

Fortran 95 support.

We do not yet have a Fortran 95 parser.

Fortran 77: Parser error when character array type has scale factor and name starting with 'p'

      program scale
      character*8 a(8)
      character*8 p(8)
      end

Fails on line 3 with

ProgramFilefortran-src: 3:20: Parsing failed.Last parsed token: TScaleFactor (3:17,3:19) 8.

Strange error. Some kind of reduce conflict so the parser thinks this is not a type declaration?

Identifiers greater than 29 letters are rejected [F90]

The specification states 31 as the maximum length [ftp://ftp.nag.co.uk/sc22wg5/N001-N1100/N692.pdf, p.18], but identifiers of length 30 and 31 are currently rejected.

program simple
  implicit none
  ! 29 characters is fine
  integer :: abcdefghijklmnopqrstuvwxyzAAA
  ! 30 breaks
  integer :: abcdefghijklmnopqrstuvwxyzAAAA
end program 

We lose track of comments between sub-program-units

module bug
integer :: x
contains
! foo
subroutine a()
integer :: x
end subroutine a
end module bug

DO LABEL WHILE not supported

Fortran 90 standard allows the following kind of thing:

      program dowhile
      implicit none
      integer :: i = 0
      DO  200  WHILE (i .lt. 10)
         i = i + 1
  200  write (*,*), i
      end

We can't parse a DO WHILE with a label in it yet. Should we allow this in F77 extended too as well as F90?

ValInteger should not have an annotation

Value nodes are meat to be embedded in ExpValue at all times, hence none of them need annotation. Enclosing expressions should have the annotations.

At some point @mrd changed this for ValVariable (I believe for renaming), it would be nice if we go back to original design as right now all captures/generations of ValIntegers end up copying the same annotation for ValVariable and its enclosing ExpValue.

Prelude.read: no parse on reals with no leading 0

I think this error is coming from fortran-src to do with the real-parsing functionality though it appears when I am using camfort (e.g., units-infer or something). Example

program f
   implicit none
   real :: x = .1
end program

camfort fails with
camfort: Prelude.read: no parse

Subroutines without arguments not parsing

In 77 and 90, subroutines with no arguments fail to parse:

subroutine foo 
implicit none
integer :: i
end

gives
ProgramFileforpar: 2:1: Parsing failed.Last parsed token: TNewline (1:15,1:15).
With arguments (e.g., subroutine foo (v)) is fine.

`genInOutAssignments` in BBlock.hs is missing a case to handle nullary functions.

Sample code:

program simple
  implicit none

  integer :: x = 2
  x = id()

  contains

  integer function id()
     id = x
  end function
 end program

Throws the genInOutAssignments error.

dataFlowSolver exception

The code in computational-physics-1/fortran77/code214.f raises a dataFlowSolver exception that I can't debug, see: fortran-src code214.f -S

genBackEdgeMap fails on recursive subroutines

Example:

program gbe
  implicit none
  contains
   recursive subroutine foo(x)
     integer :: x
     if (x == 0) then 
       x = x
     else
       call foo(x-1)
     end if
   end subroutine foo
end program gbe

fortran-src gbe.f90 -S fails with an exception genBackEdgeMap coming from the function of the same name- I guess we need to compute a fixpoint in this case when we currently are not?