pdebuyl / fortran_tester

Fortran module to test Fortran programs

Fortran tester {#mainpage}

tester is a Fortran module to test Fortran programs. It provides routines to check equality or closeness between variables and counting the errors.

A minimal example:

program test
  use tester
  implicit none

  type(tester_t) :: my_tester

  call my_tester% init()

  call my_tester% assert_equal(1+1, 2)

  call my_tester% print()

end program test

If none of the tests fail, the print method displays the message fortran_tester: all tests succeeded. Else, the program will exit with a nonzero error code, making it suitable for use as an automated test.

Author: Pierre de Buyl
License: BSD

Contributors: Peter Colberg, Stefano Szaghi, Pietro Bonfa, Elias Lettl, Giacomo Rossi, Peter Hill, Jacob Williams

Installation

fortran_tester consists of a single Fortran file. You can just drop src/tester.f90 in your Fortran project or build using CMake, FoBiS, or FPM.

Coverage information

If you read the autogenerated documentation, the coverage data should be available.

fortran_tester's Issues

API html doc

Hi Pierre,

I have just uploaded the API html doc here.

I have used FORD. I think that FORD is better than doxygen:

• designed for Fortran;
• support graph, e.g. see this;
• support LaTeX eq., e.g. see this;
• support introspection, e.g. see the link into class(test_t) to the definition of test_t here;
• the developer of FORD, Chris, is great/fast/a real Fortraner.

Do you like this DOC?

'error stop' not supported by PGI

According to fortranwiki error stop is not supported by PGI. Indeed compilation with PGI 17.10 fails at this line

fortran_tester/src/tester.f90

Line 153 in 48e344f

error stop 'fortran_tester: tests failed'

Changing that to

#if ! defined(__PGI)
      error stop 'fortran_tester: tests failed'
#else
      stop 1
#endif

Intel 15+ and Cray compilers work fine with the error stop.

fobos file

Dear @pdebuyl

It could be useful for me to have a fobos file (the makefile of FoBiS) in the project. Currently I simply do a FoBiS build that automatically builds all programs found (your tests), but for other stuff (e.g. code coverage/debug) I like to use a fobos file. I do not want to pollute your tree, it could be named/placed where you want, even a hidden .fobos file is fine for me thus having a minimal impact on your clean tree.

Do you think I can add a fobos file?

Cheers.

Dangerous mixing of logical espressions with optional arguments

The Fortran 2003 standard clearly specify that there's not an established order for the operand evaluation of a logical expression:

It is not necessary for a processor to evaluate all of the operands of an expression, or to evaluate entirely each operand, if the value of the expression can be determined otherwise.

So the expression
if (.not. present(fail) .or. (present(fail) .and. fail .eqv. .false.)) then
could lead to unpredictable result.

What's the purpose of this expression? Could you clarify it in order to split it and at first evaluate if the optional argument is present?

Support for integer with 64 bits kind

Dear @pdebuyl

I am using your library right now, it is amazing, thank you very much!

During my (brief) usage I found some issues that prevent me to exploit your library. The most important is the missing support for integer kind of 8/16 and 64 bits. I have modified your library to extend the support to all kinds with a portable definition and I have create a PR for it, see this. Feel free to not accept the PR.

Here I report the issue in details. I am using your tester for this project: essentially this library generates a 64 bits integer code... currently I have to disable the whole tests suite, see this I have to convert int64 to int32...

The PR I created fix this issue and let me to re-enable the whole tests suite of MORTIF.

Thank you very much for sharing your work.

Cheers.

P.S. your new tree arrangement is perfectly fitted with my workflow see this

Customize output

Dear Pierre,

I am using your cool library more and more... I love it. For example, I just add it into FLAP tests suite, see this.

I have another little question. Consider the following output:

tefano@zaghi(02:32 PM Tue Oct 25) on feature/group-default-values-bug-fix [+!?]
~/fortran/FLAP 17 files, 152Kb
→ ./exe/test_group
 test_group
 spectrum =  F
 domain   =  F
 grid     =  F
 test_group new -s
 spectrum =  T
 domain   =  F
 grid     =  F
 test_group new -d
 spectrum =  F
 domain   =  T
 grid     =  F
 test_group new -g
 spectrum =  F
 domain   =  F
 grid     =  T
 fortran_tester:           0  error(s) for          12 test(s)
 fortran_tester: all tests succeeded

Do you think you could like to customize the last output? I mean, now the string fortran_tester: is automatically added, but it could be easily customizable passing an optional argument to the tester init method. Having the possibility to customize this prefix could help to grep many outputs with one script. If you like the idea, I can create a pull request.

As always, thank you!

Collaboration guide lines

@pdebuyl

Dear Pierre,

I like very much this code, I would like to use it almost extensively in my codes. However, I adopt a rather cumbersome workflow that, among other things, requires a lit bit of directories-structure. As consequence, I need to do a (very little) modification to your project. However, I cannot understand what is your preferred collaboration way (I cannot find a sort of contributing guide lines)

do you accept Pull Request?

For my needs, I'll go to fork your project and doing my bad-modifications to my fork, but I would like to give you back my work if it will be of some interest for your, in the spirit of FOSS contribution. Can you tell me if you want (and how) to be able to merge my (worsening 😄 ) patches?

Thank you for sharing your work.

Travis CI exploitation

Dear @pdebuyl

In the last years I have found that Travis CI is a great help for us. I do not know if you are aware about it, but in very few words it help us to automate many boring stuff like automatic build/test/coverage-analysis/doc-deploy...

I have configured my fork of fortran_tester to exploit Travis CI, e.g. see the last log.

The Travis CI exploitation is more than add a config file (hidden) file to your project, the following mine (just as a skeleton for you):

language: generic

sudo: false

cache:
  apt: true
  pip: true
  directories:
    - $HOME/.cache/pip
    - $HOME/.local

addons:
  apt:
    sources:
      - ubuntu-toolchain-r-test
    packages:
      - gfortran-6
      - binutils
      - python-pip
      - graphviz

env:
  global:
    - MAKECOVERAGE="./scripts/makecoverage.sh"
    - MAKEDOC=""
    - CLEAN=""
    - MAKETAR=""

before_install:
  - git submodule update --init --recursive

install:
  - |
    if [[ ! -d "$HOME/.local/bin" ]]; then
      mkdir "$HOME/.local/bin"
    fi
  - export PATH="$HOME/.local/bin:$PATH"
  - export FC=/usr/bin/gfortran-6
  - ln -fs /usr/bin/gfortran-6 "$HOME/.local/bin/gfortran" && gfortran --version
  - ls -l /usr/bin/gfortran-6
  - ln -fs /usr/bin/gcov-6 "$HOME/.local/bin/gcov" && gcov --version
  # - pip install --user --upgrade pygooglechart
  # - pip install --user --upgrade graphviz
  # - pip install --user --upgrade FoBiS.py
  # - pip install --user --upgrade markdown-checklist 
  # - pip install --user --upgrade ford

script:
  - $MAKECOVERAGE

# after_success:
#   - find . -name '*.gcno' -print
#   - bash <(curl -s https://codecov.io/bash)
#   - $MAKEDOC

# before_deploy:
#   - $CLEAN
#   - $MAKETAR
#   - mv fortran_tester.tar.gz fortran_tester-$TRAVIS_TAG.tar.gz

# deploy:
#   provider: releases
#   api_key:
#     secure: 
#   file: 
#     - fortran_tester-$TRAVIS_TAG.tar.gz
#     - ./scripts/install.sh
#   skip_cleanup: true
#   overwrite: true
#   on:
#     tags: true

Do you think you like such a help?

More informative output

Dear @pdebuyl

the output of the print method is somehow too much concise, e.g.

stefano@zaghi(01:41 PM Fri Oct 14) on master [?]
~/fortran/fortran_tester 11 files, 3.3Mb
→ ./test_tester_2
 fortran_tester:           1  error(s) for           1 test(s)
fortran_tester: tests failed

It prints only the number of errors occurred. It could be useful to provide a more informative output. For example we could print which are the errors failed, the inputs provided to the test assert procedures... an approach could be to register such events into an array of strings. If you like, I can try to implement this feature.

What do you think?