noaa-gfdl / mkmf

Disclaimer

The United States Department of Commerce (DOC) GitHub project code is provided on an "as is" basis and the user assumes responsibility for its use. The DOC has relinquished control of the information and no longer has responsibility to protect the integrity, confidentiality, or availability of the information. Any claims against the Department of Commerce stemming from the use of its GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce. The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government.

mkmf: Make Makefile

mkmf is a tool written in perl5 that will construct a makefile from distributed source. A single executable program is the typical result, but I dare say it is extensible to a makefile for any purpose at all.

Features

• Understands dependencies in f90 (modules and use), the Fortran include statement, and the cpp #include statement in any type of source
• No restrictions on filenames, module names, etc.
• Supports the concept of overlays (where source is maintained in layers of directories with a defined precedence)
• Keeps track of changes to cpp flags, and knows when to recompile affected source (i.e, files containing #ifdefs that have been changed since the last invocation);
• Will run on any unix platform that has perl version 5 installed
• It is free, and released under GPL.

Pronunciation

mkmf is pronounced make-make-file or make-m-f or even McMuff (Paul Kushner's suggestion).

Installation

mkmf and list_paths can be run on any *nix type system that has C-shell (tcsh) and Perl version 5 installed.

To install, place the repository on the file system and add the bin directory to PATH.

mkmf's Issues

Catch floating point exceptions in debug build

In issue NOAA-GFDL/MOM6#194 we discussed making floating point exceptions standard. Given that there may be a performance hit it would be better to do this only in debug builds.

This issue is to add:

-fpe0 (intel)
-ffpe-trap=invalid,zero,overflow (gnu)
- (pgi)

to the debug build in the machine *.mk files.

GCC 10 requires two non-backward-compatible options

Available GCC versions on C4 are: 6, 7, 8 (default), 9, 10
Available GCC versions on C5: 10, 11 (default)

Using the current ncrc-gcc.mk template with GCC 11 on T5, @mcallic2 encountered errors such as:

Error: Type mismatch between actual argument at (1) and actual argument at (2) (REAL(8)/LOGICAL(4)).
/lustre/f2/dev/Caitlyn.Mcallister/am5/2022.01/am5p0p1_2022.04-alpha3/src/FMS/mpp/include/mpp_comm_mpi.inc:355:44:

  355 |       if( mpp_npes().GT.1 ) call MPI_BCAST( data, length*size(data(:)), MPI_CHARACTER, from_rank, peset(n)%id, error )
      |                                            1
Error: Type mismatch between actual argument at (1) and actual argument at (2) (CHARACTER(*)/LOGICAL(4)).
make[1]: *** [Makefile:151: mpp.o] Error 1
make[1]: Leaving directory '/lustre/f2/dev/gfdl/Caitlyn.Mcallister/am5/2022.01/am5p0p1_2022.04-alpha3/ncrc5.gcc-debug/exec/fms'
make: *** [Makefile:14: fms/libfms.a] Error 2

According to @thomas-robinson (and @rem1776 ), FFLAGS needs to have -fallow-invalid-boz -fallow-argument-mismatch for GCC versions 10 and later.

They are are described in the GCC 10 release notes (https://gcc.gnu.org/gcc-10/changes.html):

• Mismatches between actual and dummy argument lists in a single file are now rejected with an error. Use the new option -fallow-argument-mismatch to turn these errors into warnings; this option is implied with -std=legacy. -Wargument-mismatch has been removed.
• The handling of a BOZ literal constant has been reworked to provide better conformance to the Fortran 2008 and 2018 standards. In these Fortran standards, a BOZ literal constant is a typeless and kindless entity. As a part of the rework, documented and undocumented extensions to the Fortran standard now emit errors during compilation. Some of these extensions are permitted with the -fallow-invalid-boz option, which degrades the error to a warning and the code is compiled as with older gfortran.

These two options are not backward compatible, so we'll need to have two GCC templates or drop support for GCC versions before 10.

git-version-string does not print the blob hash if file is not in a git repository

If a file is not in a git repository, git-version-string will not print the blob (although git hash-object will).

Sample output:

> git-version-string some/file
'status:UNKNOWN'
WARNING: Not in a git repository

Desired output:

> git-version-string some/file
'status:UNKNOWN blob:e36bdb0484bab5b72b29a379a5cecebcedc3afcf'
WARNING: Not in a git repository

Update for ncrc-intel.mk needed for C5

From @nikizadehgfdl, we are planning to have a single Intel compiler template that works on both Intel and non-Intel hardware.

In the past, we had switched to the xsse2 instruction set to support run-to-run reproduciblity, and it is believed that this is no longer needed for recent Intel compilers.

The update needed is the ISA. Existing:

ISA = -xsse2

Proposed:

ISA = -march=core-avx2 -qopt-report-phase=vec -qopt-report=2 -qno-opt-dynamic-align

Note: this is two changes: 1) to avx2, and 2) generic hardware support. The first change only would be:

ISA = -xCORE-AVX2 -qopt-report-phase=vec -qopt-report=2 -qno-opt-dynamic-align

Use core-avx-i for gaea Intel templates

AVX2 is an further optimization of the AVX instruction set that allows FMS-based models to run up to ~5% faster. However, currently using it causes restart runs in model configurations (could be related to SIS2) to not reproduce, so we have decided not to use it, for now.

Add -DEBUG to compiler flags in for debug builds

It would be good to add -DDEBUG for debug builds. We could use this in MOM6 to enable additional safety checks. Presently we have things hard-coded, e.g.

define DO_SAFETY_CHECKS

or

undef DO_SAFETY_CHECKS

Erroneous REPRO/DEBUG logic in linux-intel.mk?

Lines 48-49 of linux-intel.mk read

endif
ifneq ($(DEBUG),)

which in other files (e.g. ncrc-intel.mk) read

else ifneq ($(DEBUG),)

The latter implies that REPRO and DEBUG are exclusive modes while the former suggests you can compile with both REPRO and DEBUG options. Is linux-intel.mk wrong or all the others wrong?

Intel templates do not need $MAIN_PROGRAM in link line

Most of the templates have:

LD = ftn $(MAIN_PROGRAM)

This is needed for the GCC (gfortran) and some other compilers, but not the Intel compiler.

Moreover, this can make troubleshooting compilation problems more difficult, so should be removed from the Intel templates

SHELL variable interferes with parallel make

FRE compile job logs often show this warning indicating that parallel make is not working properly:

make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.

Parallel, or recursive make, ensures that a total number of compile jobs is not exceeded. Communication among the make processes is required to enable this, described here:

https://www.gnu.org/software/make/manual/html_node/Recursion.html#Recursion

Google searches and the make documentation are clear that the child make processes must be called exactly the same or make will bail out of the parallel operation with the warning above.

@underwoo found that the SHELL variable interferes with the parallel make operation, triggering the warning. I reproduced his finding, which is also supported by this documentation on SHELL, which mentions that is not inherited to child make processes. This would seem to explain the problem.

https://www.gnu.org/software/make/manual/html_node/Choosing-the-Shell.html

Add option to include extra link options

Currently, there is not an option in mkmf that will allow a user to specify extra link options on the command line.

Update mkmf to use perl in environment rather than /usr/bin/perl

Currently, mkmf uses the /usr/bin/perl as specified in the shebang:
#!/usr/bin/perl

mkmf uses the compiled Config module that fails with an error if the running Perl version doesn't match the Perl used to compile the library.

The solution is to use the Perl in the environment, which should match the Perl used the compile Config.pm:
#!/usr/bin/env perl

Not all MPI compiler wrappers pass quotes the same

A user discovered that the -g option to add the git reference information via a cpp macro does not work for some MPI compiler wrappers. This user discovered that escaping the double quotes (") in mkmf resolves the issue.

Main branch renaming

In order to modernize this repository, the main branch of this repository should be named main.

list_paths to find symlinks

Is there a reason to not have list_paths also find symlinks to source files? This issue has come up a few times, and some users will find it useful to use symlinks.

Allowing r4 or r8 compilation on a component basis

The FMS code as of 2023.03 supports mixed-precision modeling where some components can be compiled in r4 and some can be compiled in r8. This compile option is in the mkmf templates in the FFLAGS.

https://github.com/NOAA-GFDL/mkmf/blob/master/templates/ncrc4-intel.mk#L91
https://github.com/NOAA-GFDL/mkmf/blob/master/templates/ncrc5-intel-classic.mk#L92
https://github.com/NOAA-GFDL/mkmf/blob/master/templates/ncrc5-gcc.mk#L84
https://github.com/NOAA-GFDL/mkmf/blob/master/templates/ncrc5-cce.mk#L84

This is inflexible and requires some manual work to compile a mixed precision model. A more flexible option is needed to allow users to decide which precision to compile a model component.

git-version-string adds spaces for uncommitted files

I am trying to compile with fre/test which (to my understanding) uses git-version-string in mkmf. I am testing code that I modified to see if it compiles. The problem is that when git-version-string is run, it produces output with spaces and the compiler is reading everything after the spaces as files which don't exist.

To Reproduce on Gaea:

> module load fre/test
> git-version-string /lustre/f2/dev/Thomas.Robinson/error_reproduce/mom6/src/SIS2/src/SIS_diag_mediator.F90
'ref:547c0cddeebd56c3518cc3b0e8baf44ebfa9af0b status:Modified blob:16c60ae40f3af71776a37a1224c7574dac5f60e3'

This needs to print with no spaces

Using /bin/sh in git-version-string causes problems on fresh Ubuntu systems

git-version-string uses /bin/sh which resolves to different shells (e.g. dash or bash) on different platforms.

The syntax used in the script is bash so perhaps one should use /bin/bash for the script shell.

Changing /bin/sh to /bin/bash worked for me on Ubuntu 15.04, GFDL workstation and Gaea.

Use of -fltconsistency in linux-intel.mk

FFLAGS_REPRO in linux-intel.mk contains "-fltconsistency" (Line 25) which leads to bad values for some ocean-only experiments in MOM6. Everything works on Gaea (ncrc-intel.mk) which has "-fp-model source" instead of "-fltconsistency".

Is -fltconsistency preferred? If so, we need to use in on Gaea too and then figure out why the model gives bad values. Otherwise, can we have FFLAGS_REPRO in linux-intel.mk set to the same as ncrc_intel.mk?

Change the main branch to `main`

The main branch of this repository should be renamed main to be in-line with modern/current git practices.

list_paths should be rewritten to be POSIX compliant

list_paths is currently written in csh. This should be done in a POSIX compliant shell.

intel template does not have -msse2 needed for reproducability

The option -msse2 is not in these templates even though, at GFDL, we've determined we need this option for models to reproduce between runs.