Collection of definition files for creating containerized coupled models
The containerization of an executable, be it a coupled model or the WRF code, allows you to freely move the executable to other platforms, which may run different operating systems. In effect the containerization insulates you from variations in operating systems, including future updates.
The executable is encoded in a "container image", a static file that contains the executable with its dependencies (compilers, tools, libraries and operating system). In the case of "Apptainer", this is a "sif" file, for instance "wrf.sif".
To access the "apptainer" command, type """ ml Apptainer/1.2.2 """
To run any executable, e.g. WRF, within the container, type
apptainer exec wrf.sif /software/WRF-4.1.1/main/wrf.exe
where "/software/WRF-4.1.1/main/wrf.exe" is the full path to the executable stored inside the container. Additional options can be passed to this executable if desired.
MPI calls can be delegated from inside the container to the host, which, in the case of SLURM, will manage resources. For instance,
...
#SBATCH --ntasks = 40
...
module load Apptainer
module load intel # load the Intel MPI
export I_MPI_FABRICS=ofi # turn off shm to allow the code to run on multiple nodes
# -B /opt/slurm/lib64/ binds this directory to the image when running on mahuika,
# it is required for the image's MPI to find the libpmi2.so library. This path
# may be different on a different host.
srun apptainer exec -B /opt/slurm/lib64/ wrf.sif /software/WRF-4.1.1/main/wrf.exe
will assign 40 MPI tasks to the containerized executable.
For this to work, the MPI version inside the container has to match that on the host. The images are built using
Apptainer> mpiexec --version
Intel(R) MPI Library for Linux* OS, Version 2021.8 Build 20221129 (id: 339ec755a1)
Copyright 2003-2022, Intel Corporation.
On mahuika, we have
ml Apptainer
ml intel
$ mpiexec --version
Intel(R) MPI Library for Linux* OS, Version 2021.5 Build 20211102 (id: 9279b7d62)
Copyright 2003-2021, Intel Corporation.
You will likely need to access data stored on NeSI's file system. As built, the image will mount:
- /nesi/nobackup
- /nesi/project
- /home
So you can
...
cd <wrf_input_dir>
ml intel
export I_MPI_FABRICS=ofi
srun apptainer exec -B /opt/slurm/lib64/ wrf.sif /software/WRF-4.1.1/main/wrf.exe
and run the application therein.
To access the "apptainer" command, type """ ml Apptainer/1.2.2 """
Type
sbatch wrf_build.sl
to build the wrf container. This will take definition file "wrf.def" and build the image "wrf.sif" from it.
You can check that the "sif" file built correcty by typing
apptainer shell wrf.sif
You will land in a shell environment. You may type any Unix command. For instance, check that the executable "wrf.exe" has been built:
Apptainer> ls -l /software/WRF-4.1.1/main/wrf.exe
-rwxrwxr-x 1 root root 57364712 Oct 17 20:23 /software/WRF-4.1.1/main/wrf.exe
Type "exit" to leave the shell.