mkmf template instructions in readme about isca

[jamesp]

Ah yes. OK I have a refactor somewhere that pulls the NETCDF_LIBS into an export in the environment file, the nc-config command has not proven to be as robust across machines as I'd hoped. I'll dig it out the code changes and look at putting them into the master.

With regards your second point, it's a very common scenario and there are lots of ways to do this.
Some options:

1. Remove the cb.compile() line from the script. Create another script that just creates the codebase object and compiles it. Then run that on your login node, and the experiment script on the compute.

2. Put something like this around the compile statement:

if 'compile' in sys.argv:
    cb.compile()
    sys.exit()

Then run on your login node as:
$ python held_suarez_test_case.py compile
and on the compute nodes as:
$ python held_suarez_test_case.py

3. Take a look at run_cli in isca.util instead of calling exp.run. Here is an example script using it: https://gist.github.com/jamesp/26b550fbcd79f31c6c1f17836f73155e. It allows you to control running of the experiment from the command line.
   So to compile:
   $ python held_suarez.py --compile
   And to run:
   $ python held_suarez.py --run --up-to 10 -n 16
   This is how I run on a super computer cluster: I run the compile line on the login, and submit the second line to the job queue. It also allows you to choose the number of cores you want to use at the command line rather than in the script.
   $ python held_suarez.py --help
   on the command line will show you all the available options.

[jamesp]

Thanks for the feedback Timothy, good to hear you've picked it up so quickly!

The hope was, that if you are using either ifort or gfortran to compile the code, there should not be any changes needed to mkmf.template.ia64. The complete environment should (hopefully) be configured from a file in src/extra/env.

Could you show the changes you had to make to get the model to compile? Either we can refactor them into an environment file, which would be great, or there is a compilation case we haven't come across before and we can encorporate your changes into the framework.

[timothyas]

I had to explicitly point the compiler to our netcdf directories even though the appropriate netcdf include and library paths are present in the $INCLUDE and $LD_LIBRARY_PATH environment variables. The changes were as follows for mkmf.template.ia64

git diff mkmf.template.ia64
-CPPFLAGS = -I/usr/local/include
-NETCDF_LIBS = nc-config --libs
+CPPFLAGS = -I/usr/local/include -I${NETCDF_INC} -I${NETCDF_FORTRAN_INC}
+NETCDF_LIBS = -L${NETCDF_LIB} -L${NETCDF_FORTRAN_LIB}

where we are using netcdf 4.4.1.1 and the C libraries/headers are in a different directory than the fortran libraries/headers. The netcdf environment variables are just what we set within our module system.

Also, I can open a separate issue for this, but I would like to compile on our login nodes but run on the compute nodes. The python scripts seem to want to do both at the same time with exp.run is there a way to separate these?

[timothyas]

Thanks for the help @jamesp. In my own experience I have also found nc-config to not work very nicely.

The compile + run suggestions helped out, thanks so much. Cheers