thilinarmtb / nekbench

Shell scripts for benchmarking Nek5000 across different platforms.

Hi there 👋

nekbench's Issues

Add pre-bench and post-bench scripts

Add pre-run and post-run scripts for machines so that user can set the environment before the benchmark and do other processing activities like plotting after the benchmarks. For a given machine, these can be stored in a folder under machines.

Dump ppn in `dump_metadata`

Allow users to store their default values in a configuration file.

For repeated use of NekBench, it becomes tedious to type out the command even with the added universal default values. The user should be able to assign variables in a file, let's say user.config, and NekBench will source that file before checking for required specification and running.

Remove redundant vars e.g., nb_lx1_set

Whether lx1 is set can be determined by inspecting nb_lx1_list. Currently, we are setting the flag when we alter the list, but after the user gets control of the configuration via a config file, they may not necessarily remember to set the flag. This is true for other flags. Therefore, we should remove the redundancies to reduce the risk of user error in the future.

Create a run directory for each different run

Even if the user runs the NekBench script twice with exactly the same arguments, those
data should be collected in separate run directories.

Update README to reflect changes in code

#20 changed the directory hierarchy and these changes need to be documented in README.md.
Also we have a lot of changes now. So we need to document them.

Redirect errors/warnings from bash commands to /dev/null

Example:

mkdir lelt_{nb_lelt} 2>/dev/null

Make sure the $nb_nnodes in machine file is postive

Due to the current integer division we use, this can be zero if $nb_np is less than $nb_arg4 (rpn).

Remove all the test types and run directory structure

Ask user to give a name for the run. Create a directory inside the machine and just run his
case. No need to do anything other than that.

Add a Clean option

I think a clean option that removes everything inside runs/ will be convenient.

Add an option to not run a test

The user may want to simply run a case with the specified values for the various options so that they can process the output (logfiles or fld files) themselves. We can add support for common use cases as we gain new users, but there should always be an option to just 'run'.

Use double quotes to reference variables

Add machine specific limits

Add machine specific limits (like queue limits) for each machine, so that the NekBench scripts will respect these limits when submitting jobs.

Default value for `nb_np_list` parameter.

PR #15 set the default value of nb_np_list to 1 if the user did not provide any.
One problem I see with that is the given case may not run with a single MPI rank.
May be a better way to do this may be to set a nb_np_list value depending on
the machine. We can move the definition of the variable to appropriate machine
file and set it accordingly. For example, for linux machine type this can be 4,
for cetus, this can be 64.

@kent0 : What do you think?

Variable substitution in SIZE fails.

Variable substitution in SIZE fails when the parameter is assigned an expression instead of a literal e.g., (lelt=4) is good, (lelt=lelg/lpmin+4) fails.

Allow testing of various parameters in rea

readlink -f is invalid on MacOS 10.12.6

Cleanup jobscripts/submit.* file

Move machine specific calculations in jobscripts/submit.<machine> (submission files) to the machine
files: machines/<machine>.

Create a function to print output

This should support indentation levels;

command: print "Error: Nek5000 build failed" 0
output : Error: Nek5000 build failed
command: print "Error: Nek5000 build failed" 1
output :   Error: Nek5000 build failed
command: print "Error: Nek5000 build failed" 2
output :     Error: Nek5000 build failed

Create an inheritance hierarchy for jobscripts

All the job scripts (except linux) either uses qsub or srun. So technically we only
need two scripts, we can reuse them for each different machine with machine specific
changes.

Change filename convention

I think it will be more consistent if the files adhered to the common filename convention e.g., instead of makenek.cetus, it should be cetus.makenek since makenek is the family of files and the name before the dot separator should be the unique identifier. This also has the effect of increasing user friendliness as they won't be required to type as much to edit a specific file.