hypre-space / hypredrive Goto Github PK

Improve error handling in the following scenarios:

• Filename not found (matrix, RHS, or dofmap)
• Missing level keyword when using MGR

See TODO in the code:

https://github.com/victorapm/hypre_app/blob/4f513e404b2f60c918088e490a27ce5aeb672c40/src/amg.c#L204-L210

Add interfaces to other languages

• MATLAB
• Python
• Julia
• Rust
• Fortran

Depends on #27

... in the main driver

Fix LDFLAGS when compiling with externally built Hypre library (GEOS-TPL)

To be added under test/system for the purpose of testing whole system (end-to-end) application, e.g.,

1. Does YAML parsing succeed?
2. Does hypre_app finish completion with a zero error code?

Consider using pytest for generating the program calls and checking the output

Given a linear system, possibly distributed in many files (one of each rank), we would like to allow run a graph partitioner such as parmetis, ptscotch or zoltan.

This should work for redistributing the input LS to any number of ranks. For example, we can read using 8 ranks, and redistribute to 4 or 16...

We would like to allow something like this:

general:
  warmup: no
  num_repetitions: 1

linear_system:
  type: ij
  rhs_filename: IJ.out.b
  matrix_filename: IJ.out.A

solver:
  pcg:
    relative_tol: 1.0e-9
    max_iter: 500
    print_level: 3

preconditioner:
  amg:
    -
      print_level: 1
      coarsening:
        type: HMIS
        strong_th: 0.25
      interpolation:
        prolongation_type: MM-ext+i
      relaxation:
        down_type: 16
        down_sweeps: 1
        up_type: 16
        up_sweeps: 1

    -
      print_level: 1
      coarsening:
        type: PMIS
        strong_th: 0.5
      interpolation:
        prolongation_type: standard
      relaxation:
        down_type: 8
        down_sweeps: 2
        up_type: 8
        up_sweeps: 2

  # Add more configurations as needed

Also, something like this:

general:
  warmup: no
  num_repetitions: 1

linear_system:
  type: ij
  rhs_filename: IJ.out.b
  matrix_filename: IJ.out.A

solver:
  pcg:
    relative_tol: 1.0e-9
    max_iter: 500
    print_level: 3

preconditioner:
  amg:
    - config_1.yml
    - config_2.yml

And same for solver...

hypre_app would then run all possible combinations.

This can be the early stage of an autotuner.

• Change repo address
• hypre_app -> hypredrive
• Append public APIs with HYPREDRIVE_

Add --enable-colors configure option, which will define the HAVE_COLORS macro

This should be enabled by default

The number of input arguments we accept via the YAML file, and their respective valid values, evolve over time. We need a simple way to tell users:

1. Which options they can use?
2. Which values are accepted by those options?

One idea is via command line, let's say:

$ ./hypre_app --help

  Usage: ./hypre_app input.yml

  List of valid sections for input.yml:

    general
    linear_system
    solver
    preconditioner

  For more information about which section: ./hypre_app --help <section>

Then, we might do:

$ ./hypre_app --help solver

  List of valid values for the solver:
     pcg - Preconditioned Conjugate Gradient
     gmres - Generalized Minimal RESidual
     fgmres - Flexible Generalized Minimal RESidual
     bicgstab - Bi-Conjugate Gradient STABilized

  List of valid keys under the solver section:

    max_iter: <val> - val is a positive integer number
    print_level: <val> - val is a positive integer number in {0, 1, 2}
    tol: <val> - val is a non-negative real number
    ...

Build a library out of the hypre_app project.

We should expose, via a public header file, mainly the functions needed to setup LS, preconditioner, and solver (as we have in main.c). Then, main.c would work by including only such a header

Consider adding an option to allow export only of the public debug symbols

Should use doxygen syntax and later generate the html/latex outputs.

Need to create a rtd page as well

1. Add CONTRIBUTING file
2. Add NOTICE file
3. Update README layout

The new python script would add new options to a given argument's struct, e.g.,

Suppose we want to add the field iter_setup to ILU:

 $ ./add-option.py -n "iter_setup" -t "HYPRE_Int" -a "ILU_args" -c "HYPRE_ILUSetIterSetup"

And this would update ILU_args struct with a new iter_setup option

Make sure hypre_app works with a hypre lib build externally, e.g., via GEOS-tpls

Assuming:

1. base.yml:

general:
  warmup: no
  num_repetitions: 1

linear_system:
  type: ij
  rhs_filename: IJ.out.b
  matrix_filename: IJ.out.A
  dofmap_filename: dofmap

2. solver.yml

solver:
  fgmres:
    max_iter: 500
    relative_tol: 1.0e-3
    print_level: 2

3. prec.yml

preconditioner:
  mgr:
    print_level: 1
    level:
      0:
        f_dofs: [0]
        f_relaxation: single
        g_relaxation: none
        restriction_type: injection
        prolongation_type: jacobi
        coarse_level_type: rap
    coarsest_level:
      amg:
        coarsening:
          strong_th: 0.3

We would like to do:

  $ ./hypre_app base.yml prec.yml solver.yml

and the YAML inputs would get concatenated internally as:

general:
  warmup: no
  num_repetitions: 1

linear_system:
  type: ij
  rhs_filename: IJ.out.b
  matrix_filename: IJ.out.A
  dofmap_filename: dofmap
preconditioner:
  mgr:
    print_level: 1
    level:
      0:
        f_dofs: [0]
        f_relaxation: single
        g_relaxation: none
        restriction_type: injection
        prolongation_type: jacobi
        coarse_level_type: rap
    coarsest_level:
      amg:
        coarsening:
          strong_th: 0.3
solver:
  fgmres:
    max_iter: 500
    relative_tol: 1.0e-3
    print_level: 2

We shouldn't have limitations on the number of input files.

Note the order of solver/preconditioner shouldn't matter

In reference to JOSS review thread: openjournals/joss-reviews#6654.

With regard to the state of the field, are there other solver libraries/languages that have quick solver prototyping? For example, MATLAB has rapid prototyping for choosing solvers and options. What specifically about PETSc inspires this work?

With regard to the statement of need, who is the target audience (hypre developers, hypre users, external users, application developers)?

The warmup stage is not timed and it could consist of:

• Solving a small 7-pt poisson problem
• Solve the test case itself (but do not count toward timings)

First idea seems more adequate.

Some options are missing:

• reordering
• tri_solve
• lower_jac_iters
• upper_jac_iters

Instead of passing in numbers, we would like to inform labels (suggested by Matteo), e.g.,

preconditioner:
  mgre:
    level:
      0:
        f_dofs: [wells]
      1:
        f_dofs: [water_density]
      2:
        f_dofs: [oil_density]

This will require a mapping between dof markers and labels, e.g.:

linear_system:
  dof_labels:
    wells: 3
    water_density: 1
    oil_density: 2
    pressure: 0

or

linear_system:
  dof_labels:
    3: wells
    1: water_density
    2: oil_density
    0: pressure

• Solving a sequence of linear systems (with AMG?)

Several options parsed for BoomerAMG and ILU are not being passed to their respective constructor functions.

We need to fix some issues in the syntax of our YAML strategy, which is not currently compliant with the standard.

Target one should be something like:

general:
  warmup: no
  num_repetitions: 1

linear_system:
  type: ij
  rhs_filename: IJ.out.b
  matrix_filename: IJ.out.A
  dofmap_filename: dofmap.out

solver:
  type: fgmres
  print_level: 2
  tolerance: 1.0e-6
  max_iterations: 500
  subspace_dim: 30

preconditioner:
  type: mgr
  print_level: 1
  pmax: 0
  levels:
    0:
      f_dofs: [0]
      f_relaxation:
        type: jacobi
        sweeps: 1
      g_relaxation: none
      restriction_type: injection
      prolongation_type: jacobi
      coarse_level_type: rap # coarse_level? schur_comp? reduction_strategy?
    1:
      f_dofs: [1]
      f_relaxation: none
      g_relaxation:
        type: ilu
        subtype: fill
        fill: 0
        sweeps: 1
      restriction_type: injection
      prolongation_type: jacobi
      coarse_level_type: cpr-like-bdiag

  coarse_solver: # coarsest_solver?
    type: boomeramg
    print_level: 0
    strong_th: 0.3
    pmax: 6

Implement preconditioner reuse at every "n" linear system solves

This requires hypre_app to work with a sequence of linear systems, thus depends on #3

We currently support only one linear system (matrix, rhs, dofmap) per YAML input.

The idea here is to allow users to pass in a sequence of linear systems (different matrices and vectors) to be solved by the given preconditioner/solver strategy defined in the YAML input.

Need to allow for a new linear_system structure in the YAML file. Example:

linear_system:
  directory:
    base: hypre-data/ls_
    init_suffix: 00000
    last_suffix: 00349
  matrix_filename: IJ.A.out
  rhs_filename: IJ.b.out
  dofmap_filename: dofmap.out 

This would read all:

• matrices with prefix name as IJ.A.out defined in the sub-directories starting from ./hypre-data/ls_00000 up to ./hypre-data/ls_00349.
• Same as above for the rhs vectors
• Same as above for the dofmap vectors

Allow users to oversubscribe input arguments passed from the YAML file via command line.

Example:

Suppose we have a YAML input like:

solver: gmres
  max_iter: 50
  print_level: 1
  tol: 1.0e-9

The idea is to allow users to oversubscribe, let's say the max_iter parameter, via command line such as:

$ ./hypre_app input.yml --solver:gmres:max_iter 100

Note the YAML (key, val) pairs are effectively being concatenated with the : character.

Also, we would like this to work:

  $ ./hypre_app input.yml --linear_system:execution_policy host
  $ ./hypre_app input.yml --linear_system:execution_policy device

Add execution policy option under general.

Consider a new organization structure:

├── data
├── src
    ├── solver
    ├── precon
    ├── linsys
    ├── misc
├── include
├── examples
    ├── amg
    ├── mgr
     ...
└── test
    ├── system

The folders:

1. examples: quick-start examples for users
2. data: some example matrices/vectors
3. test/system: test whole system (end-to-end) application

This new struct will hold pointers to the linear system data (matrix, vectors) and secondary info like the dofmap array.

It will adjust according to the conceptual linear system interface in use.

It will have set/get functions for accessing the internal data

After hypre-space/hypre#1046

Compute the final residual norm and report it (just after the iteration counts column)

They should start at 1, not zero, see:

https://github.com/victorapm/hypre_app/blob/9e62d73ab09411c95a3cb1a8ade91b1fb79052b8/src/fsai.c#L44-L46

We need to have simple timers based on MPI_Wtime on the most relevant stages of the driver (read LS, setup, solve)