Testing new preprocessor about esmvalcore

[jvegreg]

And maybe open a specific one to improve derivation testing? See shorturl.at/cJKZ3

[bouweandela]

Regarding testing: please avoid using sys.exit() anywhere in your code, this makes the code untestable. This is because if sys.exit() is called because a single test fails, the test execution python program will exit and no other tests will be performed and no report will be generated. Instead raise an informative exception. I have added code to main.py (pull request #91) which will log any exception that is raised and goes uncaught and after that calls sys.exit(1) to keep the current behaviour.

[valeriupredoi]

current testing average for ESMValCore is about 80-85% -> not bad! There's a couple PR's with more testing, I reckon it's safe to close this? 🍺

[nielsdrost]

@jvegasbsc I cannot seem to run the tests with coverage. What is the intended usage of the run_tests.py script? running it from the esmvaltool root runs all the tests but without any coverage, running it inside the test folder gives me coverage but allmost all the tests fail. Am I missing something?

[nielsdrost]

Instead of trying to agree on the best testing framework, we use pytest to run everything, which in turn supports all the major testing frameworks.

Note that we should update the documentation

[bouweandela]

I've started a branch for implementing tests: https://github.com/ESMValGroup/ESMValTool/tree/REFACTORING_tests

[mattiarighi]

@BenMGeo might be interested in this issue.

[nielsdrost]

We should also add unit tests covering #146 and #148

[bouweandela]

Related issue that addresses part of this task https://github.com/ESMValGroup/ESMValTool/issues/228.

[valeriupredoi]

so bad stuff, apart from derive modules and main (I have a PR that ups the testing cov for main to 86%):

esmvalcore/cmor/_fixes/cmip5/bcc_csm1_1.py           25     19    24%
esmvalcore/cmor/_fixes/cmip5/bcc_csm1_1_m.py         26     19    27%
esmvalcore/cmor/_fixes/cmip5/hadgem2_cc.py           19     12    37%
esmvalcore/cmor/_fixes/cmip5/hadgem2_es.py           19     12    37%
esmvalcore/cmor/_fixes/cmip5/mri_cgcm3.py            11      4    64%
esmvalcore/cmor/_fixes/cmip6/cesm2.py                22      9    59%
esmvalcore/cmor/_fixes/cmip6/gfdl_cm4.py             21     12    43%
esmvalcore/cmor/_fixes/cmip6/mpi_esm1_2_hr.py        21     10    52%
esmvalcore/cmor/_fixes/obs4mips/ssmi.py              10      6    40%
esmvalcore/cmor/_fixes/obs4mips/ssmi_meris.py         7      2    71%
esmvalcore/preprocessor/_download.py                 45     37    18%
esmvalcore/preprocessor/_io.py                      196     54    72%
esmvalcore/utils/draft_release_notes.py              42     29    31%

everything else is above 75%