Comments (15)
Thank you for considering this.
I will begin implementing the code coverage in Icepack. It should not require much except for adding the --coverage
certain flag to FFLAGS
/CFLAGS
in the Travis CI build environment. With this flag enabled, a series of coverage files are generated when the test suite is run. These files are inspected afterwards with gcov
.
from cice.
I've suggested some namelist combinations at
https://docs.google.com/spreadsheets/d/1dMzOC2GAsGqXJvs7xHybCQkPRneeUm78Ah1OXp34bAQ/edit?usp=sharing
Column B is the default ice_in values, and columns C through I are my proposed groupings. There is some overlap, but I don’t think that’s a bad thing. The remaining columns are existing options, except M (gx1prod, for production) which I’d like to add to the repo, and S ("individual options") which is exactly that — I didn’t want to make a separate column for each one of them.
The items in red need to be created or changed from what we currently have in cice. (And we can do things differently from what I suggest here.)
I have not included any zbgc options yet. I think @njeffery is working on that. I’m hoping they can be included in columns F through I, but maybe not because zbgc is sensitive to thermo choices. We might be able to do the skeletal-layer BGC tests in F or I.
I have not included history options other than the ones that are already being tested.
There are a few other items that aren’t covered, and I’m not sure they all need to be. Let’s see what @anders-dc's codecov comes back with. Also, I did not include the hundreds of numerical parameters in the list, only a few that are relatively new or need to be set with a particular flag, or that control whole parameterizations.
As with Icepack, we’ll probably find that some of the options do not work together, but this is a start. Also, I noticed in some of the set_nml.* files that there are namelist entries which aren’t available (yet) for CICE. I would expect those to cause the code to crash! Or the code might be doing a default rather than what we think it is…
from cice.
from cice.
@eclare108213, I will try to put some of these new options and new tests in place. I think a general question is do we want to create individual options to turn on various specific pieces and then have the test be dyneap_tdedd_kdyn1_cbubbly_.. or do we want to have some the individual options (which I think are generally useful), but then create options called testA, testB, testC (or some other name) which would group many of the settings? Any opinions?
from cice.
Just to add quickly. The testA, testB, testC thing is quite convenient but you can't really tell from the testname what is being tested. While the long, multiple option way of describing a test provides some greater insight into the test when results are posted.
from cice.
I agree, but the names get really long. We should keep small collections of namelist combinations that need to all be changed together, for everyone's convenience. I'm not sure we need every individual change in a separate file, but I'm not opposed to that. I was thinking that our tests might be named something more useful, even if they don't include every item in the name. I'm traveling for the next few days but hope to have time to look at this. In the meantime, please do whatever you think makes most sense...
from cice.
I agree than an optimal strategy might be somewhere in between. Group multiple namelist settings together to some degree then setup tests with may 3-4 options. Maybe we can group them by dynamics, physics, and a couple other groups? Another idea might be to setup options files that contain a list of other files. So files.testA might contain a list of set_nml and set_env files or something. It would be pretty easy to get the scripts to do this. Then, at least the options can be set in just one place and other "higher level" settings can pick those up. I will play around with these ideas and see what seems to work. We can continue to evolve the implementation until we find something we like.
from cice.
Looking through the alt01, alt02 etc options in the spreadsheet, I realize that alt01 is not consistent in that kdyn=0 (off) and basalstress and formdrag are on. It's not a bad idea to test that kind of combination, but it's not a useful test for basalstress and formdrag. basalstress=T also for alt03, so maybe formdrag should be moved to that one too.
from cice.
I have setup some new options in the scripts and am testing the "new" alt01 now. I can't get it to run. It blows up. I have tried a few different combinations of namelist. even just setting NICELYR=1, kcatbound=1, kitd=0, ktherm=0 with no pond tracers results in an abort. I think it might take some careful testing to come up with some cases that actually do run. Maybe what I can do is setup the test configurations as described in the document in the CICE scripts and create a PR, even if they fail. And then we can all work thru the various configurations to get some reasonable configurations working?
from cice.
Are you starting from the restart file? If so, try starting from an ice-free initial condition (I think ice_ic='none'). If the ITD or thermodynamics is different, it will blow up using the restart. If no ice grows because of the forcing, we'll need to fix that...
from cice.
I just want to bump this, I think having an easy way to visualize coverage is very important. I just noticed that the routines 'global_sub' and 'global_sum_prod' in ice_global_reductions.F90
are never called in the code with the 'lmask' argument present...
For cross-reference purposes : see CICE-Consortium/Icepack#202
from cice.
Thanks @phil-blain, this is on my task list and I have tended to defer it. I will make some time in the next month to have a look. I agree it's something that should have been setup months ago.
from cice.
A code coverage capability has been added to Icepack and CICE with PRs CICE-Consortium/Icepack#313 and #434. The results can be seen here,
https://codecov.io/gh/CICE-Consortium/Icepack
https://codecov.io/gh/apcraig/Test_CICE_Icepack
The CICE results are from a separate repository (apcraig/Test_CICE_Icepack) because submodules are not supported in codecov.io analysis, so a special repository is created for coverage testing that incorporates icepack not as a submodule. This is covered in the documentation.
For now, I think we will run the code coverage testing as needed (maybe every few months) to assess coverage and update test suites.
Both Icepack and CICE contain a badge in their README.md files that point to the coverage results and that also seems to be working.
I will be making a revision to the documentation, but believe we can close this now.
from cice.
@apcraig why not create the combined repo in the CICE-Consortium organization ? it would appear more "official" no ?
from cice.
My worry is that the community might use it for other things. Eventually, I hope codecov will support submodules and then we can test directly out of the Consortium CICE repo. Until then, I prefer this "fake" repo be hidden from the community and not appear in the Consortium organization. If others disagree with this strategy, we could easily migrate the test_cice_icepack repo.
from cice.
Related Issues (20)
- ice_in variables undefined in chapter 5 of docs HOT 4
- shortwave dEdd_snicar_ad setup HOT 10
- grid_location is missing for C and CD grids for call to seabed_stress_factor_LKD HOT 1
- model abort due to dvice negative HOT 11
- Investigate T-Test in cice.t-test.py
- NLON, ELAT not computed when TLAT, TLON, ANGLET on grid file HOT 14
- Some CMIP variables are computed using a mix of U and T quantities HOT 1
- dxT and other grid length variables
- dsnow optional argument in icepack_step_therm1 HOT 2
- evp1d performance evaluation
- Support netcdf-4 compression & chunking HOT 1
- hist_avg on multiple streams writes the same filenames when .false. HOT 9
- Potential instability related to explicit treatment of Coriolis for C-grid HOT 3
- Arguments in update_state need if present. HOT 2
- dynamics U points are active when T points are not HOT 5
- Commit/PR process HOT 3
- PIO and hdf5 failures HOT 4
- Test various restart formats and add Derecho port that uses pio spack HOT 1
- tripole initial/restart file with inconsistent values on the tripole seam HOT 15
- CICE C-grid crash on 1/12 degree tripole HOT 11
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cice.