jgcri / hectordata Goto Github PK

• code coverage test
• pkgdown
• RCMD tests

This is a possible package enhancement. Do we want to include the capabilities to go from GCAM results to hector emission input file? Unclear how useful this may be but @ouyang363 asked a question related to this and it sounds like their might be other users that would find this capability helpful.

Rough draft of the description
provide a gcam query that works with rgcam to extract results from a data base or do we want to assume the data has already been extracted. Then do the correct aggregating / unit conversion to prep the data. At which point users could either read the emissions into the hector R interface to write out to the emissions file/csv file.... 🤔

Right now the hectordata inputs are saved in ins/ds and documented in a read-me.csv, due to the size of the rcmip files they should probably be included as internal package data/committed to the repo which is kind of unideal but in the previous version loaded directly from URL but this is time-intensive and also ends up being a repetitive chunk of code. Figure out a better way to work around this...

Objective: get hectordata to pass github actions and document how the problem was solved, this is a reoccurring issue.

@bpbond could you take a look at this package and let me know what you think? So far I have only added the ssp scenarios to the package and figured we could add other scenarios as needed.

Things that would be helpful to get your input on:

• Can you take a look at the example "Using the pre-built inputs" from the READ ME? I tried to make it consistent with how we read in the inputs included with hector.
• Do you know what kind of LICENSE I should be using?
• I generated the internal package data with data-raw/generate_inputs.R does that make sense? Or should that script be located elsewhere?
• other than missing the code cover, pkgdown, & R package check running on github actions what else is missing?

Right now the generate_input_tables only has the ability to process CMIP6 RCMIP scenarios. Add the capability to generate other input tables

• DECK
• CMIP5

This is something that I've been wrestling with for a while now. For the CMIP6 scenarios do we want the scenarios to match the nomenclature of pervious Hector runs or the CMIP6 scenarios?

If we take a look at the ini files in the Hector input.

[1] "api-example.ini"                "constraints"                    "emissions"                     
 [4] "hector_rcp26_constrained.ini"   "hector_rcp26_histconstrain.ini" "hector_rcp26.ini"              
 [7] "hector_rcp45_constrained.ini"   "hector_rcp45.ini"               "hector_rcp60_constrained.ini"  
[10] "hector_rcp60.ini"               "hector_rcp85_constrained.ini"   "hector_rcp85.ini"              
[13] "hector-gcam.ini" 

The runs named hector_rcp45.ini use the scenario inputs from CMIP5 RCP 4.5 but the big difference is that in hector these are purely emission driven where as in CMIP5 these runs were driven with prescribed CO2 concentrations + emissions.

So that being said for hectordata ini files do we want to set it up so that they are all purely emission driven like the runs or do we want them to match the CMIP definition of the scenarios?

@bpbond I would be happy to hear your thoughts & the thoughts of others that want to weigh in. I am inclined to think that we should be consistent with how Hector does it but as a user that is also kind of a pain...

Add a row to the csv tables stating which version of Hector will run with the ini files??

How do we want to handle the fact that not all inputs will work with every version of Hector?

@crvernon could you help us figure out the license for hectodata?

udunits 2 is orphaned 😢 consider switching to another package

FYI @bpbond ran into this issue hector github issue where the size of the inst directory is related to the number of decimals in the emissions table can bloat the size of the directory! He suggests limiting the number of decimals printed out to 4.

• Address TODOs in the code base
• Better way to handle Zenodo, where are there extra files sneaking in?
• Pipeline for generating the inputs, drake pipeline? some sort of driver?
• gcamdata base to hector inputs?
• Working github actions (waiting on hector v3 to be on cran)
• Set up renv (waiting on hector v3 to be on cran)
• EIDR / License #14 in process
• Get final feedback #13
• Tag & release (something to consider do hector & hectordata version numbers do they need to be consistent?)
• Submit to CRAN (final step must wait on hector v3)

As discussed with @ssmithClimate and @bpbond Hector, users should be easily able to run the historical AR6 & future scenarios. While not immediately needed for Hector v3, however, this is a high-priority scenario to be able to run.

AR6 replication scenario would have to be some combination of concentration & emission driven. One idea that @ssmithClimate had would be to run Hector with the concentration constraints (concentration constraints for non CO2 GHGs) & do some sort of inverse calculation to be generated the corresponding emission time series.

Inputs for the AR6 will come from https://github.com/chrisroadmap/ar6

rn have to make inst/input/tables manually if it does not exist, would be better if some internal package data did this for users

Based on a slack conversation with @crvernon here is a check list (until a more official list or template is distributed) for R package development guidelines.

• we want to be able to access for the purpose of modification or distribution (get/set) key variables per time step
• we want to take a test driven approach where we have high and meaningful coverage
  the repo should be linked either to GitHub Actions (which I am looking more into) or right now through Travis-CI and Codecov (see the Python template here: https://github.com/IMMM-SFA/im3py
• Let me know when you have the repo up and I'll also link to Zenodo. We want to have meaningful and timely releases from beta versions up with the semantic minimum of major.minor.bug-fix where a lower case "v" precedes the version number (e.g., v0.1.0)
• Each R software should use Doxygen style function docstring (like you do already)
• We want to develop our software as packages that can be distributed and direct any fundamental development to our core software (e.g., rgcam, rgis, etc.)
• The README should include a getting started section that shows how to install, a run setup section if necessary, and an few examples.
• all lowercase package names with no spaces or separators
  function and local variable names in lower case separated by underscores; meaningful names
• constants in all CAPS underscore separated stored in a separate file
• use data.table instead of tidyverse. These can still be combined with pipes from magrittr . They are so much more efficient and readable
• Only small subsets of data for tests if needed; we do not want a large GitHub repository overhead. We will use the 'install_supplement' function I have built to fetch required inputs from a remote archive that we have minted a DOI for that matches the version of the software.
• no use of GitLFS
• add the covde coverage that gives us a percent because that is cool but more importantly a better diagnostic

After the read_remote capability is merged into rpackageutils JGCRI/rpackageutils#2 use that function to download the rcmip csv files.

Add some sort of method to make sure that the data frame contains all of the required emissions or constraints. Otherwise errors will not be triggered until trying to run the Hector core.

If we merge JGCRI/hector#643 then emissions time series will have to start at 1745, not 1750, because interpolation will no longer be allowed for the LUC, FFI, and DACCS time series.

Add capabilities to process the constraints

We want to provide users with the ability to run Hector with a hector_gcam.ini file & the input csv table.

1. a script that uses the ceds by sector data & aggregates to the global total.
2. convert from CEDS units to Hector emission units
   • compare with another table of historical (cmip5 or cmip6 would expect small differences)
3. Format as a Hector csv table csv file
   • may need to add code to pull missing emissions, concentrations, rf etc.
4. Add ini file

Outstanding Questions/Problems

• What do we do have the historical period ends? End the run early? use constant inputs for the remainder of the run?
• For the HFCs & other GHGs/aerosols excluded from the CEDS dataset what should be used?
• What additional radiative forces do we want to include? (land albedo? solar? BC on snow? contrails?)
• how are ffi and luc emissions split up for ceds? It is compatible with Hector?

The Hector package and hectordata package will eventually interact with one another, we will need to add documentation so that it is clear to hector users when they will need to use hectordata and how they will do so.

Generate ini files in some method that makes it easy to determine what emissions or concentrations to use... what changes is if the constraints are used or not... may be some function that is capable of replacing all of the strings (with the exception of the volcanic related ones)

may be what we really need is ability to turn on and off constraints.... 🤔