eco4cast / neon4cast Goto Github PK

There's a handful of functions I think are no longer used, mostly helpers that use the more cumbersome aws.s3 package to download stuff. Is it okay to remove these?

• download_scores.R: (aws.s3 method replaced by much faster method in combined_scores.R)
• download_forecast.R: (read_forecast() can read both nc and csv directly from URL)
• download_target.R: just a wrapper that converted theme names to download urls, maybe not that helpful?
• s3_helpers.R: old aws.s3-based wrappers for downloading, no longer used

I think we also might move noaa.R to a noaa4cast package, since I think that supports use cases that go beyond the neon4cast challenge.

Still core things to keep here:

metadata.R, submit.R validate.R, forecast_output_validator.R fable_helpers.R
(some of these may want to be moved to a more general forecast challenge utilities package?)

Hey @rqthomas, is it okay to post issues here? Happy to wait until this package is more developed to chime in. Or I can create a pull / merge request if you like? Trying to get the write_metadata_eml() function to work and hitting a couple of speed bumps.

Just a couple of quick observations....

1. A call to basename() might be nice here forecast_file_name_base <- tools::file_path_sans_ext(tools::file_path_sans_ext(basename(forecast_file))) if folks are caching forecast files in subdirectories as opposed to the project root;
2. num_variables still needs to be defined in the control flow for if ("statistic" %in% names(forecast))

Even with the above patches made I'm still hitting an error

Error: file missing metadata_standard_version

Not quite sure how deeply I wanted to dive into the EML package to sort that one out, and figured you might know the solution off hand.

Again, sorry if reporting here is not what is desired. Happy to delete this and engage offline too.

I'm getting the following error when trying to use stack_noaa():

It's thrown by line 85 of noaa.R. However, when I'm debugging, my out tibble is empty, so this may be an upstream issue.

Please advise!

https://github.com/eco4cast/neon4cast/blob/main/R/noaa_gefs.R

since it's not hive-partitioned, we just need to literally change the name on the partition here

Hello! I am running into an error when creating the metadata for our team's submission. We currently store the forecast submission csv and metadata in a subdirectory of our root folder to keep organized. We use the provided create_model_metadata function with the path to our forecast csv to create the metadata, which works without any issue. However, the write_metadata_eml function throws an error that '' does not exist in working directory.

This error seems to be thrown when theme is used to create file paths that do not exist in the package. I believe this is because theme expects our file path for forecast_file and metadata_yaml to begin with the challenge theme instead of subdirectories for our file path. For example, theme is assigned as "data/forecasts/phenology" for our file path, but it seems to work properly if only "phenology" is included. However, I imagine removing these subdirectories would require us to store all files in the root directory, which is not a great option organizationally. Do you have any suggestions for a workaround here, and has any other team run into this problem? Thanks so much for your help and great work putting together these functions for submission!