Comments (9)
@matthewcornell I actually see these as essential checks for the data integrity, although I recognize that they are not essential for Zoltar. Does it not make sense to add them in zoltpy? I was thinking that is where it makes sense for them, but am open to other places (or having some other python scripts run that aren't in zoltpy) as well.
from covid19-forecast-hub.
For x day ahead
targets the target_end_date
should be forecast_date + x
from covid19-forecast-hub.
Assume a numbering of weekdays where Sunday=1, Monday=2,..., Saturday=7.
For x week ahead
targets:
weekday(target_end_date)
should be 7
(i.e., a Saturday) AND
if weekday(forecast_date)<=2
target_end_date
should = forecast_date + (weekday(target_end_date) - weekday(forecast_date)) + 7*(x-1)
% this ensures that the 1-week ahead forecast is for the next saturday, and so on
if weekday(forecast_date)>2
target_end_date
should = forecast_date + (weekday(target_end_date) - weekday(forecast_date)) + 7*x
% this ensures that the 1-week ahead forecast is for the saturday after next, and so on
from covid19-forecast-hub.
I documented the checks that are done via zoltpy here: https://github.com/reichlab/covid19-forecast-hub/wiki/Validation-Checks . B/c it only checks the essential columns, I assume the above checks should be done in R (i.e., they are "statistical sanity checks," not "structural" ones).
from covid19-forecast-hub.
- Does it not make sense to add them in zoltpy? It's a judgement call. From a functional perspective, no b/c they have nothing to do with what gets sent to zoltar. But if the checks depend not just on row-level validation but on prediction-level (grouped) quantiles, then that could be a stronger argument for putting them in zoltpy, which already groups them for zoltar. However, to me it looks like they are all row-level ones.
- are we validating the FIPS locations based on the specific set of valid numbers: no. I can add that.
Reviewing, the only COVID-specific code in zoltpy is: 1) location being a FIPS code, and 2) the target names. I can refactor the code to extract these as args. This would make me feel a better about putting more validation in zoltpy.
from covid19-forecast-hub.
- Could we add another check to require point estimates?
If the forecast does not have point estimates, it breaks the visualization because it has no point to plot. Unless @nickreich we would like to be flexible in allowing point estimates?
from covid19-forecast-hub.
We probably need to split this issue up into multiple issues.
I suggest we eliminate forecast_date and target_end_date. We should use the date in the filename as the forecast_date and calculate the target_end_date using the formula above. This reduces duplication and therefore the possibilities for errors. Although I understand it eliminates the ability to make checks.
In my script to read in the processed data, I check explicitly for the 50 state fips codes using the fips file.
@matthewcornell mentioned checks in R, where are these checks?
from covid19-forecast-hub.
@jarad thanks for the pointer to the file. Re: eliminate forecast_date and target_end_date
, I'll leave that to you all, but with the setup like it is, I think changing something this major will meet resistance. re: checks in R
, I don't recall this. I haven't ported the quantile code from zoltpy to zoltr yet.
from covid19-forecast-hub.
@katiehouse3 is the requirement exactly one point estimate per location/target pair, or are multiple ones allowed?
from covid19-forecast-hub.
Related Issues (20)
- Consolidate approved licenses into one master list
- latest/ihme-covid19.zip is 0 bytes in size HOT 2
- 'Run failed: Trigger zoltar upload' messages HOT 2
- Remove legend title for state weekly reports HOT 1
- dead link to dataset dictionary
- delete visualization folder HOT 1
- delete static plots folder
- clean up travis scripts HOT 3
- delete template directory
- delete packrat issue
- clean up code folder
- [email protected]
- How online behavioural advertising works
- upload_covid19_forecasts_to_zoltar.py updates validated_file_db.json in spite of failed upload jobs
- CI failing due to Python dependencies HOT 1
- truth data truncated HOT 2
- `truth-Incident Hospitalizations.csv` not updated HOT 3
- Submissions are not being included in the ensemble HOT 1
- fix broken upload_to_zoltar.yml action
- Update main README citation info to publication not medrxiv HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from covid19-forecast-hub.