"Compute Missing/Stale Covidcast Direction" job dies on missed data_stdevs key about delphi-epidata

[capnrefsmmat]

Ah, that would make sense as the cause of the problem. I believe April 22nd was chosen only because the current direction rule was selected late at night on the 22nd in a long argument on the #covid-19-trend channel and several Zoom calls.

The conversation isn't easy to follow, but I think the reasoning is:

• The direction call should depend on the historical variation in the signal. If the signal always bounces up and down, we should only call it increasing or decreasing if the slope is greater than normal.
• Direction calls should be stable; past directions should not change when new data arrives.
• Therefore, we need to estimate the variation in the signal over some fixed time period, and use that to call all subsequent directions.

Unfortunately the fixed time period is hardcoded and David wasn't able to stay indefinitely to move us to a better procedure.

I believe we could satisfy the spirit of the rule for new signals by imposing this procedure:

• If at least one month of data is available, use the first month to estimate the standard deviation, rather than thresholding on a hardcoded SIGNAL_STDEV_MAX_DAY.
• If less than one month of data is available, do not report direction, since we have inadequate data to judge what makes the trend genuinely increasing or decreasing.

[krivard]

Same crash is happening even when no signal names are too long. Reopening and renaming.

[krivard]

The problem always occurs on a wip_ signal, and may be linked to the way we compute the “historical” stdev, since we require at least one row for a signal to have a time_value less than 20200422. This is a problematic hypothesis, because if it were true, wip_ signals should never have worked, should always have crashed the direction updater, because support for wip_ signals wasn't added until May 1.

Perhaps the WIP signals we've added so far all had historical data available going back to before 22 April?

We will need to address this before we can add any signals with a start date after 22 April. @capnrefsmmat are you familiar at all with why we decided to compute the slope threshold like this, and is there an obvious alternative we could use for new signals that don't have a historical record?

[krivard]

Direction is being removed, so this is no longer a problem.