calculation error in NYC 28-day-average daily percent positive ? about coronavirus-data

[hasedgwick]

I had thought that the 7-day values were also de-duplicated, but from what you say I gather that is done, if at all, outside of test.csv. Thank you again for your quick reply. I very much appreciate your responsiveness to my inquiry.

…

On Wed, 20 Apr 2022 at 10:39, M. Montesano ***@***.***> wrote: These are calculated from our internal systems from protected person-level data. The difference in values is rooted in the fact that in tests.csv, a person may be counted once each day ("de-duplicated by day") whereas the 28-day value de-duplicates over that time period, counting a person only once in that 28-day period - since, again, this is "people tested who tested positive" and not just "percent of tests that are positive." — Reply to this email directly, view it on GitHub <#243 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AYZG6QPTJVJRMJ6BYEFS4JDVGAJK3ANCNFSM5TXBFM3Q> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[mmontesanonyc]

Hi, thanks for your question. There's no calculation error. The 28-day value for people positive may differ from hand calculations from daily data because it is not an average of percent positive values over that 28-day period. As with other percent positivity data, it's de-duplicated by person for the time period in question.

The Increasing/Decreasing flags are triggered if the 7-day value is more than 10% above or below the 28-day value. This offers a very high level comparison of the last week of data compared to the last month of data. Slow but steady increases (as we're seeing with percent positivity) might not exceed this threshold.

Hope this helps clear up any confusion.

[hasedgwick]

Hello M. Montesano. Thanks so much for your quick response. I understand your point that the results of hand calculations will not precisely match the reported results. The discrepancy I am seeing, however, looks quite large. Please take a look at the attached screenshot, which shows a portion of the text.cvs table on GitHub. The left-hand marked column lists dates for the 28 days ending yesterday. The right-hand marked column is labeled (off the top of the screen) “PERCENT_POSITIVE_7DAYS_AVG”. I admit that we cannot calculate the 28-day average percept positive from this data, but no calculations are needed, I believe, to see the point that I am making. Just by scanning down the right-hand column, it is possible to see, I believe, that this data is not consistent with a 28-day average of 0.0449, which is what was reported yesterday. All but 4 of the 28 averages are lower than 0.0449, and most of them are quite substantially lower. Very roughly, the 28-day average looks to be somewhere in the mid 3 percent range. If this is still unconvincing to you, I would be grateful if you could point me to where the 28-day average daily percent positive is calculated, so that I can understand my error. Thanks again for your time and attention. H A Sedgwick

…

On Tue, 19 Apr 2022 at 14:44, M. Montesano ***@***.***> wrote: Hi, thanks for your question. There's no calculation error. The 28-day value for people positive may differ from hand calculations from daily data because it is not an average of percent positive values over that 28-day period. As with other percent positivity data, it's de-duplicated by person for the time period in question. The Increasing/Decreasing flags are triggered if the 7-day value is more than 10% above or below the 28-day value. This offers a very high level comparison of the last week of data compared to the last month of data. Slow but steady increases (as we're seeing with percent positivity) might not exceed this threshold. Hope this helps clear up any confusion. — Reply to this email directly, view it on GitHub <#243 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AYZG6QPKIRAPOXYWYT6AXGTVF35IVANCNFSM5TXBFM3Q> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[mmontesanonyc]

These are calculated from our internal systems from protected person-level data. The difference in values is rooted in the fact that in tests.csv, a person may be counted once each day ("de-duplicated by day") whereas the 28-day value de-duplicates over that time period, counting a person only once in that 28-day period - since, again, this is "people tested who tested positive" and not just "percent of tests that are positive."