Error in calling the nwis function for some of the catchments/station numbers about hydrotools

[stcui007]

Hi Austin and Jason, Thanks for spending time on resolving the issue. No apology is necessary as I put in the test code before filing the ticket.

…

On Fri, Sep 3, 2021 at 8:46 AM Austin Raney ***@***.***> wrote: @stcui007 <https://github.com/stcui007>, no worries. My apology if I implicitly suggested that you needed to do anything. This was just a bug in the software. #134 <#134> should resolve the issue. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#133 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACA4SRPDGGHKINHDQGCWKXTUADGRJANCNFSM5DJ5O7BA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[stcui007]

Thanks for working on this.

…

On Thu, Sep 9, 2021 at 7:35 AM Jason A. Regina ***@***.***> wrote: Fix has been deployed: https://pypi.org/project/hydrotools.nwis-client/3.0.5/ @stcui007 <https://github.com/stcui007> @aaraney <https://github.com/aaraney> @hellkite500 <https://github.com/hellkite500> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#133 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACA4SRMY7QKVEX7RCL4E6N3UBCS2VANCNFSM5DJ5O7BA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[jarq6c]

Thanks Shengting.

I successfully reproduced the error with the following:

from hydrotools.nwis_client.iv import IVDataService

service = IVDataService(value_time_label="value_time")
obs = service.get(
    sites=["01189000"],
    startDt="2015-12-01T00:00",
    endDt="2015-12-31T23:45"
)

The error seems to originate here:

hydrotools/python/nwis_client/src/hydrotools/nwis_client/iv.py

Line 262 in 908e000

dfs.loc[:, "value"] = pd.to_numeric(dfs["value"], downcast="float")

Looks like the issue is when NWIS returns an empty time series it results in an empty pandas.DataFrame. A KeyError is raised when the tool attempts to optimize the float values. As I recall, older versions of nwis_client would check for this using pandas.DataFrame.empty before attempting any optimizations. It looks like this functionality may have been lost during the asyncio transition.

I'll ping @aaraney for input.

[aaraney]

@stcui007, thanks for reporting this and thanks @jarq6c for verifying and reproducing the bug. I was able to reproduce it locally as well. I believe your thinking is correct @jarq6c, some logic was changed in #86. I will submit a PR and reference it here. Thanks!

[stcui007]

I have a check for empty dataframe in my code. When I get a returned empty dataframe, I check for it and skip the step. For the reported case, the function call doesn't come back so I can't check for it at my end.

…

On Thu, Sep 2, 2021 at 7:14 PM Austin Raney ***@***.***> wrote: @stcui007 <https://github.com/stcui007>, thanks for reporting this and thanks @jarq6c <https://github.com/jarq6c> for verifying and reproducing the bug. I was able to reproduce it locally as well. I believe your thinking is correct @jarq6c <https://github.com/jarq6c>, some logic was changed in #86 <#86>. I will submit a PR and reference it here. Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#133 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACA4SRP2O356HWLVW2WOI2DUAAHOPANCNFSM5DJ5O7BA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[aaraney]

@stcui007, no worries. My apology if I implicitly suggested that you needed to do anything. This was just a bug in the software. #134 should resolve the issue.

[jarq6c]

Fix has been deployed:
https://pypi.org/project/hydrotools.nwis-client/3.0.5/

@stcui007 @aaraney @hellkite500