extract_nwis_df() function returns a tuple with dataframe and dictionary about hydrofunctions

[mroberge]

Hi Nick!
Thanks for this question and for using hydrofunctions. I changed this function a few versions ago- I've been encouraging people to use the hf.NWIS interface instead. I can send you some code suggestions in the morning.

As you said, the tuple contains a data frame and a dictionary. The dictionary contains some metadata, but I don't remember if it has lat &long. One way to access just the dataframe is to do this:

discharge, meta = hf.extract_nwis_df(resp)

This line would replace the line where you extract values to a dataframe. You could use meta if you want or just ignore it. The rest of your code should work as is. I'll try it out in the morning!

[mroberge]

I see what you are trying to do here! It looks like you want to create a dataframe that is in the long format similar to R's 'tidy' format. I've been wanting to provide this functionality for my NWIS class for a while.

First, my code above works for your example.

Second, you said that sometimes you get two columns instead of four columns. This is because sometimes when you request data from a site it is only returning stage data instead of stage and discharge. I've never seen that before, so I'm curious. But this can be fixed by creating a more robust system for renaming your columns. Right now you just assume that you have four columns and you give them names. Instead, you could use the 'rename' method of dataframes to change the column names and create a mapper function. It would work like this: my_df.rename(mapper_function, axis=columns) now you just need a mapper function that takes the column string, looks to see if it is for qualifiers or data, and looks to see if it is for stage or discharge and return something appropriate.

Third, right now your function gives the same name to two different columns. Until we come up with a better renaming function, I would replace that line with something like this:

discharge.columns = ["discharge", "discharge-flag", 'stage', 'stage-flag']

[nlamkey]

Thanks for taking the time to look at this! My function works great now. I didn't think of doing a mapper function that's a great idea. I looked into the extra columns again, it looks like one of the sites I left out of the post was also retrieving the nitrite-nitrate data that that station also collects without being called so that through a wrench in the function so I took that station out and it separately. At any rate I accomplished what I needed to thanks for your help.

…

On Wed, Jan 12, 2022 at 11:10 AM Martin Roberge ***@***.***> wrote: I see what you are trying to do here! It looks like you want to create a dataframe that is in the long format similar to R's 'tidy' format. I've been wanting to provide this functionality for my NWIS class for a while. First, my code above works for your example. Second, you said that sometimes you get two columns instead of four columns. This is because sometimes when you request data from a site it is only returning stage data instead of stage and discharge. I've never seen that before, so I'm curious. But this can be fixed by creating a more robust system for renaming your columns. Right now you just assume that you have four columns and you give them names. Instead, you could use the 'rename' method of dataframes to change the column names and create a mapper function. It would work like this: my_df.rename(mapper_function, axis=columns) now you just need a mapper function that takes the column string, looks to see if it is for qualifiers or data, and looks to see if it is for stage or discharge and return something appropriate. Third, right now your function gives the same name to two different columns. Until we come up with a better renaming function, I would replace that line with something like this: discharge.columns = ["discharge", "discharge-flag", 'stage', 'stage-flag'] — Reply to this email directly, view it on GitHub <#112 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALP5QFTQCJMNKI7QEAH6VE3UVWYYTANCNFSM5LXF2CMA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[mroberge]

Glad to help!