iridl / python-maprooms Goto Github PK

Make sure that all widgets.py files are in sync with the new changes being made to the lms-monthly and onset maprooms.

Includes renaming of the file and some functions, adding new parameters within functions.

Also removing ison parameter from Block() to follow comment made my Aaron:

"I see. Here's where that's used. I think we should conditionally include/exclude the Block from the list in that case, instead of always including it but sometimes hiding it. (That requires a PR in the python-maprooms repo). And then remove the ison parameter from Block."

Brought up by Remi, a naming convention for cessation date that is currently misleading.

Original comment:
"that is unfortunate naming as, compared to onset, it resembles the _spell_length Parameters, wheras min_dry_days, in onset, is something else... Can fix that in another PR."

Originally posted by @remicousin in #87 (comment)

    If we need to display the values with accuracy and thus all of the decimal places I would say moving them to the description of the maproom or maybe displaying them when hovering over the input box title?

Originally posted by @drewmresnick in #177 (comment)

When option selected is 'pe' there is the error

AttributeError: module 'pingrid' has no attribute 'CORRELATION_COLORMAP_KEY'

Editing the key to match the pingrid file var which is, 'CORRELATION_COLORMAP then throws:

TypeError: to_dash_colorscale() got an unexpected keyword argument 'thresholds'

I have not finished debugging since I found this while working on something else, so I put it in this issue for now and will open a separate PR with the fix.

Besides issue #174 , there is probably more to do to split what's purely date formatting, what's purely CPT files reading... etc... and to be robust and flexible enough to cover yet another set of files that might be written differently (even though we might not be that far).

widgets.py has been copied to multiple maprooms, and sometimes edited. Harmonize the differences and use symlink as for pingrid.py, or package as a library.

First we eliminated import-time data loading by introducing "dummy" callbacks that ignore their inputs. Then we eliminated those by initializing the app with a layout function instead of a layout. But that approach ended up making way more db queries than expected, so we reverted to calling the layout function (and thus loading data) at import time. Need to either stop the layout function from being called so many times, or revert to using dummy callbacks.

Affects subseasonal forecast and onset maprooms.

These callbacks ignore their input. If the technique I suggested for eliminating these from the seasonal maproom works, apply it here too.

Originally posted by @aaron-kaplan in #85 (comment)

Need to add an except block in the dash callback to handle an error that occurs when the coordinates chosen are included in the ds, but do not have any data (all NaNs).

The exception should function to check the output of the onset function, and return an error message if the output is all NaTs.

discussion

Other plotting libraries I've used can plot a continuous function, automatically sampling at whatever resolution is appropriate for the figure being generated. I just searched and couldn't find anything like that for plotly, though.

I always have to return to the source code of this function to remind myself what it does, because the name doesn't remind me. Can we rename it something like add_seasons, label_seasons, extract_seasons, ...?

(Possibly a moot point eventually, if we switch to the approach of making season year a separate dimension.)

Originally posted by @aaron-kaplan in #87 (comment)

    @drewmresnick variable names in python traditionally use `snake_case`, not `camelCase`. On a one-person project it doesn't really make a difference, but in a project with multiple people contributing it's good to have conventions so things don't get too inconsistent.

In JavaScript it's traditional to use camelCase, and dash is a python wrapper around some JavaScript libraries, so in a dash project you may see camelCase bleeding into python code in some places. But in our own code, we should stick to snake_case.

I've not been too strict about this so far because I didn't want to get too nitpicky, but it's becoming apparent that if I don't start enforcing it now then it's going to get harder and harder to do later.

Originally posted by @aaron-kaplan in #187 (comment)

Possibly... or make it an issue. The key here is that is we are going to use the same app for other cpt forecasts (biweekly, seasonal...) so all the functions associated with reading will either have to be more fancy to accommodate different cases, or have counterparts for the different cases, and the config file will tell which case we're in.
I am just not sure what shape that will all take as I review this now. So it may make sense to leave to other PR.

Originally posted by @remicousin in #156 (comment)

Great. Now we should go through the other applications that use this pattern and make them use pingrid.error_fig too. If you don't have time to do that now (and test them), please add it to the todo list.

Originally posted by @aaron-kaplan in #113 (comment)

see output in terminal

Hi Remi,
Regarding #3 regarding rho, yes, I used a fixed number (in the case of maize, 0.55). See more info for other crops https://www.fao.org/3/x0490E/x0490e0e.htm#readily%20available%20water%20(raw)

Originally posted by @Agro-Climate in #63 (comment)

I would rather the 2nd control be the target date rather than the lead. You can do that in this PR, or make an issue to be addressed later.

Originally posted by @remicousin in #156 (comment)

Make it a callback that uses output of the start date dropdown to then calculate the possible target dates using lead times 1,2,3,4

This will probably help to convert to a function as well, so we can modulate when wanting to use this for other kind of forecasts. Can make that an issue for now.

Originally posted by @remicousin in #156 (comment)

I probably meant to print a title for the map and somehow failed to do so... we can take care of that later.

Originally posted by @remicousin in #156 (comment)

    Assuming the user and the developer are different people, this is an error that the user can't fix, so I would put the diagnostic information in the log file rather than the browser. For now, we do that either by raising an exception or with a simple `print`.

Originally posted by @aaron-kaplan in #207 (comment)

Need to add an exception to throw a warning message when the user leave a box blank, and return an empty graph until all parameters have inputs.

Once it's not subseasonal forecasts anymore, but also seasonal, should adapt those names accordingly

FBF config file can define an arbitrary number of admin levels. Apply the same pattern to onset maproom.

A suggestion from a ZMD colleague. I agree that it's a needed feature

Ingrid uses X/Y, but pingrid.py uses lon/lat, because Igor's perception was that the latter are more standard.

See some discussion and references to the CF conventions here: xarray-contrib/cf-xarray#23

For the outage data called in maproom.py, improve the SQL query.

Will need that feature at some point

If we can't get the dynamic colorscale, we should disable the choice of the percentile threshold to non-/exceed because then the map and colorscale relationship makes little sense. We should then keep it stuck at 50th percentile

    `shapes` is a dataframe, so I think 73-74 can be replaced with

shapes['label'] = df['label']

Originally posted by @aaron-kaplan in #223 (comment)

Subseasonal forecast maproom and fbfmaproom both snap position to grid. Do the same for onset maproom.

The subseasonal version of this Maproom has been merged to master. At present, I don't think there is anything in it that makes it specific to subseasonal (would work same way as for seasonal), except the datafiles reading and the retrieval and printing of S, L, target date in the app.

The next step is to have the app offer more than one S/L to viz'. The sample config file points to a folder where there is a set of S/Ls and can be used to try and set that up. Even though we don't have a sample of files for seasonal forecasts, setting up the subseasonal case should be done trying to be ready to work as well for the seasonal one.

for PRISM eBird maproom, improve on definition for color scale bar (variable ctg).

This function retrieves shapes from postgis in wkb form, and then uses shapely to convert from wkb to geojson, which is what leaflet expects. Shapely seems unnecessary here, because postgis can generate geojson directly. Try replacing ST_AsBinary with ST_AsGeoJSON in the query (which is in the config file) and replacing lines 124-129 with

return {"features": df["the_geom"]}

Originally posted by @aaron-kaplan in #85 (comment)

    Perhaps it would be less error-prone to prepend sminit to sm so we can eliminate lines 146-147 and make the first step part of the loop? (Not urgent to fix if you're in a hurry.)

Originally posted by @aaron-kaplan in #228 (comment)

When I zoom way in, I see gaps between the tiles. Most likely a bug in pingrid.py, so I'll work on that in a separate branch. In the meantime, just don't zoom too far in when you demo :-)

Originally posted by @aaron-kaplan in #85 (comment)

Unrelated to Drew's changes, I notice that when you click on the map, the location of the marker doesn't get updated until the graphs finish rendering. Could we update the marker in a separate callback instead of piggybacking it on local_plots?

Originally posted by @aaron-kaplan in #156 (comment)

Upon further investigation, the plot and table produced through ingrid are not updating (seem to be static values).

In order to compare the outputs of the new and old onset functions this must be fixed so they are properly updating.

In onset maproom, the cessation date probability of exceedence graph sometimes shows a spurious point in the middle so there is something wrong here. Note that subseas flex forecast also computes prob. of exceedence but possibly is doing more than is needed here, but might still be an inspiration to fix the problem

Once they become "ENACTS" Maprooms, the users will insist to have the admin overlays. This is done in onset.

OK to resolve this (githuhb) conversation as far as I'm concerned, but if you want to leave it open to remind you to follow up on the question, that's ok too.

Originally posted by @aaron-kaplan in #113 (comment)

    In light of previous comments above. There might be a need to have a set of functions that manipulate time. We need:

1. functions that are given the strftime format of file names and convert that in datetime objects;
2. functions that manipulate the different datetime objects, for instance to retrieve target_start and target_end from issue and lead
3. functions that print our datetime objects in strftime formats we want in the app at various locations.

I am not pretending I have the solution but hopefully it helps rationalize the whole set.

Originally posted by @remicousin in #187 (comment)

first thing first: describe what is seen when landing (maybe should us "map of landing page" rather "default map")

What about "first thing first: describe what is currently displayed?" The explanatory text can change depending on the value of the control.

Originally posted by @aaron-kaplan in #85 (comment)

I wrote a bunch of code relying on xarray but ignoring xarray's functions default behavior on those (coords/dim/axis). I also consistently used time_coord="T" as the parameter for time dimension, but it turns out xarray typically talks about dim and axis rather than coord in their function. I wonder if I should harmonize all that.

Needed for flexible forecast maprooms. Has not been needed elsewhere so far in Ingrid's

I'd think we should be able to accommodate both independently. Checkboxes to control the different layers of admin on one hand. On the other hand, infer name of the of the smallest level admin the mouse is hovering, even if that level is not a currently checked layer. In almost all admin databases, they contain the name of the geometries and the names of the higher levels they belong, so one can get the full information by querying the smallest admin level and print e.g. name3, name2, name1. There will be cases when name2, name1 are not there... We'll have to deal with these as exceptions somehow.

(The piece of code generating that will likely be reusable for the other item Aaron was mentioning: print that same information along with the graphs when clicking a gridbox, as Ingrid/uicore does.)

As to where to print the information when mousing over... I don't know. I agree that it's not great to obstruct the map with a pop-up. On the other hand, since it's meant to help figuring out where to click, it's convenient that it would be by the mouse... Maybe the bottom right corner of the map (by the "Leaflet" mark)? or right above or right below the map?

Sorry I feel like we are often make you redo things a good deal. Maybe we should have brief discussions before starting projects... On the other hand, doing something and then change to something else means more things done, thus more learning...

Originally posted by @remicousin in #171 (comment)

Turns out that out of the 4 cases I ran into: fbf's Ingrid colormap's representation; correlationcolorscale and the 2 PoE colorscales (once static); only the fbf's one has Ingrid's colormap returns something that corresponds to what start/endcolormap states. In the 3 other cases, colors order were switched, and/or replaced, and/or additional incrementations/sequencing were added...

I didn't understand this. If you want me to help look into it, please provide more details. I think we can merge this in the meantime.

Originally posted by @aaron-kaplan in #113 (comment)

    > I commented

df["the_geom"] = df["the_geom"].apply(lambda x: wkb.loads(x.tobytes()))
df["the_geom"] = df["the_geom"].apply(
lambda x: x if isinstance(x, MultiPolygon) else MultiPolygon([x])
)

and got an error right away so restored it.

I didn't mean for you to delete the wkb.loads line, only the following ones. Not important right now. I will test on devi.

Originally posted by @aaron-kaplan in #223 (comment)