ksonda / global-river-runner

Let's make a global river runner

Global River Runner

This repo is home to the API-backend of the Global River Runner project. The resources here are enough to deploy a pygeoapi server with the river-runner process. More information can be found here

The Global River Runner is a visualization simulating the path a raindrop would take, assuming it runs off into a stream and from then on to a terminating location, likely an inland water body or the ocean. A running list of interesting flow paths can be found here.

Usage

The basic operation is to issue a GET request of the form:

https://your-river-runner-domain.com/processes/river-runner/execution?lat=<y>&lng=<x>

Performance is best if your request header includes Accept-Encoding: gzip and Accept: application/json.

Some domain to keep in mind:

Domain	Version	Behind Cloudfront?
https://merit.internetofwater.app	stable	Yes
https://merit-nldi.internetofwater.app	stable	No
http://d1za2aav0xp6il.cloudfront.net	latest	Yes
https://merit-dev-z3iqgg3uaa-uc.a.run.app	latest	No

Deployment

Deployment requires an instance of pygeoapi and postgres with PostGIS installed.

The pygeoapi instance must include the river-runner process as well as several other modifications.

We have prepared docker images for both the latest versions of pygeoapi with the necessary modifications.

A copy of the hydrography database used is documented and available for download here.

Cloud

See the deploy/cloud directory for Dockerfiles and pygeoapi configurations necessary to deploy an appropriately configured pygeoapi container to the cloud service of your choice. Note the need to use environmental variables URL (your desired service URL for the pygeoapi base path), and DBHOST, DBUSER,DBNAME, DBPASSWORD (to connect to your database). The pygeoapi river-runner is designed to be a stateless application, so your database must be deployed separately.

Local

See the deploy/local directory for a docker-compose and pygeoapi configuration for a pygeoapi and postgis stack.

global-river-runner's Issues

Allegheny River is misclassified as Ohio River

Describe the data problem
The Allegheny River seems to be misclassified as the Ohio River for its entirety. Initially noticed near Pittsburgh, but the problem seems to persist upstream.

To Reproduce
https://river-runner-global.samlearner.com/?lng=-79.58416909772211&lat=40.70967444720185

Suggested fix
Allegheny River should be disaggregated from the Ohio. This is probably symptomatic of something larger, but I'm not in a position to say what.

Evidence (optional)
https://en.wikipedia.org/wiki/Allegheny_River

Potential Path Issues

I've noticed a couple of naming issues where features are over-grouped or over-extended at the end of paths, potentially related to these paths running through numerous large inland features? Not sure if the terminal flag/stream level properties might be impacted by the path running through something like the Great Lakes?

The River Runner tool now overrides names on flowlines inside of inland features, so it might be a little easier to see with on the IoW plots:

• Boundary Waters to Hudson Bay (thinks that the whole path is made up of Rainy River or the Red River): River Runner Path
• Great Lakes (thinks that everything past Huron is part of the Detroit River): River Runner Path

Many Western/Central Canadian Rivers being incorrectly aggregated into St. Mary's River/Red River of the North

Describe the data problem
More detail in this issue on the frontend repo: sdl60660/river-runner#28. Many rivers in western/central Canada seem to be over-aggregated and incorrectly assigned to a few particular features (St. Mary's River, Red River of the North, Rainy River). Oftentimes, this seems to be linked to name changes that occur after a large inland feature (opposed to a confluence).

To Reproduce
https://river-runner-global.samlearner.com/?lng=-92.93021070&lat=48.668974499

Suggested fix
The commenter in the linked issue submitted a table of suggested fixes, but the issue here appears to be a larger systemic issue that probably won't be easily remedied by making one-off fixes.

final source data

• @dblodgett-usgs @sdl60660 decide on appropriate level of simplification
• @dblodgett-usgs decide and indicate final data location. hydroshare geoconnex folder is probably not it
• @ksonda @webb-ben figure out CI/CD for local dev/Duke DB
• @dblodgett-usgs automate production of .sql and placement in final data location (rather than or in addition to .gpkg)
• @ksonda @webb-ben figure out CI/CD for production

local pygeoapi html links dont work

When deploying for local development, it appears that pygeoapi has to be configured with docker.host.internal as the hostname/url in the configs. This breaks the links in the UI. Not a big deal, worst case fixable with some alternative environmental variables in jinja templates?

Running List for Front-end

General maintenance:

• Unset yellow highlight on start run
• Clicking "x" button while in overview mode should cancel the run
• Pause then exit/cancel the run, then new run seems to cause state issues
• The "hitch"!! (caused by camera reaching first target point)

Compatibility Issues:

• Sandwich feature occurrences not working correctly
• Garbled linepath from the feature groups right now (squiggly blue lines)
• Geocoder constrained to U.S. right now
• Remove mapbounds
• Remove flowrate options, since this won't be available elsewhere
• Inset map needs to be dynamic
• Remove U.S. boundaries on click handler
• Tinker with inset map padding
• Fix ocean bounds for stopping features
• Add some sensible map zoom limits and prevent wrapping
• Playback controls are hanging on a lot of international paths for some reason
• Longer/less flowlines means smoothing coefficient should be lower and scale up more slowly at higher zoom levels
• Non-ocean stopping features (remove Canada/Mexico, find dataset of global inland water bodies)
• Parent feature stop points
• Fix error handling on round re-attempt
• Camera elevation calculation does not work at very high eleveations (e.g. himalayas)
• Add sensible error handling for unusable click values (e.g. Antarctica, the ocean)
• Fix elevation sampling issue when 3D tiles aren't rendered on start (e.g. searching for start point that's not in map bounds)
• Still having some sandwich or feature naming issues
• Bound out Antarctica
• Still having some zoom slider issues (see https://river-runner-global.vercel.app/?lng=127.05969032131696&lat=40.211533693233974)
• Perceptual speed/terrain multiplier issue
• flyTo to pre-render tiles, get correct elevation queries and smooth the flight to run transition
• Name suggestion modal
• Sub in NLDI/Firebase VAA for clicks in contiguous US (stitch together with Mexico/Canada results from last point, if necessary?)
• Edit attribution blurb
• Update README
• Linked Coordinate to Flight/Start Elevation disjointedness is a lot better, but still not great for things like Everest, this has to be fixed
• Great Salt Lake issue (hanging feature at end)
• Fix Great Lakes
• Manual override on the Saint Lawrence to replace the stopping feature with "Gulf of Saint Lawrence" or something while maintaining the existing stopping feature for the inland mapping?
• Cut out unnecessary properties on requests
• Alter async pattern to start grabbing path data during initial tile render flight
• Variable smoothing based on flowline coordinate density?
• Fix inset map issues on change in path + varying quality from highlighted feature to main path or min zoom level

Automate db update

Right now, updating the DB involves having to manually update the new sciencebase sql.gz in the bucket and then loading it into the cloudsql db. Both of these steps should be automated via a GitHub action. Still manually triggered, probably via PR to update a Dockerfile somewhere, but we shouldnt have to log into the management console to do this.

Northeastern Brazil

I noticed that several major rivers are unidentified (Capibaribe), or incorrectly identified (what is identified as Una river, is actually the Ipojuca river).

See attachment: Bacias Hidrográficas SIRGAS in KMZ, from the Agência Pernambucana de Águas e Clima APAC (Water and Climate Agency of Pernambuco) that identifies the areas of each river basin.

kmz in this zip
Bacias Hidrograficas Sirgas.zip

attribution blurb

This issue is for @dblodgett-usgs and I and @webb-ben to come up with what we want in the attribution blurb, or else documented somewhere linked from Sam's app.

First go:

Visualization by Sam Learner. Code and data for this project lives here (repo link). Raindrop flowpaths are calculated using an API(link to pygeoapi process swagger), developed by Ben Webb(link) and Kyle Onda(link) from the Internet of Water project at Duke University(link), implemented as an OGC - API Process (link) in pygeoapi (link). The underlying data is based on MERIT-Hydro(link), with a simplification and processing algorithm (link to code/sciencebase) implemented by Dave Blodegett (link) at USGS, in accordance with the HY_features river Mainstems (link to pub) concept.