hydrologicengineeringcenter / cwms-data-api-client Goto Github PK

Currently we test by checking a 401 error code with a dummy post to the catalog endpoint. From Mike Neilson:

...FYI the swagger-docs output will include security schemas if it can write, that’s better than making a request and it just pollutes the logs (the access log is one line, the failed request is like 10.)

It's been observed that if you make a change to the Python of a PythonAlgorithm that you have to close the computation (both the run windows and the definition entry screen) and open them up again for that change to be picked up.

OkHttp cache doesn't seem to do pro-active cache invalidation.
Scenario:

1. User does a GET request - OkHttp will cache it.
2. The user does a POST to modify the resource.
3. The user then re-does the first GET request.
4. OkHttp cache returns the previously cached results.

In step 4 we'd like users of cdac to get back fresh results.

From this okhttp issue: square/okhttp#853 and this documentation: https://square.github.io/okhttp/features/caching/#pruning-the-cache It appears that we are supposed to prune the cache when we want to invalidate something.

The only way to prune the cache is to call urls() and get a MutableIterator from the cache and then iterate over it and remove urls from it.

For the CDA use-case above. When a POST is done we'd like something to iterate the cache and throw away matching urls.

What does it mean to be matching? Definitely need to throw away exact matches. But what if the user just flipped a query parameter? We can't expect cdac to know the meaning of each parameter so it seems like we have to also match cache items with different query parameters.

What if the user had previous done a getAll style request and then updated one of the items and does a getAll request again?
In that case the url, even ignoring the query parameters, between the GET and the POST are not "exactly" the same.

GET: https://cwms-data.usace.army.mil/cwms-data/locations?office=SWT&unit=EN
PATCH: https://cwms-data.usace.army.mil/cwms-data/locations/BUCK
GET: https://cwms-data.usace.army.mil/cwms-data/locations?office=SWT&unit=EN

Therefore, for CDA usage, we'd want to evict everything from the same CDA Controller end-point. E.G everything that starts with: https://cwms-data.usace.army.mil/cwms-data/locations

But what about CDA Controllers that are related?
We have CDA controllers for Location Category and Location Group.
If you do a cascade delete on Location Category we'd really want the Location Group cache entries to be evicted.
Or Timeseries and TimeseriesIdentifier. Or Timeseries and Catalog. CDAC can't be expected to track all these dependencies.

Basically I think I've convinced myself that when the user does a POST, PUT, PATCH or DELETE to a CDA end-point then cdac should make OkHttp cache evict all cached responses from the root of the CDA server.

To be safe, anything that starts with https://cwms-data.usace.army.mil/cwms-data/ should be thrown away.

For CDA usage can we programmatically find an appropriate url prefix to match?

FYI, I'm about to break the clob DTO in data-api. It's the only one that uses "office" instead of "office-id".

I can go ahead and do the fix here, but I wanted to make sure y'all are aware.