is there a design flaw in request lifetimes? about data

[runspired]

HOWEVER there's no way on a request via a handler, to give the lifetime service the information ... that it needs to know when this query request document needs to be invalidated.

If you write a lifetimes service that coordinates with a handler then you immediately have the context of every request ever sent and every response ever received.

All that's needed is an app domain data structure to be sent with the request and passed to the lifetime service. it's that simple. A pointer. It enables the app domain developer to know when to invalidate documents that are not cached purely on time, header, etc etc basis, but can also be invalidated by app events.

If there is app domain information necessary for this that is not contained within the context of the request or previous requests then probably you've overthought the problem space or over-engineered the solution. Generally the only reason I could see someone wanting to do this is to dynamically change strategies for when to invalidate: an approach that would be fraught with major footguns. You'll be much better served managing a strategy based only on past and current request information and statically coded rules. This holds especially true for the near-future of EmberData in which we intend to use all of your handlers off-thread as well.

but can also be invalidated by app events

If all you mean by this is "I received a websocket notification for record added" then expose the map for invalidations in the lifetimes service to the websocket service as well.

If you mean something along the lines of "user requested a route refresh" then all that's necessary when using the route's refresh hook is to re-trigger the same queries setting the cacheOption to skip cache as described above.

If you mean something along the lines of "an invalidation event has occurred that should invalidate most/all requests" along the lines of an authentication event etc, then I'd say that sort of thing belongs clearing the cache and store entirely, which today is best done with a window reload.

Things in between probably just need to be given access to your lifetimes service to notify it of said invalidation events.

The place that you need app context is the query, to let the lifetime service know what circumstances would invalidate this document. And that needs to be done pre-fetch, but post-isHardExpired. That's where the problem is.

Today a handler can do this easily, though I still think as I mentioned above the willRequest/didRequest could assist in making this easier than having a handler and a lifetimes service be tightly coupled.

A brief example

Lets say you wanted to invalidate any query that was issued for 'user' whenever a user has successfully saved a new user record.

class Handler {
  constructor(lifetimes) {
    this.lifetimes = lifetimes;
  }
  
  request({ request }, next) {
    const { op } = request;
    
    if (op !== 'createRecord' && op !== 'query') return next(request);
    
    return next(request)
      .then((response) => {
        if (op === 'createRecord') {
          this.lifetimes.invalidateRequestsForType(request.data.record.type);
        } else if (op === 'query') {
          this.lifetimes.didRequest(request, response);
        }
        return response;
      });
  }
}

The default builder for query doesn't attach data the way that createRecord does, but if your apps did or if we updated the default to attach the primary resource type being queried (not necessarily the best strategy mind you) then instead of determining the mappings from the response data of the query you could do it from that. E.g.

class Handler {
  constructor(lifetimes) {
    this.lifetimes = lifetimes;
  }
  
  request({ request }, next) {
    const { op } = request;
    
    if (op !== 'createRecord' && op !== 'query') return next(request);
    
    return next(request)
      .then((response) => {
        if (op === 'createRecord') {
          this.lifetimes.invalidateRequestsForType(request.data.record.type);
        } else if (op === 'query') {
          this.lifetimes.didRequestForType(request.data.record.type, request, response);
        }
        return response;
      });
  }
}

[runspired]

This is a flaw in your approach to how to manage the lifetimes, but not in the lifetimes service itself.

The beauty of the lifetimes service is that it answers a few simple questions yes/no right at the point at which you are going to maybe hit network. How you get information to that point is up to the app.

Before I go further on an explanation here I will say that overall as I've worked on more complicated lifetimes strategies I too have found myself wishing for a simple way for the lifetimes service to hook into the request/response lifecycle for managing various maps of requests it might invalidate when using a non-time, non-payload based approach.

So adding something like the following to the signature is maybe in the cards, though far from required or even necessary to do all of these things. Incidentally, I also think adding a cache variable to the lifetimes service may also be required for store forking so I'm accounting for that here as well, though that ultimately depends on whether we decide that forks keep their parent's requestManager instance or create their own.

psuedo-signature

interface LifetimesService {
  isHardExpired(identifier: StableDocumentIdentifier, cache: Cache): boolean;
  isSoftExpired(identifier: StableDocumentIdentifier, cache: Cache): boolean;
  willRequest(identifier: StableDocumentIdentifier, cache: Cache): void;
  didRequest(identifier: StableDocumentIdentifier, cache: Cache): void;
}

It's maybe in the cards because it turns out that something ideal for this pattern already exists! What we actually want is something that has the context of requests and an opportunity to observe both when one is issued and when it gets a response. Sounds exactly like ...a handler! In fact the lifetimes service is sort of just that, its actually just a way to configure the behavior of the root-most handler (the cache handler).

In the above example, we wish to know when a request will not resolve from cache, and when a request that did not resolve from cache has returned new data. It turns out that a handler placed at the beginning of the handler chain has all the required context to do just this, and thus can be paired with a lifetimes service for such purposes.

Handling Race Conditions

On the broader topic of race conditions though, there are several interesting points to consider. Generally all apps have to figure out the correct balance of optimistic and pessimistic ui patterns and the best-for-them approaches to managing eventual consistency. There's no way around this, because ultimately every web client is functioning as an unreliable high-latency partial-state remote distributed replica of the database represented by your API.

We can literally never eliminate race conditions or the two generals problem, its a provably unsolvable problem even in the exceedingly rare case where you only have one single user and one single API.

In the legacy world of adapters and serializers, this problem was more pronounced because it was exceedingly rare for the cache to invalidate and go to network, which led to the pattern of using reload or backgroundReload everywhere as those were the only options available to try to make the cache a little more trusty. With the RequestManager and lifetimes approach we gain back the ability to have a trusty cache and should be able to significantly increase the amount of requests fulfilled from cache without hitting network. This is achieved because the power is returned to the app developer to determine what the right way to minimize these impossible problems is for the particular app.

For the createRecord case, most apps typically choose to solve the problems you are tackling with a time based cache, by invalidating requests specific to the current page only, or sometimes pairing these first two approaches with either local patching of known state to prevent the need to re-fetch or with a separate set of ui objects to represent recently created or added items that don't necessarily belong in a particular result set. This latter pattern is generally the better one because its expensive to invalidate everything right away, its often better to allow a small portion of the ui to not contain the new record for a small amount of time than it is to eagerly try to refetch the world.

The "record added" case is also the hardest edge case of all of them. Its far more trivial to handle "record updated" and "record removed" because its easy to know which requests the record belongs to and automatically handle that for you. Which is what EmberData does out of the box, because these two cases don't have the same impossible-to-solve problems that the added case has.

[BryanCrotaz]

I disagree. I started off thinking oh this is great, it's just a handler in the chain. However the handler doesn't have the context. It doesn't know why this request was sent. And the ED handler code actively stops me adding context to a request.

So the simple case where cache lifetime is temporal, or based on a header, that's all information that the handler has access to. But it doesn't have access to where this request came from in the app, whether that case requires optimistic or pessimistic caching, in my case which subset of createRecord should invalidate this particular query.

the power is returned to the app developer to determine what the right way to minimize these impossible problems is for the particular app

This part I agree with, but the current design doesn't allow me to do this as the app developer because the information needed is lost (in a handler), or received at the wrong time (called before or after store.request).

psuedo-signature

interface LifetimesService {
  isHardExpired(identifier: StableDocumentIdentifier, cache: Cache): boolean;
  isSoftExpired(identifier: StableDocumentIdentifier, cache: Cache): boolean;
  willRequest(identifier: StableDocumentIdentifier, cache: Cache): void;
  didRequest(identifier: StableDocumentIdentifier, cache: Cache): void;
}

This interface doesn't have the context from the application. I could get the same result by putting code in the isHardExpired and isSoftExpired hooks.

interface LifetimesService {
  isHardExpired(identifier: StableDocumentIdentifier, cache: Cache): boolean;
  isSoftExpired(identifier: StableDocumentIdentifier, cache: Cache): boolean;
  willRequest(identifier: StableDocumentIdentifier, cache: Cache, appData: unknown): void;
  didRequest(identifier: StableDocumentIdentifier, cache: Cache, appData: unknown): void;
}

store.request(request: RequestInfo, appData: unknown);

That allows the app developer to pass context down to the lifetime service so it can make a better decision.

Note

We can literally never eliminate race conditions or the two generals problem, its a provably unsolvable problem even in the exceedingly rare case where you only have one single user and one single API.

I'm not asking for that.

I am NOT asking for ED to solve the impossible caching problem. I'm asking for ED to have hooks that allow an app developer to design something that fits their circumstances.

[runspired]

So the simple case where cache lifetime is temporal, or based on a header, that's all information that the handler has access to.

This is false, it has access to the full cache, and by extension to everything related to the request past and present.

But it doesn't have access to where this request came from in the app

It never needs this, because any information you wanted to use to determine something so specific belongs being done at the point of the request. Hence using cacheOptions to skip the lifetimes service entirely in favor of self determination at the point of the query when needed. In theory point-of-request lifetime adjustment should be rare except in the case where the app knows its been asked to do a refresh.

in my case which subset of createRecord should invalidate this particular query.

If you want to invalidate queries based on createRecord, then do so, but there's no need to do that at point of request. In fact doing so at point of request would almost certainly be a mistake. Since you're using websockets you're going to need direct access to the map for invalidation on record-added scenarios anyway, so you'd want to go ahead and set that up for yourself, but for the more simple crud case two things are true: (1) issuing a createRecord request does not need to invalidate any other requests until it has completed, so your code above is the wrong time anyway and (2) its trivial to have your handler handle this case regardless of when you want to do the invalidation and to do so without passing any additional context to the request.

There is (in fact) a generic object for passing along info in the request if needed, but I haven't been steering you towards that because you absolutely don't need it and trying to use it for such would be a mistake. If all you want your handler to know is "this request is createRecord" and "it is creating a record of X type" then both of those are already part of the provided information (see for instance the output of the createRecord builder for JSON:API

data/packages/json-api/src/-private/builders/save-record.ts

Lines 164 to 173 in 4badefd

	return {
	url,
	method: 'POST',
	headers,
	op: 'createRecord',
	data: {
	record: identifier,
	},
	};
	}

)

[BryanCrotaz]

The createRecord isn't where the problem is, it's the query.

You're right, the createRecord interaction with the cache is post-fetch, which can be achieved already.

The place that you need app context is the query, to let the lifetime service know what circumstances would invalidate this document. And that needs to be done pre-fetch, but post-isHardExpired. That's where the problem is.

[runspired]

@BryanCrotaz at this point I'm not sure what you are trying to do then but it sounds super specific and if it is super specific you should just mark the request to skip cache at the point of request.

[BryanCrotaz]

I'll try and explain again.

There's a query. An app domain query. Its results are valid for x seconds. So far so easy, all handled by the ED design, as the lifetime service, or a handler has access to all the data it needs to achieve this. Any algorithm that only involves an individual request can be made to work.

Now something happens in the app that causes that document to be invalid. But which documents? This is where the lifetime service steps in - it knows what data is tangled each request, and when a particular piece of data changes, or an event happens, or anything app domain side that the dev cares about, it can invalidate the document on a future request. Again, App Domain stuff, a custom lifetime service.

HOWEVER there's no way on a request via a handler, to give the lifetime service the information (again, app domain information, I'm not asking ED to implement a custom cache strategy), that it needs to know when this query request document needs to be invalidated.

All that's needed is an app domain data structure to be sent with the request and passed to the lifetime service. it's that simple. A pointer. It enables the app domain developer to know when to invalidate documents that are not cached purely on time, header, etc etc basis, but can also be invalidated by app events.

[BryanCrotaz]

you should just mark the request to skip cache at the point of request.

If I did that then the query would never be cached, which would get rid of the whole point of using ED in the first place. If I didn't need caching, I could just fetch every time straight into a pojo.

[runspired]

If I did that then the query would never be cached, which would get rid of the whole point of using ED in the first place. If I didn't need caching, I could just fetch every time straight into a pojo.

There's a few layers to which this is untrue, I'll dive into both.

First, is there value in caching even if you hit network every time? The answer is yes, because the cache is really just a map and you still need a map to handle resolution for cycles in graphs (e.g. relationships). Without such resolution handling (and thus deduping), your payloads often become massive and if you do dedupe then you're reinventing solutions to hard problems on the other side that are otherwise already handled for you!

Second, and more importantly, I was on my phone for that answer and probably could have worded that answer better. By skip cache I don't mean to opt out of caching entirely, I mean that you should opt out of resolving that particular request from cache for that particular fetch of it if that info is already known. The result will still update the cache and a subsequent request after that one can still re-use the result if desired.

E.g.

store.request(query('user', {}, { reload: true }));

or

store.request(query('user', {}, { backgroundReload: true }));