[apollo-link-offline] What's your idea? about apollo-link

[lachenmayer]

Hey folks, I've tried to piece together a basic implementation for react-native that has similar behavior to apollo-offline.
(pinging Malpaux/apollo-offline#14)

You can check it out in this gist: https://gist.github.com/lachenmayer/2e364a5ca9ae0918eb032867d0c6720d

It's a combination of:

• apollo-link-queue,
• apollo-link-retry,
• and a custom optimisticFetchLink function.

When the device goes offline, requests will be queued, and will be unqueued when it goes back online. (apollo-link-queue)

Any request that fails with a network error will be retried (currently infinitely). (apollo-link-retry)
Note that this could be either because the device is offline, or the backend is simply not reachable (eg. if it's down).

However, if we can resolve the query from the cache, it will not be retried, and instead the data in the cache will be used as the response. This is implemented in the optimisticFetchLink function.

In my opinion, this "optimistic fetch" behavior is one of the most important parts of apollo-offline, and any future apollo-link-offline implementation should support this. This enables the user to keep using the app as usual while offline, as long as the data has been fetched & persisted at some point. In my opinion, this should be default behavior of the network-and-cache fetch policy, but unfortunately it does not look like this will change anytime soon (see apollographql/react-apollo#604 (comment)).

If you save the gist as offlineLink.js, you can use it as follows:

import { InMemoryCache } from 'apollo-cache-inmemory'
import { ApolloClient } from 'apollo-client'
import { createHttpLink } from 'apollo-link-http'

// get this from https://gist.github.com/lachenmayer/2e364a5ca9ae0918eb032867d0c6720d
import { createOfflineLink } from './offlineLink'

const cache = new InMemoryCache()
const networkLink = createHttpLink()

const offlineLink = createOfflineLink({ cache })

const link = ApolloLink.from([
  // ... + other links ...
  offlineLink,
  networkLink,
])

const client = new ApolloClient({
  cache,
  link,
}

Known issues / missing bits

• This only works with react-native for now. The isOnline check should be abstracted away to make this cross-platform.
• If you only have a partial response in the cache, you will get a rejected promise somewhere down the line telling you that the response is incomplete. I haven't yet looked into how to fix that, or indeed how apollo-offline solves this. If you want to try fixing this, I would appreciate any pointers on this.
• You currently can't change any of the retry settings - this should be added as an extra config parameter.
• You have no control over the optimistic fetching. You should be able to set this independently for each query. I'm not a big fan of apollo-offline's implementation of this (setting a {__online_: true} query variable), but there should definitely be some way to do this.
• There are obviously also no tests.

I'd appreciate for anyone who's looking for a solution on this to try this out, and we can then possibly turn this into a proper npm module with typings, tests, docs & the whole lot. This behavior is super important in my opinion, and it's a real shame that apollo-client 2 has no proper solution for this yet.

Nice one ✌️

[giautm]

As an offline app:

• All graphql requests are added to the queue. (can be stored in the cache).
• When the application receives the online signal, it informs the link to resubmit the previous request.

I imagine that it is used like this:

import { NetInfo } from 'react-native';
import OfflineLink from 'apollo-link-offline';

const offlineLink = new OfflineLink({
  isOnlineAsync: () => NetInfo.isConnected.fetch(),
});

NetInfo.isConnected.addEventListener('change', (isConnected) => {
  if (isConnected) {
    offlineLink.resubmit();
  }
});

[masull]

Is Apollo working on an AWS AppSync equivalent? We already have a GraphQL server and need offline client caching for queries and mutations using our server, not AWS solutions (i.e. Lamda, DynamoDB, Elastic Search).

[helfer]

@giautm I've implemented pretty much exactly what you describe for a small project of mine a little while ago, and decided to publish it as a really simple package: apollo-link-queue. If any of you have ideas for improvement, I'd love to get your suggestions or contributions.

[jamesreggio]

For those of you arriving here from a Google Search, I've written up a summary of all of the existing offline technologies available today for Apollo Client 2.0.

https://github.com/benseitz/apollo-link-offline/issues/1#issuecomment-371678922

[HariSeldon23]

I think this would be an amazing piece of functionality. The way I see it potentially happening, would be to have Apollo-link-offline define a set of Queries and Mutations that need to be used offline.

Once those are defined, in the background of the application, we could fetch all the data from the Queries and store those in local storage. For the Mutations, we'd need all the fields created in the local storage so that we can store any data that is created offline.

This way we can still perform functionality while offline while storing the data in a local cache.
We'd also pass Googles PWA tests if we could link to Service Workers
Then when the app comes back online, then all the mutations that occurred on the local cache can be passed back through to the API.

Obviously there'd need to be some form of Concurrency protection in place.

My worry is User A (offline) updates Table X while User B (online) has updated Table X while User A was offline. We'd need some form of Concurrency rules here. I guess it could be date driven, but it doesn't solve the issue if User A overrides User B's data without User B being aware.

Or we could just fail the Mutation and let the User manually update when back online.

Apollo Client 2 is amazing, but this piece of functionality would be the absolute dogs bollocks

[danieljvdm]

@2WheelCoder I like the idea of a PR to apollo client for a new cache. Perhaps apollo-offline-cache which can extend apollo-cache-inmemory? I'd be willing to help work on that. Want to open an issue there?

[PaulBrachmann]

@nicocrm I very much agree with you. Persisting queued queries/mutations should definitely be something a potential apollo-link-offline takes care of.

@benseitz I too think it'd be a great idea to start apollo-link-offline as a community effort and would very much like to participate in its development.

@lachenmayer You are absolutely right, the amount of code shared by apollo-offline and the planned apollo-link-offline package is probably going to be next to none. I do however think, that many of the underlying concepts of the former still apply when developing an offline toolkit set in the Apollo 2.0 universe.

In any case I'd be happy to discuss ideas for apollo-link-offline, give feedback on the implementation and - as soon as my schedule frees up a bit - contribute some code as well.

[2WheelCoder]

@danieljvdm I’m in the same place and I think it does make sense to handle persistence directly on the cache after just running into the same issue you did. The HttpLink just isn’t suited for it since the observable completes after the request but (usually) before the cache updates.

While it may be worth opening up a PR on apollo-cache-inmemory I also wonder if it would be better to fork it into a new project. Apollo 2’s modularity around the cache and link seem to suggest that if you need a different kind of cache you can just build it. Maybe an issue on apollo-client is the next step (I don’t think apollo-cache-inmemory has its own repo)?

Apologies for getting a little off-topic from links here.

[MillerGregor]

@smithaitufe,
Yes, you can use apollo-cache-persist.

[HariSeldon23]

@2WheelCoder @danieljvdm what about using LocalForage for the persistence https://github.com/localForage/localForage If you guys open a PR I'd be willing to lend a hand

[HariSeldon23]

Interesting article that could be relevant to this discussion https://blog.logrocket.com/building-an-offline-first-app-with-react-and-rxdb-e97a1fa64356?t=now

It looks really promising using rxdb as the offline database. This is compatible with React, React-Native, Vue, Angular, pretty much most popular options.

The issue I find though is that it needs to sync with a noSQL database like PouchDB (derivative of CouchDB). In my use case I'm using a relational SQL database (Postgres) and all my data validation occurs at the database level. If I had to use this system, then I'd need to maintain validation via the jsonschema.

It doesn't follow DRY principles, and would definitely introduce issues.

Another option I've found is Kinto http://docs.kinto-storage.org/en/stable/index.html which works with Postgres so should avoid some of the extra layers of complexity that rxdb would introduce. Kinto uses HTTP, so we could theoretically just plug it into Apollo-link-http https://github.com/apollographql/apollo-link/tree/master/packages/apollo-link-http

[ngaller]

@benseitz I am definitely in favor of that. There is a lot of interest in the feature so I am sure there is room for a community project - as long as I am not duplicating or fragmenting existing effort I can create a new team and repo for apollo-link-offline and invite all interested. I have some personal interest as well as a client really pushing for that feature so I'll have some hours to burn on it. I'll ask on the apollo-offline repo to see if they want to take the lead on it since they have a lot more experience.

[lachenmayer]

I definitely have personal interest in this too - would be happy to chat further with you on this @nicocrm. I haven't noticed any lost requests so far on react-native but tbh I haven't really tested all this stuff properly (only with queries so far & it seems to run everything just fine).

I feel like it would make sense to start a repo for this. @MLPXBrachmann who built apollo-offline mentioned above that he won't have time to spend on improving apollo-offline in the next couple of weeks, and I believe it does make most sense to call it apollo-link-offline.

I don't think there's any code that's going to be reusable from apollo-offline as it's very redux-specific.

[benseitz]

@MLPXBrachmann @lachenmayer @nicocrm This sounds like the right way to do so!
Since anyone can open a public channel on the Apollo Slack. I would suggest opening one called apollo-link-offline. Maybe this makes communication between us a little bit easier. All important decisions should still be documented in GitHub Issues.

Are you okay with me opening both the repo and slack channel or does one of you want to do that?

[PCreations]

What would you consider as the typical use case ?

[holman]

I'd be a huge fan of this as well. I know there's been varying attempts at tackling this with 1.0's Redux store, but making it a full citizen of the Apollo 2.0 ecosystem with a consistent approach would be pretty marvelous.

Most of what was covered here already makes a lot of sense to me- basically I want to persist my data to local storage, and then yank it from there into Apollo after the fact. Secondarily, queuing up mutations.

As kind of a tertiary goal, being able to suck in even more data in the background would also be great, and something I'll be looking to do regardless. I'd actually like to store almost all of the user's data locally as well, just to make common query operations much faster, and then rehydrate for full accuracy, network allowing. So some sort of solution for that eagerly-load-for-later angle would be great, particularly as Service Workers are starting to become viable with Safari finally having them in tech preview.

[2WheelCoder]

I too am excited about this. @danieljvdm Got a bit of a head start on persistence over in an issue in the apollo-client repo.

I think the RetryLink will be useful for managing network errors and retries, though I'd like to see some sort of falloff like redux-offline had.

[danieljvdm]

This is pretty cool. The issue that @2WheelCoder referenced is an early and perhaps naive approach geared more towards persisting the cache rather than full offline support (queuing queries/mutations and such).

It still could be a decent place to start and build on top of. What I pointed out in that issue is that I currently don't have a good way of monitoring the cache for changes. I actually tried to implement @2WheelCoder 's initial suggestion earlier today but ran into similar problems. The issue with creating an ApolloLink for "monitoring store changes" is that it only lives in the context between a client action and then the subsequent cache write. So I can intercept the request before it goes out (middleware) and intercept it before it gets written to the cache (afterware) but I don't know when it has been written - I can only guess (my current hack is a 1000ms timeout).

Another fatal flaw with using a link for offline persistence is that it doesn't account for subscriptions. A subscription link will only fire when the subscription is opened, not on subsequent events (maybe there's a way to do this that I missed?).

This is basically where I am right now, and I'm pretty stuck - I think to get any further I may have to open a PR to apollo-cache-inmemory.

[sedubois]

@holman would you have a reference about service worker in safari tech preview?

[holman]

@sedubois it dropped (kinda suddenly) a month or two ago. It's in tech preview, but there's still quite a bit to go.

[2WheelCoder]

@danieljvdm Yeah, I'll open an issue if you want to get a PR going. I don't have much time over the next week and a half, but am happy to jump in after that.

@Eishpirate Definitely agree LocalForage is great. I think the way redux-persist managed saving the store was pretty solid and can probably act as a decent pattern to follow in apollo-offline-cache.

[lancygoyal]

Folks, i am using redux persist package to save store data in react native AsyncStorage. Hows i do this now? Any idea?

[timLoewel]

@Eishpirate: does localforage work for react native? Looking at the compatibility chart of the library, I can not find anything about react native. https://github.com/localForage/localForage/wiki/Supported-Browsers-Platforms

It would be very sad if the offline feature of apollo would not work for apps..

[HariSeldon23]

@timLoewel it's a very good point. Hadn't really thought as far ahead as React Native. Don't believe it explicitly does.

This may cause unexpected issues further down the line. Not sure though if there's a viable alternative without having to reinvent the wheel

[MillerGregor]

Sounds like a lot of the work will be related to caching.

How about redux? (I'm serious)

An apollo-link-offline package, based on the apollo-offline work and...
An apollo-cache-redux package. Use redux-offline/redux-persist just as apollo-offline does now.

Apollo created the inmemory-cache as the default, generic solution - I can understand why they didn't want to be tied so closely with another library. But the redux/redux-offline/redux-persis solution is so battle-tested... it's still a great choice. Development is very active for all three.

EDIT: apollo-cache-redux exists now, thanks to @rportugal

[smithaitufe]

Please what is the state of this issue? Is there something we can use for offline stuffs that is from Apollo and not related to Redux Offline?

[smithaitufe]

@Gregor1971
Thanks for this reference. I will try it out and submit my observations.
On a surface look, it looks cool and easy to implement.

How does this differ from apollo-link-queue

[MillerGregor]

@smithaitufe,

How does this differ from apollo-link-queue?

It looks like apollo-link-queue does intelligent things based on connection status. apollo-cache-persist saves the cache; allowing, for example, users to start and run your app while disconnected. Without persisting the cache, you app would need connectivity to start up,

We'll likely want both, or better yet, @lachenmayer's solution above (which uses apollo-link-queue) and a cache that's persistent. (we're chatting in the link repos, so the most of the focus here is on that)

[PaulBrachmann]

@lachenmayer From first looks, your approach looks very appealing.

I absolutely agree with you on the necessity of something like apollo-offline's optimistic fetch feature. It (or rather the lack of it) largely was what inspired me to start apollo-offline.

I myself am not quite satisfied with how I had to implement selectively enabling/disabling the optimistic fetch feature in apollo-offline. Query variables weren't the first choice, but they seemed like the most practical one. What would you propose?

The next couple of weeks are going to be quite stressful for me. After that though, I'd be more than happy to contribute to the implementation of an up-to-date offline first solution for Apollo - may that be a new version of the apollo-offline package or something like apollo-link-offline.

[lachenmayer]

Thanks @MLPXBrachmann!
I feel that the optimistic fetching should be controlled using fetchPolicy - if I don't want anything to come from the cache, I should be able to use network-only.

[smithaitufe]

@lachenmayer @MLPXBrachmann

I am more than ready and willing to contribute.

[ngaller]

I too am looking for options to implement offline behavior in an Apollo app. The approach from @lachenmayer looks super promising, but as it relies on apollo-link-retry the mutations that have not been applied successfully will not be persisted, so if the page is refreshed any data that has not been committed to the server will be discarded (I assume that is the same thing on react-native if the app is suspended). Is there any work or at least discussion on that aspect?

[benseitz]

@nicocrm I see... At the moment the discussion around offline support ist mostly in this issue. I think apollo themselves have other priorities right now, so we should create a community project apollo-link-offline. This will hopefully lead to more discussion and progress than right now 😄

[ngaller]

For sure, thank you!

[benseitz]

I added you three as collaborators to the Repo. And you can join the #apollo-link-offline channel in Apollo Slack

Of course everyone is invited to collaborate on the GitHub Repo and discuss on Slack :)
I'm very excited 💯

[geminiyellow]

hope it working.

@masull That would be absolutely amazing. I tried and didn't think firebase or parse platform work for me, so having this feature would be glorious.
Having a hard time setting everything up with lots of packages... not fun at all :)