feathers-plus / feathers-offline-realtime Goto Github PK

We have implemented optimistic mutation as described in the docs:

	/**
	 * Creating a new message using optimisic mutation
	 * https://docs.feathersjs.com/guides/offline-first/optimistic-mutation.html
	 */
	static actOnCreateMessage = async (obj) => {
		const { conversationID, text }= obj;
		const messagesRealtime = new Realtime(
			this.client.service('api/chat/v1/messages'),
			{
				query: { conversationID },
				publication: record => record.conversationID === conversationID,
				uuid: true
			}
		);

		this.client.use('clientMessages', optimisticMutator({ replicator: messagesRealtime }));
		const clientMessages = this.client.service('clientMessages');

		await messagesRealtime.connect();

		try {
			const response = await clientMessages.create({ conversationID, text });
			return { response };
		} catch (error) {
			return { error };
		}
	}

So as this works fine when connected to the internet, the app breaks with a timeout when offline:

Error: Timeout of 5000ms exceeded calling api/chat/v1/messages::find

Though the documentation already contains a section for periodic inspection
https://docs.feathersjs.com/guides/offline-first/configure-realtime.html#example-using-periodic-inspection

I think it'll be really useful if the engine can do it on it's own internally. First check if the client transport is rest and if so start longpolling with a sensible default of N seconds, which can obviously be overridden via options.

It could seem like magic at first, but I guess that's the entire point of this realtime engine, to simply replicate the records without us having to worry about how its doing so.

Because socket.io uses 'component-emitter'. 'events' has not been updated lately.

The before client-side hooks run before the optimistic-mutator adapter is called during which it calls the remote service.

There is 2 not necessarily compatible needs here. First the id, data, params when the adapter is called is what is used to call the remote service. Second, the client-side hooks are trying to duplicate as much as they can the processing of the server-side hooks, in order to have the value of the optimistically updated data similar to the what the server will produce.

Perhaps we ought to have a useThePresentValueOfTheHookToCallTheServer hook for use on the client.
It would stash the hook's current id, data, params. The adapter would use those stashed values if they existed; working the same way as now if they did not.

Currently feathers-offline-realtime when taking the first snapshot does not update the isLoading prop in the relevant store created by feathers-redux

I don't see IndexDb being used in your code and was wondering how you store the offline data?

I'm evaluating the offline realtime solution for my new react native app. I would like to let you know about some things I came about. Maybe they are already solved without me realizing that.

The things that I'm missing are:

• some retry logic for optimistic mutations or an interface for customizing this (maybe pass a custom shareable retry function (serviceName, record, tryNo) => -1, 0..x ms in options)
• start with a partial data set of historical data, that is already locally on the storage of the device
• recheck, correct and update/remove data from the past xx days / past xx records at startup
• make the records store pluggable by defining a store interface (Use case: store the local records in redux, also using normalizr to split nested records that need updates, too)
• some information about resilience (What if the connection drops for some seconds - in a car or train maybe - will feathers-offline-realtime get out of sync? Is there a way to detect this and handle it?)

I think many of these are common things. Hope it helps.

Actual behavior

I have two redux-thunk actions: one to fetch my user's post list, and another one to create a post. Both use an offline service, configured as indicated in docs.

const postsOffline = feathers.service('postsClient')

const fetchPosts = userId => dispatch => {
  return postsOffline.find({ author: userId })
    .then(data => dispatch(fetchPostsSuccess(data)))

const createPost = post => dispatch => {
  return postsOffline.create(post)
    .then(post => dispatch(createPostSuccess(post)))

When I create a post, createPostSuccess is dispatched with the new post ( the local copy ) as the payload, and the reducer return [...state, action.payload], so everything works fine.
But when I navigate to another route and then come back to posts page, the fetchPosts action is dispatched but as the payload it has the posts list with the new one created ( by the server ) AND the local copy of the same new post, so this new post is duplicated.

Expected behavior

When fetchPosts is dispatched, posts list is returned with the new post ( the one created by the server if there's connection or the local copy if there's not, NOT BOTH ).

I can't figure out what happen, any suggestion?
Thanks for the great work.

🚨 You need to enable Continuous Integration on all branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because we are using your CI build statuses to figure out when to notify you about breaking changes.

Since we did not receive a CI status on the greenkeeper/initial branch, we assume that you still need to configure it.

If you have already set up a CI for this repository, you might need to check your configuration. Make sure it will run on all new branches. If you don’t want it to run on every branch, you can whitelist branches starting with greenkeeper/.

We recommend using Travis CI, but Greenkeeper will work with every other CI service as well.

You cannot be guaranteed a clean connection termination over TCP (https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Connection_termination).

By default, a Socket.IO client will send a heartbeat to the server every 15 seconds (heartbeat interval), and if the server hasn't heard from the client in 20 seconds (heartbeat timeout) it will consider the client disconnected.

In mobile apps using a wireless network connection the heartbeat interval shouldn't be too short otherwise your battery will be drained fast (e.g. the Node.js default server-to-client heartbeat interval of ~25 seconds is not very good). https://developer.android.com/training/efficient-downloads/efficient-network-access.html#RadioStateMachine

When server calls socket.disconnect(), the reason given to disconnect handler is: io server disconnect. And when server stops, the reason is: transport close.

It seems like the guides and docs are currently in Feathers Auk's documentation site, which is now outdated as it is superseded by Feathers Buzzard. Also, the GitHub pages for feathers-plus/offline-realtime seems to be nonexistent.

As this package still works great, I suggest moving the guides and docs to the latest version of feathers' site, so that it will be easier for others to find.

P.S. I found a more detailed version of the documentation in the older v0.1.0 tag too.

On emited replication events and in subscriber, is the remove action necessary? Can it be replaced by mutate action with eventName removed?

While working on my take on feathers-offline-owndata (+ownnet) I stumbled upon a controversy concerning the use of uuid contra id.

When updating a record multiple times with the optimistic mutator e.g. (extracted and extended from the optimistic-mutator-online.test.js suite)

service.create({id:0, uuid:1000, order:0});
service.update(1000, {id:0, uuid:1000, order:99});
service.update(1000, {id:9, uuid:1000, order:99});
service.update(1000, {id:99, uuid:1000, order:99});

This results in three different documents(rows) due to the logic first looking in local storage for uuid(=1000) as the key and sending an update request to backend with id(=0) as the key. The same thing happens for the next two updates except now the id is 9 and 99 respectively resulting in two extra documents(rows). The problematic code can be found in optimistic-mutator.js in the _update method. And similar problems in _create, _patch, and _remove too.

If the above lines of code are executed on a service without the optimistic mutator activated, then you will get an error like NotFound: no record found for id '1000' which is what I would expect to get.

I believe the use of uuid ought to be transparent to the developer (i.e. only used behind the scenes). Another solution could be to enforce the use of uuid as the key on all collections(/tables). Either way, the code needs some attention. Currently, I've opted for the first solution in feathers-offline-owndata and it works.

BTW to whom or how do I submit my attempt feathers-offline-owndata? Please drop me the address/link on michael (at) hillerstrom (dot) name

With feathers-offline-realtime version 0.1.1 the crypto node module was added as a dependency, what breaks react native compatibility. The result is more or less this error.

Version 0.1.0 works fine with React Native.

Is there a cdn for this repository equal to the feathers.js Client?

Publication on the server emits an emit to inform the client the publication has been handled. This can be replaced with the ack option.

socket.send([...args][, ack])
args
ack (Function)
Returns Socket
Sends a message event. See socket.emit(eventName[, ...args][, ack]).

#socket.emit(eventName[, ...args][, ack])
eventName (String)
args
ack (Function)
Returns Socket
Emits an event to the socket identified by the string name. Any other parameters can be included. All serializable datastructures are supported, including Buffer.

socket.emit('hello', 'world');
socket.emit('with-binary', 1, '2', { 3: '4', 5: new Buffer(6) });
The ack argument is optional and will be called with the server answer.

socket.emit('ferret', 'tobi', (data) => {
console.log(data); // data will be 'woot'
});

// server:
// io.on('connection', (socket) => {
// socket.on('ferret', (name, fn) => {
// fn('woot');
// });
// });
#socket.on(

publication: data => data.username === 'john';
// or
publication: clientPublications.addPublication(feathersClient, 'messages', {
  module: commonPublications,
  name: 'query',
  params: { username: 'john' },
});

Is what actually filters the records both for snapshots and service events.

query: { username: 'john' }

Is used to return a minimal number of records during the snapshot. This is for performance purposes. options.publication still filters them further.

Having both publication and query is ugly. Can we specify just one in common use cases?

Idea #1: If we have options.query but no options.publication , can we could set options.publication = require('./commonPublications').query(options.query). This does not handle communicating the publication to the server. We'd have to call clientPublications.addPublication(() to do that, and trust the server Publications = {} contains commonPublications.query. Further how do we knoe the user wants the server to do filtering as we cannot have a isServer option.

Idea #2: Such defaults are confusing, fragile and awkward to document. Let people get used to specifying both options.query and options.publication.

I'm for #2 ATM.

Steps to reproduce

EDIT / TLDR;

I have commented out all requires and uses of debugmodule and now it works:

I've added the module to a @stencil/core app with an import statement. Rollup is trying to bundle things up but fails. I then commented out all lines of code in your module until I found out that this line fails:

var _debug = require('debug');

inside base-replicator.js.

Looking at the mode_modules of your module it indeed is missing.

Then I tried to npm install inside your module which gave me the error below.

Expected behavior

1. I can import the module into an app that uses a web bundler, in my case rollup.
2. I can npm install on the distributed npm package without error

Actual behavior

npm install on the deployed version of the module gives me this:

> npm run compile


> [email protected] compile /Users/matthias/Documents/Projekte/bitflower/Case OS/Prototype/v0.2/caseos-ui/node_modules/feathers-offline-realtime
> shx rm -rf lib/ && babel -d lib/ src/

src/ doesn't exist
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! [email protected] compile: `shx rm -rf lib/ && babel -d lib/ src/`
npm ERR! Exit status 2
npm ERR! 
npm ERR! Failed at the [email protected] compile script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/matthias/.npm/_logs/2018-02-09T12_49_47_873Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! [email protected] prepublish: `npm run compile`
npm ERR! Exit status 2
npm ERR! 
npm ERR! Failed at the [email protected] prepublish script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/matthias/.npm/_logs/2018-02-09T12_49_47_963Z-debug.log

System configuration

Tell us about the applicable parts of your setup.
The module is part of a Stencil (Ionic hybrid apps) app.

Module versions (especially the part that's not working):

{
  "_from": "feathers-offline-realtime",
  "_id": "[email protected]",
  "_inBundle": false,
  "_integrity": "sha1-BQvY+HuJTZ8f/3pGhhLgO6aJXZs=",
  "_location": "/feathers-offline-realtime",
  "_phantomChildren": {},
  "_requested": {
    "type": "tag",
    "registry": true,
    "raw": "feathers-offline-realtime",
    "name": "feathers-offline-realtime",
    "escapedName": "feathers-offline-realtime",
    "rawSpec": "",
    "saveSpec": null,
    "fetchSpec": "latest"
  },
  "_requiredBy": [
    "#USER",
    "/"
  ],
  "_resolved": "https://registry.npmjs.org/feathers-offline-realtime/-/feathers-offline-realtime-0.1.2.tgz",
  "_shasum": "050bd8f87b894d9f1fff7a468612e03ba6895d9b",
  "_spec": "feathers-offline-realtime",
  "_where": "/Users/matthias/Documents/Projekte/bitflower/Case OS/Prototype/v0.2/caseos-ui",
  "author": {
    "name": "Feathers contributors",
    "email": "[email protected]",
    "url": "https://feathersjs.com"
  },
  "bugs": {
    "url": "https://github.com/feathersjs/feathers-offline-realtime/issues"
  },
  "bundleDependencies": false,
  "contributors": [],
  "dependencies": {
    "component-emitter": "1.2.1",
    "debug": "^2.6.8",
    "feathers-commons": "0.8.7",
    "feathers-errors": "2.8.1",
    "feathers-offline-snapshot": "^0.0.1",
    "feathers-query-filters": "2.1.2",
    "md5": "2.2.1",
    "shortid": "2.2.8",
    "uberproto": "1.2.0",
    "uuid": "3.1.0"
  },
  "deprecated": false,
  "description": "Offline-first realtime replication with optimistic updates.",
  "devDependencies": {
    "babel-cli": "^6.24.1",
    "babel-core": "^6.24.1",
    "babel-plugin-add-module-exports": "^0.2.1",
    "babel-preset-es2015": "^6.24.1",
    "chai": "^4.0.0",
    "feathers": "^2.1.3",
    "feathers-hooks": "^2.0.1",
    "feathers-memory": "^1.1.0",
    "istanbul": "^1.1.0-alpha.1",
    "mocha": "^3.4.2",
    "semistandard": "^11.0.0",
    "shx": "^0.2.2"
  },
  "directories": {
    "lib": "lib"
  },
  "engines": {
    "node": ">= 4.6.0"
  },
  "homepage": "https://github.com/feathersjs/feathers-offline-realtime",
  "keywords": [
    "feathers",
    "feathers-plugin"
  ],
  "license": "MIT",
  "main": "lib/",
  "name": "feathers-offline-realtime",
  "repository": {
    "type": "git",
    "url": "git://github.com/feathersjs/feathers-offline-realtime.git"
  },
  "scripts": {
    "changelog": "github_changelog_generator && git add CHANGELOG.md && git commit -am \"Updating changelog\"",
    "compile": "shx rm -rf lib/ && babel -d lib/ src/",
    "coverage": "istanbul cover node_modules/mocha/bin/_mocha -- --opts mocha.opts",
    "lint": "semistandard src/**/*.js test/**/*.js --fix",
    "mocha": "mocha --opts mocha.opts",
    "prepublish": "npm run compile",
    "publish": "git push origin --tags && npm run changelog && git push origin",
    "release:major": "npm version major && npm publish",
    "release:minor": "npm version minor && npm publish",
    "release:patch": "npm version patch && npm publish",
    "start": "npm run compile && node example/app",
    "test": "npm run compile && npm run lint && npm run coverage",
    "watch": "babel --watch -d lib/ src/"
  },
  "semistandard": {
    "sourceType": "module",
    "env": [
      "mocha"
    ]
  },
  "version": "0.1.2"
}

NodeJS version:
8.9.4

Operating System:
macOS 10.13.3

Browser Version:
Chrome 64

Module Loader:
rollup (@stencil/core)