GithubHelp home page GithubHelp logo

pouchdb-community / relational-pouch Goto Github PK

View Code? Open in Web Editor NEW
404.0 17.0 68.0 31.4 MB

Store relational data in PouchDB/CouchDB

License: Apache License 2.0

JavaScript 15.77% Shell 1.19% HTML 0.55% TypeScript 82.49%

relational-pouch's Introduction

Relational Pouch Build Status

var db = new PouchDB('mydb');
db.setSchema([
  {singular: 'author', plural: 'authors', relations: { books: {hasMany: 'book'}}},
  {singular: 'book', plural: 'books', relations: {author: {belongsTo: 'author'}}}
]);
db.rel.save('author', {
  name: 'George R. R. Martin', id: 1, books: [6, 7]
}).then(function () {
  return db.rel.save('book', { title: 'A Game of Thrones', id: 6, author: 1});
}).then(function () {
  return db.rel.save('book', {title: 'The Hedge Knight', id: 7, author: 1});
}).catch(console.log.bind(console));

Relational Pouch is a plugin for PouchDB that allows you to interact with PouchDB/CouchDB like a relational data store, with types and relations.

It provides an enhanced API on top of PouchDB that is probably more familiar to fans of relational databases, and maybe even easier to use. At the same time, though, you still have CouchDB's awesome indexing and sync capabilities.

This plugin also uses clever tricks to avoid creating secondary indexes. This means that even if you have complex entity relations, your database operations should still be very fast.

The main goal of this is to provide an API that is as similar to Ember Data and json:api as possible, while still being performant and Pouch-like.

This plugin is largely what powers Ember Pouch.

Installation

In the browser

Download from GitHub, or use Bower:

bower install relational-pouch

Then include it after pouchdb.js in your HTML page:

<script src="pouchdb.js"></script>
<script src="pouchdb.find.js"></script>
<script src="pouchdb.relational-pouch.browser.js"></script>

In Node.js

npm install relational-pouch

And then attach it to the PouchDB object:

var PouchDB = require('pouchdb');
PouchDB.plugin(require('relational-pouch'));
PouchDB.plugin(require('pouchdb-find'));

Typescript

This package contains its own type definitions. Due to the nature of setSchema, which alters the database on which it is called, typescript needs to know about these changes. This is done by returning a new type. So working with this plugin should look something like this:

import Pouch from 'pouchdb-core';
//import some adapter
import find from 'pouchdb-find';
import rel from 'relational-pouch';

Pouch
  //.plugin(someadapter)
  .plugin(find)
  .plugin(rel);

const baseDB = new Pouch(...);//adapter options
const relDB = baseDB.setSchema(...);//schema options

let relDoc = await relDB.rel.find('sometype', 'someid');

//non relation pouch API is still available
let doc = await relDB.get('someid');

API

Summary

db.setSchema(schema)

Call this after you initialize your PouchDB, in order to define your entities and relationships:

var db = new PouchDB('mydb');
db.setSchema([
  {
    singular: 'post',
    plural: 'posts',
    relations: {
      author: {belongsTo: 'author'},
      comments: {hasMany: 'comment'}
    }
  },
  {
    singular: 'author',
    plural: 'authors',
    relations: {
      posts: {hasMany: 'post'}
    }
  },
  {
    singular: 'comment',
    plural: 'comments',
    relations: {
      post: {belongsTo: 'post'}
    }
  }
]);

This is a synchronous method that does not return a Promise.

You can define one-to-one, one-to-many, and many-to-many relationships using any combination of belongsTo and hasMany that you want. For more examples, read the Ember guide to models, which is what inspired this.

You need to explicitly define the singular and plural forms of your entities, because I'm not a big fan of applying magic Anglocentric defaults to everything.

db.rel.<method>

Once you call setSchema, your db will be blessed with a rel object, which is where you can start using the rest of this plugin's API.

documentType

Rarely, you might want to have two different views over the same underlying data. Use documentType to create a view which reads the same data as another type:

var db = new PouchDB('mydb');
db.setSchema([
  {
    singular: 'post',
    plural: 'posts',
    relations: {
      author: {belongsTo: 'author'},
      comments: {hasMany: 'comment'}
    }
  },
  {
    singular: 'postSummary',
    plural: 'postSummaries',
    documentType: 'post'
  }
]);

Here, when you load a "postSummary", it will return the same core record as "post", but will not resolve the relationships.

Be careful when using this feature โ€” it is probably best to treat a type declaring a documentType as read-only. Do all creates/updates via the main type.

db.rel.save(type, object)

Save an object with a particular type. This returns a Promise.

db.rel.save('post', {
  title: 'Rails is Omakase',
  text: 'There are a lot of a-la-carte software...'
});

Result:

{
  "id": "14760983-285C-6D1F-9813-D82E08F1AC29",
  "rev": "1-84df2c73028e5b8d0ae1cbb401959370"
}

You can optionally specify an id, otherwise relational-pouch will create an id for you. (What's the difference between id and _id? See below.)

db.rel.save('post', {
  title: 'Rails is Unagi',
  text: 'Delicious unagi. Mmmmmm.',
  id: 1
});

Result:

{
  "id": 1,
  "rev": "1-0ae315ee597b22cc4b1acf9e0edc35ba"
}

You'll notice the special field rev, which is a revision identifier. That'll come into play later.

id and rev are reserved field names!

This plugin uses id and rev. You shouldn't use those fields for anything else. An id can be any string or integer.

id vs _id

id belongs to relational-pouch, whereas _id belongs to PouchDB/PouchDB. relational-pouch uses id for references to related records. You can put the same value for id as _id, but if one is autogenerated you might want to autogenerate the other, too. Use-cases vary so do what works for you! See also: db.rel.parseDocID and db.rel.makeDocID, covered in this README below.

db.rel.find(type)

Find all objects with a given type. Returns a Promise.

db.rel.find('post');

Result:

{
  "posts": [
    {
      "title": "Rails is Unagi",
      "text": "Delicious unagi. Mmmmmm.",
      "id": 1,
      "rev": "1-0ae315ee597b22cc4b1acf9e0edc35ba"
    },
    {
      "title": "Rails is Omakase",
      "text": "There are a lot of a-la-carte software...",
      "id": "14760983-285C-6D1F-9813-D82E08F1AC29",
      "rev": "1-84df2c73028e5b8d0ae1cbb401959370"
    }
  ]
}

The list will be empty if it doesn't find anything. The results are sorted by id.

db.rel.find(type, id)

Find an object with the given type and id. Returns a Promise.

db.rel.find('post', 1);

Result:

{
  "posts": [
    {
      "title": "Rails is Unagi",
      "text": "Delicious unagi. Mmmmmm.",
      "id": 1,
      "rev": "1-0ae315ee597b22cc4b1acf9e0edc35ba"
    }
  ]
}

db.rel.find(type, ids)

Find multiple objects with multiple ids. Returns a Promise.

db.rel.find('post', [3, 2, 1]);

Result:

{
  "posts": [
    {
      "title": "Rails is Unagi",
      "text": "Delicious unagi. Mmmmmm.",
      "id": 1,
      "rev": "1-0ae315ee597b22cc4b1acf9e0edc35ba"
    },  
    {
      "title": "Maybe Rails is more like a sushi buffet",
      "text": "Heresy!",
      "id": 2,
      "rev": "1-6d8ac6d86d01b91cfbe2f53e0c81bb86"
    }
  ]
}

If an id isn't found, it's simply not returned. Notice that above, there is no object with an id of 3.

find results are always returned ordered by id. The order of your ids array will not necessarily be reflected in the returned array of objects.

db.rel.find(type, options)

Find all objects with a given type and limit the results via the passed in options. Returns a Promise.

db.rel.find('post',{startkey: 1, limit: 2});

Result:

{
  "posts": [
    {
      "title": "Rails is Unagi",
      "text": "Delicious unagi. Mmmmmm.",
      "id": 1,
      "rev": "1-0ae315ee597b22cc4b1acf9e0edc35ba"
    },  
    {
      "title": "Maybe Rails is more like a sushi buffet",
      "text": "Heresy!",
      "id": 2,
      "rev": "1-6d8ac6d86d01b91cfbe2f53e0c81bb86"
    }
  ]
}

The following options based on the options for PouchDB batch fetch are available:

  • startkey & endkey: Get documents with IDs in a certain range (inclusive/inclusive).
  • limit: Maximum number of documents to return.
  • skip: Number of docs to skip before returning (warning: poor performance on IndexedDB/LevelDB!).

db.rel.del(type, object)

Deletes the given object. Returns a Promise.

db.rel.del('post', {id:1, rev:"1-0560dbb11ead319c9f5bc1f667ea8e84"});

Result:

{"deleted":true}

The minimum you need to delete something is an id and a rev. The easiest pattern is to just find it before deleting it:

db.rel.find('post', 1).then(function (post) {
  return db.rel.del('post', post);
});

db.rel.putAttachment(type, object, attachmentId, attachment, attachmentType)

Adds an attachment to the given object. Returns a Promise. See PouchDB Attachments but note that ._attachments is instead .attachments in relational-pouch.

var attachment = new Blob(['Is there life on Mars?']);
// Or in Node.js: new Buffer('Is there life on Mars?')
db.rel.putAttachment('post', {id:1, rev:"1-..."}, 'file', attachment, 'text/plain');

This returns the new rev:

"2-...."

db.rel.getAttachment(type, id, attachmentId)

Gets an attachment for the given document id. Returns a Promise to a Blob (or Buffer for Node).

db.rel.getAttachment('post', 1, 'file').then(function (attachment) {
  // convert the Blob into an object URL and show it in an image tag
  $('img').attr('src', URL.createObjectURL(attachment));
});

db.rel.removeAttachment(type, object, attachmentId)

Remove an attachment from the given object. Returns a Promise.

var attachment = new Blob(['Is there life on Mars?']); // new Buffer('Is there life on Mars?') for node
db.rel.putAttachment('post', {id:1, rev:"1-0560dbb11ead319c9f5bc1f667ea8e84"}, 'file', attachment, 'text/plain').then(function (res) {
  var post = res.posts[0];
  db.rel.removeAttachment('post', post, 'file');
});

Or continuing from the putAttachment example:

db.rel.removeAttachment('post', {id: 1, rev:"2-09d5c5bd86fc170c064b296773044ea9"} , 'file');

This returns the new rev:

"3-...."

db.rel.parseDocID(docID)

Parses a raw CouchDB/PouchDB doc _id into an object containing a type and id field. Basically only useful for working with the db.changes() feed, so you can tell what changed from a "relational" perspective rather than from the raw CouchDB/PouchDB perspective.

This method is synchronous, so it directly returns the object rather than a Promise.

db.rel.parseDocID("author_1_0000000000000019");

Returns:

{
  "type": "author",
  "id": 19
}

So e.g. with changes() you could do:

db.changes().then(function (changes) {
return changes.results.map(function (change) {
  return db.rel.parseDocID(change.id);
});

Result is e.g.:

[
  {"type": "author", "id": 19},
  {"type": "book", "id": 1},
  {"type": "book", "id": 2},
  {"type": "book", "id": 3}
]

db.rel.makeDocID(parsedID)

Creates a valid _id from an object with type and id properties, such as parseDocID generates. The format is <type>_<id type>_<id>. The <id type> depends on the id. If the id is undefined the value is 0, if the id is a number, the value is 1, if the id is a string the value is 2, and if the id is an object the value is 3.

db.rel.makeDocID({ "type": "author", "id": 123 });
// author_1_0000000000000123
db.rel.makeDocID({ "type": "author", "id": "onetwothree" });
// author_2_onetwothree

Useful if you need to perform operations with the underlying database, e.g.:

var _id = db.rel.makeDocID({ "type": "author", "id": 19 });
db.get(_id).then(function (doc) {
  var parsedId = db.parseDocID(doc._id);
  doc.data.type = parsedId.type;
  doc.data.id = parsedId.id;
  return doc.data;
});

db.rel.isDeleted(type, id)

Returns a Promise that resolves to true if document is deleted, false if it still exists and null if it is not in the database

db.rel.parseRelDocs(type, pouchDocs)

Parses pouch documents that should be in this schema. Loads extra relations if needed. This function is useful when you are loading data from db.find for example, instead of from db.rel.

Example:

db.find(selector).then(function (data) {
  return db.rel.parseRelDocs(type, data.docs);
});

Returns data as db.rel.find(type) would return it.

It requires a type parameter, which could be gotten from the id of the first document using db.rel.parseDocID(docID), but most of the time you will know which type you are trying to find with the db.find call.

db.rel.findHasMany(type, belongsToKey, belongsToId)

Uses db.find to build a selector that searches for the documents of type type that have a value of belongsToId set in the field belongsToKey. This can be used in a higher level API to use asynchronous hasMany relations that don't store the hasMany side.

Example:

db.rel.findHasMany('post', 'author', '1');

Returns a Promise that will resolve to an array of posts that have author 1 as db.rel.find(type) would.

Since db.find requires the use of indexes, you need to setup an index for this extra lookup. The fields required by this index are _id and the field specified in the belongsToKey, prefixed with data.. The _id field is used to filter the related items by type. Without it another type with the same field could also be returned. So the example above would give:

db.createIndex({index: { fields: ['data.author', '_id'] }});

For performance reasons the queried field is used first here. As this is to be expected to return a smaller set that filtering on type first. If your database however has a lot more document types that has data.author fields too, you may find that switching the order and using id as the first filter will give faster results.

Managing relationships

Entity relationships are encoded using the Ember Data Model format, which is a slight simplification of json:api.

One-to-one relationships

An author has exactly one profile, and vice-versa:

db.setSchema([
  {
    singular: 'author',
    plural: 'authors',
    relations: {
      'profile': {belongsTo: 'profile'}
    }
  },
  {
    singular: 'profile',
    plural: 'profiles',
    relations: {
      'author': {belongsTo: 'author'}
    }
  }
]);

db.rel.save('author', {
  name: 'Stephen King',
  id: 19,
  profile: 21
}).then(function () {
  return db.rel.save('profile', {
    description: 'nice masculine jawline',
    id: 21,
    author: 19
  });
}).then(function () {
  return db.rel.find('author');
});

Result:

{
  "authors": [
    {
      "name": "Stephen King",
      "profile": 21,
      "id": 19,
      "rev": "1-bf705a912bf672b30ad262b33a19c5c3"
    }
  ],
  "profiles": [
    {
      "description": "nice masculine jawline",
      "author": 19,
      "id": 21,
      "rev": "1-ef86a08ea3243ea59302ceaa04afd59f"
    }
  ]
}

Many-to-one relationships

An author has many books:

db.setSchema([
  {
    singular: 'author',
    plural: 'authors',
    relations: {
      'books': {hasMany: 'book'}
    }
  },
  {
    singular: 'book',
    plural: 'books',
    relations: {
      'author': {belongsTo: 'author'}
    }
  }
]);

db.rel.save('author', {
  name: 'Stephen King',
  id: 19,
  books: [1]
}).then(function () {
  return db.rel.save('author', {
    name: 'George R. R. Martin',
    id: 1,
    books: [6, 7]
  });
}).then(function () {
  return db.rel.save('book', {
    title: 'It',
    id: 1,
    author: 19
  });
}).then(function () {
  return db.rel.save('book', {
    title: 'A Game of Thrones',
    id: 6,
    author: 1
  });
}).then(function () {
  return db.rel.save('book', {
    title: 'The Hedge Knight',
    id: 7,
    author: 1
  });
}).then(function () {
  return db.rel.find('author');
});

Result:

{
  "authors": [
    {
      "name": "George R. R. Martin",
      "books": [
        6,
        7
      ],
      "id": 1,
      "rev": "1-04e165889a4a9303a6dc07a54cee9741"
    },
    {
      "name": "Stephen King",
      "books": [
        1
      ],
      "id": 19,
      "rev": "1-38580117cb4a1ddb2c7151453a7f9129"
    }
  ],
  "books": [
    {
      "title": "It",
      "author": 19,
      "id": 1,
      "rev": "1-1b7ea74936a8034aee7da27ffd36a63f"
    },
    {
      "title": "A Game of Thrones",
      "author": 1,
      "id": 6,
      "rev": "1-a6f0dc69fc79d5565639074b5defa52d"
    },
    {
      "title": "The Hedge Knight",
      "author": 1,
      "id": 7,
      "rev": "1-4988aa3215070c71e1505a05f90bb60f"
    }
  ]
}

Many-to-many relationships

Peter Straub actually co-wrote The Talisman with Stephen King. So a book can have many authors, and an author can have many books:

db.setSchema([
  {
    singular: 'author',
    plural: 'authors',
    relations: {
      'books': {hasMany: 'book'}
    }
  },
  {
    singular: 'book',
    plural: 'books',
    relations: {
      'authors': {hasMany: 'author'}
    }
  }
]);

db.rel.save('author', {
  name: 'Stephen King',
  id: 19,
  books: [1, 2]
}).then(function () {
  return db.rel.save('author', {
    name: 'Peter Straub',
    id: 2,
    books: [2, 3]
  });
}).then(function () {
  return db.rel.save('book', {
    title: 'It',
    id: 1,
    authors: [19]
  });
}).then(function () {
  return db.rel.save('book', {
    title: 'The Talisman',
    id: 2,
    authors: [19, 2]
  });
}).then(function () {
  return db.rel.save('book', {
    title: 'Ghost Story',
    id: 3,
    authors: [2]
  });
}).then(function () {
  return db.rel.find('author');
});

Result:

{
  "authors": [
    {
      "name": "Peter Straub",
      "books": [
        2,
        3
      ],
      "id": 2,
      "rev": "1-92901c8e3e0775765777bfcbe8f4c2dd"
    },
    {
      "name": "Stephen King",
      "books": [
        1,
        2
      ],
      "id": 19,
      "rev": "1-d70d9fe033f583493029372c88ae21d0"
    }
  ],
  "books": [
    {
      "title": "It",
      "authors": [
        19
      ],
      "id": 1,
      "rev": "1-96751a2a5bb7b0fd70564efe6856dbd6"
    },
    {
      "title": "The Talisman",
      "authors": [
        19,
        2
      ],
      "id": 2,
      "rev": "1-9faf8c4f72db782dacce16a7849d156b"
    },
    {
      "title": "Ghost Story",
      "authors": [
        2
      ],
      "id": 3,
      "rev": "1-7564a1195f143e24ebf24d914c60d6be"
    }
  ]
}

Async relationships

Just like with Ember Data, you can define relationships to be async, which means that dependent objects aren't automatically sideloaded. This can reduce your request time and payload size.

For instance, let's say you want to load all authors, but you don't want to load their books, too. You can do:

db.setSchema([
  {
    singular: 'author',
    plural: 'authors',
    relations: {
      books: {hasMany: {type: 'book', options: {async: true}}}
    }
  },
  {
    singular: 'book',
    plural: 'books',
    relations: {
      author: {belongsTo: {type: 'author', options: {async: true}}}
    }
  }
]);

By default, async is consider false. So this:

...
  books: {hasMany: 'book'}
...

is equivalent to this:

...
  books: {hasMany: {type: 'book', options: {async: false}}}
...

Now let's try with {async: true}. You'll notice that, when we fetch the list of authors, only the book ids will be included, not the full books:

return db.rel.save('author', {
  name: 'Stephen King',
  id: 19,
  books: [1, 2, 3]
}).then(function () {
  return db.rel.save('book', {
    id: 1,
    title: 'The Gunslinger'
  });
}).then(function () {
  return db.rel.save('book', {
    id: 2,
    title: 'The Drawing of the Three'
  });
}).then(function () {
  return db.rel.save('book', {
    id: 3,
    title: 'The Wastelands'
  });
}).then(function () {
  return db.rel.find('author');
});

Result:

{
  "authors": [
    {
      "name": "Stephen King",
      "books": [
        1,
        2,
        3
      ],
      "id": 19,
      "rev": "1-9faf8c4f72db782dacce16a7849d156b"
    }
  ]
}

This can cut down on your request size, if you don't need the full book information when you fetch authors.

Thanks to Lars-Jรธrgen Kristiansen for implementing this feature!

Don't save hasMany

By default relational-pouch will store the child ids of an Many-to-one relationship as a property on the many side (the parent). This can lead to extra conflicts, since this goes against the normal "documents are changes" way that Couch works best. Edits to documents are best to be self contained, and changing a parent document because a child is inserted can result in problems with multiple users.

A way to fix this is to specify to relational-pouch that the parent actually does not store this array of children. But instead relational-pouch should use a db.find query to search for them.

db.setSchema([
  {
    singular: 'author',
    plural: 'authors',
    relations: {
      books: {hasMany: {type: 'book', options: {queryInverse: 'author'}}}
    }
  },
  {
    singular: 'book',
    plural: 'books',
    relations: {
      author: {belongsTo: 'author'}
    }
  }
]);

This will tell relational-pouch to not save the book ids on the author, and use a query using db.find to look for the related books. Since this uses db.rel.findHasMany(type, belongsToKey, belongsToId) internally, you also need an index as specified there, where belongsToKey is the field specified in the queryInverse option.

For async relations this queryInverse will not work at this moment and the child id array will not be present on the result. You can use the db.rel.findHasMany(type, belongsToKey, belongsToId) for this scenario instead. If you also don't give relational-pouch the child ids when calling db.rel.save you could also completely remove the hasMany side of the relation from the schema .

Advanced

Deeply nested relationships are also possible. Everything just ends up being sideloaded in the same JSON object response.

{
  "lions" : [...],
  "tigers" : [...],
  "bears" : [...]
}

When you save, you must explicitly provide the ids of dependent objects, and they must be saved independently. There is no cascading at all.

You can attach the full entity object with an id to another object, but if you include an object without an id, it will be ignored.

db.setSchema([
  {
    singular: 'author',
    plural: 'authors',
    relations: {
      profile: {belongsTo: 'profile'},
      books: {hasMany: 'books'}
    }
  },
  {
    singular: 'profile',
    plural: 'profiles',
    relations: {
      author: {belongsTo: 'author'}
    }
  },
  {
    singular: 'book',
    plural: 'books',
    relations: {
      author: {belongsTo: 'author'}
    }
  }
]);

var profile = {
  description: 'nice masculine jawline',
  id: 21,
  author: 19
};
var book1 = {
  id: 1,
  title: 'The Gunslinger'
};
var book2 = {
  id: 2,
  title: 'The Drawing of the Three'
};
var book3 = {
  id: 3,
  title: 'The Wastelands'
};
db.rel.save('profile', profile).then(function () {
  return db.rel.save('book', book1);
}).then(function () {
  return db.rel.save('book', book2);
}).then(function () {
  return db.rel.save('book', book3);
}).then(function () {
  return db.rel.save('author', {
    name: 'Stephen King',
    id: 19,
    profile: profile,
    books: [book1, book2, book3]
  });
}).then(function () {
  return db.rel.find('author');
});

Result:

{
  "authors": [
    {
      "name": "Stephen King",
      "profile": 21,
      "books": [
        1,
        2,
        3
      ],
      "id": 19,
      "rev": "1-308a75619dc1b96bece7b6996d36d18b"
    }
  ],
  "profiles": [
    {
      "description": "nice masculine jawline",
      "author": 19,
      "id": 21,
      "rev": "1-7bd39e62046a0816f9c5a3836a548ec8"
    }
  ],
  "books": [
    {
      "title": "The Gunslinger",
      "id": 1,
      "rev": "1-f3a305eae85642ce74412141ec0ae0bf"
    },
    {
      "title": "The Drawing of the Three",
      "id": 2,
      "rev": "1-1c94deba48af8c1c2df1c5545246846b"
    },
    {
      "title": "The Wastelands",
      "id": 3,
      "rev": "1-a4a96e3f9e2cb3d516605fa46bbed080"
    }
  ]
}

The plugin is not smart enough to infer bidirectional relationships, so you have to attach the relation to both object. E.g. in the above example, each book explicitly has its author set, and the author explicitly has his books set. If you want to add a new book, you would need to save() the book, add it to the author's list of books, and then save() the author.

Managing revisions ("rev")

When you update an existing object, you'll need to include the rev, or else you'll get a 409 conflict error. This is standard CouchDB/PouchDB behavior, so the common idiom is:

db.rel.find('post', 1).then(function (res) {
  // do whatever you want to do to update the post
  return db.rel.save('post', res.posts[0]).catch(function (err) {
    if (err.code === 409) { // conflict
      // handle the conflict somehow. e.g. ask the user to compare the two versions,
      // or just try the whole thing again
    } else {
      throw err; // some other error
    }
  });
});

This also applies to deletions:

db.rel.find('post', 1).then(function (res) {
  return db.rel.del('post', res.posts[0]).catch(function (err) {
    if (err.code === 409) { // conflict
      // handle the conflict
    } else {
      throw err; // some other error
    }
  });
});

To avoid getting into a long discussion of why you have to do this: suffice it to say, when you build a client-server sync architecture, you are building a distributed system. Distributed systems are hard, and managing conflicts is just a reality when you have multiple computers that aren't perfectly in sync.

You will have to deal with conflicts sooner or later. With PouchDB and CouchDB, you simply pay that cost up-front.

Jan Lenhardt has a nice writeup on this.

Attachments

Thanks to bterkuile, this plugin also support attachments! Attachments are simply added inline, in the normal PouchDB way, but as

doc.attachments

rather than

doc._attachments

I.e. It follows the same convention as doc.id and doc.rev.

How does it work?

A relational Pouch/Couch is just a regular database that has been partitioned by type.

So for instance, a document with type "pokemon" and id "1" might have an actual _id like "pokemon_1", whereas a "trainer" with id "2" might have an actual _id like "trainer_2". Simple, but effective.

What is important is that this plugin leverages the very efficient allDocs() API, rather than the slower query() API. Also, it joins related documents by simply making extra requests, rather than using native map/reduce joined documents. And it's smart enough to group the requests, so the data is fetched in the fewest posisble number of requests.

Although this method may seem naรฏve, in practice you get much better performance, because building secondary indexes in Pouch/Couch takes quite a bit of time. Whereas if you just use allDocs(), it uses the built-in index on _id, so you don't have to wait for an index to be built.

Testing

In Node

This will run the tests in Node using memory and http adapter:

npm test

if you don't have a admin party setup you can specify admin credentials in the RELATIONAL_POUCH_DB_AUTH environment variable like this:

RELATIONAL_POUCH_DB_AUTH=user:password@

You can also check for 100% code coverage using:

npm run coverage

If you don't like the coverage results, change the values from 100 to something else in package.json, or add /*istanbul ignore */ comments.

If you have mocha installed globally you can run single test with:

TEST_DB=local mocha --reporter spec --grep search_phrase

The TEST_DB environment variable specifies the database that PouchDB should use (see package.json).

In the browser

Run npm run dev and then point your favorite browser to http://127.0.0.1:8001/test/index.html.

The query param ?grep=mysearch will search for tests matching mysearch.

Automated browser tests

You can run e.g.

CLIENT=selenium:firefox npm test
CLIENT=selenium:phantomjs npm test

This will run the tests automatically and the process will exit with a 0 or a 1 when it's done. Firefox uses IndexedDB, and PhantomJS uses WebSQL.

Changelog

4.0.0

  • Breaking change: To prevent us from having to do cloning of input documents, we have changed the save, putAttachment and removeAttachment API. These functions no longer return the complete document. The attachment functions only return the new rev value, while the save will also return the id. So after these promises resolve you have to manually update your in app data to reflect this new revision (and possibly id) if you want to update the document later. You can use something like the following:
    let updatedData = await db.rel.save('post', post);
    Object.assign(post, updatedData);
    or
    post.rev = await db.rel.putAttachment('post', post, 'file', fileData);
  • This library now uses Typescript, Webpack and Babel in its build setup. The build creates files in 2 output directories: lib and dist.
    • The lib directory will contain the output of tsc in esnext mode. So this can be used by Webpack and other module aware systems. These will require Babel transformations if you want to use them, but this way you can specify your own target.
    • The dist directory contains 2 files, pouchdb.relational-pouch.browser.js and pouchdb.relational-pouch.node.js. These are compiled by webpack with targets ">2%, not ie 11" and "node 10". This should be sufficient for now, but otherwise you can build your own with Webpack.

relational-pouch's People

Contributors

akofman avatar broerse avatar bterkuile avatar dependabot[bot] avatar egoossens avatar floer32 avatar garrensmith avatar gr2m avatar jkleinsc avatar jlami avatar larsjk avatar msenevir avatar mshick avatar nolanlawson avatar nruth avatar rsutphin avatar stevebest avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

relational-pouch's Issues

Handle relations when deleting/updating an object.

I didn't inspect the code to know whether it is possible but would that make any sense to handle relations when deleting or updating an object ?

For instance, if we have a one-to-many relation between books and an author, If we delete one of these books, we should also remove its reference from the books array in the artist object.

Thoughts ?

PouchDB v5.4.5

I get an exception Cannot read property 'Promise' of undefined from pouch-utils.js when I upgrade to the latest version of PouchDB:

Promise = window.PouchDB.utils.Promise

Sync issues with attachments on IE 10

Hello,

I am building an Ionic app using this plugin to store my data.
I came across a problem when replicating an attachment, containing a base64string of a picture.
First, i add an attachment to the pouchdb, then, i try to replicate the attachment to a remote CloudantDB which returns an error.

In andoid or IOS, everything works fine, but when i try to replicate an attachment on an IE10 device, an error occurs.

The only logging i am getting is the following:
@dbService | syncData | Error while replicating data to Remote Db
ERROR: Object doesn't support property or method 'readAsBinaryString'

I execute following code after i saved my attachment:
Db.getDb().replicate.to(Db.getRemoteDb()).on('complete', function (result) { console.log('@dbService | syncData | Data replicated to Remote Db'); }).on('error', function (error) { console.log('@dbService | syncData | Error while replicating data to Remote Db'); console.log('ERROR: ' + error); });

I also retrieve the following logging when i try to replicate the attachment but I totally dont understand how the filereader plugin might possibly have anything to do with this issue.
Exception was thrown at line 267, column 9 in ms-appx://be.axi.sramvp1/www/plugins/cordova-plugin-file/www/FileReader.js 0x800a01b6 - JavaScript runtime error: Object doesn't support property or method 'readAsBinaryString'

To summarize, saving the attachment to the pouchdb just works fine. Only replicating to the remote cloudantDB doesn't work.

If anyone else came across this issue, please let me know.

Thanks in advance

Handle cases where the database is in an inconsistent state

This can happen during replication, e.g. if a parent gets written before a child or a child gets written before a parent. I guess the proper thing to do would be to catch 404s and return nothing for that dependency.

I admit, though, that we are working against the grain of how CouchDB is designed.

Add a query method to allow chaining from ember-pouchdb/query

First, thanks a lot for the tool set you put together.

I feel like query in ember-pouch is broken. I amend your tests and the relational context is lost (querying smasher also returns tacoSoup)
cf.

This issue lead me here where I would like to add a query method to call from ember-pouchdb instead of the db.find() which can returned not relationally valid data.

I am currently giving it a go and will probably create a PR soon.

Is possible validate the integrity of data in a relationship?

Is possible validate the integrity of data in a relationship?

For example, in the documentation, an author has many books.

db.setSchema([
  {
    singular: 'author',
    plural: 'authors',
    relations: {
      'books': {hasMany: 'book'}
    }
  },
  {
    singular: 'book',
    plural: 'books',
    relations: {
      'author': {belongsTo: 'author'}
    }
  }
]);

If I add an author with a book code that does not exist. Is possible validate?

db.rel.save ('author', {
ย ย  name: 'Stephen King'
ย ย  id: 19
ย ย  books: [100]
}) ....

In this case, the item 100 does not exist.

Thank you.

Expose serialize and deserialize methods for the keys

So you can connect pouch relational directly to couchdb, where the docs are not stored in the same database, but in there own database. So the type is not automatically added to the id, if for instance the db adapter is 'http'.

Fewer HTTP requests

Right now it makes a lot. For local it doesn't matter, but for remote CouchDBs, it can be a drag.

Saving a doc with inner objects behavior

Suppose we have a javascript object with a inner object that we want to save as a doc in pouchdb . In my case it has more lines and a couple of relationships, but for simplicity :

{
  id : "doc1",
  name : "Document 1",
  props : {
    id : 1,
    prop1 :  "hello",
    prop2 : "world"
  }
}

In normal pouch, we can put this to save a doc like:

Image of normal pouchdb

But, if when we try to save with db.rel.save, it only stores the id of object, skipping all other content:
Image of relational-pouchdb

If before call db.rel.save, I do doc.props = [doc.props]; it will be stored as an array with the entire object. It has a big side effect doing this when we retrive in client, console.log($scope.doc) prints all doc correclty (including prop as an array with one object), but doing console.log($scope.doc.props) throws undefined.

So, what's happening when saving an object inside doc? It is the normal behavior?

Merge 1.4.0 ?

Hello,

I just noticed that from the npm repo, the last version of this project is 1.4.0 which is different from the master branch. Would it be possible to merge it to master ?

I can't find any branch for this version, only a tag and it's not really convenient in order to compare, fork or fix properly some issues.

Thanks !

db.rel.find(type, id) returns all 'type' docs, not only doc with that id.

Also happens with db.rel.find(type, ids). If you take the example from readme:

          var db = new PouchDB('mydb');
          db.setSchema([
              {singular: 'author', plural: 'authors', relations: { books: {hasMany: 'book'}}},
              {singular: 'book', plural: 'books', relations: {author: {belongsTo: 'author'}}}
          ]);
          db.rel.save('author', {
              name: 'George R. R. Martin', id: 1, books: [6, 7]
          }).then(function () {
              return db.rel.save('book', { title: 'A Game of Thrones', id: 6, author: 1});
          }).then(function () {
              return db.rel.save('book', {title: 'The Hedge Knight', id: 7, author: 1});
          }).catch(console.log.bind(console));

And then you want find book with id 7:

          db.rel.find('book',7).then(function(doc){
              console.log(doc);
          });

You got as result all books:

{
    "books" : [
        {
            "title" : "A Game of Thrones",
            "author" : 1,
            "id" : 6,
            "rev" : "1-16ce4f20eae39bd0655f00f81973dd14"
        },
        {
            "title" : "The Hedge Knight",
            "author" : 1,
            "id" : 7,
            "rev" : "1-7d35db00e3b7df172e33173eded5ab72"
        }
    ],
    "authors" : [
        {
            "name" : "George R. R. Martin",
            "books" : [
                 6,
                 7
            ],
            "id" : 1,
            "rev" : "1-c4c9b023321d67787d737421837988e3"
        }
    ]
}

Multiple relations possible?

Would something like this work (especially the relations between obj and objList):

let schema = [
            {
                singular: 'obj', plural: 'objs', relations: {
                'objList': { belongsTo: 'objList' },
                'attributesEditable': { belongsTo: 'attributesEditable' },
                'attributesViewable': { belongsTo: 'attributesViewable' },
            },
            },
            {
                singular: 'objList', plural: 'objLists', relations: {
                'objs': { hasMany: 'obj' },
                'obj': { belongsTo: 'obj' },
            },
            },
            {
                singular: 'attributesEditable', plural: 'attributesEditables', relations: {
                'obj': { belongsTo: 'obj' },
            },
            },
            {
                singular: 'attributesViewable', plural: 'attributesViewables', relations: {
                'obj': { belongsTo: 'obj' },
            },
            },
        ];

What I want:
image

My App starts with an entry point which is an obj and this has one objList with many obj in it and so on.

Why adding id type to id string?

In the serialize function, id type added to id parameter. Why this is necessary?
I think it's for the find function.

function serialize(type, id) {
  // simple collation that goes like this:
  // undefined < numbers < strings < object
  var res = type.replace('_', '') + '_';
  if (typeof id === 'number') {
    // zpad
    id = id.toString();
    while (id.length < MAX_INT_LENGTH) {
      id = '0' + id;
    }
    res += TYPE_NUM + '_' + id;
  } else if (typeof id === 'undefined') {
    // need lowest possible value
    res += TYPE_UNDEF;
  } else if (typeof id === 'object') {
    // need highest possible value
    res += TYPE_OBJ;
  } else { // string
    res += TYPE_STRING + '_' + id;
  }
  return res;
}

Search with multiple fields or selectors

Hi
I am not able to search with multiple fields like (name,description) by using this plugin as we can do with pouchdb-find plugin but that plugin is also not working as relation-pouch is saving data in one field in JSON form.

is there any way to do it ?

plugin not compatible with requirejs

Recently attempted to add local storage with ember-pouch which depends on relational-pouch. My project is using requirejs and no matter what I attempted, loading this plugin through require would not function.

Removing all references to requirejs and simply including this library as <script inside the page has cured the issue.

db.rel.find(type) not returning all data

For some reason, I'm not getting my full set of results when using this.

Are there any caveats to how this returns data? I've looked at the data in the database and it looks very similar to items that are being returned. The ids have the correct format i.e job_2_SOMEGUID.

I've done queries using pouch-find and get 170 results but only 144 with db.rel.find()

docs: removeAttachment readme say it adds an attachment

I think there's a typo in the readme

db.rel.removeAttachment(type, object, attachmentId)

Adds an attachment to the given object. Returns a Promise.

var attachment = new Blob(['Is there life on Mars?']); // new Buffer('Is there life on Mars?') for node
db.rel.putAttachment('post', {id:1, rev:"1-0560dbb11ead319c9f5bc1f667ea8e84"}, 'file', attachment, 'text/plain').then(function (res) {
  var post = res.posts[0];
  db.rel.removeAttachment('post', post, 'file');
});

Result:

{
  "posts": [
    {
      "id": 1,
      "rev": "3-...."
    }
  ]
}

I don't really know what this does, but it seems like it should say remove instead of add, and I don't know if the result example is right or needs editing.

Navigation properties on returned entity

Hi,

I'd like to do something like: customer.customersource.name. Is this possible?

I know when I do db.rel.find('customer', 'my_id') it will return the customer and the related entities but it would be nice if it could also attach a property to the model for navigation purposes.

Readme file info about find/save and find/del pattern inaccurate

Forgive me if I missed something, but it seems I can't reproduce the find/save and find/del patterns described in the Readme file.
The find method returns a collection object even when specifying an id. Shouldn't the first item be extracted before it can be fed to the del/save method?

db.rel.find('post', 1).then(function (post) {
  return db.rel.del('post', post.posts[0]).catch(function (err) {
    if (err.code === 409) { // conflict
      // handle the conflict
    } else {
      throw err; // some other error
    }
  });
});

Object is deleted but still exist in database

Hello,
I tried to delete an object using this method:

delete(id){
    console.log("Id : " + id);
    this.db.rel.find('destination', id).then(doc => {
      return this.db.rel.del('destination', doc).then(data => {
        console.log(data);
      });
    });
  }

The result I get from the console:
{deleted: true}

But after I refresh the app the object appears again.

Implement getById method, or expose serialize function

I was trying to put together a convenience/wrapper method in my app that just returns the requested object -- to make the data structure easier to work with for editing and other situations I would prefer only having the necessary object, rather than an object with embedded array and relations ( i.e., { posts: [ { id: 1} ] } ). The docs actually refer to a non-existent method db.rel.get for cases where you need the doc revision to perform a delete.

Having access to the internal serialize function in some form would allow me to build my own method using a basic db.get (the serialize scheme is easy enough, but I would rather not reimplement it in my app code). It could also be wrapped up to be symmetrical with parseDocID like:

var id1 = 'post_1_abc123'
var id2 = db.rel.makeDocID(db.rel.parseDocID(id1));
id1 === id2 // true

I also noticed the Ember DS.Store class has a getById(type, id) method, so perhaps there are other ways you might want to implement functionality like this that do not involve exposing serialize.

I could do a PR for a makeDocID method, but being newbish with this module getById might be a bigger challenge right now.

Use Relational Pouch with requirejs

Hi,

I am new to PouchDB and requirejs and would like to use Relational Pouch with requirejs. Unfortunately I always get the error: 'Uncaught TypeError: undefined is not a function' as soon as I want to use a relational pouch function. Looks like relational pouch is not not accessible, although it is is being loaded.

Here is my code: https://github.com/Iomegan/Pouchpouchdb-plugin-requirejsD/blob/master/scripts/client.js (and a test project)

Does Relational Pouch even work with requirejs? PouchDB alone works fine with.

How to set async option for individual method calls?

In the docs, it's possible to set async to true or false in the setSchema method:

books: {hasMany: {type: 'book', options: {async: true}}}

But how can I set it to false for individual method calls like find?

For example:

// In some cases I want to use async: false
db.rel.find('books', {async: false});

// and in other cases I want to use async: true
db.rel.find('books', {async: true});

factory.db.rel.del & PouchDB Sync

Hello,
Thank you for your work !

I have an issue with factory.db.rel.del
When I delete with this function I guess it's normal that in the PouchDatabase Document Store
The document get a "deleted":true

And In the Sync CouchDB Database I can see the datas disapear.

But they stay in the local PouchDB with "deleted":true.

and Unfortunately, I cannot re-save the same objects
I have a ""Document update conflict", Error 409

Even if those docs were deleted before

db.rel.find doesn't have an option return attachments (only metadata)?

Hi guys,

I am trying to fetch all the items with its attachments using db.rel.find. I've tried to add an option but no luck.

db.rel.find('post', { attachments: true })

I know there's a way to get attachments by using db.rel.getAttachment(type, id, attachmentId). But this one only gets an attachment of a single item.

Many-to-many relationships pulling too many documents

First of all, I started testing pouchdb in an ionic 2 proof of concept app only a few days ago, so my knowledge is very limited (and I started to evaluate relational-pouch about 2 hours ago...)

I created the following schema (which, if I am not mistaken, should be a many-to-many relation):

           {
                "singular": "department",
                "plural": "departments",
                "relations": {
                    "checklists": { "hasMany": "checklist" }
                }
            }, {
                "singular": "checklist",
                "plural": "checklists",
                "relations": {
                    "departments": { "hasMany": "department" }
                }
            }

I created 10 'department' documents of the form { id: 1 ... 10, checklists: [1], ... }
I created 1 checklist: { Id: 1, departments: [1,2,3,4,5,6,7,8,9,10], ... }

When running the query db.rel.find('department', [1,3,5,7,9]), all departments (id: 1 to 10) are returned (the single checklist is also returned, which is expected).
This is not the behavior I expected. It seems that because the checklist points to all departments, the departments are pulled along even if they are not in the query list of ids. If I modify the checklist to { id: 1, departments": [1,3,5,7,9] } to match the query, then it "works" (meaning only departments with ids 1,3,5,7,9 are returned).

Not sure if I am doing something wrong, if it's a bug, or if it's by design. Is there a way to limit documents to what was requested in the list of ids?

Can't find an object if one of it's 'belongsTo' parents doesn't reference it anymore.

When a relation is broken, it's not even possible to find the child element anymore.
I agree that relations shouldn't get broken, but I'm not sure this is the best behaviour.
Perhaps only returning the object, without it's parent would be better.

describe('broken relation test', function() {

    var testPouch2;

    before(function() {

        var authorModel = {
            singular:'author',
            plural:'authors',
            relations:{
                books:{hasMany:'books'}
            }
        };
        var bookModel = {
            singular:'book',
            plural:'books',
            relations:{
                author:{belongsTo:'author'}
            }
        };
        var librarySchema = [authorModel, bookModel];
        testPouch2 = new PouchDb('https://user:[email protected]/library');
        testPouch2.setSchema(librarySchema);

        var john = {
            name:'John Doe',
            books: [
// Breaking the relation, 'ISBN12341234',
            'ISBN5759759'
            ],
            id:'jdoe'
        };
        var book1 = {
            title:'Great book',
            id:'ISBN5759759',
            author:'jdoe'
        };
        var book2 = {
            title:'Awesome book',
            id:'ISBN12341234',
            author:'jdoe'
        };

        return testPouch2.rel.save('book', book1)
        .then(function() {
            return testPouch2.rel.save('book', book2);
        })
        .then(function() {
            return testPouch2.rel.save('author', john);
        });             

    });

    it('should return at least the book that we\'re looking for', function() {
        return testPouch2.rel.find('book', 'ISBN12341234')
        .then(function(books) {         
            expect(books['books'][0].id).to.equal('ISBN12341234');
// This fails because only the author and the book it references are returned.
        });
    });
});

Just curious what you're opinion is on this behaviour.
I'll try submitting a PR if I find the time to dive into the rel#find method's code.

Unprefixed ids

This question is related to this.

I'm converting data between a pre-existing flat format and the nested data key which is used by relational pouch using transform-pouch which works great, but the issue I'm seeing is that relational-pouch expects all _ids to be prefixed with (in my case) note_2_.

None of the existing data follows this convention, they all just use standard uuids. Converting the id's with transform-pouch does not help here, presumably because relational-pouch relies on the prefix to lookup data. Is there any way around this which does not involve migrating the ids of all my existing data?

Perhaps I should be writing a PR to transform-pouch to add handlers which only apply to replication?

Can not query documents

Hey guys, how to fix this?

Input

db.rel.find('author')

output

(node:57648) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): TypeError: db.find is not a function
(node:57648) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code

Support to bulk operations

First of all, sorry for the inconvenience because I really dont know if this is an issue or an architecture decision not to offer bulk operations.

I am currently working with a large amount of documents and I found useful to keep the bulkDocs method supported by PouchDB. Why not exposing bulk save and bulk delete methods as described below?

  function _bulkSave(type, objs) {
    var typeInfo = getTypeInfo(type);
    var pouchDocs;
    return Promise.resolve().then(function () {
      pouchDocs = objs.map(function (obj) {
        return toRawDoc(typeInfo, obj);
      });
      return db.bulkDocs(pouchDocs);
    }).then(function (pouchRes) {
      var res = {};
      res[typeInfo.plural] = objs.map(function (obj, i) {
        return extend(true, obj, {
          id: deserialize(pouchRes[i].id),
          rev: pouchRes[i].rev
        });
      });
      return res;
    });
  }

  function _bulkDel(type, objs) {
    var typeInfo = getTypeInfo(type);
    var pouchDocs;
    return Promise.resolve().then(function () {
      pouchDocs = objs.map(function (obj) {
        var pouchDoc = toRawDoc(typeInfo, obj);
        return {
          _id: pouchDoc._id,
          _rev: pouchDoc._rev,
          _deleted: true
        };
      });
      return db.bulkDocs(pouchDocs);
    }).then(function () {
      return objs.map(function () {
        return {deleted: true};
      });
    });
  }

  function bulkSave(type, objs) {
    return Promise.resolve().then(function () {
      return _bulkSave(type, objs);
    });
  }

  function bulkDel(type, objs) {
    return Promise.resolve().then(function () {
      return _bulkDel(type, objs);
    });
  }

  db.rel = {
    bulkSave: bulkSave,
    bulkDel: bulkDel
  };

Also, I did some changes to ember-pouch adapter so it could use these bulk operations and accelerate insertions and deletions for large amount of data.

Thanks

Explanation about docID

Hello,

It's more a question than an issue. I'd just like to understand what the meaning of the number between underscores when creating a DocID :

db.rel.makeDocID({ "type": "author", "id": 19 });
Returns:
"author_1_0000000000000019"

What the meaning of 1 ?

Thanks a lot.

Performance of loading a lot of results

First of all - thanks for this plugin. It may be really useful, especially for local DBs on mobile devices, where it allows to limit the stored data, compared to "one big object" approach.

I have a use case that requires some more logic - we retrieve all objects, to run a filter function on them and use only a subset. RelationalPouch queries all objects with all their relations and so on, so the resulting memory usage and execution time become too significant.

I have made a fork that tries to solve this issue, by allowing an array of constraints: https://github.com/bamlab/relational-pouch/tree/feature/where-and-join
The solution can be extended/improved but this is the first idea. It could be further extended to only retrieve selected relations, hence the branch name.

Could you give me some feedback? What do you think of the problem itself and my proposed solution?

How do I do sort on the relational data

Just curious what is the best way to support sorting ? ex. I have multiple comments on a blog, but when I find this blog, I'd like comments to be sorted by timestamp, how do I do this ? Thanks.

Consider async relationship fetching

We are using the ember-pouch adapter and have two model: Category and Product.
Did some testing with 34 categories and a total of 1168 products. Fetching and parsing the categories is very slow because it fetches all the products as well.
Category fetch: 4.380ms
Product fetch: 2352.893ms
Parse: 2458.889ms

I really only need to load products when entering a category/:category_id route. Is it possible to load products async?

If not maybe fetching the all products with "startKey-endKey" and removing when parsing would be faster than a "keys:Array[1168]" fetch.

Addition of a flag in the schema to define local types

It may be a bit convoluted but it would be nice to specify a type that is local only in the schema so that the schema can be shared and sync-ed partially.

My issue case: I would like to create a user model with attributes. I would to take advantage of pouch features but I don't want user attributes to be synchronised the usual way (instead I want to rely on pouch-auth to update the _users record).

Working in Ember, if that's a feature worth exploring, it will be great to have that available in Ember-pouch.

Let me know what your thoughts are before I dig into the potential implementation.

Thanks

How to upsert with relational Pouch ?

Hello,

here is what I tried, but unfortunately I encounter sometimes an infinite loop of 409 errors....
Example on angular with $q.

Can you help me to improve upserting with relational-pouch ?

          var deltaFunc = function (doc) {
                doc.counter = doc.counter || 0;
                doc.counter++;
                return doc;
            };

            //upsert when a db.rel.save catch a 409 error.
            var upsert = function(doc, type) {
                return $q.when(db.rel.find(type, doc.id))
                    .then(function (result) {
                        var firstProperty = Object.keys(result)[0];
                        var doc2 = result[firstProperty][0]; // I get the doc here.
                        console.warn(doc.rev, doc2.rev);
                        console.warn(doc, doc2);
                        if(doc.rev != doc2.rev){
                            doc.rev = doc2.rev;
                            doc = deltaFunc(doc);
                            return tryAndPut(doc, type);
                        }else{
                            console.log("SAME Revisions !! :( ");
                            delete doc.rev; // rev deletion for a fresh save in order to create a new REV.
                            doc = deltaFunc(doc);
                            return tryAndPut( doc, type, del);
                            //return $q.when(factory.localDB.rel.find(type, doc.id)); (invetigating)
                        }
                    })
                    .catch(function (err) {

                        if (err.status !== 404) { // some error other than "not found"
                            throw err;
                        }
                        console.error(err);
                        return {_id : doc.id}; // default doc
                    });
            };


            var tryAndPut = function(doc, type) {
                console.log("tryAndPut", doc);
                return $q.when(db.rel.save(type, doc))
                    .catch(function (err) {
                        if (err.status !== 409) { // some error other than "conflict"
                            console.error("some error other than conflict",err);
                            throw err;
                        }else{
                            console.error("tryAndPut",err);
                            return upsert(doc, type, del);
                        }
                    }).then(function(res){
                        console.log("SUCCESS RE-SAVE "+ type +" OK",  res );
                        return res;
                    });
            };

Modeling a rooted tree graph

Hi there,

I'm exploring some options for how I'm going to structure the related data in my PouchDB / CouchDB database, and I'm struggling to understand the limits of what relational-pouch can do in my use case. I suspect that this is way out of scope for what relational-pouch was built to handle, but I wanted to make sure.

I was unsure whether this heading alluded to what I'm trying to do or not.

Basically, let's assume my data model has 30-40 document types, any of which can have a parent_id that links them to exactly one parent document, of an arbitrary type, in a non-cyclic fashion. For any given document, if I follow the chain of parent_ids all the way to its end, I'll reach a common ancestor (the root).

If I query relational-pouch for a leaf document, can / will relational-pouch automatically, recursively, side-load all the ancestors in the response? Or will I need to do that at the application level?

How would I handle a situation where I need to delete a non-leaf node? If I don't save the relationship bi-directionally (since that would lead to parent nodes getting hammered with revisions), how would I go about re-linking all those descendants to the deleted document's parent?

Here's an example that might help illustrate the situation I'm grappling with:

// root node that everything points back to
{ _id: "country_1", type: "country"}

// children of root
{_id: "state_1", type: "state", parent_id: "country_1"}
{_id: "state_2": type: "state", parent_id: "country_1"}

// grandchildren of root
{_id: "city_1": type: "city", parent_id: "state_1"}
{_id: "city_2": type: "city", parent_id: "state_1"}
{_id: "city_3": type: "city", parent_id: "state_2"}
{_id: "city_4": type: "city", parent_id: "state_2"}

// assets of various types can belong to federal, state or municipal entities
// these would be either children, grandchildren or great-grandchildren of root
{_id: "asset_a_1", type: "asset_a", parent_id: "country_1"}
{_id: "asset_a_2", type: "asset_a", parent_id: "state_2"}
{_id: "asset_b_1", type: "asset_b", parent_id: "city_3"}
{_id: "asset_b_2", type: "asset_b", parent_id: "city_2"}
{_id: "asset_b_3", type: "asset_b", parent_id: "city_2"}
{_id: "asset_c_1", type: "asset_c", parent_id: "city_3"}
{_id: "asset_c_2", type: "asset_c", parent_id: "city_4"}
{_id: "asset_c_3", type: "asset_c", parent_id: "city_4"}

When I pull up asset_c_2, would relational_pouch resolve all the way back to country_1, or would it only resolve city_4?

If I wanted to delete state_2, this would orphan city_3, city_4, asset_a_2, asset_c_1, asset_c_2 and asset_c_3, since state_2 is their link to the root node. So, before deleting state_2, I would need to find all documents whose parent_id points to state_2 and link them to country_1. Can / does relational-pouch handle any of this logic, or would I need to do all of this myself?

My feeling is that, because the parent_id points to an arbitrary document type, relational-pouch probably can't handle this in the first place since you have to define belongsTo relationships by individual types in your schema. But I can't be the only person who's ever tried to model a graph in a *ouchDB. Is there a better-suited library I haven't heard of? Do I need to write my own?

P.S. Thanks for making ember-pouch work. It's saved me weeks of headaches :)

FindHasMany is limited to 25 (default) hits

When using ember-pouch (or relational-pouch), when following a hasMany link, it doesn't set an unlimited limit, so I only get the first 25 children

For instance when listing all managers, that have a managerFor relation to technicians (inverse via manager) results in a _find where manager = the id. However, since no limit is included, I only ever get the 25 first technicians for the manager.

The code in question is: https://github.com/pouchdb-community/relational-pouch/blob/master/lib/index.js#L425-L438

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.