dfahlander / dexie-relationships

Dexie relationship plugin

Dexie.js relationship plugin

Dexie.js is a wrapper library for indexedDB - the standard database in the browser.

Dexie relationship plugin provides an API to ease the loading of relational data from foreign tables

Installation

npm:

npm install dexie-relationships --save

bower:

bower install dexie-relationships --save

API Example

Schema

Note the use of -> which sets the foreign keys.

import Dexie from 'dexie'
import relationships from 'dexie-relationships'

var db = new Dexie('MusicBands', {addons: [relationships]})

db.version(1).stores({
    genres: 'id, name',
    bands: 'id, name, genreId -> genres.id',
    albums: 'id, name, bandId -> bands.id, year'
});

Seed the data

db.transaction('rw', db.bands, db.albums, db.genres, () => {
    // Genres
    db.genres.bulkPut([{
        id: 1,
        name: "Rock"
    },{
        id: 2,
        name: "Schlager"
    }])

    // Bands
    db.bands.bulkPut([{
        id: 1,
        name: 'Beatles',
        genreId: 1
    },{
        id: 2,
        name: 'Abba',
        genreId: 2
    }])
    
    // Albums
    db.albums.bulkPut([{
        id: 1,
        name: 'Abbey Road',
        year: 1969,
        bandId: 1
    }, {
        id: 2,
        name: 'Let It Be',
        year: 1970,
        bandId: 1
    }, {
        id: 3,
        name: 'Super Trouper',
        bandId: 2,
        year: 1980
    }, {
        id: 4,
        name: 'Waterloo',
        bandId: 2,
        year: 1974
    }])
})

Usage

db.bands
  .where('name').startsWithAnyOf('A', 'B') // can be replaced with your custom query
  .with({albums: 'albums', genre: 'genreId'}) // makes referred items included
  .then(bands => {
      // Let's print the result:
      bands.forEach (band => {
          console.log (`Band Name: ${band.name}`)
          console.log (`Genre: ${band.genre.name}`)
          console.log (`Albums: ${JSON.stringify(band.albums, null, 4)}`)
      });
})

NOTE: The properties that are set onto the result ('albums' and 'genre' in this case) will not be visible when callilng JSON.stringify(band), because they are marked as non-enumerable. The reason for this is to prevent the properties to be redundantly stored back to the database if calling db.bands.put(band).

Result

Band Name: Abba
Genre: Schlager
Albums: [
    {
        "id": 3,
        "name": "Super Trouper",
        "bandId": 2,
        "year": 1980
    },
    {
        "id": 4,
        "name": "Waterloo",
        "bandId": 2,
        "year": 1974
    }
]
Band Name: Beatles
Genre: Rock
Albums: [
    {
        "id": 1,
        "name": "Abbey Road",
        "year": 1969,
        "bandId": 1
    },
    {
        "id": 2,
        "name": "Let It Be",
        "year": 1970,
        "bandId": 1
    }
]

dexie-relationships's Issues

Cannot find module 'dexie-relationships'

Sorry for this, I am new in javascript.

I have a project with Angular 2 generated with angular-cli latest version. and "typescript": "~2.0.0"

I install:

npm install dexie-relationships --save

and I import:

import relationships from 'dexie-relationships'

When I build with ng serve

ERROR in C:/frontend/src/app/database.ts (2,27): Cannot find module 'dexie-relationships'.)

Need help getting related items.

I am just getting into indexedDB and am having some growing pains. The project is an Agular 8 project so my code snippets will be typescript. I will try to post a striped down version of the code needed to resolve this issue so you don't have to read all the extra junk not related.

The issue

I have an order table that needs to included items from my parts table that have a column orderId.
I am using the function getById to get the order.

My db class

import { Injectable } from '@angular/core';
import Dexie from 'dexie';
import relationships from 'dexie-relationships';
import { OrderPart } from '../models/order-part.model';
import { Order } from '../models/order.model';

@Injectable()
export class AppDbService extends Dexie {

  orderParts: Dexie.Table<OrderPart, number>;
  orders: Dexie.Table<Order, number>;

  constructor() {
    super('MyApp', { addons: [relationships] });

    var db = this;

    db.version(1).stores({
      orderParts: '&id, orderId -> orders.id',
      orders: '&id'
    });

    this.orderParts = this.table('orderParts');
    this.orders = this.table('orders');

    db.orderParts.mapToClass(OrderPart);
    db.orders.mapToClass(Order);
  }

  async getById(id: number): Promise<T> {
    return await this.orders.where('id').equals(id).with({ parts: 'orderParts' }).then(orders => {
      return orders[0];
    });
  }
}

the Order class

export class Order {
  constructor(id?: number) {
    this.id = id;
  }

  public id: number;
  public parts: OrderPart[] = [];
}

the OrderPart class

export class OrderPart {
  constructor(id?: number, partId?: number, quantity?: number, price?: number, orderId?: number) {
    this.id = id;
    this.partId = partId;
    this.quantity = quantity;
    this.price = price;
    this.orderId = orderId;;
  }

  public id: number;
  public partId: number;
  public quantity: number;
  public price: number;
  public orderId: number;
}

oneToMany with no matches should get empty array instead of undefined

Basically, I would like to assert that when expecting an "albums" property, it should always be there. If no items matches, it should be empty instead of not defined at all:

db.bands.add({
        id: 3,
        name: 'Band without albums',
        genreId: 1
})

const results = await db.bands.with({albums: 'albums'});

// Here's the assertion I would like to have:
assert (results.every(band => band.albums && Array.isArray(band.albums)));

Should improve build process

Should try buble + rollup instead of babel + webpack.
An alternative would be keeping babel but replace webpack with webpack2-beta.

If we do this we will have better outputs with smaller footprints in the dist folder, such as:

• dexie-relationships.js
• dexie-relationships.min.js
• dexie-relationships.es.js

We would also support stuff like watch without having to develop a build system like I had to in Dexie.

Error when including related table that does not have rows corresponding to the foreign key

Given the schema

db.version(1).stores({
    bands: '++id, name',
    albums: '++id, bandId -> bands.id, name'
})

If I add a band, but no corresponding album:

await db.bands.put({
    name: "My Band"
})

and then query the DB for bands, including albums by relation;

const bands = await db.bands.with({albums: 'albums'})

I´d expect to get one band with an empty array of albums, but instead I get the error

Promise.js:836 Unhandled rejection: Error: Could not lookup foreign key where albums.bandId == band.albums. The content of the failing key was: 1.

Is this the expected behaviour? Seems a bit odd that you cannot have 0 entities in a one to many relation.

Property 'with' does not exist in Typescript

I rewrite official example in https://stackblitz.com/edit/typescript-hfut4q.

How to get sub relations

I am trying to get the an order with a list of orderParts and each order part ref the part it is derived from as the user need to be able to customize the part details for an order. when I run the below function getById I get the order with the list of orderParts but the part is not included on each orderPart.

Is there a way to get sub relations?

My db class

import { Injectable } from '@angular/core';
import Dexie from 'dexie';
import relationships from 'dexie-relationships';
import { OrderPart } from '../models/order-part.model';
import { Order } from '../models/order.model';

@Injectable()
export class AppDbService extends Dexie {

  orderParts: Dexie.Table<OrderPart, number>;
  orders: Dexie.Table<Order, number>;
  parts: Dexie.Table<Part, number>;

  constructor() {
    super('MyApp', { addons: [relationships] });

    var db = this;

    db.version(1).stores({
      orderParts: '&id, orderId -> orders.id, partId -> parts.id',
      orders: '&id',
      parts: '&id'
    });

    this.orderParts = this.table('orderParts');
    this.orders = this.table('orders');
    this.parts = this.table('parts');

    db.orderParts.mapToClass(OrderPart);
    db.orders.mapToClass(Order);
    db.parts.mapToClass(Part);
  }

  async getById(id: number): Promise<T> {
    return await this.orders.where('id').equals(id).with({ parts: 'orderParts' }).then(orders => {
      return orders[0];
    });
  }
}

the Order class

export class Order {
  constructor(id?: number) {
    this.id = id;
  }

  public id: number;
  public parts: OrderPart[] = [];
}

the OrderPart class

export class OrderPart {
  constructor(id?: number, partId?: number, quantity?: number, price?: number, orderId?: number) {
    this.id = id;
    this.partId = partId;
    this.quantity = quantity;
    this.price = price;
    this.orderId = orderId;;
  }

  public id: number;
  public partId: number;
  public quantity: number;
  public price: number;
  public orderId: number;
}

the Part class

export class Part {
  constructor(id?: number, partNumber?: string, description?: string, price?: number) {
    this.id = id;
    this.partNumber = partNumber;
    this.description = description;
    this.price = price;
  }

  public id: number;
  public partNumber: string;
  public description: string;
  public price: number;
}

Does not work with Webpack

I'm using typescript with webpack, and when instantiating Dexie, the export that's used is relationships.default.

import Dexie from 'dexie';
import relationships from 'dexie-relationships'

export class AppDatabase extends Dexie {
    constructor(){
        super('AppDatabase', {addons: [relationships]});
    }
}

After compilation and packing, {addons: [relationships]} turns into what is essentially {addons: [relationships.default]}.

Is this an error with the module, or an export?

When foreign key is null, we get an error

Could not lookup foreign key where ... The content of the failing key was: null.

In my opinion the relation property should just be null there.

Nested relationships

Hi just wondering if you can help?

My schema looks like this:

db.version(6).stores({
      route: 'id,name',
      journey: 'id,date,route -> route.id',
      fare: 'id',
      stop: 'id,name,route -> route.id',
      time_set: 'id,stop -> stop.id',
      departure: 'id,journey -> journey.id,time_set -> time_set.id',
    });

And I am trying to make a query like this:

this._dexieService.departure
              .filter((departure): boolean => {
                return departure.journey === journey_id;
              })
              .with({
                journey: 'journey',
                time_set: 'time_set',
                time_set__stop: 'stop',
              });

But that last line in the with is obviously not working (One of a number of different syntaxes I've tried). Essentially I want to have an object inside an object inside an object, to represented the 3-level nested relationship between departures->time_sets->stops (and then also ->routes)

Just wondering if it is at all possible to have this kind of output?

dexie-relationships do not work with require in electron

dexie-relationships do not work with require in electron, but dexie alone does

var Dexie = require('dexie')
var relationships = require('dexie-relationships')
var db = new Dexie('MyProject', {addons: [relationships]})

Line 3 is in error.
Here is the error comming in the node_modules/dexie/dist/dexie.js file :

dexie.js works well alone, so I suggest it may come from the webpack formatting ?

Thanks for help

Can't use orderBy or sortBy as well?

I'm having trouble figuring out how to use sortBy() or orderBy() with with():

This works fine:

db.tables.where('someIdx').equals(someIdx)
	.with({other: 'otherId'})
	.then(function(results) {
		...
	});

Adding sortBy fails:

db.tables.where('someIdx').equals(someIdx).sortBy('date')
	.with({other: 'otherId'})
	.then(function(results) {
		...
	});

Currently, my only workaround is to manually sort the results manually, which seems incorrect.

I understand why this occurs, because sortBy() returns an array (rather than staying as a collection). Unfortunately, dexie-relationships also converts the data to an array and makes the whole thing impossible to use...

It would be better if the data could be kept as a collection. (imho sortBy() suffers the same issue)

No bower support since version 1.2.4

Now, since build outputs have been removed from the repo (to make it easier to collaborate and merge from different branches), bower package wont work anymore.

People being dependent on bower, please comment on this. Might use the same workaround as I do in Dexie - to create a release script that automatically checks in build output onto a separate branch and uses that branch as the github release = bower release.

If no one is using bower, we should just delete bower.json from this repo.

Many to many?

Is many to many possible with this? What the syntax would be?

'Dexie.override is not a function' when deriving Dexie

index.js refers to Dexie as db.constructor, which is not true when db is an instance of a subclass to Dexie. We need to change that and import Dexie from 'dexie' instead.

Relationships based on Object

Hi, currently creating relations requires explicit ID in parent object (if I understand corretly based on docs/tutorials). There will be possibility to create relations based stricte on object? E.g.

class FooA {
    id?: number;
    someParam: string;
    otherParam: FooB;
}

class FooB {
    id?: number;
    someOtherParam: string;
}

class MyDB extend Dexie {
    ...
    constructor() {
        ...
        this.version(1).stores(
            fooA: '++id, someParam, otherParam -> otherParam.id',
            fooB: '++id, someOtherParam'
        );
        ...
    }
}

Currently after try to save FooA object I get in indexedDB row with full data (json) of FooA and FooB, but I want to have only ID of FooB in FooA.

Relationships from within arrays?

I have a lot of Dexie tables that have an array with foreign keys. For example, a shopping bag with items:

{
    shoppingBagId: 1000,
    weight: 42,
    bagContents: [
        {
            itemId: 299,
            quantity: 1,
        },
        {
            itemId: 792,
            quantity: 1,
        },
        {
            itemId: 458,
            quantity: 1,
        },
    ]
}

This was all fine when I my application was always online and my relational backend did all the joining, but now that I'm moving to an offline mode, suddenly all the relational goodness has to be baked in into the IndexedDB key store. But I'm rambling.. so ideally I would make those itemId's point to my item table.

{
    shoppingBagId: 1000,
    weight: 42,
    bagContents: [
        {
            itemId: 299, -> points to the item table
            quantity: 1,
        },
        ...
    ]
}

Is this anywhere within the realm of possibilities? Also, it makes me wonder at which point the key store becomes a relational database again ;-).

Imcompatibility when using dexie-observable

Hi there,

I've noticed that when using dexie-observable and dexie-relationships together, an error is raised as soon as you call db.on('changes', (changes) => {}

Uncaught (in promise) TypeError: Cannot read property 'subscribe' of undefined.

To reproduce simply add this import in your file that initializes the Dexie isntance:
import 'dexie-observable';

Thanks!

Does not work for multiple indexes

Hello! Thank you very much for this helpful and useful addon and for the Dexie!
It would be even better, if it would working with multiple indexes.
For example "*groupsIds -> group.id"

May be I have some problem with architecture of my app's DB, but I need that for example one item could be a member of 2 or more groups. (user is admin and buyer at same time, or MacBook is notebook and PC and topseller at the same time).

getForeignKeys() method doesn't cut off prefixes of fields before searching by index. But may be the problem is some deeper in core...

Thanks

Problem with relations One-to-many

Hi,
When I have 2 bands in the same genre

    // Bands
    this.db.table('bands').bulkPut([{
        id: 1,
        name: 'Beatles',
        genreId: 1
    },{
        id: 2,
        name: 'Abba',
        genreId: 2
    },{
        id: 3,
        name: 'The Police',
        genreId: 1
    }])

Recovery only 1 object with same genre.

I found in the code and I discored that line # 182 attribute is foreignKey

For this reason 2 objects can not share the same foreignKey ( lookup[row[targetIndex]] = row; )

Console Log
lookup: Object {1: Object, 2: Object}

===============
return Promise.all(queryPromises).then(function (queryResults) {
foreignTableNames.forEach(function (tableName, idx) {
var foreignTable = usableForeignTables[tableName];
var result = queryResults[idx];
var targetIndex = foreignTable.targetIndex;
var foreignIndex = foreignTable.index;
var column = foreignTable.column;

      // Create a lookup by targetIndex (normally 'id')
      // and set the column onto the target
      var lookup = {};
      rows.forEach(function (row) {
        lookup[row[targetIndex]] = row;
      });
      console.log( 'lookup: ', lookup );

      // Populate column on each row
      result.forEach(function (record) {
        var foreignKey = record[foreignIndex];
        var row = lookup[foreignKey];
        console.log( record[foreignIndex] );

Wrong behavior where two foreign keys targeting the same table

Hi,

I tried to declare two foreign key targeting the same table, but I got the following issue :

stores = {
    'albums': 'id',
    'bands': 'id, firstAlbumId -> albums.id, lastAlbumId -> albums.id'
};

db.bands.with({
    firstAlbum: 'firstAlbumId',
    lastAlbum: 'lastAlbumId'
}).then(function (bands) {
    bands.forEach(function (band) {
        console.log('band : ' + band.name)
        console.log('first :' + band.fisrtAlbum);
        console.log('last : ' + band.lastAlbum);
        // in this case, fisrt album is always undefined
    });
});

db.bands.with({
    lastAlbum: 'lastAlbumId',
    firstAlbum: 'firstAlbumId'
}).then(function (bands) {
    bands.forEach(function (band) {
        console.log('band : ' + band.name)
        console.log('first :' + band.fisrtAlbum);
        console.log('last : ' + band.lastAlbum);
        // in this case, last album is always undefined
    });
});

The second relationship overwrite the first, because foreign tables are declared in an object.
Then, I fixed it locally by declaring usableForeignTables as an array of objects containing tableName as property in addition of existing properties : it works well.

You can see below the new "with" method for comparison (sorry, I only edited the dist version).

  /**
   * Iterate through all items and collect related records
   *
   * @param relationships
   *
   * @returns {Dexie.Promise}
   */
  db.Collection.prototype.with = function (relationships) {
    var this$1 = this;

    var baseTable = this._ctx.table.name;
    var databaseTables = db._allTables;

    // this holds tables that have foreign keys pointing at the current table
    var usableForeignTables = [];

    // validate target tables and add them into our usable tables object
    Object.keys(relationships).forEach(function (column) {
      var tableOrIndex = relationships[column];
      var matchingIndex = this$1._ctx.table.schema.idxByName[tableOrIndex];

      if (matchingIndex && matchingIndex.hasOwnProperty('foreignKey')) {
        var index = matchingIndex;
        usableForeignTables.push({
          column: column,
          index: index.foreignKey.targetIndex,
          tableName: index.foreignKey.targetTable,
          targetIndex: index.foreignKey.index,
          oneToOne: true
        });
      } else {
        var table = tableOrIndex;

        if (!databaseTables.hasOwnProperty(table)) {
          throw new Error('Relationship table ' + table + ' doesn\'t exist.');
        }

        if (!databaseTables[table].schema.hasOwnProperty('foreignKeys')) {
          throw new Error('Relationship table ' + table + ' doesn\'t have foreign keys set.');
        }

        // remove the foreign keys that don't link to the base table
        var columns = databaseTables[table].schema.foreignKeys.filter(function (column) { return column.targetTable === baseTable; });

        if (columns.length > 0) {
          usableForeignTables[table] = {
            column: column,
            index: columns[0].index,
            tableName: table,
            targetIndex: columns[0].targetIndex
          };
        }
      }
    });

    return this.toArray().then(function (rows) {
      //
      // Extract the mix of all related keys in all rows
      //
      var queries = usableForeignTables
        .map(function (foreignTable) {
          // For each foreign table, query all items that any row refers to
          var tableName = foreignTable.tableName;
          var allRelatedKeys = rows
            .map(function (row) { return row[foreignTable.targetIndex]; })
            .filter(isIndexableType);

          // Build the Collection to retrieve all related items
          return databaseTables[tableName]
              .where(foreignTable.index)
              .anyOf(allRelatedKeys);
        });

      // Execute queries in parallell
      var queryPromises = queries.map(function (query) { return query.toArray(); });

      //
      // Await all results and then try mapping each response
      // with its corresponding row and attach related items onto each row
      //
      return Promise.all(queryPromises).then(function (queryResults) {
        usableForeignTables.forEach(function (foreignTable, idx) {
          var tableName = foreignTable.tableName;
          var result = queryResults[idx];
          var targetIndex = foreignTable.targetIndex;
          var foreignIndex = foreignTable.index;
          var column = foreignTable.column;

          // Create a lookup by targetIndex (normally 'id')
          // and set the column onto the target
          var lookup = {};
          rows.forEach(function (row) {
            lookup[row[targetIndex]] = row;
          });

          // Populate column on each row
          result.forEach(function (record) {
            var foreignKey = record[foreignIndex];
            var row = lookup[foreignKey];
            if (!row) {
              throw new Error(
                "Could not lookup foreign key where " +
                tableName + "." + foreignIndex + " == " + baseTable + "." + column + ". " +
                "The content of the failing key was: " + (JSON.stringify(foreignKey)) + ".");
            }

            if (foreignTable.oneToOne || !row.hasOwnProperty(column)) {
              // Set it as a non-enumerable property so that the object can be safely put back
              // to indexeddb without storing relations redundantly (IndexedDB will only store "own non-
              // enumerable properties")
              Object.defineProperty(row, column, {
                value: foreignTable.oneToOne ? record : [record],
                enumerable: false,
                configurable: true,
                writable: true
              });
            } else if (!foreignTable.oneToOne) {
              row[column].push(record);
            }
          });
        });
      }).then(function () { return rows; });
    });
  };

By the way, thank you for this usefull plugin, I just started using it in my company and except this issue, it works well !

I hope it will be fixed soon in order to use it from npm :)

Loading relationships with `get()`?

The current use case (AFAICT) for with() is to use it where you would call .toArray().

However, I'd like to use it on .get() queries and in places where I don't want to invoke .toArray(). Implicitly invoking toArray() limits query building scenarios.

Rail's ActiveRecord handles this by "queueing up" query filters, then having an evaluate method (similar to toArray()). Is there a similar way to do this, so with() will work with get(), plus enable some more complex query builders such as the following use cases:

Use Case 1: Executing a scalar query

let song = await db.songs.with({ album: 'albumId' }).get(3);

Use Case 2: Composing a collection query

let findSongs = (albumId) => {
  let query = db.songs
      .where('albumId')
      .equals(albumId)
      .with({ album: 'albumId' });

  return query;
};

let page = (query, number, limit) => {
  return query.offset(number*limit).limit(limit);
};

let songs = await page(findSongs(3), 2, 10)).toArray();

Use Case 3: Composing queries that are "doubly" async

let intersect = async (table, queries) => {
  let results = await Promise.all(
    queries.map(q => q.primaryKeys()));

  let reduced = results
    .reduce((a, b) => {
      let set = new Set(b);
      return a.filter(k => set.has(k));
    });

  return table.where(':id').anyOf(reduced);
};

let query1 = db.songs
    .where('albumId')
    .equals(3)
    .with({ album: 'albumId' });

let query2 = db.songs
    .where('genreId')
    .equals(5)
    .with({ album: 'albumId' });

let aggregateQuery = await intersect(query1, query2);
let songs = await aggregateQuery.toArray();

If the answer is "yes" or @dfahlander has some pointers on how to do this, I'm 💯 to write a PR!

Does not work with TypeScript

I'm not sure if it's TypeScript's or the module's fault, but this is the error I'm getting when I try to import and use the module.

error TS2322: Type 'typeof "[PROJECT]/node_modules/dexie-relationships/dist/index"' is not assignable to type '(db: Dexie) => void'.
  Type 'typeof "[PROJECT]/node_modules/dexie-relationships/dist/index"' provides no match for the signature '(db: Dexie): void'.

I'm importing via import relationships from 'dexie-relationships'. I'm using the latest release (1.2.6) and the latest Dexie (2.0.0-beta.10). Any advice?

Support for Dexie v3

Hi, I would like to know what are the plans for support and maintenance of this repo.

I've seen that the last commit here was over 3 years back and latest supported Dexie version is v2 alpha looking at the peer dependencies in package.json, but the current Dexie version is v3 and v4 is already in alpha.

There are also unresolved issues related to old Dexie version such as this one which would be great to get fixed as well.

Could you please let us know if you are still willing to maintain this library and if it's possible to open PRs for this repo?
Or is it better to fork it and take over if you don't have any time for it?

Thanks!