GithubHelp home page GithubHelp logo

sequelize-transparent-cache / sequelize-transparent-cache Goto Github PK

View Code? Open in Web Editor NEW
183.0 183.0 33.0 912 KB

Simple to use and universal cache layer for Sequelize

License: Creative Commons Attribution 4.0 International

JavaScript 100.00%
cache caching caching-library memcached nodejs redis sequelize

sequelize-transparent-cache's Introduction

sequelize-transparent-cache

Build Status Coverage Status JavaScript Style Guide Code Climate npm version Dependency Status

Simple to use and universal cache layer for Sequelize.

  • Abstract: does not depends on underlying database, or cache specific
  • Transparent: objects returned from cache are regular Sequelize instances with all your methods
  • Explicit: all calls to cache comes through cache() method
  • Lightweight: zero additional dependencies

Installation

Install sequelize-transparent-cache itself:

npm install --save sequelize-transparent-cache

Find and install appropriate adaptor for your cache system, see "Available adaptors" section below. In this example we will use ioredis

npm install --save sequelize-transparent-cache-ioredis

Example usage

const Redis = require('ioredis')
const redis = new Redis()

const RedisAdaptor = require('sequelize-transparent-cache-ioredis')
const redisAdaptor = new RedisAdaptor({
  client: redis,
  namespace: 'model',
  lifetime: 60 * 60
})

const sequelizeCache = require('sequelize-transparent-cache')
const { withCache } = sequelizeCache(redisAdaptor)

const Sequelize = require('sequelize')
const sequelize = new Sequelize('database', 'user', 'password', {
  dialect: 'mysql',
  host: 'localhost',
  port: 3306
})

// Register and wrap your models:
// withCache() will add cache() methods to all models and instances in sequelize v4
const User = withCache(sequelize.import('./models/user'))

await sequelize.sync()

// Cache result of arbitrary query - requires cache key
await User.cache('active-users').findAll({
  where: {
    status: 'ACTIVE'
  }
})

// Create user in db and in cache
await User.cache().create({
  id: 1,
  name: 'Daniel'
})

// Load user from cache
const user = await User.cache().findByPk(1);

// Update in db and cache
await user.cache().update({
  name: 'Vikki'
})

Look for all examples applications in examples folder.

Methods

Object returned by cache() call contains wrappers for limited subset of sequelize model or instance methods.

Instance:

Model:

In addition, both objects will contain client() method to get cache adaptor.

Available adaptors

You can easy write your own adaptor. Each adaptor must implement 3 methods:

  • get(path: Array<string>): Promise<object>
  • set(path: Array<string>, value: object): Promise<void>
  • del(path: Array<string>): Promise<void>

Checkout existed adaptors for reference implementation.

sequelize-transparent-cache's People

Contributors

c10h22 avatar danielhreben avatar greenkeeper[bot] avatar ignacioblit avatar jsarafajr avatar p-fedyukovich avatar shanderlam avatar siddhantj19 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

sequelize-transparent-cache's Issues

Add support for find and findAll methods

Hello,

Myself and I believe many more users of this library would benefit of having the possibility of transparently caching more sophisticated queries, such as the result of a find, findOne and findAll methods.

I do understand that some of the core logic within this library relies in using cache keys composed of the model name and an id. This may be the reason why supporting this methods could be kinda cumbersome.

Yet I wanna propose something that may make this work without having to change this core logic on how caching keys are composed.

What I propose is that we use a hash composed of the options argument of the find methods as the id argument and keep the model name as is.

It would be something like this:

find (options) {
    const id = hash(options)
    return cache.get(client, model, id)
        .then(instance => {
          if (instance) {
            return instance
          }
          return model.find.apply(model, arguments)
              .then(instance => cache.save(client, instance, id))
        })
    }

cache.save could check if id is set, and override the default behaviour when this happens.

For the hash function we could use something like this:

https://www.npmjs.com/package/object-hash

Or simply stringify the JSON.

@DanielHreben let me know if you would approve such change and functionality. I don't mind tackling this issue myself as I would put it to good use, yet I wanna be sure you will merge it before staring.

Question: Why allow reads out of cache?

Been looking into the niche world of "open source projects that tie together redis abstractions over mysql abstractions." I'm using redis and sequelize now, although changing to ioredis wouldn't be a big deal. Came across your library and like that it uses composition and has writes baked in.

In order to use yours, I'd need to submit a PR to support static destroy, but that's not too bad (probably win/win). I can delete a user, and thus would need to delete all individual entries of a "entity" for a user.

So I was wondering why user.cache().update was done rather than hijacking user.update in some way. My concern is that it would be too easy for new devs on a project to accidentally write user.update instead of user.cache().update and let that slip through.

Support for Sequelize 5

Hi Team,

I'm looking to confirm if this library support Sequelize 5 as I'm keen to add it to our project. If not, any thought on what is needed to make it compatible? Happy to take a stab at the conversion.

Thanks!

Not working with freezeTableName

Getting error when using findByPk:

error: Error: MemberOrganization has not been defined
    at Sequelize.model (/Users/austin/Dev/RupieNetwork/node_modules/sequelize/lib/sequelize.js:371:13)
    at Object.keys.forEach (/Users/austin/Dev/RupieNetwork/node_modules/sequelize-transparent-cache/lib/cache/util.js:46:27)
    at Array.forEach (<anonymous>)
    at loadAssociations (/Users/austin/Dev/RupieNetwork/node_modules/sequelize-transparent-cache/lib/cache/util.js:35:35)
    at dataToInstance (/Users/austin/Dev/RupieNetwork/node_modules/sequelize-transparent-cache/lib/cache/util.js:12:15)
    at client.get.then.data (/Users/austin/Dev/RupieNetwork/node_modules/sequelize-transparent-cache/lib/cache/index.js:60:12)
    at propagateAslWrapper (/Users/austin/Dev/RupieNetwork/node_modules/async-listener/index.js:504:23)
    at /Users/austin/Dev/RupieNetwork/node_modules/async-listener/glue.js:188:31
    at /Users/austin/Dev/RupieNetwork/node_modules/async-listener/index.js:541:70
    at /Users/austin/Dev/RupieNetwork/node_modules/async-listener/glue.js:188:31

Traced it down to this block in utils line 38:

  if (model.associations[key].hasOwnProperty('options')) {
      const modelName = model.associations[key].options.name.singular
      associations.push({
        model: model.sequelize.model(modelName),
        as: key
      })
    }

My model looks like this:

const OrganizationxRolexProject = sequelizeConfig.define(
  'OrganizationxRolexProject',
  {
    RoleID: Sequelize.INTEGER
  }
);

and the association looks like this:

   Project.belongsToMany(models.Organization, {
    as: 'MemberOrganizations',
    through: models.OrganizationxRolexProject,
    foreignKey: 'ProjectID',
    otherKey: 'OrganizationID'
  });

TL;DR; model.associations[key].options.name.singular isn't the right value for the actual model, and only provides the as association alias, so model.sequelize.model(modelName) throws.

error with lerna bootstrap

While trying npm i in the root of the repo. I'm getting following error. Any possible fix?

> undefined postinstall /home/siddhant/Academics/Projects/Gsoc/sequelize-transparent-cache
> lerna bootstrap

lerna notice cli v3.20.2
lerna info Bootstrapping 5 packages
lerna info Installing external dependencies
lerna ERR! yarn install --mutex network:42424 --non-interactive exited 1 in 'sequelize-transparent-cache-ioredis'
lerna ERR! yarn install --mutex network:42424 --non-interactive stdout:
yarn install v1.19.0
[1/4] Resolving packages...
[2/4] Fetching packages...
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

lerna ERR! yarn install --mutex network:42424 --non-interactive stderr:
warning Waiting for the other yarn instance to finish (5614)
error Incorrect integrity when fetching from the cache

lerna ERR! yarn install --mutex network:42424 --non-interactive exited 1 in 'sequelize-transparent-cache-ioredis'
lerna WARN complete Waiting for 2 child processes to exit. CTRL-C to exit immediately.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! undefined postinstall: `lerna bootstrap`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the undefined postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/siddhant/.npm/_logs/2020-03-17T15_20_34_284Z-debug.log


Includes are not being reinstated when converting a cache object to an instance

I'm noticing that when you save an object with association, it is not correctly reinstated on the instance.

Model

export default (sequelize, DataTypes) => {
  const Account = sequelize.define(
    'Account',
    {
      id: {
        type: DataTypes.UUID,
        defaultValue: DataTypes.UUIDV4,
        primaryKey: true
      },
      photo: { type: DataTypes.STRING(255) },
      firstName: {
        type: DataTypes.STRING(255)
      },
      lastName: {
        type: DataTypes.STRING(255)
      },
      email: {
        type: DataTypes.STRING(255),
        unique: true,
        required: true,
      },
    },
    {
      tableName: 'Accounts',
      timestamps: true,
    }
  );

  Account.prototype.toJSON = function toJSON() {
    const values = { ...this.get() };
    delete values.password;

    if (values.sites) {
      values.sites = values.sites.map(site => {
        const { siteId, siteKey } = site.toJSON();
        return {
          siteId,
          siteKey
        };
      });
    } else {
      values.sites = [];
    }
    return values;
  };

  Account.associate = models => {
    Account.hasMany(models.AccountSite, {
      foreignKey: 'accountId',
      constraints: true,
      onDelete: 'cascade',
      as: 'sites'
    });
  };

  return Account;
};

sequelize-transparent-cache

function dataToInstance(model, data) {
  if (!data) {
    return data;
  }
  console.log(data); // includes associated content
  const instance = model.build(data, { isNewRecord: false });
  console.log(instance.toJSON()); // Doesn't include associated content
  if (data.updatedAt) {
    instance.setDataValue('updatedAt', data.updatedAt);
  }

Another variant on this project

Hey, I lost track of this project a long while ago and based my own implementation on the original code base. I have moved to Graphql and my version has a number of features geared towards dealing with the "dataloader" concept that is common in that field. I'm wondering if there is any way I can contribute?

My own version has some additions that I don't see here:

  • The ability to get a cached association on an instance
  • The ability to clear cache keys based on a regular expression
  • The ability to hash a where statement

What I implemented assumes basically:

  • people retrieve lists and individual objects
  • lists have pagination and other where specific aspects
  • you need to be able to clean the cache in a smart way. Update object x, and all the related list elements need to be cleaned as well.

In order to achieve this for lists, I basically define a cache key, e.g. list. and add any where statement as an add on hash, e.g. list%some_hash%. And when I update an individual row, in addition to cleaning the item, I clean list using a regular expression. I also offer a debug output to help in the debugging.

It makes the cache powerful but also leads to some fragility because you can get unexpected issues if you don't know what you're doing. But it would be nice to attempt to abstract this logic to a level that anyone could use it confidently.

Current release appears to be broken at node <10

I just installed the package and, upon starting, I am getting:

0|dev    | 2019-11-28 20:54 +00:00: /home/andrew/server/node_modules/sequelize-transparent-cache/src/cache/util.js:39
0|dev    | 2019-11-28 20:54 +00:00:       } catch { // TODO: Fix issue with JSON and BLOB columns
0|dev    | 2019-11-28 20:54 +00:00:               ^
0|dev    | 2019-11-28 20:54 +00:00: 
0|dev    | 2019-11-28 20:54 +00:00: SyntaxError: Unexpected token {
0|dev    | 2019-11-28 20:54 +00:00:     at createScript (vm.js:80:10)
0|dev    | 2019-11-28 20:54 +00:00:     at Object.runInThisContext (vm.js:139:10)
0|dev    | 2019-11-28 20:54 +00:00:     at Module._compile (module.js:617:28)
0|dev    | 2019-11-28 20:54 +00:00:     at Module._extensions..js (module.js:664:10)
0|dev    | 2019-11-28 20:54 +00:00:     at Object.require.extensions.(anonymous function) [as .js] (/usr/lib/node_modules/babel-cli/node_modules/babel-register/lib/node.js:152:7)
0|dev    | 2019-11-28 20:54 +00:00:     at Module.load (module.js:566:32)
0|dev    | 2019-11-28 20:54 +00:00:     at tryModuleLoad (module.js:506:12)
0|dev    | 2019-11-28 20:54 +00:00:     at Function.Module._load (module.js:498:3)
0|dev    | 2019-11-28 20:54 +00:00:     at Function._load (/usr/lib/node_modules/pm2/node_modules/@pm2/io/src/metrics/httpMetrics.ts:190:35)
0|dev    | 2019-11-28 20:54 +00:00:     at Module.require (module.js:597:17)
0|dev    | 2019-11-28 20:54 +00:00:     at require (internal/module.js:11:18)
0|dev    | 2019-11-28 20:54 +00:00:     at Object.<anonymous> (/home/andrew/server/node_modules/sequelize-transparent-cache/src/cache/index.js:1:44)
0|dev    | 2019-11-28 20:54 +00:00:     at Module._compile (module.js:653:30)
0|dev    | 2019-11-28 20:54 +00:00:     at Module._extensions..js (module.js:664:10)
0|dev    | 2019-11-28 20:54 +00:00:     at Object.require.extensions.(anonymous function) [as .js] (/usr/lib/node_modules/babel-cli/node_modules/babel-register/lib/node.js:152:7)
0|dev    | 2019-11-28 20:54 +00:00:     at Module.load (module.js:566:32)
0|dev    | 2019-11-28 20:54 +00:00:     at tryModuleLoad (module.js:506:12)
0|dev    | 2019-11-28 20:54 +00:00:     at Function.Module._load (module.js:498:3)
0|dev    | 2019-11-28 20:54 +00:00:     at Function._load (/usr/lib/node_modules/pm2/node_modules/@pm2/io/src/metrics/httpMetrics.ts:190:35)
0|dev    | 2019-11-28 20:54 +00:00:     at Module.require (module.js:597:17)
0|dev    | 2019-11-28 20:54 +00:00:     at require (internal/module.js:11:18)
0|dev    | 2019-11-28 20:54 +00:00:     at Object.<anonymous> (/home/andrew/server/node_modules/sequelize-transparent-cache/src/methods/class.js:1:15)

Typo?

Maintainers needed

Hi all! I'm not actively using Sequelize anymore, but there is clearly a high demand for the functionality provided by this project.
I'm looking for 1-3 members who will be keen to take ownership of this package. Let me know here if you are interested.

Cache logging

It would be extremely useful to have an ability to monitor which values are saved, retrieved, updated and deleted when a developer works with a cache.

Something like that:

import { withCache } from 'sequelize-transparent-cache';

const cacheOptions = { 
  logger: true
 };

const SomeSequelizeModel = sequelize.define(...);
const CachedModel = withСache(SomeSequelizeModel, cacheOptions);

And after that, each cache related action will be logged to a console or any other place.

What nice to have in logs:
1 - Timestamp in order to compute approximate ttl.
2 - Cache hits / misses.
3 - When a new cache entry was created / deleted / updated.

Cache include query not possible ?

I have a query with (INNER JOIN) include models, and when i get data from cache, i get only first level of depth (Model) of my query. Why ?


Model.cache('cache-key').findAll({
...
                    include: [{
		        model: Model2,
....
		        include: [{
			      model: Model3,
			      ....
			    }]
		    }]

Nested includes only returns one level down from the cache

Currently, I'm using sequelize-transparent-cache, ioredis and redis to run the following...

const include = [{
    model: Parent,
    include: [{ model: Child }]
}];
Model.cache(cacheKey).findAll({ include: include });

Please excuse the brevity of the above example.
The first query returns the nested object, however, the "hit" from the cache only returns the parent include. Thanks in advance for your help on this ;-)

Unable to cache the data in Redis using cache().create()

I'm getting an error when trying to cache the data in Redis using the model.cache().create(data) function.

This is the error that I'm getting:-

error: node_redis: The SET command contains a invalid argument type.
Only strings, dates and buffers are accepted. Please update your code to use valid argument types.. cause: node_redis: The SET command contains a invalid argument type.
Only strings, dates and buffers are accepted. Please update your code to use valid argument types. method=POST, url=/api/users, command=SET, args=[model:users:52797, {"uuid":"a63e54841a19406a84fa1dcd963b3a17","status":"inactive","email_verified":false,"phone_number_verified":false,"id":52797,"username":"test11x1xxx2xe","email":"[email protected]","first_name":"test","last_name":"test","updated_at":"2020-08-12T07:56:59.618Z","created_at":"2020-08-12T07:56:59.618Z"}, 0=EX, 1=3600]

The error says that the Redis SET command is being passed invalid argument

Please, let me know what am I missing in the code implementation, or is there any issue with the library version?

The library version I'm using:-
sequelize - 4.33.4
sequelize-transparent-cache - 2.3.0
sequelize-transparent-cache-ioredis - 2.3.0
redis - 3.0.2
ioredis - 4.17.3

findAndCountAll not working

TypeError: Channel.cache(...).findAndCountAll is not a function

list(req, res) {
    const { page, limit, offset, order } = getPagination(req.query);
    // const Model = Channel.cacher.model(`Channel`).ttl(60);
    return Channel
      .cache(`list-channels-${qs.escape(req.query.q)}-${qs.escape(page)}-${qs.escape(limit)}-${qs.escape(offset)}-${qs.escape(order)}`)
      .findAndCountAll({
        order: [
          ['name', order],
        ],
        limit,
        offset,
        distinct: true,
      })
      .then((data) => {
        const response = getPagingData(data, page, limit);
        res.send(response);
      })
      .catch((err) => { 
        res.status(500).send({
          message:
            err.message || "Some error occurred while retrieving channels."
        });
       });
  },

License info is missing

We're considering add sequelize-transparent-cache adaptor for Hazelcast IMDG, but one thing seems to be weird. The repo seems to be missing license information. Is it on purpose? If not, could you tell me if that's a permissive license and add LICENSE file into the repo?

Unable to cache instance if model's schema does not have id field

Hello,

I noticed that cache's keys are formed from an array of model name and instance id :
const key = [ getInstanceModel(instance).name, instance.id ]
If we already have an id field in our model like :
const user = await User.cache().create({ id: 1, name: 'Daniel' })
it works as expected as in our cache we will have an entry with key "models:user:1" (supposing our namespace is models)

but imagine this case:

const user = await User.cache().create({ email: '[email protected]', name: 'Daniel' })

this leads to an entry in cache with key "models:user:" as lib expect to have field "id"

Typescript types support ?

Hi there,
Is it planned to add typescript definitions ?
to .cache() for example ?

I would definitely love to create a MR to implements all that
But I have no idea where to start from to extends an actual Sequelize Model
to add a cache function.

Manually clearing the cache for a model

I was wondering if there is a way to clear the cache for a specific model, or with a certain path prefix?
We have models that are queried and cached, but can be updated en-masse, using the class update, increment, decrement, and Sequelize query methods. As such, the values in the cache will need to be cleared on these occurrences. Is there a way to do this through this library, or will I need to go directly to redis and do it manually?

TypeError: instance.get is not a function (when using "raw: true" with findAll)

query:
const result = Model1.cache('key').findAll({ raw: true })

Stacktrace:

middleware TypeError: instance.get is not a function
middleware at instanceToData (/home/saiban/techclan/admin-portal/node_modules/sequelize-transparent-cache/src/cache/util.js:2:19)
middleware at Array.map ()
middleware at Object.saveAll (/home/saiban/techclan/admin-portal/node_modules/sequelize-transparent-cache/src/cache/index.js:35:36)
middleware at /home/saiban/techclan/admin-portal/node_modules/sequelize-transparent-cache/src/methods/class.js:53:38
middleware at processImmediate (internal/timers.js:456:21)
middleware From previous event:
middleware at /home/saiban/techclan/admin-portal/node_modules/sequelize-transparent-cache/src/methods/class.js:53:14
middleware at runMicrotasks ()
middleware at processTicksAndRejections (internal/process/task_queues.js:97:5) +0ms
middleware Error Caught in Async Middleware TypeError: instance.get is not a function
middleware at instanceToData (/home/saiban/techclan/admin-portal/node_modules/sequelize-transparent-cache/src/cache/util.js:2:19)
middleware at Array.map ()
middleware at Object.saveAll (/home/saiban/techclan/admin-portal/node_modules/sequelize-transparent-cache/src/cache/index.js:35:36)
middleware at /home/saiban/techclan/admin-portal/node_modules/sequelize-transparent-cache/src/methods/class.js:53:38
middleware at processImmediate (internal/timers.js:456:21)
middleware From previous event:
middleware at /home/saiban/techclan/admin-portal/node_modules/sequelize-transparent-cache/src/methods/class.js:53:14
middleware at runMicrotasks ()
middleware at processTicksAndRejections (internal/process/task_queues.js:97:5) +7ms

Sequelize v4.41.0 deprecation of findById

This sequelize release deprecated the findById method and I thought you should know, and maybe update the API to include the new findByPk method to the cache object.

Literal expression doesn't work

For example

await Comment.cache(`count-page`).findAll({
		    attributes: [
		    	[Sequelize.literal('COUNT(Comment.id)'), 'count'],
		    	[Sequelize.literal('AVG(Comment.vote)'), 'avg']
		    ],
		    where: {
		    	'approved' : true
		    }
		})

Wrong caching key when a primary attribute of a model is Serial

There is a bug with creating a cache key when any primary attribute is Serial.

Prerequisites:

  1. Your PG server should be up and running.
  2. All dependencies are installed.
    npm i pg (sometimes needed for Sequelize)
    npm i sequelize
    npm i sequelize-transparent-cache-variable
    npm i sequelize-transparent-cache
const Sequelize = require('sequelize')
const VariableAdaptor = require('sequelize-transparent-cache-variable')
const sequelizeCache = require('sequelize-transparent-cache')

const variableAdaptor = new VariableAdaptor({})
const { withCache } = sequelizeCache(variableAdaptor)

// Feel free to pass your own credentials
const sequelize = new Sequelize('postgres', 'postgres', '', {
  host: 'localhost',
  dialect: 'postgres',
  operatorsAliases: false,

  pool: {
    max: 5,
    min: 0,
    acquire: 30000,
    idle: 10000
  }
})

const Documents = withCache(sequelize.define(
  'Documents',
  {
    name: {
      type: Sequelize.TEXT,
      notNull: true
    }
  }
))

async function go () {
  try {
    await sequelize.authenticate()
    await sequelize.sync()

    console.log('Connection has been established successfully.')
  } catch (err) {
    console.error('Unable to connect to the database:', err)
  }

  const payload = { name: 'Doc #1' }

  const builtModel = await Documents.build(payload) // builtModel.id ---> null
  const created = await Documents.cache().insertOrUpdate(payload) 

  console.log(created.id === builtModel.id) // both ids are null

  process.exit(0)
}

go()

Please, see the following line. As you can see model.build is being used on the line #27, it returns built model without an id (id = null).

Next, go to this line and put the console.log there and print variable key. Once you run the code snippet provided above you will see something like that:
[ 'Documents', null ]

This issue could be especially dangerous when you have a complex primary key (serial field + varchar field for example)

createdAt,updatedAt not caching

I cached response from db into redis. but when I tried to get the cached data, then createdAt, updatedAt not returning. any help?

The dataToInstance method is very performance-consuming

After querying a large number of records through findAll, getAll hits the cache. At this time, dataArray.map(data => dataToInstance(model, data)) will be executed. This process is considered to be costly. The response time of 100+ data without cache is 0.0x seconds, After being cached, the response time is >0.9 seconds

Sequelize col not working

when i cache model the [col('category.id'),'id'], and [col('category.name'),'name'], is empty in response json

exports.getAllUsers= async function(req,res){
   await db.User.cache('aa').findAll({
        attributes:[
          'id',
          'name',
          'lname',
        ],
        include: [
          {
           model: db.SubModel1,
           as: 'posts',  
           attributes:[          
           [col('category.id'),'id'],
           [col('category.name'),'name'],
           ],
           separate:true,
           include:[{model:db.SubModel2,as:'submodel2',attributes:[]}]
          }
        ]
    }).then(async u=>{
      if(u){
      res.json(u);
      }else{
          res.json('Error');
      }
    }).catch(e=>{
      console.log(e);
      Logger.error(e,req);
      res.json({error:'!!Error'}); 
    });
}

generateRecursiveInclude function doesn't work with models for different schema

I personally don't feel the library should be caching includes as that would mean the primary key is returning more than the object that it represents. If you need to cache the entire database response then you likely need a custom implementation anyways. It's likely possible for this to work correctly, but you're going to continue to battle every error people have for their custom implementations and includes.

Either keep it simple or support every option that sequelize provides. My solution was to remove it entirely.

diff --git a/node_modules/sequelize-transparent-cache/src/cache/util.js b/node_modules/sequelize-transparent-cache/src/cache/util.js
index ce2cdf5..c8e50fe 100644
--- a/node_modules/sequelize-transparent-cache/src/cache/util.js
+++ b/node_modules/sequelize-transparent-cache/src/cache/util.js
@@ -6,8 +6,7 @@ function dataToInstance (model, data) {
   if (!data) {
     return data
   }
-  const include = generateIncludeRecurse(model)
-  const instance = model.build(data, { isNewRecord: false, raw: false, include })
+  const instance = model.build(data, { isNewRecord: false, raw: false })
   restoreTimestamps(data, instance)
   return instance
 }
@@ -51,25 +50,6 @@ function restoreTimestamps (data, instance) {
   })
 }
 
-function generateIncludeRecurse (model, depth = 1) {
-  if (depth > 5) {
-    return []
-  }
-  return Object.entries(model.associations || [])
-    .filter(([as, association]) => {
-      const hasOptions = Object.prototype.hasOwnProperty.call(association, 'options')
-      return hasOptions
-    })
-    .map(([as, association]) => {
-      const associatedModel = model.sequelize.model(association.target.name)
-      return {
-        model: associatedModel,
-        include: generateIncludeRecurse(associatedModel, depth + 1),
-        as
-      }
-    })
-}
-
 module.exports = {
   instanceToData,
   dataToInstance

Included models not returning with timestamps

I'm currently using sequelize-transparent-cache and redis to retrieve data using the following call

let quiz = await Quiz.cache().findByPk(quizId, { include: [ 'nodes', 'results' ], where: { categoryId: categoryIds } });

On first call (grabbing direct from the DB) this is fine. However, on subsequent calls (from the cache), the included nodes or results do not have any of the timestamps. Our JSON schema serializer (fast-json-stringify) then complains, because the schema requires the createdAt and updatedAt keys.

The data inside redis does infact have the timestamps on the included associations, but it seems it's not being returned back to the instances

Setting TTL for every query

Hello,
is it possible or do you plan add options to better working with TTL? For example

const user = await User.cache({ttl: 60}).findByPk(1);

Because I don't want using same TTL for all models. What you recommend?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.