Keyv v5 - Features and Release Plans 🎉 about keyv

[jaredwray]

Hi All - Keyv Hooks are now up and running and we will be releasing v5.0.0-alpha.0 on NPM this weekend for testing. Just FYI.

[jaredwray]

@hugo082 - thanks and yes we are highly considering to move to a more stable model instead of using require. Will update as we get closer to alpha versions in the next couple weeks.

[jaredwray]

@gperdomor - thanks for your suggestion and I will plan to look at moving @keyv/redis to the standard driver as that does make sense. Working on getting v5 out and then will look at that.

[hugo082]

Hi,

Thanks for the update!

With the current version, there is an import design issue that breaks the compatibility with bundlers (used by Next.js even for backend environments). please refer to #45 for more details.

To summarize, here is the part of the code that is responsible of the compatibility break. We are using require(adapters[adapter]) which is a require with expression and produces the following warning: the request of a dependency is an expression.

Is it possible to upgrade the v5 to be compatible with bundlers and remove this part of the code? It will produce a breaking change similar to:

import KeyvMemcache from '@keyv/memcache';
import Keyv from 'keyv';

// don't work anymore
const keyv = new Keyv('memcache://user:pass@localhost:11211');

// works
const keyv = new Keyv({ store: new KeyvMemcache('memcache://user:pass@localhost:11211') });

// proposals
const keyv = new Keyv(new KeyvMemcache('memcache://user:pass@localhost:11211'));
const keyv = new Keyv(KeyvMemcache, 'memcache://user:pass@localhost:11211');

[lukechilds]

Awesome progress, looking forward to seeing v5!

[jaredwray]

Awesome progress, looking forward to seeing v5!

@lukechilds - thanks so much one thing we will be discussing is if we want to go with the proposal as the require() functionality is going to cause issues with other libraries. Thoughts?

[jaredwray]

@hugo082 - updated with proposal

[jaredwray]

Added in information around hooks and statistics.

[lukechilds]

@lukechilds - thanks so much one thing we will be discussing is if we want to go with the proposal as the require() functionality is going to cause issues with other libraries. Thoughts?

Yeah if you wanna support bundling / frontend then I think probs best to remove dynamic require. I do think inferring deps via URI is pretty slick API design if it's possible to keep though. Potentially there could be some workaround that keeps the current behaviour without doing a dynamic require. Maybe some kind of shim that statically requires each dep on a best effort basis like:

storage-adaptors.js

try {
  module.exports.sqlite = require('sqlite')
} catch {}

try {
  module.exports.redis = require('redis')
} catch {}

then you could require the shim at runtime without error, check which storage adaptor you want to use, and return an error if it's undefined.

Potentially that could still cause issues with bundlers if they can't find redis at bundle time or whatever I'm not sure. Maybe defining them as peer deps could help? Could be worth investigating if you want to keep the URI API though.

[lukechilds]

But yeah if it's gonna be too much of a headache or isn't possible I think only accepting a storage adapter instance makes sense like:

const keyv = new Keyv({ store: storageAdaptor })

[hugo082]

Potentially that could still cause issues with bundlers if they can't find redis at bundle time or whatever I'm not sure. Maybe defining them as peer deps could help? Could be worth investigating if you want to keep the URI API though.

The main issue I see with the try / catch approach is that the bundle cannot statically check so will try to bundle all dependencies (redis + sqlite).
There is maybe a solution to keep the same API by using TS Template literal types (which was introduced in typescript 4.1). I'm not sure it works but it may looks like

type SQLiteURI = `sqlite://${string}`;
type RedisURI = `redis://${string}`;
type URI = SQLiteURI | RedisURI;

const isRedisUri = (uri: URI): uri is RedisURI => uri.startsWith('redis://');
const isSQLiteUri = (uri: URI): uri is SQLiteURI => uri.startsWith('sqlite://');

const uri = 'redis://...'; // <= as to be a literal type, not only string

const clientFromUri = async (uri: URI) => {
  if (isRedisUri(uri)) {
    return await import('sqlite')
  } else if (isSQLiteUri(uri)) {
    return await import('redis')
  }

  assertType<never>(uri);
};

But it will still create a breaking change as the uri property as to be a literal type URI, not a string

[jaredwray]

Hi All - I wanted to update you on the progress leading up to v5 release. We have now split the repo into two branches:

• main - this is now the v5 branch and we will be adding to it there. We do plan to put a link to the v4 branch.
• v4 - this is now the v4 branch and the goal is to maintain this through 2024.

The current branch is now completely in typescript and also has updated types. Our next step is to work on publishing the v5 alpha version for people to start testing and then roll out the rest of the features hooks, stats, browser compatibility, compression fixes, offline / tiering mode (multi storage adapters), and namespace enhancements.

[gperdomor]

Hi all... I just discover this package and looks awesome... I have on doubt, any particular reason why you use ioredis instead official redis package?

Maybe the V5 can introduce a new store with the official package?

[gperdomor]

@jaredwray or maybe rename the current redis implementation to @keyv/ioredis in v5, in that way a new package @keyv/redis with official redis package can be created, I can help if you want :D

[gperdomor]

Hi again @jaredwray, it's me again... I want to know if there is any update regarding to the V5 release or if existe any beta version available?

[gperdomor]

seems like ioredis eventually will be deprecated according to this comment, so now it's more important a node-redid adapter 🙏🏻 redis/ioredis#1870 (comment)

[jaredwray]

seems like ioredis eventually will be deprecated according to this comment, so now it's more important a node-redid adapter 🙏🏻 redis/ioredis#1870 (comment)

Correct and we will be moving to the redis driver

[jaredwray]

Hi All! Wanted to update this thread as https://www.npmjs.com/package/keyv/v/5.0.0-rc.1 is now published as next and you can download it via npm install keyv@next to get the latest. Please note, you will also want to use the latest storage adapters if using typescript as it is compatible with v5. We will be publishing them over the next 24 hours and will tag the storage adapters with next. 🎉

This includes ESM and CJS support (CJS will get phased out most likely with Keyv v6), hooks, statistics, and moving to async/await with typescript and the following breaking change:

URI and Proposed Breaking Change 🔥

After feedback from the community and looking into this more (thanks @hugo082) we have decided to longer support the uri on the constructor. This would be a major breaking change 😬 but we believe everything else will be worth it.

import KeyvMemcache from '@keyv/memcache';
import Keyv from 'keyv';

// don't work anymore
const keyv = new Keyv('memcache://user:pass@localhost:11211');

// works
const keyv = new Keyv({ store: new KeyvMemcache('memcache://user:pass@localhost:11211') });

// proposals moving forward. 
const keyv = new Keyv(new KeyvMemcache('memcache://user:pass@localhost:11211'));
const keyv = new Keyv(KeyvMemcache, 'memcache://user:pass@localhost:11211');
const keyv = new Keyv({ store: new KeyvMemcache('memcache://user:pass@localhost:11211') });

Finally, After more consideration we are going to get v5 live and past RC (we are planning that to be in June after testing) the plan moving forward is to do the final feature sets since they are non-breaking.

I will be closing this thread down and adding issues for the following features when it makes sense to track:

• Storage Tiering - we are considering not adding this in because it seems to really be a feature that we want to support in caching systems such as cache-manager and cacheable long term but would love your feedback 👍 or 👎
• Namespacing - this is currently being planned out and should be non breaking change but will be later.
• Simplified Compression Adapters - this will be added as an enhancement as time permits this year.