balihoo / lru-cache-promise

lru-cache with promises and async fetching

lru-cache-promise

lru-cache with promises

This library wraps Isaac Z. Schlueter's lru-cache and adds a getAsync function.

Usage:

var cache = require("lru-cache-promise")();

var fetchFunction = function(key) {
  /* Do something async here to fetch the value */
};

cache.getAsync("some key", fetchFunction)
.then(function(value) { console.log(value) });

getAsync returns an A+ (specifically bluebird) promise.

• If the value has already been cached, the promise will resolve immediately
• If not, fetchFunction will be called with the requested key. fetchFunction must return either a concrete value or a promise.

getAsync ensures only one call to fetchFunction for a given key, allowing the calling code to be make many concurrent calls for the same key without negatively impacting the system from which the values are fetched.

lru-cache-promise's Issues

Memory leak on calls before CacheItem is resolved

Once cache-item is resolved, it holds onto its array of "resolvers". Let's say you attempted to access multiple items before the resolution. Any references the fetchFunction has are kept even after the item resolves, because the item still has references to the resolvers which have references to the fetchFunction which has references to its implicit values.

This can easily be seen with the following example:

const LRU = require('lru-cache-promise');
const uuid = require('uuid');

var cache = LRU({maxAge: 1000 * 60 * 60 * 2});

function get_message(message) {
    return message;
}


setInterval( () => {
    let key = uuid.v1();

    for(var i = 0; i < 5; i++) {
        var value = `Data value for key on iteration ${i} could have been ${uuid.v1()}`;

        cache.getAsync(key, (key) => {
            return get_message(value);
        })
            .then((cache_item) => {
                console.log(`Fetched: ${cache_item}`);
            });
    }

}, 10000);

One would expect the CacheItem to resolve to just the first value- and it does- but you'll find it also keeps references to the other generated strings.

• Cache item "real" value -> Data value for key on iteration 0 could have been <uuid>
• Still referenced in cache item -> Data value for key on iteration 1 could have been <uuid>
• Still referenced in cache item -> Data value for key on iteration 2 could have been <uuid>
• Still referenced in cache item -> Data value for key on iteration 3 could have been <uuid>
• Still referenced in cache item -> Data value for key on iteration 4 could have been <uuid>

cache-item should resolve initial fetched value in the same fashion as the others

As it stands, the first promise returned by "getAsync" will resolve last. This is because every resolved item is being pushed into an array of resolvables except the last one, which is simply returned. Unfortunately, this results in the odd behavior where the first "getAsync" request is resolved after the other resolvables iterated upon.

The first call should likely behave as the others, going into the resolution array.

This would mean cache-item.fetch might look something like the following:

    fetch: ->
        Promise.try =>
            if @_status is status.fetched or not @fetchFunction
                # Return the current value
                return @value

    # Add a promise to the list of promises awaiting fetch completion
    p = new Promise (resolve, reject) =>
        @resolvers.push resolve
        @rejectors.push reject

    # Call the fetch function
    if @_status is not status.fetching
        Promise.resolve @fetchFunction @key
        .then (value) =>
            @value = value
            @_status = status.fetched

            r value for r in @resolvers
                value

        .catch (err) =>
            @_status = status.fetchFailed
            r err for r in @rejectors

            throw err
    
    @_status = status.fetching

    return p

As you can probably tell, I don't really know coffeescript. But hopefully that code is relatively clear. The main thing here is to be consistent with the resolve functions, and not to treat the first one differently.

EDIT: As an addendum to this, after some experimentation, Promise.each would need to be used for the iteration rather than just a for loop. Again, not sure how to do this in coffeescript.