encryptio / go-meetup Goto Github PK

entry values can have their memory reused once they are not in use by any Get and are not in the tree (have been evicted or expired.)

This is needed for fully linearizable caches; the idea being that you can cache data, but do a less expensive validation step at each get to make sure the data is up to date. Once you know that, you can either re-get it from the backend or return it as-is.

This would be intended for use by a background cleaner goroutine (managed outside of go-meetup itself) to reduce memory usage for caches that are not near their size limits.

For example, this code:

c := meetup.New(meetup.Options{
    ExpireAge: time.Minute,
})

var i int64
for i := int64(0); ; i++ {
    c.Get(strconv.Itoa(i))
    time.Sleep(time.Second)
}

Will use an unbounded amount of memory.

If we know we're going to block on readiness (!e.Ready) and we call c.startFill(key, e), we should instead directly call fill to avoid the overhead of starting a goroutine.

Extension of #1

The concurrency limiter chooses the next request to run psuedorandomly, due to the use of a channel write to implement concurrency limiting.

This behavior can stall individual reads for an arbitrary amount of time, adding a large variation in latency for requests.

Instead of writing to a full channel, something with fairer semantics should be implemented; for example, a deque.

Currently, eviction in (*Cache).fill iterates over all entries and removes ones that have not been recently used (since the last eviction pass hit those pairs.)

If fill checked for expired values as it iterated and removed those, regardless of whether they had been recently used or not, then it would need the same or fewer iterations to free space for values, only remove values which cannot be used anyway, and thus make better use of the cache space.