[Question] over time can TinyLFU make it more difficult to add cache entries? about ristretto

[karlmcguire]

It sounded to me that if the value calculated by new entries is higher than the lowest entry in a random sample, the new entry gets admitted. Otherwise not.

Correct.

won't the number of discarded items go up over time? Imagine if there are only reads for a minute or 10minutes, the frequency of most items increase and they gain a higher value than any new incoming entries.

TinyLFU prevents this with two "freshness" mechanisms:

• Counter Resets: every N increments all counters are halved. Because a counter increment occurs for every Read, in your example of only reads for an extended period of time this mechanism would work as fast as possible (compared to a mixed workload).
• 4-bit Counters: not only are these great for memory usage but due to the small range of values these actually perform better in hit ratio benchmarks due to the opportunities it provides for new items to be admitted (@ben-manes has ran these).

I'm not confident in how the value is calculated, so I apologies if this is way off. Also curious if you have metrics for discarded (new items that was never added) metrics?

No problem, this is a common question, and the freshness mechanisms are a small but important aspect of Ristretto.

We do have metrics for new items that are rejected here.

[karlmcguire]

Here's an example of checking the rejectSets metric:

// create cache with Metrics on
cache, err := NewCache(&Config{
    NumCounters: 1e7,  // 10M
    MaxCost:     1 << 30,  // 1GB
    BufferItems: 64,
    Metrics:     true,
})
if err != nil {
    panic(err)
}

// do your testing
//
//

// get the number of rejected items
rejected := cache.metrics().Get(rejectSets)

Currently we're not publicly exposing the metrics. I don't see a reason not to, so we'll probably add those to the public API.

[ben-manes]

Also an admission window can be used to delay the decision, allowing a new entry the chance to build up its frequency. That avoids consecutive misses in recency traces. The amount of space allocated to the window can be dynamically tuned based on the workload.

[karlmcguire]

The concise answer to this question is: freshness mechanisms (addressed above) and windowing (addressed in #46). I'll close this for now.