dbaarda / dlfucache Goto Github PK

This requires good feedback on the cache performance that indicates if a higher or lower T value would perform better.

It's complicated partly because it's nearly impossible to know what the optimal performance of the cache should be, and the signals available when we only keep the decaying hit-count for each entry doesn't give a lot of signal. The optimal hit-rate depends on the load

One possible measure of the cache performance is "fit", where we keep a decaying average of the hit-count values for cache hits. If the current cache hits reflect the current distribution of entry counts, then the average hit-count should be;

Pentry = entry_count/count_sum
Centry = entry_count * Pentry = entry_count^2 / count_sum
Cmean = sum(entry_count^2) / count_sum = count_sum2 / count_sum
fit = count_avg / Cmean 

Where;

Pentry is the probability of hitting a particular entry 'e'.
Centry is the expected count-sum contribution from hits on a single entry.
Cmean is the expected average hit-count.
fit is the ratio of the average hit-count to the expected hit-count.

in practice this does seem to give a fairly good indication of when the cache is working well, but it doesn't tell you if T should be increased or decreased when it is not working well.