Enhance cache with BP-Wrapper optimizations about ristretto

[kk3399]

i am thinking to get the batching part work on top of this one https://github.com/golang/groupcache/tree/master/lru and compare before, after hit ratio and ops/sec

@mangalaman93 i see that this one has hit ratio https://github.com/dgryski/trifles/tree/master/cachetest
does this provide ops/sec https://github.com/dgraph-io/benchmarks/blob/master/cachebench/cache_bench_test.go ?

here is what the plan looks like -

• buffered lossy (ring buffer) channel for reads per client - https://content.pivotal.io/blog/a-channel-based-ring-buffer-in-go
• buffered non lossy channel for writes initially one per client, but may add more upto max if the channel is full
• dynamically merge and remove from "merged channels" identifying channel close - https://blog.golang.org/pipelines
• when finally applying changes, write channel gets priority over reads or the perceived order needs to be preserved?

[kk3399]

@mangalaman93 true 😞

looks like the golang CI is running on top of master branch, should it be running on the PR branch instead? #11

also i signed CLA after creating the PR, should i abandon the PR and create it again?

[ben-manes]

Read buffer
https://github.com/ben-manes/caffeine/blob/master/caffeine/src/main/java/com/github/benmanes/caffeine/cache/BoundedBuffer.java

Striping:
https://github.com/ben-manes/caffeine/blob/master/caffeine/src/main/java/com/github/benmanes/caffeine/cache/StripedBuffer.java

Write Buffer
https://github.com/ben-manes/caffeine/blob/master/caffeine/src/main/java/com/github/benmanes/caffeine/cache/MpscGrowableArrayQueue.java

Schedule after read
https://github.com/ben-manes/caffeine/blob/91eec88be93f16549ce82dd2f17959fd8eba75dd/caffeine/src/main/java/com/github/benmanes/caffeine/cache/BoundedLocalCache.java#L1106

Schedule after write
https://github.com/ben-manes/caffeine/blob/91eec88be93f16549ce82dd2f17959fd8eba75dd/caffeine/src/main/java/com/github/benmanes/caffeine/cache/BoundedLocalCache.java#L1310

[ben-manes]

buffered non lossy channel for writes initially one per client, but may add more upto max if the channel is full

I used a single write buffer because the throughput is limited by the hash table operation, since its synchronized per entry. I didn't observe the write buffer as a contention point.

when finally applying changes, write channel gets priority over reads or the perceived order needs to be preserved?

I processed reads before writes to update the policy prior to removal. The replay order probably doesn't matter, especially when later moving off LRU so it is less predictable in tests.

[mangalaman93]

@kk3399 when you add the base LRU to repo, I will move the benchmarks, both performance and hit ratio benchmarks, to this repo too.

[kk3399]

@mangalaman93 looks like neither branch creation not commits on master are allowed?

[mangalaman93]

I think you can just fork the repo and work in your own fork and then, create PRs. That'd be the simplest approach.