blemale / scaffeine

Thin Scala wrapper for Caffeine (https://github.com/ben-manes/caffeine)

Scaffeine

A thin Scala wrapper for Caffeine (https://github.com/ben-manes/caffeine).

Browse the API docs for the latest release.

Motivations

Caffeine is an awesome Java caching library. It has an impressive performance and a neat Java 8 API.

However the API does not play very well with Scala. So this is the thinner wrapper we can came with to make Caffeine easy and idiomatic to use in Scala.

API

Cache

"Cache" should "be created from Scaffeine builder" in {
    import com.github.blemale.scaffeine.{ Cache, Scaffeine }
    import scala.concurrent.duration._

    val cache: Cache[Int, String] =
      Scaffeine()
        .recordStats()
        .expireAfterWrite(1.hour)
        .maximumSize(500)
        .build[Int, String]()

    cache.put(1, "foo")

    cache.getIfPresent(1) should be(Some("foo"))
    cache.getIfPresent(2) should be(None)
  }

LoadingCache

"LoadingCache" should "be created from Scaffeine builder" in {
    import com.github.blemale.scaffeine.{ LoadingCache, Scaffeine }
    import scala.concurrent.duration._

    val cache: LoadingCache[Int, String] =
      Scaffeine()
        .recordStats()
        .expireAfterWrite(1.hour)
        .maximumSize(500)
        .build((i: Int) => s"foo$i")

    cache.get(1) should be("foo1")
  }

AsyncLoadingCache

 "AsyncLoadingCache" should "be created from Scaffeine builder with synchronous loader" in {
    import com.github.blemale.scaffeine.{ AsyncLoadingCache, Scaffeine }
    import scala.concurrent.duration._

    val cache: AsyncLoadingCache[Int, String] =
      Scaffeine()
        .recordStats()
        .expireAfterWrite(1.hour)
        .maximumSize(500)
        .buildAsync((i: Int) => s"foo$i")

    whenReady(cache.get(1)) { value =>
      value should be("foo1")
    }
  }

"AsyncLoadingCache" should "be created from Scaffeine builder with asynchronous loader" in {
    import com.github.blemale.scaffeine.{ AsyncLoadingCache, Scaffeine }
    import scala.concurrent.duration._

    val cache: AsyncLoadingCache[Int, String] =
      Scaffeine()
        .recordStats()
        .expireAfterWrite(1.hour)
        .maximumSize(500)
        .buildAsyncFuture((i: Int) => Future.successful(s"foo$i"))

    whenReady(cache.get(1)) { value =>
      value should be("foo1")
    }
  }

Download

Download from Maven Central or depend via SBT:

"com.github.blemale" %% "scaffeine" % "<version>" % "compile"

Snapshots of the development version are available in Sonatype's snapshots repository.

scaffeine's Issues

Refactor sbt build

• Extract release process from the main build.sbt

AsyncLoadingCache .get does not wrap exceptions in Future

When using the AsyncLoadingCache and using .get()it calls underlying.get(key).toScala. The underlying .get() call can fail and throw an exception for instance when the update method can not connect to an external service to fetch the new data.

This results in needing to wrap each call to the AsyncLoadingCache .get() method in a Try to handle the possibility it will fail. As the return type is Future[T] we would expect no exceptions to be thrown from this code, but instead handle it as a failed Future.

Try(cache.get(item)).recover {
  case ex: Exception => Future.failed(ex)
 }.get

This is not ideal, and would be nice to have this behaviour built into Scaffeine itself.

scaffeine/src/main/scala/com/github/blemale/scaffeine/AsyncLoadingCache.scala

Line 41 in 0f3df2a

def get(key: K, mappingFunction: K => V)(implicit ec: ExecutionContext): Future[V] =

API Docs are empty

Link from the main page leads to:

Need ability to provide different loaders for the values

getIfPresent why mapping to Some instead of wrapping?

Hey,

i suppose wrapping the future in getIfPresent with Some(future) would be better than mapping the result in Some.

I have following use case:
I want to check if a key is present in the cache. The only way i found was using the method getIfPresent.
When matching the Future against None everything is fine, cause this will returned immediatley. But in case it exists, the code has to wait until the Future initialized to get the Some value.

In my proposed way this situation would be immediatley at hand.

Feel free to correct me if i am wrong with my assumption :)

EDIT:
forgot to mention that i am using AsyncLoadingCache

Upgrade sbt to version 1.x

At the time of writing the current version of sbt is 1.2.8

Can I use JSR-107 Annotations with Scaffeine?

Hi. Is it possible to use annotations like @CacheResult if I use Scaffeine? Can you give some examples on how to do it? Thanks.

Update for caffeine 2.6.1

Caffeine is now on version 2.6.1. Can this get updated to support it? Thanks!

EDIT: Just made a PR with these changes.

Support other effects beside `Future` for async caches

Hi !

Everything is in the title I think : would it be possible to add the possibility to support other effects types for async caches than Future ?
At work we are commited to the typelevel stack and macking scaffeine caches transition between Future and IO is very boilerplate-ish. I imagine it is the same with ZIO or Monix users.

This could be done either thru

• tagless final : not sure how this would work work the underlying Java API but more generic than
• effect system specific modules which may take some work to maintain

If you think this an accceptable evolution for the lib I'm ready to improve my FOSS karma by doing the PR ( for IO only since it's the only one I am familiar with ^^ )

Fix links to java classes in scaladoc

Links to Caffeine and JDK classes are broken in Scaladoc.

Compile to scala 2.12

http://www.scala-lang.org/news/2.12.0

Implement the async `asMap()` view from caffeine 2.7

Unexpected Scaffeine().buildAsyncFuture behavior

Hi,

I have a problem with async loading cache. From the doc, I see that If the asynchronous computation fails then the entry will be automatically removed. . This implies that it will also never be cached.
However I can still see the error being cached for some short period of time, see below.

import com.github.blemale.scaffeine.{AsyncLoadingCache, Scaffeine}

import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.DurationInt
import scala.concurrent.{Await, Future}
import scala.util.Random

object AsyncCacheProblem extends App {

  var firstCall = true
  val sleep     = 0L

  val cache: AsyncLoadingCache[String, Int] = {
    def log(str: String) = Future(println(str))
    Scaffeine()
      .expireAfterWrite(10.seconds)
      .buildAsyncFuture[String, Int] { str =>
        for {
          _    <- log("Generate random")
          bool <- Future(Random.nextBoolean())
          result <- if (bool || firstCall) {
                      firstCall = false
                      Future.failed(new RuntimeException("BOOM"))
                    } else log(s"Got $str").flatMap(_ => Future(str.size))
        } yield result
      }
  }

  def getSleep = Future(Thread.sleep(sleep)).flatMap(_ =>
    cache.get("foo").map(result => Right(result)).recover(ex => Left(ex))
  )

  def repeatFuture[U](count: Int)(fut: => Future[U]): Future[List[U]] =
    (1 to count).foldLeft(Future.successful[List[U]](Nil)) { (f, _) =>
      f.flatMap { x =>
        fut.map(_ :: x)
      }
    } map (_.reverse)

  val fut = repeatFuture(10)(getSleep)
  println(Await.result(fut, 20.seconds))

}

This code will print

Generate random
List(Left(java.lang.RuntimeException: BOOM), Left(java.lang.RuntimeException: BOOM), Left(java.lang.RuntimeException: BOOM), Left(java.lang.RuntimeException: BOOM), Left(java.lang.RuntimeException: BOOM), Left(java.lang.RuntimeException: BOOM), Left(java.lang.RuntimeException: BOOM), Left(java.lang.RuntimeException: BOOM), Left(java.lang.RuntimeException: BOOM), Left(java.lang.RuntimeException: BOOM))

The cache function is being called once, and the error is being cached. However if I increase the timeout between the subsequent calls, for example set val sleep = 100L it works fine, and it prints:

Generate random
Generate random
Generate random
Generate random
Got foo
List(Left(java.lang.RuntimeException: BOOM), Left(java.lang.RuntimeException: BOOM), Left(java.lang.RuntimeException: BOOM), Right(3), Right(3), Right(3), Right(3), Right(3), Right(3), Right(3))

At each call, the function is invoked again, generating a random value at each step. Eventually, it gets the value, caches it, and never runs the function again.

It is most likely a Caffeine issue looking into how thin the Scaffeine wrapper is, but I haven't reproduced it yet on pure Caffeine cache, will do it once I have some time. If I will be able to reproduce, I will a Caffeine issue as well, but I'm adding it here for visibility as well

What is the ExecutionContext used for?

From looking at the code it seems to be used only in the map inside allLoader. Is it used anywhere else?

Can be factory method typed?

Is possible to make typed api something like

Object Scaffeine
def apply[K,V]: Scaffeine[K, V] = ?

Caffeine 2.7

The latest release adds AsyncCache, split off from AsyncLoadingCache, to provide a manual version. This also includes an async asMap() view.

Redundant implicit ExecutionContext parameters in AsyncLoadingCache

AsyncLoadingCache.get(), getFuture(), and probably others do NOT need or use an ExecutionContext, yet they demand it as an implicit parameter, so now one has to provide/import some ExecutionContext at every calling site for no reason at all, which is nonfatal, but annoying.

I imagine you added this parameter because in other methods, e.g. getAll(), you used Future.map(), which actually demands an ExecutionContext, and then you just slapped it on everywhere else for good measure, which is understandable.

The unused parameters better be removed, but as some users may have already specified the parameter explicitly, if you don't want to break things for them, it might be better to specify a default value for the parameter instead, e.g. null.

But then, even map()s which actually make use of ExecutionContext are just an implementation detail of your library. Just a way you post-convert results that came from java Futures. The sanest thing to do here would be not to demand an ExecutionContext from user, but either of:

1. Use an ExecutionContext derived from the Executor that the user specified during cache construction with the Caffeine.executor() method.
2. Better still, execute your post-conversions in the same thread as the original computations, since they're logically a part of them.

For option 1 you can just obtain the user-defined executor from the cache and convert it with ExecutionContext.fromExecutor().

But I like option 2 the most. The user has already expressed their will with regard to which thread the computations should run on by specifying (or leaving the default) Executor during cache construction. Jumping through additional threads and rescheduling hoops just so that a toScala executes on a java collection is suboptimal, imo.

It can be implemented with a special executor. Scala's Futures actually use a class like that internally.

import java.util.concurrent.Executor
import scala.concurrent.ExecutionContext

class CurrentThreadExecutor extends Executor {
  override def execute(r : Runnable) {
    r.run()
  }
}

object CurrentThreadExecutor {
  implicit val instance = ExecutionContext.fromExecutor(new CurrentThreadExecutor)
}

All that is left is to supply the instance in all wrapper methods.

Whichever option you choose, the resulting EC would preferably go as default value for the implicit parameter, if you don't want to break existing users and keep the flexibility of allowing users to reschedule post-conversions to other threads, though I fail to see a use for it.

[BUG] Unbounded growth of the cache when writing in a tight loop (weight-based cache)

I was using the weight-based Caffeine cache and I noticed the bug where if we hit the cache very hard in a tight loop, the cache will grow unbounded since the eviction cannot keep up.

Please see the simple test code lists below

package concurrency

import com.github.benmanes.caffeine.cache.RemovalCause
import com.github.blemale.scaffeine.{Cache, Scaffeine}

import java.util.concurrent.atomic.AtomicInteger

sealed class MyCache {
  /** The weigh function for Caffeine cache. */
  private def weigh(key: Int, value: String) = {
    value.length
  }

  private val numEvictedEntries = new AtomicInteger()

  /** Record the number of entries being evicted. */
  private def evictionListener: (Int, String, RemovalCause) => Unit =
    (key: Int, value: String, cause: RemovalCause) => {
      numEvictedEntries.addAndGet(1)
    }

  /** Caffeine cache */
  private val cache: Cache[Int, String] =
    Scaffeine()
      .recordStats()
      .weigher(weigh)
      .maximumWeight(100)
      .evictionListener(evictionListener)
      .build()

  /** Get the number of evicted entries. */
  def getNumEvicted: Int = numEvictedEntries.intValue()

  def read(key: Int): Option[String] = cache.getIfPresent(key)

  def write(key: Int, value: String): Unit = cache.put(key, value)
}

object HitCache extends App {

  println("START")

  val myCache: MyCache = new MyCache

  for (index <- 0 until 10000) {
    val key = index
    val value = "A * 10" + index.toString

    myCache.write(key, value)

    // Test if the write succeeded
    val optionString = myCache.read(key)
    assert(optionString.isDefined) // BUG -- assertion failed
    assert(optionString.get == value)
  }

  println("OK")

}

ThisBuild / scalaVersion := "2.12.10"
...
libraryDependencies += "com.github.blemale" % "scaffeine_2.12" % "5.2.1"

The bug only affects the weight-based cache, it does not affect the size-based cache.

Java 8 support

The CacheLoader and AsyncCacheLoader fail on Java 8 projects. I see in the fork network, someone else ran into the same issue and this is the commit they added: BinaryMan32@badd0c3

However, I also see on Caffeine's releases page, that the new version 3 does not support Java 8 and to rather use version 2 for Java 8 support https://github.com/ben-manes/caffeine/releases/tag/v3.0.0

So potentially we need a version of Scaffeine that runs on the latest V3 and another version that runs on the latest V2 for Java 8.

Caffeine 3.0

FYI, this major release sets the baseline to JDK11 to allow us to remove sun.misc.Unsafe in favor of VarHandles. There were a few minor API improvements, such as to refresh. A rarely used feature, CacheWriter, was removed as a poor concept that is better served by Map computations and an new synchronous eviction listener. For most users this should be drop-in or a very minor update, so upgrading here shouldn't be much effort.

Cache refreshAfterWrite is not working ?

Hi,

I am trying to understand why my reload function is not called after the refreshAfterWrite duration so, I implemented this simple test:
`
import cats.implicits.catsSyntaxOptionId
import com.github.blemale.scaffeine.{AsyncLoadingCache, Scaffeine}
import com.typesafe.scalalogging.LazyLogging
import org.mockito.ArgumentMatchersSugar
import org.scalatest.concurrent.ScalaFutures.whenReady
import org.scalatest.matchers.must.Matchers
import org.scalatest.wordspec.AnyWordSpecLike

import java.util.concurrent.TimeUnit
import scala.concurrent.Future
import scala.concurrent.duration.FiniteDuration

class ScalaCaffeineSpec extends AnyWordSpecLike with Matchers with ArgumentMatchersSugar with LazyLogging {

val refreshAfter: FiniteDuration = FiniteDuration(100, TimeUnit.MILLISECONDS)

val asyncCache: AsyncLoadingCache[Int, Int] =
Scaffeine()
.maximumSize(1)
.refreshAfterWrite(refreshAfter)
.expireAfterAccess(FiniteDuration(1, TimeUnit.HOURS))
.recordStats()
.buildAsyncFuture(
loader = (: Int) => Future.successful(1),
reloadLoader = ((: Int, _: Int) => reload).some
)

private def reload: Future[Int] = Future.successful(2)

"Scala AsyncLoadingCache cache" must {
"load" in {
whenReady(asyncCache.get(1))(_ mustBe 1)
}

"reload at least once" in {
  Thread.sleep(refreshAfter.toMillis + 100)
  whenReady(asyncCache.get(1))(_ mustBe 2)
}

}

}
`

the reload fails, any idea ?

Expiration API feedback

I have the foundation in place to support variable expiration. I'm hoping to work on the integration and expose it over the next few weeks. It would be nice to hear your feedback. The implementation uses a TimerWheel, which is an O(1) alternative to a O(lg n) priority queue.

See the full API for JavaDoc. In short it is,

interface Expiry<K, V> {
  long expireAfterCreate(K key, V value, long currentTime);
  long expireAfterUpdate(K key, V value, long currentTime, long currentDuration);
  long expireAfterRead(K key, V value, long currentTime, long currentDuration);
}

The intent is to avoid allocations by using primitives, so it is slightly less friendly by using nanos instead of Duration objects. Its too bad value types are still a long ways off, so performance concerns leaks into the API. The currentDuration is available so that users can indicate no change and no expiration is modeled as returning a large duration (e.g. Long.MAX_VALUE).

How would you model this and make it idiomatic in Scala?

Bump Caffeine version to 2.3.0

ExecutionContext for AsyncCache

I have read issue 6 #6

I am wondering why AsyncLoadingCache.buildAsyncFuture() doesn't use a loader of type loader: (K1, scala.concurrent.ExecutionContext) => Future[V1].

AsyncCacheLoaderAdapter could convert the incoming java.util.concurrent.Executor to ExecutionContext.

The Future based loader I specify needs an ExecutionContext, and it should preferably come from the cache.

What do you think?

Should I just construct the cache with the same ExecutionContext I use for the loader function?

LoadingCache lambda returning null

Hello! kindly pardon me if this come across as a dumb question or lack of understanding of how the caching works. according to this, if the lambda for LoadingCache returns [null], but it seems that through Scaffeine, when i return the null value from the lambda, a NPE is thrown instead of not assigning value to they key.

 val cache: LoadingCache[String, T] = Scaffeine()
    .build[String, T]((key: String) => null) // throws NPE

is this by design? or am i missing something?

Switch builder methods from Duration to FiniteDuration

First of all great wrapper!

I was using some of the methods like refreshAfterWrite and expireAfter and for testing (and since it should expire for a specific case) added Duration.Inf as argument.

But this results always in the following error:

toNanos not allowed on infinite Durations
java.lang.IllegalArgumentException: toNanos not allowed on infinite Durations
	at scala.concurrent.duration.Duration$Infinite.fail(Duration.scala:217)
	at scala.concurrent.duration.Duration$Infinite.toNanos(Duration.scala:220)

Since it's not allowed, it might be better to make the API more specific and use FiniteDuration or allow it and set it then to Long.maxValue as suggested by the Caffeine project.

If you want i can make a PR if times allows it from my side :)

Ticker does not work with free Futures

Hi,

I am trying to implement a fakeTicker for testing purposes but somehow ticker works with constant Futures (like Future.successful) but does not work with ongoing Futures that use ExecutorContext.

Ticker: FakeTicker.scala

import com.github.benmanes.caffeine.cache.Ticker
import java.util.concurrent.atomic.AtomicLong
import scala.concurrent.duration.Duration

class FakeTicker extends Ticker {
  private val nanos                  = new AtomicLong()
  private val autoIncrementStepNanos = new AtomicLong()

  override def read(): Long =
    nanos.getAndAdd(autoIncrementStepNanos.get())

  def advance(duration: Duration): FakeTicker = {
    advance(duration.toNanos)
    this
  }

  def advance(nanoseconds: Long): FakeTicker = {
    nanos.addAndGet(nanoseconds)
    this
  }

  def setAutoIncrement(duration: Duration): Unit = {
    this.autoIncrementStepNanos.set(duration.toNanos)
  }
}

Working example:

import com.github.blemale.scaffeine.{ AsyncLoadingCache, Scaffeine }
import org.scalatest.freespec.AsyncFreeSpec

import scala.concurrent.Future
import scala.concurrent.duration._
import scala.util.Random

class CacheSpec extends AsyncFreeSpec {
  val fakeTicker = new FakeTicker

  case class Expired(ttl: Long, data: String)

  val cache: AsyncLoadingCache[String, Expired] = Scaffeine()
    .executor(scala.concurrent.ExecutionContext.global)
    .ticker(fakeTicker)
    .refreshAfterWrite(1.hour)
    .expireAfter(
      create = (_: String, response: Expired) => response.ttl.hours,
      update = (_: String, response: Expired, _: FiniteDuration) => response.ttl.hours,
      read = (_: String, _: Expired, duration: FiniteDuration) => duration
    )
    .buildAsyncFuture[String, Expired](load(_))

  def load(s: String): Future[Expired] = {
    Future.successful(Expired(4, Random.nextString(10)))
  }

  "get" - {
    "should pass" in {
      for {
        first <- cache.get("test")
        _ = fakeTicker.advance(5.hours)
        second <- cache.get("test")
      } yield {
        assert(first != second)
        succeed
      }
    }
  }
}

Does not work on free Futures:

import com.github.blemale.scaffeine.{ AsyncLoadingCache, Scaffeine }
import org.scalatest.freespec.AsyncFreeSpec

import scala.concurrent.Future
import scala.concurrent.duration._
import scala.util.Random

class CacheSpec extends AsyncFreeSpec {
  val fakeTicker = new FakeTicker

  case class Expired(ttl: Long, data: String)

  val cache: AsyncLoadingCache[String, Expired] = Scaffeine()
    .executor(scala.concurrent.ExecutionContext.global)
    .ticker(fakeTicker)
    .refreshAfterWrite(1.hour)
    .expireAfter(
      create = (_: String, response: Expired) => response.ttl.hours,
      update = (_: String, response: Expired, _: FiniteDuration) => response.ttl.hours,
      read = (_: String, _: Expired, duration: FiniteDuration) => duration
    )
    .buildAsyncFuture[String, Expired](load(_))

  def load(s: String): Future[Expired] = {
    // Changed from `Future.successful` to `Future`
    Future(Expired(4, Random.nextString(10)))
  }

  "get" - {
    "should pass" in {
      for {
        first <- cache.get("test")
        _ = fakeTicker.advance(5.hours)
        second <- cache.get("test")
      } yield {
        assert(first != second)
        succeed
      }
    }
  }
}

Question: Is cache.get(key, buildFunct) thread safe?

I have a simple (no Loading or AsyncLoading) Cache instance.
Is cache.get(key, buildFunction) a thread safe operation? I'm concerned about the case when buildFunction has to be run.
In my case buildFunction will perform a side effect to return the initialization value.
Can two threads race on generating this initial value?
Thanks!

2.12 build

Hey there,

is there any chance of having a 2.12 build? I would really appreciate that.

Thanks a lot,
TH

Upgrade scala to version 2.11.12 and 2.12.8

Is % compile necessary for sbt example in README?

The README has this example for including scaffeine with sbt:

"com.github.blemale" %% "scaffeine" % "3.1.0" % "compile"

Sorry this may be a silly question, but I was wondering what % "compile" did here. Does it do anything different than leaving it out?