tofu-tf / tofu

Functional programming toolbox

ToFu

Functional programming toolkit aimed at taming the complexity of Tagless Final approach.

Quick Start

see the docs on the microsite

see the examples in the examples directory

To use the whole utils pack just add to your build.sbt:

libraryDependencies += "tf.tofu" %% "tofu" % "latest version in badge"

Of course, you can also specify an exact list of tofu modules that you want to add to your dependencies (used in place of "tofu"):

• tofu-kernel for core independent utils
  • tofu-kernel-cats-mtl for interop between tofu-kernel and Cats MTL
• tofu-core-* for core utils (replace suffix * with ce2 or ce3 depends on which cats-effect version you use)
• tofu-core-higher-kind for higher kinded utils
• tofu-concurrent for concurrent utils
• tofu-config for config utils
• tofu-data for data utils
• tofu-derivation for derivation utils and derevo annotations
• tofu-doobie for Doobie utils
• tofu-enums for Enumeratum utils
• tofu-env for Env (a variation of a Reader Monad based on Monix Task)
• tofu-fs2-interop for interop with fs2
• tofu-streams for streaming utils
• tofu-logging for the whole set of logging utils (derivation, layout, structured, util, interop)
  • tofu-logging-derivation for logging derivation only
  • tofu-logging-layout for logging layout only
  • tofu-logging-structured for logging structured only
  • tofu-logging-util for logging util only
  • tofu-logging-refined for interop between tofu-logging and Refined
  • tofu-logging-shapeless for interop between tofu-logging and Shapeless tag
  • tofu-logging-log4cats for interop with Log4Cats
• tofu-memo for caching utils
• tofu-observable for monix.reactive.Observable utils
• tofu-zio-interop for interop with ZIO (core, logging)
  • tofu-zio-core for ZIO instances only
  • tofu-zio-logging for ZIO logging only

Adopters

Proud user of ToFu? Feel free to add your company!

Contributing

Please note we use the following labels for automated release descriptions:

• chore if your PR does not change any types and runtime semantics
• fix if your PR merely fixes incorrect behavior

Formatting

We have an automated check for style conformance. You can run sbt checkfmt before PR. If you have any trouble during this check, just run sbt fmt and commit again.

Copyright

Copyright the maintainers, 2019-2024

Logos made with love by @impurepics

tofu's Issues

Implement Agent

Implement new concurrent datatype Agent mostly like RefM from ZIO and looking like Ref + Semaphore (Atom + Gatekeeper) for everything else

It should have at least following methods

trait Agent[F[_], A]{
  // return current value, this will never block
  def get: F[A]
  // update value with effectful transformation, wait for the new result
  def updateM(f: A => F[A]): F[A]
  // enqueue transformation, return immediately
  def fireUpdateM(f: A => F[A]): F[Unit]
  // modify value with effectful transformation, wait for the new result
  def modifyM[B](f: A => F[(A, B)]): F[B]
}

Analogue of `adaptError` from MonadError for Handle

It would be nice to have something like

def adaptError[A](fa: F[A])(pf: PartialFunction[E, E]): F[A]

in Handle typeclass

Create summoners for specialized Readers

Make it possible to summon specific contextual instances via

HasContext[F, C]
HasLocal[F, C]    
HasContextRun[F, G, C]

Add convenient syntax for working with F[Either[E, A]]

We interested in few handy methods, that would allow to do basic things with F[Either[E, A]] with additional context Monad[F] without wrapping it in EitherT
possible list is

def orElse(f: => F[Either[E, A]]): F[Either[E, A]]
def catchAll(e: E => F[A]): F[A]
def absolve(implicit R: Raise[F, E]): F[A]
def flatMapR[B](f: A => F[B]): F[Either[E, B]]
def flatMapL[E](f: E => F[E1]): F[Either[E1, A]]
def doubleFlatMap[B](f: A => F[Either[E, B]]): F[Either[E, B]]

Execute instance for ZIO

It would be nice to add tofu.Execute typeclass intance for ZIO

Bifunctor alternative for Env?

Separate type for typed error (not fixed to Throwable) can be useful in many situations, including expressing absence of error (as Nothing) as well as representing some domain errors.
Monix contributors are already working on BIO alternative to Task (https://github.com/monix/monix-bio).
Should we provide such an alternative to Env? Probably it would be wiser to build it upon Monix BIO implementation (or not).

Make WithRun a subtype of Unlift

Add missing instance derivations for ContextT

• Parallel
• Timer
• anything else?

Logging-refined module

Hi
I day-by-day using tofu i found out that i need to manually create a snippet with implicit conversion to support loggable instances for refined types.
I would like to create PR if you don't mind =)

Add docs for logging module

I’m quite annoyed by constant complaining in Scala Group about tofu documentation. Let’s change that, starting from logging.

Add abstractions for streaming datatypes

We need to provide common ground for streaming datatypes, considering following implementations:
https://fs2.io/
https://zio.dev/docs/datatypes/datatypes_stream
https://monix.io/docs/3x/reactive/observable.html
https://github.com/travisbrown/iteratee

Make docs

Write and publish nice docs using mdoc/microsite

Incorrect EmbedLogging implementation

EmbedLogging passes all parameters into the underlying implementation as a a single array.

object Demo extends App {
  val zioLogs = ZLogs.uio.byName("demo")

  implicit val logs = Embed[Logging].embed(zioLogs)

  def run(args: List[String]) = info"${1} ${2} ${3}".as(0)
}

The code above outputs [1,2,3] {} {} instead of 1 2 3. I think the problem is in the write method:

def write(level: Logging.Level, message: String, values: LoggedValue*): F[Unit] =
  underlying.flatMap(_.write(level, message, values))

Should be

underlying.flatMap(_.write(level, message, values: _*))

Timeout fallback effect could be executed even on successful completion

This comment describes the issue

Ambiguous implicit values of Fire for Env

Using Fire with Env fails due to ambiguous implicit instances, that come from EnvInstances and StartInstances.

import cats._
import cats.syntax.applicative._

import tofu._
import tofu.env._

type MyEnv[A] = Env[Unit, A]
object MyEnv extends EnvSpecializedFunctions[Unit]

def f[F[_]: Fire: Applicative]: F[Unit] = Applicative[F].unit

f[MyEnv]

This results in

ambiguous implicit values:
 both method envInstance in trait EnvInstances of type [E]=> tofu.env.EnvFunctorstance[E]
 and method concurrentInstance in trait StartInstances of type [F[_]](implicit F: cats.effect.Concurrent[F])tofu.Fire[F]
 match expected type tofu.Fire[Playground.MyEnv]

https://scastie.scala-lang.org/9UKp4m7ZRN23ydHGetWLzA

Function allowing to get back into exceptional context in `RestoreTo`

Wouldn't it be useful to have a function allowing to get back from G[_] to F[_] in RestoreTo?

Possibly fix unstable concurrent test for ReadWrite

this build was unsuccessful due to unpredictable fiber scheduling, it would be nice to search the way to stabilize it

Deprecate and/or remove tofu.concurrent.syntax.limitedTraverse

Cats-effect 2.0.0 already has this option in Concurrent along with corresponding syntax under cats.effect.syntax.paralleln: link.
There's no need in our implementation, we should probably deprecate and then remove (or just remove) it.

Refine tofu.syntax.context

1. It would be nice to have

def askF[F[_]] = ...

similarly to ask syntax.

2. Also context might have the following variant:

def context[F[_], C](implicit ctx: HasContext[F, C]): F[C] = ctx.context

to distinguish among several Context[F] instances in the scope

Ambiguous implicit for tofu.Raise

In some cases compilation fails with "ambiguous implicit values" error, for example: https://scastie.scala-lang.org/gGXylEdUQu66Us6W5df1ug
It is not always possible not to depend on Sync, so it is necessary to have implicits both from tofu.zioInstances.implicits and zio.interop.catz. But importing tofu.zioInstances.implicits._ somehow solves the problem in this case.

Logging with Marker

Apparently, while trying to log values with additional marker using logging provided by Logs.sync, we encounter java.lang.StackOverflowError

Code example:

  case class User(name: String, age: Int)
  object User{
    implicit val userShow: Show[User] = user => s"name=${user.name}, age=${user.age}"

    implicit val userLoggable: Loggable[User] = Loggable.show[User]
  }

  val testUser = User("John", 42)
  val marker = MarkerFactory.getMarker("some.marker")

  class MyService[F[_]: Logging: Monad] {
    def doLogging(): F[Unit] =
      for {
//         works
        _ <- Logging[F].info(s"User: ${testUser.show}", testUser)  
//         doesn't work
        _ <- Logging[F].infoWithMarker(s"User: ${testUser.show}", marker, testUser)
      } yield ()
  }

  object MyService {
    def apply[I[_]: Functor, F[_]: Monad](implicit logs: Logs[I, F]): I[MyService[F]] = {
      logs.forService[MyService[F]].map(implicit l => new MyService[F])
    }
  }

  implicit val logs: Logs[IO, IO] = Logs.sync[IO, IO]

  val io: IO[Unit] = for {
    service <- MyService[IO, IO]
    _       <- service.doLogging()
  } yield ()

  io.unsafeRunSync()
}

Result:

Exception in thread "main" java.lang.StackOverflowError
at tofu.logging.impl.LoggingImpl.writeMarker(LoggingImpl.scala:26)
at tofu.logging.LoggingBase.infoWithMarker(Logging.scala:49)
at tofu.logging.LoggingBase.infoWithMarker$(Logging.scala:48)
at tofu.logging.impl.LoggingImpl.infoWithMarker(LoggingImpl.scala:8)
at tofu.logging.impl.LoggingImpl.writeMarker(LoggingImpl.scala:29)
at tofu.logging.LoggingBase.infoWithMarker(Logging.scala:49)
at tofu.logging.LoggingBase.infoWithMarker$(Logging.scala:48)
at tofu.logging.impl.LoggingImpl.infoWithMarker(LoggingImpl.scala:8)

I believe the problem is with tofu.logging.impl.SyncLogging. There are no overriding of traceWithMarker, infoWithMarker... methods.

Provide instances of loggable for java.time data classes

It would be nice to have instances of those classes.
Maybe new API for formatting dates could be born from this issue :)

Provide instances of loggable for java.sql time-related classes

It would be really nice to have Loggable instances for java.sql.{Date, Timestamp, Time}.
Because not every database driver supports java.LocalDate and sometimes you just need to apply some transformations on data while logging it's content

Sentry integration with logging

It is nice to have some way to integrate sentry with tofu logging, but I couldn't find any ways to do it. Is there any?
If not, how can I help to implement this?

Fix TofuBracketMVarOps.bracketUpdate

Current method is nonsense, we should transform it to a method, accepting A => F[A] and putting result on success,

Behavior should be related to datatype described in #95

Typeclass for horizontal composition in optics module

Given Property[A, B] and Property[A, C] we need a way to get Property[A, (B, C)].
Looks like separate typeclass and instances (at least for Property and Contains) will help.

Add implicit def for extracting subcontext from ReaderT

Here is example.
Scactie

Given all the lenses (A Contains B) it would be nice to have implicit def that will extract subcontext like ReaderT[M, A, *] => ReaderT[M, B, *]

Documentation roadmap

The goal of this issue is to track an overall progress over tofu documentation.
I propose the following structure:

Core

• Error management (split into Raise, Handle/Restore pages?)
• Forking and racing (split into Fire, Race, Start pages?)
• Execute
• Guarantee
• Timeout
• Void

Contextual

• Env
• Context (rename page from HasContext to Context?)

Concurrent

• Agent
• Atom
• Daemon
• GateKeeper
• ReadWrite
• QVar
• MakeDeferred
• MakeMVar
• MakeRef
• MakeSemaphore

Optics

• Hierarchy
• Contains
• Equivalent
• Extract
• Folded
• Items
• Property
• Subset
• Updated

Utilities

• Config
• Console
• Logging
• Memo

Any comments are highly appreciated!

LogginImpl stackoverflow

import cats.effect.IO
import org.slf4j.LoggerFactory
import tofu.logging.Logging
import tofu.logging.impl.LoggingImpl

object HerdingAnimals {
  def main(args: Array[String]): Unit = {
    implicit val logger: Logging[IO] = new LoggingImpl[IO](LoggerFactory.getLogger(getClass.getName)) {}
    logger.info("test").unsafeRunSync()
  }
}

Runtime:

Exception in thread "main" java.lang.StackOverflowError
	at tofu.logging.impl.LoggingImpl.write(LoggingImpl.scala:17)
	at tofu.logging.LoggingBase.info(Logging.scala:41)
	at tofu.logging.LoggingBase.info$(Logging.scala:41)
	at tofu.logging.impl.LoggingImpl.info(LoggingImpl.scala:8)
	at tofu.logging.impl.LoggingImpl.write(LoggingImpl.scala:20)
	at tofu.logging.LoggingBase.info(Logging.scala:41)
	at tofu.logging.LoggingBase.info$(Logging.scala:41)
	at tofu.logging.impl.LoggingImpl.info(LoggingImpl.scala:8)
	at tofu.logging.impl.LoggingImpl.write(LoggingImpl.scala:20)
	at tofu.logging.LoggingBase.info(Logging.scala:41)
	at tofu.logging.LoggingBase.info$(Logging.scala:41)

Add convenient syntax for working with F[Option[A]]

We interested in few handy methods, that would allow to do basic things with F[Option[A]] with additional context Monad[F] without wrapping it inOptionT
possible list is

def getOrElseF(fa: => F[A]): F[A]
def orElseF(fa: => F[Option[A]]): F[Option[A]]
def orThrow(err: => E)(implicit FE: Raise[F, E]): F[A]
def flatMapOpt[B](f: A => F[B]): F[Option[B]]
def doubleFlatMap[B](f: A => F[Option[B]]): F[Option[B]]

Logging interop modules don't get published

Modules in logging/interop have never been released. Need to add them to a proper place in build.sbt.

ContextT scala doc

It looks like an unfinished sentence. https://github.com/TinkoffCreditSystems/tofu/blob/337a8f0ab58b4cac28f4954f3fe31d1f1655dab1/concurrent/src/main/scala/tofu/concurrent/ContextT.scala#L10

Implement timestamp generation syntax

Currently, there is no way to generate timestamps with Tofu.
Adding syntax for this would be beneficial

Conflicting tofu.Errors implicit inference

Error instance hierarchy refactoring
led to imposibility of .raise syntax usage without specifying types explicitly if there is at least an ApplicativeError instance in the context.

https://scastie.scala-lang.org/PyLHexCiQG2TtIkey41dZQ

Implement whenOpt and unlessOpt

Implement methods like whenA and unlessA, but returning F[Option[A]] in the tofu.syntax.monadic

ContextShift instance for ContextT

Request for ContextShift instance for ContextT; tried to implement myself:

 implicit def ContextTContextShift[F[+_], C[_[_]]](implicit cs: ContextShift[F]): ContextShift[ContextT[F, C, *]] = new ContextShift[ContextT[F, C, *]] {
    def shift: ContextT[F, C, Unit] = ContextT.lift(cs.shift)
    def evalOn[A](ec: ExecutionContext)(fa: ContextT[F, C, A]): ContextT[F, C, A] = ContextT.provideF(ctx => cs.evalOn(ec)(fa.run(ctx)))
  }

Make Execute[F].deferFuture work as before

Execute object has been removed in 0.8.0 which is unfortunate.

Create front page

Create a front page in this repo or another with main information about repo, probably some blog to bee seen at tofu.tf

Make `raise` syntax work with `ContravariantRaise` instances

Unable to find implicit for Restore[F] despite of Errors[F] are found

For example, unable to use tofu.handle retry syntax when ApplicativeThrow in scope:
https://scastie.scala-lang.org/SghpwBxgQd6KDntxEStmJg

Tofu philosophy

At this point tofu states to be a set of functional recipies.

I think, we should establish a more specific philosophy, emphasize the certain problems, which tofu is aimed to solve.
A clearly defined set of use-cases would help to answer the "Why do I need tofu?" question and to determine the direction of future development.

Inconsistency in tofu.syntax.handle

Having some Handle[F, E], I can handle errors of type E1 <: E with the handleWith method, but not with handle: https://scastie.scala-lang.org/jzaAE02dRwuYlnD7nLaQ7Q

This is because the handleWith method requires Handle[F, E1], which can be derived from Handle[F, E] using handleDowncast. But the handle method requires HandleTo[F, F, E1], which can not be derived.

I think the easiest way to fix this is to require Handle[F, E] in the handle method. But I don't know if there might be cases where HandleTo[F, F, E] is available, but Handle[F, E] is not.

Function allowing to traverse with key

It would be nice to have something like:

traverseKey(f: A => F[B]): F[(A, B])
traverseFilterKey(f: A => F[Option[B]]): F[(A, B)]

Derivation of loggable does not respect @masked annotation

Example:
https://scastie.scala-lang.org/mqg5DCneS2KUAWOPgRjKrg

Daemon.exit hangs in some cases

The exit methon can hang forever if the daemon is canceled immediately after it is created. The following code throws a TimeoutException, in my case on the second or third iteration: https://scastie.scala-lang.org/ZEIyjIhUQjORqtfSpAf6Yg. No exception is thrown if:

• the call to cancel is removed
• the call to exit is removed
• or some delay is added between the calls to daemonize and cancel.

I think the problem is with this line: https://github.com/TinkoffCreditSystems/tofu/blob/master/concurrent/src/main/scala/tofu/concurrent/Daemon.scala#L38. The underlying fiber can be canceled before it even starts, so the block inside the guaranteeCase will not be executed and the Deferred will not be completed with a value.

Support covariance in syntax

i.e. for all syntax methods such as:

def flatTap[B](f: C => F[B])(implicit F: FlatMap[F]): F[C] = F.flatTap(fa)(f)

Replace with:

def flatTap[G[x] >: F[x], B](f: C => G[B])(implicit G: FlatMap[G]): G[C] = G.flatTap(fa)(f)

Motivation? The following works with covariant syntax:

Some(1).flatTap(_ => None)

But doesn't work with naive LEGACY syntax encodings.

Logging-shapeless module

Hi
I day-by-day using tofu i found out that i need to manually create a snippet with implicit conversion to support loggable instances for tagged types.
I would like to create PR if you don't mind =)
(sorry for copy-pasting)

Derive `Embed` instance for non-representable traits

Right now Embed is derived only as subtype of RepresentableK

But having trait such as

def Foo[F]{
   def foo(input: F[Int]): F[String]
}

it is still possible to derive an instance of Embed as follows

implicit val fooEmbed: Embed[Foo] = new Embed[Foo]{
    final def embed[F[_]: FlatMap](ffoo: F[Foo[F]]) = new Foo[F]{
        def foo(input: F[Int]) = ffoo.flatMap(_.foo(input))
   }
}

Consider changing scalafmt rules

I do support the need for a forced formatter in a project with many contributors, but the current ruleset is awful and doesn't work properly in IDEA (has some wrong fields).

Implement functions for Option\Either

We need something like

def some[A]: Subset[Option[A], A]
def none[A]: Subset[Option[A], Unit]
def left[A, B]: Subset[Either[A, B], A]
def right[A, B]: Subset[Either[A, B], B]

in the tofu.optics.functions