Slow build times with Quill, can our approach be improved? about zio-quill

[guizmaii]

@Aleks67B @jilen Can you please give a concrete example of code before/after maybe? I'm also suffering from long compilation times with Quill and I'm not sure to understand @jilen suggestion

Also, I think that it'd be good to document this trick. Lots of people are suffering from these compilation times. (I can make the PR to document it, no worries)

[Aleks67B]

@guizmaii

Sure, taking the example from the initial comment regarding the ProductId, our previous implicit mapping was:

case class ProductId(id: Long)

protected implicit val productIdDecoder: MappedEncoding[Long, ProductId] = MappedEncoding(ProductId.apply)
protected implicit val productIdEncoder: MappedEncoding[ProductId, Long] = MappedEncoding(_.id)

Updated with explicit Encoders and Decoders:

protected val ctx: PostgresMonixJdbcContext[SnakeCase.type]

protected implicit val productIdDecoder: ctx.Decoder[ProductId] = ctx.mappedDecoder(MappedEncoding(ProductId.apply), ctx.longDecoder)
protected implicit val productIdEncoder: ctx.Encoder[ProductId] = ctx.mappedEncoder(MappedEncoding(_.id), ctx.longEncoder)

[guizmaii]

BTW, if anyone is reading this, here's a generic implementation of this trick for zio-prelude newtypes:

// File: `io.example.extensions.scala`
import zio.prelude._
import io.getquill.MappedEncoding
import io.getquill.context.jdbc.JdbcContextTypes

object extensions {

  trait QuillEncoding[A] {
    self: Newtype[A] =>

    def quillEncoder(ctx: JdbcContextTypes[?, ?])(implicit encoder: ctx.Encoder[A]): ctx.Encoder[Type] =
      ctx.mappedEncoder[Type, A](MappedEncoding(unwrap), encoder)

    def quillDecoder(ctx: JdbcContextTypes[?, ?])(implicit decoder: ctx.Decoder[A]): ctx.Decoder[Type] =
      ctx.mappedDecoder[A, Type](MappedEncoding(wrap), decoder)
  }

}

// File: `io.example.domain.scala`
import zio.prelude._
import io.example.extensions._

object domain {
  
  // Subtype example 
  type ProductId = ProductId.Type
  object ProductId extends Subtype[Long] with QuillEncoding[Long]

  // Newtype example
  type AnotherId = AnotherId.Type
  object AnotherId extends Newtype[String] with QuillEncoding[String]

}


// File: `io.example.db.Codecs.scala`
import io.getquill.context.jdbc.*
import io.example.domain._

/**
 * Note that I'm using PG. Some of the traits I extend here come from the PG extensions of Quill.
 * If you're not using PG, you might need to replace/remove some of them.
 */
trait Codecs
    extends Encoders
    with Decoders
    with ObjectGenericTimeEncoders
    with ObjectGenericTimeDecoders
    with BooleanObjectEncoding
    with UUIDObjectEncoding
    with ArrayDecoders
    with ArrayEncoders { self: JdbcContextTypes[?, ?] =>

  // IntelliJ might not be happy with the implicits required by the `quillEncoder` and `quillDecoder` methods so we tell it to shut up with the following weird comment:
  /*_*/
  implicit val encoderProductId: Encoder[ProductId] = ProductId.quillEncoder(self)
  implicit val decoderProductId: Decoder[ProductId] = ProductId.quillDecoder(self)
  implicit val encoderAnotherId: Encoder[AnotherId] = AnotherId.quillEncoder(self)
  implicit val decoderAnotherId: Decoder[AnotherId] = AnotherId.quillDecoder(self)
  /*_*/
}

// Finally, you need to mixin this `Codecs` trait to your Quill context. Here's an example:
// File: `io.example.db.DBLive.scala`
import io.getquill.context.jdbc.*
import io.getquill.*
import io.example.domain._

trait DB {
  ...
}

object DB {
  def live: ULayer[DB] = ...
}

final class DBLive(dataSource: DataSource) extends DB {
  import DBLive.Context.*
  
  ...
}

object DBLive {

  /**
   *  Note the `with Codecs` here. That's what you need to add to your context
   */
  
  // noinspection TypeAnnotation
  @SuppressWarnings(Array("scalafix:ExplicitResultTypes"))
  private[db] object Context extends PostgresZioJdbcContext(naming = Literal) with Codecs {
    import extras.*
    
    private[db] def products  = quote(querySchema[Product]("products"))
    
    def selectProductsById(id: ProductId) = 
      quote {
        products.filter(_.id === lift(id))
      }
  
  }
}

(Note that the above code wasn't written in an IDE nor was compiled so it might be slightly incorrect but you have the general idea. The QuillEncoding and its methods are correct. It's copy/pasted from my project)

[jilen]

You may first try to use the -Yprofile-trace to get some insights. (Note seems only works on jdk 8, higher version may need some java9 module system tweak )

We have some evidence that replacing the strongly-typed IDs with bare longs improves query compilation time

And if you can confirm MappedEncoding cause the slow down.
You can just define the val xidEncoder: Encoder[Xid] = mappedEncoder ... directly to reduce the implicit resolution time.

[Aleks67B]

Thank you for the suggestion @jilen . We have removed our implicit MappedEncodings and changed to defining Encoders and Decoders directly. This has resulted in a ~30% reduction in compile time.

[guizmaii]

Thanks a lot @Aleks67B

[guizmaii]

@Aleks67B Sidenote question: why are you using case classes to implement your typesafe ids and not zio-prelude newtypes? AFAIK, case classes have an overhead at runtime (instantiation) which newtypes don't have

Example:

import zio.prelude._

type ProductId = ProductId.Type
object ProductId extends Subtype[Long] // Can also be `... extends Newtype[Long]` depending on your needs