The Original Sin of Scala Ecosystems

Mateusz Kubuszok

About me

Agenda

  • a (de)motivating example

  • more examples from history

  • pattern?

  • what can we do about it

A (de)motivating example

My idea

  • OSS clone of Reddit

  • Scala 2.13

  • Cats + Cats Effect 2 + FS2 + Http4s

  • Monix as IO, TaskLocal for MDC

  • Tapir

  • Jsoniter

  • Scala Newtype

  • March 2021 - last commit

  • everything works

  • September 2022 - updated dependencies

  • migration took about 2-3 weeks

Why?

Cats Effect 2 to 3

def myModule[F[_] : ConcurrentEffect
                  : ContextShift
                  : Timer]: Resource[F, MyModule] = ...

someIO.unsafeRunSync()
def myModule[F[_]: Async]: Resource[F, MyModule] = ...

import cats.effect.unsafe.implicits.global
someIO.unsafeRunSync()

Also:

  • forced updates of libraries which depended on CE: FS2, Http4s, Doobie

  • there is no Monix for Cats Effect 3 ATM

Monix and Local

// Based on https://olegpy.com/better-logging-monix-1/
final class MonixMDCAdapter extends LogbackMDCAdapter {
  private val map = monix.execution.misc.Local(
    java.util.Collections.emptyMap[String, String]())
  // methods delegating to map
}
object MonixMDCAdapter {
  def configure(): Unit = {
    val field = classOf[org.slf4j.MDC]
      .getDeclaredField("mdcAdapter")
    field.setAccessible(true)
    field.set(null, new MonixMDCAdapter)
  }
}
// set up Logback
MonixMDCAdapter.configure()
// propagating changes to Local within Task
task.executeWithOptions(_.enableLocalContextPropagation)
object IOGlobal {
  private val threadLocal = ThreadLocal.withInitial(
    () => Map.empty[IOLocal[_], Any]
  )
  // IOLocalHack.get : IO[Map.empty[IOLocal[_], Any]]
  def propagateState[A](thunk: => IO[A]): IO[A] =
    IOLocalHack.get.flatMap { state =>
      threadLocal.set(state); thunk }
}
def configureAsync(tc: Async[IO]) = new Async[IO] {
  // extract IOLocal and set it in threadLocal
  // in every operation which might use it
  def suspend[A](hint: Sync.Type)(thunk: => A) =
    tc.suspend(hint)(propagateState(tc.pure(thunk))).flatten
  def handleErrorWith[A](fa: IO[A])(f: Throwable => IO[A]) =
    tc.handleErrorWith(fa)(e => propagateState(f(e)))
  def flatMap[A, B](fa: IO[A])(f: A => IO[B]) =
    tc.flatMap(fa)(a => propagateState(f(a)))
  def tailRecM[A, B](a: A)(f: A => IO[Either[A, B]]) =
    tc.tailRecM(a)(b => propagateState(f(b)))
  // and plain redirect to tc for everythin else
}

FS2 Kafka

type EventBusProducer[F[_], Event] = Pipe[
  F,
  (UUID, Event),
  ProducerResult[UUID, Event, Unit]
]
// changed the order of parameters in ProducerResult
type EventBusProducer[F[_], Event] = Pipe[
  F,
  (UUID, Event),
  ProducerResult[Unit, UUID, Event] // <-- here
]

Http4s

import org.http4s.server.blaze.BlazeServerBuilder
import org.http4s.util.{ CaseInsensitiveString => CIString }
// Blaze moved to a separate dependency
import org.http4s.blaze.server.BlazeServerBuilder
// CIString became deprecated alias
import org.typelevel.ci.CIString

Tapir

val endpoint: Endpoint[I, E, O, R]
val endpoint: Endpoint[A, I, E, O, R]
Http4sServerOptions.default[F].copy[F](
    decodeFailureHandler = ...,
    logRequestHandling = LogRequestHandling[F[Unit]](
      doLogWhenHandled = ...,
      doLogAllDecodeFailures = ...,
      doLogLogicExceptions = ...,
      noLog = Applicative[F].unit
    )
  )
Http4sServerOptions.customiseInterceptors[F]
  .decodeFailureHandler(...)
  .serverLog(
    DefaultServerLog[F](
      doLogWhenReceived = ...,
      doLogWhenHandled = ...,
      doLogAllDecodeFailures = ...,
      doLogExceptions = ...,
      noLog = Sync[F].unit,
      logWhenHandled = true,
      logAllDecodeFailures = false,
      logLogicExceptions = true
    )
  ).options

Other Examples from History

Scala 2.12 to 2.13

def convert[Coll[_], A, B](coll: Coll[A])(
  f: A => B
)(
  implicit bf: CanBuildFrom[Coll[A], A, Coll[B]]
): Coll[B] = ...

list.to[Vector]
def convert[Coll[X] <: Iterable[X], A, B](coll: Coll[A])(
  f: A => B
)(
  implicit factory: Factory[B, Coll[B]]
): Coll[B] = ...

list.to(Vector)

Happy experiments

  • scala-parallel-collections

  • scala-parser-combinators

  • scala-continuations

Neverending 0.x version

  • Circe - 7 years since creation (2015), still 0.x

  • Doobie - 7 years between 0.1 (2014) and 1.0-RC1 (2021)

  • Http4s - 5 years since creation (2017), 0.x and unstable milestones 1.0

  • Chimney - 5 years since creation (2017), still 0.x

Major version update

  • Cats Effect 1.0 (2018) to Cats Effect 2.0 (2019) - 1 year

  • Cats Effect 2.0 (2019) to Cats Effect 3.0 (2021) - 2 years

  • ZIO 1.0 (2020) to ZIO 2.0 (2022) - 2 years

Monad of the Year

  • Scala’s Future

  • Scalaz Future

  • Scalaz Task

  • Monix Task

  • Scalaz IO → ZIO

  • free → freer → eff

  • monad transformers

  • tagless final

  • MTL

  • ZIO with ZLayers

Some observations?

  • the "new is always better" attitude

  • which prioritizes greenfield over maintenance

  • relatively (to other languages) frequent breaking changes

  • in statically typed FP world API changes are very invasive

  • how much of your API is defined with the types you directly control?

Thesis

In Scala we have a glory-driven development.

What Can We Do About It?

Language

  • Scala 3 LTS initiative

  • MiMa

  • thinking about tooling when new features are introduced

Library maintainers

  • not staying on early-semver forever

  • committing to major version for a long time

  • deciding when the API would be "good enough" to stop rewriting it without some graceful migration strategy

Library users

def listUsers(
  fetchUsers: IO[List[User]],
  printUser: User => IO[Unit]
): IO[Unit] =
  fetchUsers.flatMap(users => users.traverse(printUser))
def listUsers[F[_]: Monad, G[_]: Traverse](
  fetchUsers: F[G[User]],
  printUser: User => F[Unit]
): F[Unit] =
  fetchUsers.flatMap(users => users.traverse(printUser))
package com.mycompany.distributed

export akka.*
opaque type Result[-E, +A] = ...

object Result:
  // utilities

  extension[E, A](result: Result[E, A])
    // extension methods

Thank You!