Scala flatmap and Parallel programming

We usually use flatmap and map in scala. Of course, if we use future we chaining by using flatmap or map… etc. But is this run parallel? In fact (as you know) it’s not parallel. If you chaining some monads, it runs sequentially.

Future chaining is not parallel

Let’s see some example. Guess that, there is some db dependent function like beneath

Each takes 2 seconds, 2 seconds, 3 seconds. And we want to make userInfo by calling functions. You can call like this.

As you can see, It takes 7 seconds (2 sec + 2 sec + 3 sec). It is not parallel.

How to be parallel?

by function

There easy 2 ways to be parallel. Solve this by function. You can make like this.

It takes 5 seconds(2 seconds + max(2 seconds, 3 seconds)). It’s because, before chaining, Future already started. And after getting an email address, 2 seconds, we only need to wait for 1 second to get accountInfo because it runs parallel.

by using the Cartesian product and applicative

But it has to make function. It’s annoying. We can simply make this by using cats.

It also takes 5 seconds, But it is simpler than by using a function.

Implicit in Scala

In Scala, Implicit is very important. But it is very ambiguous and hard to understand. And it is learning curve to enter Scala world. This post is basic about Scala Implicit.

Implicit is operated by the compiler. If there is some error, for example, Instance type is not compatible or need implicit parameter Compiler is looking for Implicit conversion or Implicit Instance. And this post is about making implicit Conversion or Instance.

Implicit Conversion

implicit def

We can convert some instance to other instance automatically by using implicit. First, we can use ‘Implicit Conversion’. Let’s say that, there are some instances like that.

If we want to convert ‘ValueCaseClass’ to ‘PrintableClass’, you can use by using implicit def like this.

In this code,


val printableClass1: PrintableClass = ValueCaseClass("Implicit Def")

printableClass1’s type is PrintableClass but we set ValueCaseClass. It is compile error. But, Compile is looking for implicit conversion and finally find implicit definition ‘convertValueCaseClassToPrintable’ and get PrintableClass.

implicit class

You can also convert by using implicit class.

In this code,


val printableClass2: PrintableClass = ValueCaseClass("Implicit class").toPrintable

ValueCaseClass don’t has toPrintable. Compiler looking for implicit class and find ‘ConvertableValueCaseClass’ and there is ‘toPrintable’.

Implicit Instance

In Scala, def can get implicit parameter.

In code above, ‘print’ need ‘printableClass’ parameter. Compiler automatically pass implicit instance ‘printableClass’.

Implicit Order

Above, there we can get the implicit instance. And I think it is very useful. We can get Implicit Instance in many ways. Let see an example.

Can you guess which string is printed in the console? The answer is ‘Implicit Instance in Same Conjure’ Implicit order is

  1. In same conjure
  2. Imported instance
  3. companion object
  4. default imported

So, if we delete

def getString(implicit implicitInstance: Instance[String]): String = implicitInstance.value

, “Implicit Instance in imported” will be printed. And if we delete

import Imported.implicitIntInstance1

, “Implicit Instance in Companion Object” will be printed

Default implicit instance by companion object.

Let’s pay attention to the companion object implicit instance. We can make default instance by implicit instance in companion object.

Above code


print(int) // Type: Int - 1
print(hello) // Type: Hello - Hello(ktz)

Function ‘print’ is passed default implicit instance in companion object


implicit val IntPrinter: Printer[Int] = new Printer[Int] {
  override def print(value: Int): String = s"Type: Int - $value"
}

implicit val PersonPrinter: Printer[Hello] = new Printer[Hello] {
  override def print(value: Hello): String = s"Type: Hello - $value"
}

But next ‘print(int)’ is passed ‘injectedIntPrinter’. Like this, we can inject any instance by importing some instance.

Tagged Type in Scala

Human always makes mistake. Also, software-engineer makes mistake. Guess there are some codes like this.

Two function calls


getHashCode(userId, deviceUuid, carSerialId) // Right Answer = res0: String = 13-1236-12348

getHashCode(deviceUuid, userId, carSerialId) // Wrong Answer = res1: String = 1235-14-12348

both compile. But the first one is a right answer, but secondly is wrong. Then What is a good way to help myself not make a mistake? At first thinking, It is very good way to make case class.

This way, we can reduce mistakes. But we lost many things. First, we have to write many boilerplate. We have to make case class every Id and other Long types. And we lost that Id is the Long type. So, if we want to use userId or carSerialId or DeviceUuid, we have to call member. And cannot assign to Long type variable! To solve these problems, we can use Tagged Type.

Tagged Type

Tagged Type is tagging to some type A and Define as a subtype of A. For example, if we want to define userId as a subtype of Long, we can declare UserId as a Tagged Type of Long. Here is following an example.

Now, UserId, DeviceUuid, CarSerialId is Subtype of Long. We can use this type as Long type. For example, add Number to UserId by just using ‘+’ operator. or can assign to Long. But, It is UserId Type, DeviceUuid cannot be passed as a parameter in UserId Position.

Tagged Type Eraser

Sometimes you want to override function by tagged type like this.

But if you override like this, Compiler says that ‘Not Compiled because tagged type erased after compile’. Yes, Tagged Type is erased after compile time. Then how can we solve the problem? You can solve this by using Either.

Of course, you can use Coproduct in shapeless

By this way, You can keep self from make mistake.

Scala Implicitly

As I study about Scala, I open see implicitly. But, every time I see this, I just ignore just thinking like “It’s syntactic sugar of implicit”. This time, I write this post to really under stand this.

In Scala, implicitly is just this.


def implicitly[T](implicit e: T) = e

That’s all! implicitly is a function just get implicit e instance and return e. Then, what is an advantage of using implicitly?

Example 1

You can use implicit like this.

implicit val optionInt: Option[Int] = Some(1)
implicit val optionBoolean: Option[Boolean] = Some(true)

def getImplicitInt(implicit oInt: Option[Int]): Int = oInt.get
def getImplicitBoolean(implicit oBoolean: Option[Boolean]): Boolean = oBoolean.get

getImplicitInt
// res0: Int = 1
getImplicitBoolean
// res1: Boolean = true

Now, you can use just same by using implicitly


implicit val optionInt: Option[Int] = Some(1)
implicit val optionBoolean: Option[Boolean] = Some(true)

def getImplicitlyA[A: Option]: A = implicitly[Option[A]].get

getImplicitlyA[Int]
// res2: Int = 1
getImplicitlyA[Boolean]
// res3: Boolean = true

Implicitly can use in such situation.

Implicitly can use when you want to get implicit instance which have 1 type parameter.

As you can see above, by using ‘Implicitly[Option[Int]]’, you can get implicit instance ‘Option[Int]’. It has 1 type parameter Int.

Example 2

Then how about implicitly for the type which has no type parameter? For example, implicit can do like this.


implicit val string: String = "Hello"
implicit val boolean: Boolean = true
implicit val int: Int = 0

def getImplicitT[T](implicit t: T): T = t

getImplicitT[Int]
// res0: Int = 0

getImplicitT[String]
// res1: String = Hello

getImplicitT[Boolean]
// res2: Boolean = true

But, if you do the same with implicitly, errors occur.


def getImplicitlyT[T]: T = implicitly[T]

getImplicitlyT[Int]
getImplicitlyT[String]
getImplicitlyT[Boolean]

// compile error: not enough arguments for method implicitly: (implicit e: T)T. Unspecified value parameter e. def getImplicitlyT[T]: T = implicitly[T] ^

Yes, because I mentioned above, implicitly can be used the type which has a type parameter. Then how can we use implicitly in this situation? The answer is ‘make Id type’.


type Id[A] = A

def getImplicitlyT[T : Id]: T = implicitly[Id[T]]

getImplicitlyT[Int]
// res3: Int = 0

getImplicitlyT[String]
// res4: String = Hello

getImplicitlyT[Boolean]
// res5: Boolean = true

I make new type Id which has one type parameter. And Id[A] is same as A. So, you can use just like implicit

Context bound

Right side of function ‘getImplicitlyAt’ has type parameter(A : Option). It looks like type bound. But it is called Context bound. Type bound is ‘<:’.You can see type bound in Scala School. In context bound, it doesn’t mean that A is Option. Instead, A will be a type passing to Option.

Applicative in Scala Cats

I always heard about Applicative, but I don’t know what really is. So, I write this post to understand myself. Below picture is from adit.io and example is from mastering advanced Scala.

What is Applicative?

During programming, you will encounter this situation.

def getSome(a: Int): Option[Int] = Some(a)

def getNone(a: Int): Option[Int] = None

def add(a: Int, b: Int): Int = a + b

val aOpt = getSome(1)

val bOpt = getSome(2)

And I want to get aOpt and bOpt add together using add. But aOpt and bOpt is Option. So, the usual way to do is by using flatMap and map like this.


aOpt.flatMap(a => bOpt.map(b => add(a, b)))
// res0: Option[Int] = Some(3)

In here, we can use Applicative.

What is Applicative?

I don’t know that my understanding of Applicative is right. But I think, Applicative is

applicative_just.png

Very famous picture about Applicative.

In here there is a wrapped value: 2. And there is a wrapped function. We unwrapped both function and value, and process some operation and wrap it again! And we can also adapt these things to above example.


import cats.instances.option._
import cats.Applicative

Applicative[Option].map2(aOpt, bOpt)(add)
// res1: Option[Int] = Some(3)

We can also use like this.


import cats.syntax.all._

(aOpt |@| bOpt).map(add)

Monad extends Applicative


@typeclass trait Monad[F[_]] extends FlatMap[F] with Applicative[F] {
  override def map[A, B](fa: F[A])(f: A => B): F[B] =
  flatMap(fa)(a => pure(f(a)))
}

As you can see, Monad extends Applicative. And Applicative extends Functor. So, hierarchy is ‘Monad <: Applicative <: Functor’

Tip – Traverse

While studying with Mastering Advanced Scala, there are some tips in Applicative, So, I write down this. If there is a List[Int] and want all element adapt ‘getSome’ function, for example


val ints = List(1,2,3,4,5)
ints.map(getSome)
// res3: List[Option[Int]] = List(Some(1), Some(2), Some(3), Some(4), Some(5))

The result is ‘List(Some(1), Some(2), Some(3), Some(4), Some(5))’ and if you want to make ‘Some(List(1, 2, 3, 4, 5))’ you can use Traverse like this.


import cats.Traverse
import cats.instances.list._
import cats.instances.option._

Traverse[List].traverse(ints)(getSome)

// res4: Option[List[Int]] = Some(List(1, 2, 3, 4, 5))

This will return None if any element return None


def getSomeOrNone(a: Int): Option[Int] =
  if(a % 2 == 0) Some(a)
  else None

Traverse[List].traverse(ints)(getSomeOrNone)

// res5: Option[List[Int]] = None

 

Cartesian Product in Scala Cats

During work, I have some problem handling three future at the same time. First future return a long value and second and third future receive the parameter from the first return and return a tuple.

The code is like this.


for {
  long <- futureLong()
  int <- futureInt(long)
  char <- futureChar(long)
} yield (int, char)

Of course for comprehension operates step by step, so, futureInt is called after futureLong is finished and futureChar is called after futureInt is finished. So, It takes too much time to get a final return value.

So, I changed codes like this.


val longVal: Future[Long] = futureLong()

val intVal: Future[Int] = for {
  long <- futureLong()
  int <- futureInt(long)
} yield int

val charVal: Future[Char] = for {
  long <- futureLong()
  char <- futureChar(long)
} yield char

val forComp: (Int, Char) = Await.result(for {
  int <- intVal
  char <- charVal
} yield (int, char), 5 second)

Yes, It works as I want. But, there is duplicated codes like, ‘long <- futureLong()’. During thinking a long time, my coworker, Liam, recommends using  Cartesian Product.

What is Cartesian Product?

Cartesian Product is very simple. It’s just a product of two Sets. For example, if there are two sets A = {1,2,3} and B = {‘a’, ‘b’, ‘c’}, Cartesian Product of A X B is

{(1, ‘a’), (1, ‘b’), (1, ‘c’), (2, ‘a’), (2, ‘b’), (2, ‘c’), (3, ‘a’), (3, ‘b’), (3, ‘c’)}

Yes. that’s all. Of course Cartesian Product didn’t hold commutative law. For example Cartesian Product of B X A is

{(‘a’, 1), (‘a’, 2), (‘a’, 3), (‘b’, 1), (‘b’, 2), (‘b’, 3), (‘c’, 1), (‘c’, 2), (‘c’, 3)}

And different from A X B.

Cartesian Product in cats

Above example can be implemented in Scala.


import cats.instances.list._
import cats.syntax.all._

val ints: List[Int] = List(1, 2, 3, 4)
val chars: List[Char] = List('a', 'b', 'c')

(ints |@| chars).tupled
// res0: List[(Int, Char)] = List((1,a), (1,b), (1,c), (2,a), (2,b), (2,c), (3,a), (3,b), (3,c), (4,a), (4,b), (4,c))
(chars |@| ints).tupled
// res1: List[(Char, Int)] = List((a,1), (a,2), (a,3), (a,4), (b,1), (b,2), (b,3), (b,4), (c,1), (c,2), (c,3), (c,4))

(ints |@| chars).map(_ + _.toString)
// res2: List[String] = List(1a, 1b, 1c, 2a, 2b, 2c, 3a, 3b, 3c, 4a, 4b, 4c)

In Cats Cartesian Product Syntax is ‘|@|’.(Some people call it Oink, and others call Pizza box) If you call tuple, it will return List Tuple. You can also map it.

Cartesian Product in Future

Ok, let’s return to the previous problem. I need to call futureInt and futureChar at a same time. And using three for comprehension is too dirty. So, how and I solve this with Cartesian Product? It’s very simple.


for {
  long <- futureLong()
  (int, char) <- (futureInt(long) |@| futureChar(long)).tupled
} yield (int, char)

Very simple right? You can test whether this expression really operates at the same time.

In this test, the first test occurs TimeoutException and second and third pass test well. Then, we can make a conclusion that, last expression also run at the same time.

Writer Monad in cats

I already wrote a post about reader monad and It’s time to write about Writer Monad. First, I thought that Writer Monad is opposite of Reader Monad. But it’s quite different from Reader Monad. Let’s say that Writer Monad is…

Writer Monad is stack something while operating.

Hm… I think it’s not a very well summarized sentence. If any reader has nicer summarized one, comment me. Below example is from mastering advanced Scala.

Example

type Writer id defined in cats like this.

type Writer[L, V] = WriterT[Id, L, V]

As you can see, Writer Monad accept type L & V. type L is a type which stacks something. and V is a result(return) value. Id position in WirterT is something wraps the result. Here is Id so, never mind.

You can simply use Writer Monad like.

import cats.data.Writer

def greetW(name: String, logged: Boolean): Writer[List[String], String] =
  Writer(List("Compose a greeting"), {
    val userName = if(logged) name else "User"
    s"Hello $userName"
  })

def isLoggedW(name: String): Writer[List[String], Boolean] =
  Writer(List("Checking if user is logged in"), name.length == 3)

import cats.instances.list._

val name = "Joe"

val resultW: Writer[List[String], String] = for {
  logged <- isLoggedW(name)
  greeting <- greetW(name, logged)
} yield greeting
val (log, result) = resultW.run

// log: List[String] = List(Checking if user is logged in, Compose a greeting)
//result: String = Hello Joe

Yes. It’s very simple. It just stacks some strings in L(List[String]) and returns value in V(String). I think it’s very useful.

How is it works?

Of course, you can have curious that how’s is it works? As you can see in ‘for comprehension’, there is nowhere to pass List[String]. But after finishing for comprehension, it splits out log: List[String] and result: String.

Let’s remind this.

For comprehension is syntactic sugar of flatMap.

You can also write like this.

val flatMapResult: Writer[List[String], String] =
  isLoggedW(name).flatMap(logged => greetW(name, logged))

It’s the same result of for comprehension statements.

Ok. It’s flatMap. Then, Let’s deep dive to WriterT.flatMap

final case class WriterT[F[_], L, V](run: F[(L, V)]) {
        ...
  def flatMap[U](f: V => WriterT[F, L, U])(implicit flatMapF: FlatMap[F], semigroupL: Semigroup[L]): WriterT[F, L, U] =
    WriterT {
      flatMapF.flatMap(run) { lv =>
        flatMapF.map(f(lv._2).run) { lv2 =>
          (semigroupL.combine(lv._1, lv2._1), lv2._2)
        }
      }
    }
        ...
}

In this code, never mind another but line:7. In Line 7, they combine two List[String]. Stacking is occurred in here. WriterT is different from others. value run’s type is F[(L, V)] and it does flatMap with it. So we can stack List[String]

WriterT Example

WriterT is Monad Transformer of Writer. So, you can use like this.

import cats.data.WriterT
import cats.instances.future._
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

def greetWFuture(name: String, logged: Boolean): WriterT[Future, List[String], String] =
  WriterT(Future(List("Compose a greeting"), {
    val userName = if(logged) name else "User"
    s"Hello $userName"
  }))

def isLoggedWFuture(name: String): WriterT[Future, List[String], Boolean] =
  WriterT(Future(List("Checking if user is logged in"), name.length == 3))

val resultWFuture: WriterT[Future, List[String], String] = for {
  logged <- isLoggedWFuture(name)
  greeting <- greetWFuture(name, logged)
} yield greeting

val futureResult: Future[(List[String], String)] = resultWFuture.run

import scala.concurrent.Await
import scala.concurrent.duration._

val (logAwait, resultAwait) = Await.result(futureResult, 2 second)

It’s very simple. We can use Writer Monad inside another Monad.