Scala flatmap and Parallel programming

We usually use flatmap and map in scala. Of course, if we use future we chaining by using flatmap or map… etc. But is this run parallel? In fact (as you know) it’s not parallel. If you chaining some monads, it runs sequentially.

Future chaining is not parallel

Let’s see some example. Guess that, there is some db dependent function like beneath

Each takes 2 seconds, 2 seconds, 3 seconds. And we want to make userInfo by calling functions. You can call like this.

As you can see, It takes 7 seconds (2 sec + 2 sec + 3 sec). It is not parallel.

How to be parallel?

by function

There easy 2 ways to be parallel. Solve this by function. You can make like this.

It takes 5 seconds(2 seconds + max(2 seconds, 3 seconds)). It’s because, before chaining, Future already started. And after getting an email address, 2 seconds, we only need to wait for 1 second to get accountInfo because it runs parallel.

by using the Cartesian product and applicative

But it has to make function. It’s annoying. We can simply make this by using cats.

It also takes 5 seconds, But it is simpler than by using a function.

Implicit in Scala

In Scala, Implicit is very important. But it is very ambiguous and hard to understand. And it is learning curve to enter Scala world. This post is basic about Scala Implicit.

Implicit is operated by the compiler. If there is some error, for example, Instance type is not compatible or need implicit parameter Compiler is looking for Implicit conversion or Implicit Instance. And this post is about making implicit Conversion or Instance.

Implicit Conversion

implicit def

We can convert some instance to other instance automatically by using implicit. First, we can use ‘Implicit Conversion’. Let’s say that, there are some instances like that.

If we want to convert ‘ValueCaseClass’ to ‘PrintableClass’, you can use by using implicit def like this.

In this code,


val printableClass1: PrintableClass = ValueCaseClass("Implicit Def")

printableClass1’s type is PrintableClass but we set ValueCaseClass. It is compile error. But, Compile is looking for implicit conversion and finally find implicit definition ‘convertValueCaseClassToPrintable’ and get PrintableClass.

implicit class

You can also convert by using implicit class.

In this code,


val printableClass2: PrintableClass = ValueCaseClass("Implicit class").toPrintable

ValueCaseClass don’t has toPrintable. Compiler looking for implicit class and find ‘ConvertableValueCaseClass’ and there is ‘toPrintable’.

Implicit Instance

In Scala, def can get implicit parameter.

In code above, ‘print’ need ‘printableClass’ parameter. Compiler automatically pass implicit instance ‘printableClass’.

Implicit Order

Above, there we can get the implicit instance. And I think it is very useful. We can get Implicit Instance in many ways. Let see an example.

Can you guess which string is printed in the console? The answer is ‘Implicit Instance in Same Conjure’ Implicit order is

  1. In same conjure
  2. Imported instance
  3. companion object
  4. default imported

So, if we delete

def getString(implicit implicitInstance: Instance[String]): String = implicitInstance.value

, “Implicit Instance in imported” will be printed. And if we delete

import Imported.implicitIntInstance1

, “Implicit Instance in Companion Object” will be printed

Default implicit instance by companion object.

Let’s pay attention to the companion object implicit instance. We can make default instance by implicit instance in companion object.

Above code


print(int) // Type: Int - 1
print(hello) // Type: Hello - Hello(ktz)

Function ‘print’ is passed default implicit instance in companion object


implicit val IntPrinter: Printer[Int] = new Printer[Int] {
  override def print(value: Int): String = s"Type: Int - $value"
}

implicit val PersonPrinter: Printer[Hello] = new Printer[Hello] {
  override def print(value: Hello): String = s"Type: Hello - $value"
}

But next ‘print(int)’ is passed ‘injectedIntPrinter’. Like this, we can inject any instance by importing some instance.

Tagged Type in Scala

Human always makes mistake. Also, software-engineer makes mistake. Guess there are some codes like this.

Two function calls


getHashCode(userId, deviceUuid, carSerialId) // Right Answer = res0: String = 13-1236-12348

getHashCode(deviceUuid, userId, carSerialId) // Wrong Answer = res1: String = 1235-14-12348

both compile. But the first one is a right answer, but secondly is wrong. Then What is a good way to help myself not make a mistake? At first thinking, It is very good way to make case class.

This way, we can reduce mistakes. But we lost many things. First, we have to write many boilerplate. We have to make case class every Id and other Long types. And we lost that Id is the Long type. So, if we want to use userId or carSerialId or DeviceUuid, we have to call member. And cannot assign to Long type variable! To solve these problems, we can use Tagged Type.

Tagged Type

Tagged Type is tagging to some type A and Define as a subtype of A. For example, if we want to define userId as a subtype of Long, we can declare UserId as a Tagged Type of Long. Here is following an example.

Now, UserId, DeviceUuid, CarSerialId is Subtype of Long. We can use this type as Long type. For example, add Number to UserId by just using ‘+’ operator. or can assign to Long. But, It is UserId Type, DeviceUuid cannot be passed as a parameter in UserId Position.

Tagged Type Eraser

Sometimes you want to override function by tagged type like this.

But if you override like this, Compiler says that ‘Not Compiled because tagged type erased after compile’. Yes, Tagged Type is erased after compile time. Then how can we solve the problem? You can solve this by using Either.

Of course, you can use Coproduct in shapeless

By this way, You can keep self from make mistake.

Play with Lagom (0) – What is (Not Completed)

As you can see obove, it’s not completed. As I study Lagom, I can say more about these topic. And fill it more!

These days, Internet Technology is growing up and there occur many buzzwords around. ‘Reactive’, ‘Micro Service’, ‘Event Sourcing’, ‘CQRS’… Many so called ‘Progressive Company’ cry out these buzzwords. And I also want to know about this. After moving to Kakao, I usually use Play, Akka, Finatra.(Personally, Finatra is a very good framework to make a productive micro server.) And looking forward to finding a very good solution to manage micro services in our service. And these series is a very short journey about Lagom.(Because I just want to taste Lagom. Of course, if I decide that it is the real solution to our service, it will be long journey 🙂 )

What is Reactive?

One of the hard waves around here is ‘Reactive’. You can read and sign to ‘Reactive Manifesto’ in here. Ok, after sign to Manifesto, Let’s summary what is Reactive System.

  • Reactive is Responsive. Service has to be responsive at all time.
  • Reactive is Resilience. Service has to be responsive at to time of failure occur.
    • It means that, if an error occurs, the error is isolated and not affect to other component or service.
  • Reactive is Elastic. Service has to be scaled elastically when system load.
    • It means that, if the service loaded, for example, Akka system assign more thread to the loaded actor.
  • Reactive is Message Driven. Reactive call remote system by targeting message.

Reactive Programming vs Reactive System.

This topic is not related to the main subject. But it is very important to distinguish between Reactive Programming and Reactive System. It is referenced from this article. It’s just summary about this article.

The reactive system is the way to build service. Above description about reactive is about Reactive system. Reactive system passes messages to the target. By message passing and mail box, we can assign more resources to some component(or micro server). We can scale horizontally or vertically.

The Reactive Programming is the way to handle events. We catch events from event stream and handle by chaining functions. We catch event compare to Reactive System which operated by message passing. Reactive programming is usually async,(In fact, have to be async) we can assign to another job which needs computing resource.

Reactive Programming is fit to Reactive System. Words are very awkward, But it means very natural. Reactive Programming is tightly coupled between Event and computing by chaining. And this makes less resilience. And Reactive system is key to resolve this problem.

What is CQRS?

CQRS is very simple. In service, we separate Command (Write) and Query. (Read) Martin Fowler describes it very well in this article.

it’s like read replica in SQL, Application request writes and read in different API or server.

cqrs
Martin Fowler – CQRS

As you can see, Service Interface(API Gateway) request read to Query Model, and write to Command Model.

What is Event Sourcing?

Event Sourcing is making event as a sequence of data. Let’s think like this. We make some data ‘a’ to 3 by adding 1 and 2. In common architecture, We only store that a is 3. But in Event Sourcing, we store that

  1. ‘add 1’ event occur,
  2. ‘add 2’ event occur.

So, we can get that a is 3. It means that we can get system service when a is 1 or a is 0. This can restore the service any time, any status.

Scala Implicitly

As I study about Scala, I open see implicitly. But, every time I see this, I just ignore just thinking like “It’s syntactic sugar of implicit”. This time, I write this post to really under stand this.

In Scala, implicitly is just this.


def implicitly[T](implicit e: T) = e

That’s all! implicitly is a function just get implicit e instance and return e. Then, what is an advantage of using implicitly?

Example 1

You can use implicit like this.

implicit val optionInt: Option[Int] = Some(1)
implicit val optionBoolean: Option[Boolean] = Some(true)

def getImplicitInt(implicit oInt: Option[Int]): Int = oInt.get
def getImplicitBoolean(implicit oBoolean: Option[Boolean]): Boolean = oBoolean.get

getImplicitInt
// res0: Int = 1
getImplicitBoolean
// res1: Boolean = true

Now, you can use just same by using implicitly


implicit val optionInt: Option[Int] = Some(1)
implicit val optionBoolean: Option[Boolean] = Some(true)

def getImplicitlyA[A: Option]: A = implicitly[Option[A]].get

getImplicitlyA[Int]
// res2: Int = 1
getImplicitlyA[Boolean]
// res3: Boolean = true

Implicitly can use in such situation.

Implicitly can use when you want to get implicit instance which have 1 type parameter.

As you can see above, by using ‘Implicitly[Option[Int]]’, you can get implicit instance ‘Option[Int]’. It has 1 type parameter Int.

Example 2

Then how about implicitly for the type which has no type parameter? For example, implicit can do like this.


implicit val string: String = "Hello"
implicit val boolean: Boolean = true
implicit val int: Int = 0

def getImplicitT[T](implicit t: T): T = t

getImplicitT[Int]
// res0: Int = 0

getImplicitT[String]
// res1: String = Hello

getImplicitT[Boolean]
// res2: Boolean = true

But, if you do the same with implicitly, errors occur.


def getImplicitlyT[T]: T = implicitly[T]

getImplicitlyT[Int]
getImplicitlyT[String]
getImplicitlyT[Boolean]

// compile error: not enough arguments for method implicitly: (implicit e: T)T. Unspecified value parameter e. def getImplicitlyT[T]: T = implicitly[T] ^

Yes, because I mentioned above, implicitly can be used the type which has a type parameter. Then how can we use implicitly in this situation? The answer is ‘make Id type’.


type Id[A] = A

def getImplicitlyT[T : Id]: T = implicitly[Id[T]]

getImplicitlyT[Int]
// res3: Int = 0

getImplicitlyT[String]
// res4: String = Hello

getImplicitlyT[Boolean]
// res5: Boolean = true

I make new type Id which has one type parameter. And Id[A] is same as A. So, you can use just like implicit

Context bound

Right side of function ‘getImplicitlyAt’ has type parameter(A : Option). It looks like type bound. But it is called Context bound. Type bound is ‘<:’.You can see type bound in Scala School. In context bound, it doesn’t mean that A is Option. Instead, A will be a type passing to Option.

Applicative in Scala Cats

I always heard about Applicative, but I don’t know what really is. So, I write this post to understand myself. Below picture is from adit.io and example is from mastering advanced Scala.

What is Applicative?

During programming, you will encounter this situation.

def getSome(a: Int): Option[Int] = Some(a)

def getNone(a: Int): Option[Int] = None

def add(a: Int, b: Int): Int = a + b

val aOpt = getSome(1)

val bOpt = getSome(2)

And I want to get aOpt and bOpt add together using add. But aOpt and bOpt is Option. So, the usual way to do is by using flatMap and map like this.


aOpt.flatMap(a => bOpt.map(b => add(a, b)))
// res0: Option[Int] = Some(3)

In here, we can use Applicative.

What is Applicative?

I don’t know that my understanding of Applicative is right. But I think, Applicative is

applicative_just.png

Very famous picture about Applicative.

In here there is a wrapped value: 2. And there is a wrapped function. We unwrapped both function and value, and process some operation and wrap it again! And we can also adapt these things to above example.


import cats.instances.option._
import cats.Applicative

Applicative[Option].map2(aOpt, bOpt)(add)
// res1: Option[Int] = Some(3)

We can also use like this.


import cats.syntax.all._

(aOpt |@| bOpt).map(add)

Monad extends Applicative


@typeclass trait Monad[F[_]] extends FlatMap[F] with Applicative[F] {
  override def map[A, B](fa: F[A])(f: A => B): F[B] =
  flatMap(fa)(a => pure(f(a)))
}

As you can see, Monad extends Applicative. And Applicative extends Functor. So, hierarchy is ‘Monad <: Applicative <: Functor’

Tip – Traverse

While studying with Mastering Advanced Scala, there are some tips in Applicative, So, I write down this. If there is a List[Int] and want all element adapt ‘getSome’ function, for example


val ints = List(1,2,3,4,5)
ints.map(getSome)
// res3: List[Option[Int]] = List(Some(1), Some(2), Some(3), Some(4), Some(5))

The result is ‘List(Some(1), Some(2), Some(3), Some(4), Some(5))’ and if you want to make ‘Some(List(1, 2, 3, 4, 5))’ you can use Traverse like this.


import cats.Traverse
import cats.instances.list._
import cats.instances.option._

Traverse[List].traverse(ints)(getSome)

// res4: Option[List[Int]] = Some(List(1, 2, 3, 4, 5))

This will return None if any element return None


def getSomeOrNone(a: Int): Option[Int] =
  if(a % 2 == 0) Some(a)
  else None

Traverse[List].traverse(ints)(getSomeOrNone)

// res5: Option[List[Int]] = None

 

Cartesian Product in Scala Cats

During work, I have some problem handling three future at the same time. First future return a long value and second and third future receive the parameter from the first return and return a tuple.

The code is like this.


for {
  long <- futureLong()
  int <- futureInt(long)
  char <- futureChar(long)
} yield (int, char)

Of course for comprehension operates step by step, so, futureInt is called after futureLong is finished and futureChar is called after futureInt is finished. So, It takes too much time to get a final return value.

So, I changed codes like this.


val longVal: Future[Long] = futureLong()

val intVal: Future[Int] = for {
  long <- futureLong()
  int <- futureInt(long)
} yield int

val charVal: Future[Char] = for {
  long <- futureLong()
  char <- futureChar(long)
} yield char

val forComp: (Int, Char) = Await.result(for {
  int <- intVal
  char <- charVal
} yield (int, char), 5 second)

Yes, It works as I want. But, there is duplicated codes like, ‘long <- futureLong()’. During thinking a long time, my coworker, Liam, recommends using  Cartesian Product.

What is Cartesian Product?

Cartesian Product is very simple. It’s just a product of two Sets. For example, if there are two sets A = {1,2,3} and B = {‘a’, ‘b’, ‘c’}, Cartesian Product of A X B is

{(1, ‘a’), (1, ‘b’), (1, ‘c’), (2, ‘a’), (2, ‘b’), (2, ‘c’), (3, ‘a’), (3, ‘b’), (3, ‘c’)}

Yes. that’s all. Of course Cartesian Product didn’t hold commutative law. For example Cartesian Product of B X A is

{(‘a’, 1), (‘a’, 2), (‘a’, 3), (‘b’, 1), (‘b’, 2), (‘b’, 3), (‘c’, 1), (‘c’, 2), (‘c’, 3)}

And different from A X B.

Cartesian Product in cats

Above example can be implemented in Scala.


import cats.instances.list._
import cats.syntax.all._

val ints: List[Int] = List(1, 2, 3, 4)
val chars: List[Char] = List('a', 'b', 'c')

(ints |@| chars).tupled
// res0: List[(Int, Char)] = List((1,a), (1,b), (1,c), (2,a), (2,b), (2,c), (3,a), (3,b), (3,c), (4,a), (4,b), (4,c))
(chars |@| ints).tupled
// res1: List[(Char, Int)] = List((a,1), (a,2), (a,3), (a,4), (b,1), (b,2), (b,3), (b,4), (c,1), (c,2), (c,3), (c,4))

(ints |@| chars).map(_ + _.toString)
// res2: List[String] = List(1a, 1b, 1c, 2a, 2b, 2c, 3a, 3b, 3c, 4a, 4b, 4c)

In Cats Cartesian Product Syntax is ‘|@|’.(Some people call it Oink, and others call Pizza box) If you call tuple, it will return List Tuple. You can also map it.

Cartesian Product in Future

Ok, let’s return to the previous problem. I need to call futureInt and futureChar at a same time. And using three for comprehension is too dirty. So, how and I solve this with Cartesian Product? It’s very simple.


for {
  long <- futureLong()
  (int, char) <- (futureInt(long) |@| futureChar(long)).tupled
} yield (int, char)

Very simple right? You can test whether this expression really operates at the same time.

In this test, the first test occurs TimeoutException and second and third pass test well. Then, we can make a conclusion that, last expression also run at the same time.