DEV Community: Riccardo Cardin

...And Monads for (Almost) All: The Reader Monad

Riccardo Cardin — Wed, 11 Mar 2020 11:29:33 +0000

Who follows me from the beginning perfectly knows my obsession for the dependency management in programming languages. I have already written of dependency injection mechanisms in modern programming languages. In Eat that cake! I wrote about the Cake Pattern and how to use Scala self types to resolve the problem of dependency injection. In Resolving your problems with Dependency Injection, I introduced the problem of the dependency resolution. In Resolve me, Implicitly, I showed how to use Scala implicits to implement a dependency injection mechanism. Now its time to speak about how functional programming tries to solve the dependency management issue, using the Reader Monad.

The problem

Imagine you have a type whose responsibility is to manage the persistence of stocks information. Let we call it StockRepository. The repository can retrieve all the stocks present in a wallet, can sell a quantity of a stock or can buy some amount of stock. It follows the definition of such a type.

trait StockRepository {
  def findAll(): Map[String, Double]
  def sell(stock: String, quantity: Double): Double
  def buy(stock: String, amount: Double): Double
}

The repository implements what we can call persistence logic.

Then, we have a type that uses the StockRepository to give to its clients some business logic built upon the above persistence logic. Let's call it Stocks. Imagine that we want to give access to the three functions of the repository, plus a fourth function that invest money in the stock that has the lowest quotation.

findAll()
sell(stock: String, quantity: Double)
buy(stock: String, amount: Double)
investInStockWithMinValue(amount: Double)

So, Stocks has a dependency upon StockRepository. How can we express such fact in the code? We don't want to use the constructor injection mechanism or anything related to it. We want to stay functional.

Dependency management within functions

The trivial solution

An option is to pass a reference of the repository to each function that need to access to its methods.

object Stocks {
  def findAll(repo: StockRepository): Map[String, Double] = repo.findAll()
  def sell(stock: String, quantity: Double, repo: StockRepository): Double = 
    repo.sell(stock, quantity)
  def buy(stock: String, amount: Double, repo: StockRepository): Double = 
    repo.buy(stock, amount)
}

This trick does its dirty job, but it pollutes the signature of each function that needs some external dependency. Our example has only one dependency, but in real life, dependencies are often more than one.

Using currying to isolate dependencies

The currying process can help us to make things a little better. Imagine isolating the dependency parameters using a curried version of the previous functions.

object Stocks {
  def findAll()(repo: StockRepository): Map[String, Double] = repo.findAll()
  def sell(stock: String, quantity: Double)(repo: StockRepository): Double = 
    repo.sell(stock, quantity)
  def buy(stock: String, amount: Double)(repo: StockRepository): Double = 
    repo.buy(stock, amount)
}

As you know, the currying process allows us to partially applied a function, obtaining as the result of the partial application a new function with fewer inputs.

In mathematics and computer science, currying is the technique of translating the evaluation of a function that takes multiple arguments into evaluating a sequence of functions, each with a single argument.

Let's take an example.

Let the function def add(a: Int, b: Int): Int = a + b that adds to integers. If we apply currying to the function add we obtain the following new function.

`def add(a: Int) = (b: Int) =>  a + b`

The return type of the function add is not anymore a pure Int but now it is a function from Int => Int.

If we apply the currying reasoning to the functions of the Stocks module, we obtain the following definition.

object Stocks {
  def findAll(): StockRepository => Map[String, Double] = repo => repo.findAll()
  def sell(stock: String, quantity: Double): StockRepository => Double = 
    repo => repo.sell(stock, quantity)
  def buy(stock: String, amount: Double): StockRepository => Double = 
    repo => repo.buy(stock, amount)
}

We remove the ugly StockRepository parameter from the signature of our function! Yuppi yuppi ya! However, it is complicated to compose functions with the last signature we had :( Imagine that we want to implement the function investInStockWithMinValue using the function we developed so far. A possible implementation is the following.

def investInStockWithMinValue(amount: Double): StockRepository => Unit =
  (repo: StockRepository) => {
    val investment = Stocks.findAll()
      .andThen(s => s.minBy(_._2))
      .andThen{ case (stock, _) => stock }
      .andThen(s => Stocks.buy(s, amount)(repo))
  }

It is not simple to follow what is going on. The use of the function andThen does not help the reader to understand the main workflow of the function, because it is not semantically focused on the operation it is carrying on. Moreover, in the last line, there is a very ugly function application, Stocks.buy(s, amount)(repo) that waste our code with a detail that is not related to the business logic but only to the implementation.

We can do better than this. Much better.

The Reader monad

What if we encapsulate the curried function inside a data structure? Using such an approach is precisely the idea behind the Reader Monad.

We have our function, let's say f: From => To, where From and To are respectively the starting type (domain) and the arriving type (codomain) of the function. As we just said, we put a data structure around our function.

case class Reader[From, To](f: From => To) { /* ... */ }

We want to apply in some way the function enclosed inside our data structure. We add the application function.

def apply(input: From): To = f(input)

To improve the readability of our code, we want to have a function different from andThen to compose a function f. Given a function g: To => NewTo, we need a function to compose g with f inside a Reader. This function is called map.

def map[NewTo](transformation: To => NewTo): Reader[From, NewTo] =
  Reader(c => transformation(f(c)))

The flatMap function composes f with functions z: To => Reader[From, NewTo]. This function is equal to the last application of the andThen method in our previous example.

def flatMap[NewTo](transformation: To => Reader[From, NewTo]): Reader[From, NewTo] =
  Reader(c => transformation(f(c))(c))

In other words, the flatMap function serves to compose two functions sharing the same dependency. In our example, using a flatMap, we can compose the functions findAll and buy both sharing the dependency among a StockRepository. Using a simple map, we would obtain the nesting of a Reader into another Reader.

val quantity: Reader[StockRepository, Reader[StockRepository, Double]] = 
  Stocks.findAll()
    .map { stocks => 
      val minStock = stocks.minBy(_._2)
      Stocks.buy(minStock, 1000.0D)
    }

Quite annoying. Using a flatMap, instead, we can flatten the result type, and everything goes ok. The function application in the flatMap definition does the trick.

val quantity: Reader[StockRepository, Double] = 
  Stocks.findAll()
    .flatMap { stocks => 
      val minStock = stocks.minBy(_._2)
      Stocks.buy(minStock, 1000.0D)
    }

Finally, we need an action to lift a value of type To in a Reader[From, To]. In other words, we want to be able to create from a value of type To a function that receives a value of type From and returns a value of type To. This function is not a member of the Reader monad itself. It is more like a factory method.

def pure[From, To](a: To): Reader[From, To] = Reader((c: From) => a)

The whole Reader type is something similar to the following.

object ReaderMonad {
  case class Reader[From, To](f: From => To) {
    def apply(input: From): To =
      f(input)

    def map[NewTo](transformation: To => NewTo): Reader[From, NewTo] =
      Reader(c => transformation(f(c)))

    def flatMap[NewTo](transformation: To => Reader[From, NewTo]): Reader[From, NewTo] =
      Reader(c => transformation(f(c))(c))
  }
  def pure[From, To](a: To): Reader[From, To] = Reader((c: From) => a)
}

It happens that the type Reader satisfies with its functions apply, map and flatMap the minimum properties needed to be a monad. The description of the monad laws is behind the scope of this post.

Moreover, because of the presence of the function map and flatMap, we can use the type Reader in a fashion way to simplify our code.

Finally, using the monad

First of all, we change the Stocks type with the Reader monad.

object Stocks {
  def findAll(): Reader[StockRepository, Map[String, Double]] = Reader {
    repo => repo.findAll()
  }
  def sell(stock: String, quantity: Double): Reader[StockRepository, Double] = Reader {
    repo => repo.sell(stock, quantity)
  }
  def buy(stock: String, amount: Double): Reader[StockRepository, Double] = Reader {
    repo => repo.buy(stock, amount)
  }
}

As you can see, every direct dependency was removed and substituted with the Reader[StockRepository, _] type.
Now, it's time to return to our previous investInStockWithMinValue function. Using the methods we defined on the monad, we can rewrite the function as follows.

def investInStockWithMinValue(amount: Double): Reader[StockRepository, Double] = 
  Stocks.findAll()
    .map(stocks => stocks.minBy(_._2))
    .map { case (stock, _) => stock }
    .flatMap(stock => Stocks.buy(stock, amount))

Using the syntactic sugar available from the Scala language, we can rewrite the above code use a for-comprehension statement.

def investInStockWithMinValueUsingForComprehension(amount: Double): Reader[StockRepository, Unit] =
  for {
    stocks <- Stocks.findAll()
    minStock <- ReaderMonad.pure(stocks.minBy(_._2)._1)
    _ <- Stocks.buy(minStock, amount)
  } yield ()

I love the for-comprehension construct because it is self-explanatory :)

Who is responsible for resolving the dependencies, declared through the Reader monad? The answer is simple, that is the main method.

def main(args: Array[String]): Unit = {
  val stockRepo = new StockRepository {
    override def findAll(): Map[String, Double] = Map("AMZN" -> 1631.17, "GOOG" -> 1036.05, "TSLA" -> 346.00)
    override def sell(stock: String, quantity: Double): Double = quantity * findAll()(stock)
    override def buy(stock: String, amount: Double): Double = findAll()(stock) / amount
  }
}

investInStockWithMinValueUsingForComprehension(1000.0D).apply(stockRepo)

What if I have more than one dependency?

Very often, we have functions that depend by more than one single dependency. For example, think that you want to add a rate change service to the functions of the Stock type. Using RateChangeService, it is possible to buy and sell in a currency that is different from dollars.

def buy(stock: String, amount: Double, currency: String)(repo: StockRepository, changer: RateChangeService) = {
  val dollarAmount = changer.changeToDollar(amount, currency)
  repo.buy(stock, dollarAmount)
}

The Reader monad we just analyzed handles only one dependency at time. Should we try away all the suitable types we developed until now? No, we shouldn't. If you depend on more than one type, you can create a new container type, something similar to a context.

case class Context(val repo: StockRepository, val changer: RateChangeService)

In this way, we reduce our function to depend on a single type again, our Context type. Ball, game, set.

def buy(stock: String, amount: Double): Reader[Context, Double] = Reader {
    ctx => {
      val dollarAmount = ctx.changer.changeToDollar(amount, currency)
      ctx.repo.buy(stock, amount)
    }
}

Conclusions

In this post, we analyzed how to declare dependencies in functions. We begin from the simplest possible solution, and we composed step by step a more elegant and practice solution that is called the Reader monad. Finally, we showed how the monad could simplify the code through the use of the for-comprehension construct.

The code of the Reader monad is available on my GitHub, reader-monad. I also developed a version of the monad in Kotlin, for the lovers of this emergent programming language, reader-monad-kotlin.

Rerefences

The Pimp My Library Pattern

Riccardo Cardin — Tue, 17 Dec 2019 08:05:58 +0000

Originally posted on: Big ball of mud

Which is the main problem you have as a developer when you use libraries that you don't own? You can't change them. If something is missing in the public API of a library, there is no chance to extend it. Using some excellent old object-oriented programming, you can overcome this problem by writing much boilerplate code. In the JVM ecosystem, modern programming languages, such as Scala, Kotlin or Groovy, try to give a solution to library extension, the Pimp My Library Pattern. Let's go, and see what I am talking.

The problem

Let's begin with a very extreme example. Imagine you want to add a method Integer type in Java that allows you to translate an int into an instance of the java.time.Period class. For whom that don't know this class, a Period represents a time, using days, months, years.

All that we want to achieve is having something like the following.

final Period days = Integer.valueOf(42).days();

In more evolved languages, such as Scala or Kotlin, the above statement would look like the following.

val days = 42.days;

In Java, you have no many such possibilities to achieve the goal. Since we cannot nor we want to modify directly the Integer type, and we do not want to use inheritance on a concrete type, the only remaining possibility is to implement a method somewhere that receives in input an Integer and returns a Period.

class Integer2Period {
    private Integer integer;
    Integer2Period(int integer) {
        this.integer = integer;
    }
    Period days() {
        return Period.ofDays(integer);
    }
}
var days = new Integer2Period(42).days();

Meh. We used a wrapper or some variance of the Object Adapter Pattern, but we are very far from the objective we originally had.

Let's see how modern JVM languages, such as Kotlin, Scala and Groovy answer this problem.

Scala

Scala was the language that first introduced the Pimp My Library pattern. The pattern was introduced by the Scala language's dad, Martin Odersky, in his article Pimp my Library, in the far 2006.

The pattern allows extending a type adding methods to it without using any form of inheritance. Using the pattern, we can add a method to the Int type without extending from it.

The Scala language implements the pattern through implicit conversions. First of all, we need to declare an implicit class that allows the compiler to convert our primary type into a new type that adds the method we want to have. In our case, the primary type is the Int type.

package object extension { 
  implicit class ExtendedInt(val integer: Int) extends AnyVal {
    def days = Period.ofDays(integer)
  }
}

Inside the package extension, or in any package, explicitly importing the package extension, we can use the method defined in the type ExtendedInt as a method defined for the Int type.

val days = 42.days;

The tricks that make the magic are two:

The declaration of the implicit type inside a package object forces the compiler to automatically import the ExtendedInt type in all the files that belong from it.
The class ExtendedInt is declared as implicit.
The class ExtendedInt is a subclass of the type AnyVal. From Scala 2.10, extending from AnyVal allows the compiler to perform some code optimisations. It's called Custom Value Classes.

If I don't interpret the result of the javap command wrongly on the .class files generated by the Scala compiler, Scala adopts a conversion to the bytecode of the implicit class similar to the wrapper approach I gave for Java.

public final class org.rcardin.extension.package$ExtendedInt {
  public int integer();
  public java.time.Period days();
  public int hashCode();
  public boolean equals(java.lang.Object);
  public org.rcardin.extension.package$ExtendedInt(int);
}

The method int integer() is a getter of a private attribute and the constructor of the class takes as input a variable of type int. The decompiled implicit Scala class has the same structure as the Java class Integer2Period.

The standard library extensively uses the pattern. All the type defined with the suffix Ops implement the pattern. Have a look at the StringOps type for an example.

Kotlin

Also, the newbie JVM-based language, Kotlin, has its implementation of the Pimp my library pattern. In Kotlin slang, the pattern implementation it's called Extension functions. The pattern was introduced in the language to contrast the fact that the majority of the libraries a Kotlin developer could use are in Java, and not in Kotlin.

The syntax needed to declare an extension function is less verbose than the syntax used in the Scala language ( :O ).

fun Integer.days(): Period = Period.ofDays(this)

In our example, the Integer type is also called the receiver type. Whereas, the this reference on the right of the assignment symbol is called the receiver object. The this reference refers to the integer instance on which the extension method is called. To preserve encapsulation, you can access only to the public methods of the receiver object.

The compiler does not import the extension methods by default. As any other Kotlin entity, you need to import them before using explicitly.

Under the hood, the compiler translates every extension method in a static method having the receiver object as its first parameter. The name of the enclosing class is equal to the name of the file that declares the extension function.

Suppose that we declared the Integer.days function in a file called IntegerUtil.kt, then the Kotlin compiler compile our code into a static method inside a class called IntegerUtilKt.

class IntegerUtilKt {
    public static Period days(Integer receiver) {
        return Period.ofDays(receiver);
    }
}

It's very similar to the solution we gave for the Java language.

The translation that the Kotlin compiler performs on extension functions allows us to call them also on nullable types. No method is called directly on the receiver object passed as the first parameter to a static method.

So, extension functions and the Kotlin type system allow us to declare something like the following.

fun String?.isNullOrBlank(): Boolean = this == null || this.isBlank()

You can safely use the above method in if-statements, to control if a nullable object contains a null reference or not.

val possiblyEmptyString: String? = // Obtaining the string reference
if (possiblyEmptyString.isNullOrBlank()) { // No NullPointerException!!!
    // Do something smart
}

Awesome. Kotlin continues to surprise me every day.

Conclusions

Sometimes a library contains almost all that you need, but it lacks some feature that you desire. Extension using the regular object-oriented mechanisms is not a possibility in such cases. Many JVM-based languages give you the possibility to achieve the goal to add the methods you need to an existing library without modifying it.

The Pimp my library pattern is the mechanism to make the magic happen. Scala uses implicit objects and conventions to implement such a pattern. In contrast, Kotlin has a more natural approach that integrates very well with the Kotlin type system concerning the handling of null references.

Moreover, let's say that also Groovy implements the pattern, using Extensions and Categories.

Where are you Java? Will you ever join the party?

If you want, download the code of the Scala example from my repository on GitHub: pimp-my-library.

References

Terminate a Spring Boot application that uses Kafka-Streams and MongoDB

Riccardo Cardin — Tue, 12 Nov 2019 09:05:29 +0000

Terminate a Spring Boot application that uses Kafka-Streams and MongoDB

Nov 12 '19 Comments: 1 Answers: 1

I have a Spring Boot application that uses Kafka-Streams. In details, there is a stream that filters the messages it receives using the results of a query performed in MongoDB. The code is something similar to the following.

final KStream<String, String> stream =
    kStreamBuilder.stream(Serdes.String(), Serdes.String(), inputTopic)
                  .filter((s, message) -> service.hasSomeProperty(message))

…

Open Full Question

Optional Is the New Mandatory

Riccardo Cardin — Mon, 07 Oct 2019 13:07:12 +0000

Originally posted on: Big ball of mud

Since the beginning of the "computer programming era", developers had searched for a solution for one of the biggest mistake made in computer science, the invention of the null reference. Since functional programming became mainstream, a solution to this problem seems to arise, the use of the optional type. In this post, I will analyze how this type is used in different programming languages, trying to understand the best practices to apply in each situation.

Returning the nothing

Let's start from a concrete example. Think about a repository that manages users. One classic method of such type is the method def findById(id: String): User, whatever an id could be. This method returns the user that has the given id, or nothing if no such user exists in your persistence layer.

How can we represent the concept of nothing? It is common to use the null reference to accomplish this task.

val maybeUser = repository.findById("some id")
val longName = maybeUser.name + maybeUser.surname

The problem is not in returning a null reference itself. The problem is using a reference containing a null. The above code, written in Scala, rises a NullPointerException during the access of the methods name and surname if the user reference is equal to null.

So, what's a possible solution? A first attempt can be to return a List[User]. The signature of the method becomes the following, def findById(id: String): List[User]. An empty list means that there is no
user associated with the given id. A not empty list means that there is a user associated with the given id. The problem is that the user is exactly one, whereas the semantic of lists if to store zero or more elements.

So using a list seems to be convenient, but not semantically correct. The use of an empty list to represent the nothing produces programs that are hard to read (violating the Principle of least astonishment) and to maintain.

However, the idea behind the use of lists is perfect. How can we refine it?

The Option type

The answer to our question is in the type that Scala calls Option. Other programming languages call this type in different ways, such as Optional in Java, or Maybe in Haskell.

The Option[T] type is just like a list that can be empty or can contain exactly one element. It has two subtypes that are None and Some[T]. So, an Option object that contains something is an instance of the Some[T] subtype, whereas an empty option is an instance of None. There is only one instance of the None type that in Scala is defined as an object.

Option is indeed an Algebraic Data Type, but the definition of such a concept is behind the scope of this post.

It is possible to use the factory method Option(value: T), declared in the Option object to build a new instance of an Option class. The factory method will create a new Option instance: If value is equal to null, then the None object is returned, Some(value) otherwise.

Using the Option type, our previous method changes its signature in def findById(id: String): Option[User]. Ok, interesting. However, how can I use the user value contained inside the option type? There are many ways to do that. Let's look at them.

Getting the value out of the option type

One of the possible choices is to try to get the value out of the option. The API of the Option type lists the methods isDefined and get. The first check if an option contains a value, and the second allows you to extract that value. The usage pattern raising from the use of the above methods is the following.

val maybeUser: Option[User] = repository.findById("some id")
var longName: String;
if (maybeUser.isDefined) {
  longName = maybeUser.get.name + maybeUser.get.surname
}

Despite the use of a var reference, the above pattern is to avoid. If you call the get method on a None object, you will obtain an error at runtime. So, you don't improve the code that much from the version that checks for the null directly.

For this reason, the Scala Option type provides a variant of the get method, which is getOrElse. This method allows you to provide a default value to return in case of an empty option. Imagine that our User type provides a gender: Options[String] method, which eventually returns the gender of a user. It is possible to use the following safer pattern using getOrElse.

val maybeUser: User = // Retrieving a user
val gender: String = maybeUser.gender.getOrElse("Not specified")

Using pattern matching

In Scala, it is possible to use pattern matching on Algebraic Data Types. The Option type is not an exception in this sense. Although it is not such idiomatic, this approach does what it promises us — raw and straightforward.

val maybeUser: Option[User] = repository.findById("some id")
val longName: String = maybeUser match {
  case Some(user) => user.name + user.surname
  case None => "Not defined"
}

Using pattern matching is a little more verbose than using the next approach I am going to show you, but it is still more elegant than checking the existence of a value using the isDefined method.

Using the Option type as a list

The suggested approach by the Scala community is to use the Option type, just as if it was a list. This approach takes advantage of the fact that the Option type is indeed a monad (also, in this case, explaining what a monad is in functional programming is behind the scope of this post).

The Option type, like lists, exposes a lot of useful methods to transform e manage the Option inner value, without even the need to extract it. Among the others, such methods are the map, flatMap and filter functions.

The map method allows you to transform the value owned by an Option object if any.

val maybeUser: Option[User] = repository.findById("some id")
val longName: Option[String] = maybeUser.map(user => user.name + user.surname)

The flatMap method allows you to compose functions that in turn, return an object of type Option.

val maybeUser: Option[User] = repository.findById("some id")
// Using a simple map function, we obtain an Option[Option[String]]...uhh, ugly!
val gender: Option[String] = maybeUser.flatMap(user => user.gender)

Finally, the filter method allows you to discard any value that does not fulfil a given predicate.

val maybeUser: Option[User] = repository.findById("some id")
val maybeNotARiccardo: Option[User] = maybeUser.filter(user => user.name != "Riccardo")

All the above methods, if invoked on a None object, return None automatically, preserving any method call concatenation.

Note that we can combine the use of the map, flatMap and filter method using for-comprehension, giving to our code a look more like Haskell.

val maybeUser: Option[User] = repository.findById("some id")
val maybeNotARiccardo: Option[User] = maybeUser.filter(user => user.name != "Riccardo")
val maybeTheGenderOfANoRiccardo: Option[String] = maybeNotARiccardo.flatMap(user => user.gender)

The above code can be rewritten using a simple for...yield expression.

val maybeTheGenderOfANoRiccardo: Option[String] =
  for {
    user <- repository.findById("some id")
    gender <- user.gender if user.name != "Riccardo")
  } yield (gender)

Last but not least, if you need to apply some side-effects if a value is present in an Option object, you can use the foreach method. This function cycles on the single value of the option, if present, and executes a procedure.

val maybeUser: Option[User] = repository.findById("some id")
maybeUser.foreach(user => println(s"$user.name $user.surname"))

Option type in other languages

The Option type is not present only in Scala. Many programming languages widely use it.

Java

Since Java 8, the language introduced the Optional<T> type in the mainstream JVM language. It is very similar to its Scala cousin.

Also, the Java API defines different factory methods to build a new instance of Optional: Optional.of which throws an exception if the given value is null, and Optional.ofNullable, that returns an Optional.empty in case of null value.

A small difference if the behaviour of the map method. If the function used by map returns a null value, the method interprets this as the Optional.empty value. The map method behaves like the flatMap method, in this particular case.

Optional<User> maybeUser = repository.findById("some id");
Optional<String> maybeUserName = maybeUser.map(user -> {
    if ("Riccardo".equals(user.getName()) {
        return user.getName();
    } else {
        return null;
    }
});

The above code returns an Optional.empty value if the name of the user is equal to the String "Riccardo".

Another interesting method of the Java Optional type is orElseThrow. This method allows us to choose which exception to rising in case of an empty Optional object.

It worth reporting the answer that Brian Goetz gave to the StackOverflow question, Should Java 8 getters return optional type?.

NEVER call Optional.get unless you can prove it will never be null; instead use one of the safe methods like orElse or ifPresent. In retrospect, we should have called get something like getOrElseThrowNoSuchElementException or something that made it far clearer that this was a highly dangerous method that undermined the whole purpose of Optional in the first place. Lesson learned.

Kotlin

The Kotlin programming language does not have built-in support for something similar to the Scala Option type or the Java Optional type. The reason why is mainly that the language support compile-time null checking.

In Kotlin every type has two variants. Continuing our previous example, Kotlin defines both the User, and the User? type. A reference of the first type cannot be null, whereas an instance of the User? type can also hold the null value. Any non-nullable type is a child type of the corresponding nullable type.

In other words, Kotlin attempts to solve the problem by forcing you to handle null references. Kotlin forces you to handle the exception or to take full responsibility. For example, you can't use the dereferencing operator, the . (dot), if you are managing nullable type. The following code won't even compile.

val maybeUser: User? = repository.findById("some id")
val longName = maybeUser.name + maybeUser.name

Kotlin tries to prevent NullPointerException for you. The safe call operator, ?. must be used in this case. It checks if the reference is null, and if it is not, it calls the method on it, propagates the null value otherwise.

val maybeUser: User? = repository.findById("some id")
val longName: String? = maybeUser?.name + maybeUser?.name

The power of the safe call operator shines with chained methods calls. In this case, it also reminds very strictly the use of the methods map and flatMap defined in the Option type.

val maybeGender: String? = 
  repository.findById("some id")?.gender

The above method is equal to the following Java counterpart.

String maybeGender = 
    repository.findById("some id")
              .flatMap(user -> user.getGender())
              .getOrElse(null);

If you don't mind about NPEs, you can always use the !!. operator, that does not perform any check on reference nullability.

Finally, also Kotlin defines the same behaviour of the orElse method in Scala or Java for nullable types, using the Elvis operator, ?:. If we want to return a particular value of the gender if something is null during the gender resolution process, we can proceed as the following code shows.

val maybeTheGender: String = 
  repository.findById("some id").?gender ?: "Not defined"

As you can see, the type of the last maybeTheGenderOfANoRiccardo variable is not nullable anymore, because the elvis operator eliminates the possibility for the reference to be null.

Conclusions

In this post, we analyzed a modern solution to deal with the null value and in general, the absence of information. We started to see how the Scala programming language defines Option[T] type, listing the best practice of its use. Then, we saw how the good old Java approaches to the problem, the Optional<T> type, concluding that it is very similar to the solution proposed in Scala.

Finally, we looked at a different solution, the one proposed by Kotlin. Kotlin does not have any optional type, because it tries to handle the nullability of references at compile-time. Although this approach is exquisite and as powerful as the approaches proposed in Scala and Java, it poorly composes with the other constructs used in functional programming. For this reason, Kotlin has its library for functional structures, such as the optional type, which is ARROW.

Bonus: Are you planning to use an option value as a parameter for a method? Please, don't do that (Why should Java 8's Optional not be used in arguments).

References

It's a Kind of Magic: Kinds in Type Theory

Riccardo Cardin — Fri, 22 Feb 2019 09:21:34 +0000

Originally posted on: Big ball of mud

I am not an expert of functional programming, nor type theory. I am currently trying to enter the world of Haskell after I mess around the Scala world for a while. When I read about Kinds, I remember I was surprised, and a little bit scared. Is it possible that types have types? So what does this mean? In this little introductory post, I will try to explain to myself what I understood during my journey through kinds in my Haskell travelling.

The beginning of all the sadness

To make things simple, we start from the Java programming language. In Java, you can define a class (a type), which take as input parameter another type. It is called a generic class. For example, think to the List<T> type. There is a type parameter T that one should provide to the compiler when dealing with a List to obtain concrete and usable types. Then, we have List<Integer>, List<String>, List<Optional<Double>>, and so on.

What about if we provide not a concrete type as the value of the type parameter? What about List<List>, for example? Well, the Java compiler transforms this type into the more safer List<List<Object>>, warning you that you are using List in a non-generic way. And yes, List<Object> is an awful concrete type.

So, it seems that the Java type system allows filling type parameters only with concrete types. Fair enough. What about the Scala programming language? Well, Scala introduces in the game a feature that is called Higher Kinded Types (HKT). HKTs allow us to define things like the following.

trait Functor[F[_]] {
  def map[A, B](fa: F[A])(f: A => B): F[B]
}

What does that strange symbol, F[_], mean? It means that Functor is something (a type class, indeed), which type parameter can be filled with any other generic type that accepts one and only one type parameter, i.e. List[T], Option[T], and so on.

The Scala vision of types comes from how the type system of Haskell was thought. Also in Haskell, you can define a functor.

class Functor f where  
    fmap :: (a -> b) -> f a -> f b

Different sysntax, same semantic.

So, in some programming languages, we need a way to distinguish between the available types. As we just saw, some types are concrete, and types that need some information to become concrete. Type theory comes in help to us, with Kinds, i.e. types of types.

However, first, we need to specify the small bricks we need to build the road through kinds' definition.

Types, and other stuff

Values

Starting from the beginning, we found values. Values are the information or data that our programs evaluate. Examples of values are things like 1, "hello", true, 3.14, and so on. Each value in a program belongs to a type. A value can be assigned to a name, that is called a variable.

Types

Types give information to our program about how to treat values at runtime. In other terms, they set contraints to values. Saying that a variable has type bool, e: bool, it means that it can have only two values, that is true or false.

So, a type is a compile-time constraint on the values an expression can be at runtime.

In statically typed programming languages, the definition of types for values is made at compile time. You must give to each value (or variable) a type while you are writing your code. In dynamically typed programming languages, the type of values is determined at runtime, that is during the execution of the program.

In functional programming languages, in which functions are first-order citizens, each function has an associated type too. For example, the function sum, that takes two integers and returns their sum, has type (Int, Int) => Int in Scala, or Int -> Int -> Int in Haskell.

Type constructors

As we previously said, there can be "types" that are parametric. Generic types uses type parameters to exploit this feature. So, we can define a "type" using a type parameter such as List[T], where T can be instantiated with any concrete type we want.

Using type parameters is an attempt to justify the fact that, let's say, the behaviour of a List[Int] is very similar to the behaviour of a List[String]. It is nothing more than a way to generify and abstract over types.

So, in the same way (value) constructors take as input a list of values to generate new values, type constructors take as input a type to create new types. List[T], Map[K, V], Option[T], and so on are all classified as type constructors. In Java, we call them generics. In C++, they call them templates.

Now, we have two different "kinds" of objects dealing with types: The former is represented by concrete types; The latter by type constructors. We have just said that we need to define the type of a type :O

Some Kind (of monsters)

Kinds are the types of objects that are related to types, more or less. A kind is more of an arity specifier. In their simplest form, we use the character * to refer to kinds.

* is the kind of simple and concrete types. A concrete type is a type that doesn't take any parameters.

What about type constructors? Let's take a step back. A (value) constructor is nothing more than a function that takes a tuple of values in input and returns a new value.

class Person(name: String, surname: String) = {
    //...
}

The above is nothing more than a function of type (String, String) => Person

Now, lift this concept to type constructors: Values are now represented by types, and (value) constructors by type constructors. Then, a type constructor is a function that takes in input a tuple of types and produces a new type.

Think about a list. It's definition is List[T]. So, it takes a type in input to produce a concrete type. It is similar to a function having only one parameter. List, Option and so on, has kind * -> *. Map[K, V] has kind * -> * -> *, because it needs two concrete types in input to produce a concrete type.

The fact that type constructors are similar to functions is underlined by the use of currying to model situations, in which more than one parameter takes place.

In Haskell, you can ask the kind of a type to the compiler, using the function :k.

ghci> :k List  
List :: * -> *

However, a question should arise into your mind: What the hell are kinds useful for? The answer is typeclasses.

Type Classes

A type class is a concept that not all the programming languages have. For example, it is present in Haskell, and (in some way) Scala, but not in Java.

A type class defines some behaviour, and the types that can behave in that way are made instances of that type class. So when we say that a type is an instance of a type class, we mean that we can use the functions that the type class defines with that type.

A type class is very similar to the concept of interface in Java. In Scala, it is obtained using a trait. In Haskell, there is a dedicated construct that is the class construct. In Haskell, there are type classes for a lot of features that a type could have: The Eq type class marks all the types that be checked for equality; The Ord type class marks all the types that can be compared; The Show type class is used by the types that can be pretty-printed in the standard output.

Our Functor that we defined earlier is, in fact, a type class. The fmap function defines the behaviour of the type of class.

Type classes are not native citizens of Scala, but they can be simulated quite well. The example below declares the type class Show.

trait Show[A] {
  def show(a: A): String
}

While in Haskell type classes have native support, so the compiler recognises them and generates automatically the binary code associated with them, in Scala you need to revamp some intricated mechanisms, involving implicits (see Type classes in Scala for more details on the topic).

So, to let a type to belong to a type class contained in the standard library, in Haskell you have simple to declare it in the type definition, using the deriving keyword.

data List a = Empty | Cons a (List a) deriving (Show, Read, Eq, Ord)

In general, to make a type to belong to a type class you have to provide the implementation of the methods (behaviours) of the type class. The following code shows a possible implementation of the Functor type class by the Maybe type (Option in Scala). As you can see, there are dedicated constructs of the language, as instance and where, to support type classes.

instance Functor Maybe where  
    fmap f (Just x) = Just (f x)  
    fmap f Nothing = Nothing

Anyway, describing abstract behaviour, type classes are inherently abstract too. The way the abstraction is achieved is using type parameters. Usually, type classes declare some constraints on the type represented by the type parameter.

The problem is that type classes can declare some strange type parameters. Reasoning on the kind of types, allows us to understand which type can be used to fulfil the type parameter.

Let's do a simple example. Looking back at the Functor type class definition, the type f must have a kind * -> *, which means that the type class requests types that take only one parameter. The function fmap applies f to a type a and a type b, which are concrete types in absence of any other indication. As previously said, Such kind of objects are similar to the List, Maybe (Option in Scala, and Optional in Java), Set type constructors.

What if we want to apply such type class to the Either type constructor? Both in Haskell and Scala, it is defined as Either[L, R], defining two type parameters, respectively left and right. For the definition we gave of a kind, Either has kind * -> * -> *. The kind of Either and of the type parameter requested by the Functor class are not compatible, so we need to partially apply Either type, to obtain a new type having kind * -> *.

For sake of completeness (and for those of you that know Haskell syntax), the partial application to make Either a member of Functor results in something like the following. Here, we mapped the case of the right type parameter.

instance Functor (Either a) where  
    fmap f (Right x) = Right (f x)  
    fmap f (Left x) = Left x

In conclusion, Kinds let us know how to use type classes, and type constructors, to obtain concrete type, sharing their behaviour.

Conclusions

Our journey through an edulcorated extract of type theory has finished. We saw many different concepts along the way. Many of these should be more detailed, but a post would not have been enough. We should have understood that we need kinds only when we have to manage Higher Kinded Types in Scala or Haskell (or in any programming language that provides a version of such types).

Kinds help you to understand which type can be a member of a type class. Long story short :)

References

...And Monads for (Almost) All: The State Monad

Riccardo Cardin — Mon, 26 Nov 2018 11:23:05 +0000

Originally posted on: Big ball of mud

This article starts a series on a topic that is very hot among the developers' community: functional programming. In details, we will focus on monads. This very complex object coming from the Category Theory is so important in functional programming that is very hard to program without it in this kind of paradigm. Its main aim is to make simpler the composition of pure functions, that are mathematical functions that cannot have any side effect. We start with a classic: The State Monad. As its name suggests, using this object, we will be able to manage the state of a program effectively, without breaking one of the main principles of the paradigm, immutability.

Disclaimer: I will not describe the very basics of functional programmings, such as immutability and referential transparency. The material I present has the primary objective to fix the concepts and the reasoning that are needed to understand the basic Monads.

I will use Scala, because I don't know Haskell (but, it is in my todo list) and because it is a language I love (coming from Java ecosystem). You can agree, or in disagreement with me. I don't mind ;)

The context

First of all, we need a program with some state to mutate during its execution. As an example, let's model a simple representation of a stock portfolio. A stock portfolio is a map that associate stock's names to some stocks owned.

type Stocks = Map[String, Double]

In our representation of reality, a client can perform three operations on her stock portfolio: To buy more stocks; To sell owned stocks; To get the number of stocks owned. As we know that in the functional programming object are immutable, the three functions cannot have any side effect on their input. So, all of the functions must also return the updated bank account.

// Buys an amount (dollars) of the stock with given name. Returns the number
// of purchased stocks.
def buy(name: String, amount: Double, portfolio: Stocks): (Double, Stocks) = {
  val purchased = amount / Prices(name)
  val owned = portfolio(name)
  (purchased, portfolio + (name -> (owned + purchased)))
}
// Sells a quantity of stocks of the given name. Returns the amount of
// dollars earned by the selling operation.
def sell(name: String, quantity: Double, portfolio: Stocks): (Double, Stocks) = {
  val revenue = quantity * Prices(name)
  val owned = portfolio(name)
  (revenue, portfolio + (name -> (owned - quantity)))
}
// Returns the quantity of stocks owned for name.
def get(name: String, portfolio: Stocks): Double = portfolio(name)

We imagine to live in a colored world, full of funny and unicorns, so our portfolio always contains some stocks for the given name.

The problem

Now that we defined our essential functions, we want to create another function that composes them to create a new use case:

Selling all the stocks for a given company
Using the sell's revenue, buying stocks of another company
Returning the number of stocks of the first type owned, and the quantity of the newly purchased stocks

Using nothing more than the functions get, sell, and buy, we can develop the new function as follows.

def move(from: String, to: String, portfolio: Stocks): ((Double, Double), Stocks) = {
    val originallyOwned = get(from)(portfolio)
    val (revenue, newPortfolio) = sell(from, originallyOwned)(portfolio)
    val (purchased, veryNewPortfolio) = buy(to, revenue)(newPortfolio)
    ((originallyOwned, purchased), veryNewPortfolio)
  }

Despite the simplicity of the above code, you will have undoubtedly noticed that we have to pass manually the correct reference to the updated stocks portfolio to every step of our algorithm. This kind of programming is very tedious and error prone.

We need a mechanism to compose in a smarter way functions that deal with the status of an application. The best would be not to have to worry at all about passing the status of our application from one function call to another.

Awkwardness like this is almost always a sign of some missing abstraction waiting to be discovered.

The State Monad helps us in this way.

The path through the Monad

Citing the book "Learn You a Haskell for Great Good!"

We’ll say that a stateful computation is a function that takes some state and returns a value along with some new state.

Our buy and sell functions already behave in this sense. They return both a value and a new instance of state. However, the function get returns only a value, without updating the state. No problem. It is easy to change its definition to let the function return a value and a state: Use the same input state. So the definition of the function get changes into the following.

def get(name: String, portfolio: Stocks): (Double, Stocks) = (portfolio(name), portfolio)

The type of the stateful computation quoted above is S => (A, S), where S is the state and A is the value resulting from the execution of the function. The functions of this type are called state actions of state transitions. Hence, we can rewrite the buy, sell, and get functions as follows.

First of all, let's use some currying to divide inputs into two groups, and to isolate the state.

def buy(name: String, amount: Double)(portfolio: Stocks): (Double, Stocks)
def sell(name: String, quantity: Double)(portfolio: Stocks): (Double, Stocks)
def get(name: String)(portfolio: Stocks): (Double, Stocks)

If we pass to these functions only the first group of inputs, we obtain precisely a set of functions of type S => (A, S).

def buyPartial(name: String, amount: Double): Stocks => (Double, Stocks) = 
  buy(name, amount)
// And so on...

The next step is to remove the currying at all. It is not elegant, and we don't like its syntax ;)

def buy(name: String, amount: Double): Stocks => (Double, Stocks) = portfolio => {
  val purchased = amount / Prices(name)
  val owned = portfolio(name)
  (purchased, portfolio + (name -> (owned + purchased)))
}
// And so on...

Just to improve the readability of our three functions, let's define the following type alias.

type Transaction[+A] = Stocks => (A, Stocks)

Our three functions always return a Double as output. However, to be more elastic, we need the type parameter A to let a function on Stocks type to returns any value.

With the new type alias, our functions become the following.

def buy(name: String, amount: Double): Transaction[Double] = portfolio => {
  val purchased = amount / Prices(name)
  val owned = portfolio(name)
  (purchased, portfolio + (name -> (owned + purchased)))
}
def sell(name: String, quantity: Double): Transaction[Double] = portfolio => {
  val revenue = quantity * Prices(name)
  val owned = portfolio(name)
  (revenue, portfolio + (name -> (owned - quantity)))
}
def get(name: String): Transaction[Double] = portfolio => {
  (portfolio(name), portfolio)
}

// And finally...
def move(from: String, to: String): Transaction[(Double, Double)] = portfolio => {
  val (originallyOwned, _) = get(from)(portfolio)
  val (revenue, newPortfolio) = sell(from, originallyOwned)(portfolio)
  val (purchased, veryNewPortfolio) = buy(to, revenue)(newPortfolio)
  ((originallyOwned, purchased), veryNewPortfolio)
}

Well, the situation has improved for nothing. What we need now is a better way to compose our functions that we can obtain making the last step through the definition of the State monad: The definition of map and flatMap functions.

The final step: combinator functions

Well the power of programming is composition, the ability to combine different operations to obtain our target result.

The final step we need to do to build our version of the State monad is to define a set of functions that let us combine states smartly, i.e. Transaction[+A] instances.

The first function we need is map. As you may already know, the map function allows applying a function to the type contained in a generic data structure, obtaining a new version of it.

In our case, the map function is used to transform a Transaction[A] in a Transaction[B].

def map[A, B](tr: Transaction[A])(f: A => B): Transaction[B] = portfolio =>
  (a, newPortfolio) = tr(portfolio)
  (f(a), newPortfolio)

However, the problem with the map function is that it tends to accumulate the generic type. For example, the result of the repeated application of map to a Transaction[_], map(/*... */)(a => map(/*...*/)(/*...*/) is Transaction[Transaction[_]]. So, we need a function that behaves like map, but that can flatten the accumulated types: The flatMap function. Instead of taking a function from A to B as a parameter, the flatMap function takes a function from A to Transaction[B] as a parameter.

Remembering the type Transaction[+A] = Stocks => (A, Stocks), the flatMap function is defined indeed in this way.

def flatMap[A,B](tr: Transaction[A])(f: A => Transaction[B]): Transaction[B] = portfolio =>
  (a, newPortfolio) = tr(portfolio)
  f(a)(newPortfolio)

Very well. The last step we miss is to define a function that lifts a value of type A to the type Transaction[A]. Think about this function as a factory method in Object-Oriented Programming.

def apply[A](value: A): Transaction[A] = portfolio => (A, portfolio)

Using the combinators we just defined, we can rewrite the move method more cleanly.

def move(from: String, to: String): Transaction[(Double, Double)] =
  flatMap(get(from))(
    originallyOwned => flatMap(sell(from, originallyOwned))(
      revenue => map(buy(to, revenue))(
        purchased => (originallyOwned, purchased)
      )
    )
  )

The presence of the map and the flatMap functions allows us to improve further the readability of the code of the move function, just applying the syntactic sugar of the for-comprehension construct. With the for...yield syntax, the function becomes the following.

def move(from: String, to: String): Transaction[(Double, Double)] =
  for {
    originallyOwned <- get(from)
    revenue <- sell(from, originallyOwned)
    purchased <- buy(to, revenue)
  } yield {
    (originallyOwned, purchased)
  }

Wow! Now, the code looks like the good imperative code, but it maintains all the remarkable feature of functional code. Supercalifragilisticexpialidocious!!!

Summing up

It's time to abstract the above concepts into our monad construct. First of all, we need a type for our monad. Our stocks example uses the type Stocks to store the state of the application. We need to substitute it with a generic type variable, S. Transaction[A] abstracts to type State[S, A].

type State[S, A] = S => (A, S)
// And so...
type Transaction[A] = State[Stocks, A]

Once we defined the State[S, A] type, the unit, map, and flatMap functions becomes the following.

def unit[S, A](a: A): State[S, A] = state => {
  (a, state)
}
def map[S, A, B](sm: State[S, A])(f: A => B): State[S, B] = state => {
  val (value, newState) = sm(state)
  (f(value), newState)
}
def flatMap[S, A, B](sm: State[S, A])(f: A => State[S, B]): State[S, B] = state => {
  val (value, newState) = sm(state)
  f(value)(newState)
}

So, this is our State Monad! Awesome! Moreover, the definition of type Transaction regarding the type State allows us to maintain the code of our business function as it is. No change is needed.

If you prefer, you can define a trait for the type State. If so, the functions unit, map, and flatMap become methods of the trait, instead of lonely functions.

Reinventing the wheel

There are a lot of good functional libraries out of there for the Scala language. Both Scalaz and Cats have their implementation of the State Monad. So, the above is only an exercise to acquire a deeper understanding of the State Monad concepts, but it is not ready for production use ;)

Conclusions

In our journey, we started from an application that needs to share some state. Many functions modify this state. Following the functional programming constraints, we tried to avoid the use of mutable state. Using curryfication and first-order functions, we built a framework that allowed us to share the state between different functions. Then, we introduced some combinators, to make the process of composing the previous functions in a more natural, and less error-prone way.

Our final gratification was the definition of the State Monad.

The code I used in this post is available in my GitHub repository, state-monad-example

References

The Secret Life of Objects: Inheritance

Riccardo Cardin — Mon, 06 Aug 2018 15:43:30 +0000

Originally posted on: Big ball of mud

This post is the second part of a series of posts concerning the basic concepts of Object-Oriented Programming. The first post has focused on Information Hiding. This time, I focus on the concept of Inheritance. Whereas most people think that it is easy to understand, the inheritance is a piece of knowledge that is very difficult to master correctly. It subtends a lot of other object-oriented programming principles that are not always directly related to it. Moreover, it is
directly related to one of the most potent forms of dependency between objects.

Out of there, many articles and posts rant against Object-Oriented Programming and its principles, just because they don't understand them. One of these posts is Goodbye, Object Oriented Programming, which treats inheritance and the others object-oriented principles with a very biased and naive eye.

So, let's get this party started, and let's demystify the inheritance in object-oriented programming once for all.

The definition

A good point where to start to analyze a concept is its definition.

Inheritance is a language feature that allows new objects to be defined from existing ones.

Given that we have a class for each object

an object's class defines how the object is implemented, i.e. the internal state and the implementation of its operations. [..] In contrast, an object's type only refers to its interface, i.e. the set of requests to which it can respond.

So,

It is important to understand the difference between class inheritance and interface inheritance (or subtyping). Class inheritance defines an object's implementation in terms of another object's implementation. In short, it's a mechanism for code sharing. In contrast, interface (or subtyping) describes when an object can be used in place of another.

The big misunderstanding

By now, we understood there are two types of inheritance: Class inheritance and interface inheritance (or subtyping). When you used the former, you're performing code reuse, that is you know a class as a piece of code you need, and you are extending from it to reuse that code, redefining all the stuff you don't need.

class AlgorithmThatReadFromCsvAndWriteOnMongo(filePath: String, mongoUri: String) {
  def read(): List[String] = {
      // Code that reads from a CSV file
  }
  def write(lines: List[String]): Unit = {
      // Code that writes to MongoDb
  }
}
class AlgorithmThatReadFromKafkaAndWriteOnMongo(broker: String, topic: String, mongoUri: String)
  extends AlgorithmThatReadFromCsvAndWriteOnMongo(null, mongoUri) {

  def read(): List[String] = {
      // Code that reads from a Kafka topic
  }
}
class AlgorithmThatReadFromKafkaAndWriteOnMongoAndLogs(broker: String, topic: String, mongoUri: String, logFile: String)
  extends AlgorithmThatReadFromKafkaAndWriteOnMongo(broker, topic, mongoUri) {

  def write(lines: List[String]): Unit = {
      // Code that writes to MongoDb and to log file
  }
}

Clearly, the class AlgorithmThatReadFromKafkaAndWriteOnMongo inherits from AlgorithmThatReadFromCsvAndWriteOnMongo only to reuse the code that writes on MongoDB. The drawback is that we have to pass null to the constructor of the parent class. Something is not good here.

The banana, monkey, jungle problem

The problem with object-oriented languages is they’ve got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.

This quote is by Joe Armstrong, the creator of Erlang. For this principle, every time you try to reuse some class, you need to add dependencies also to its parent class, and to the parent class of the parent class, and so on.

Using the previous example, if you want to reuse the class AlgorithmThatReadFromKafkaAndWriteOnMongoAndLogs, you also need to import classes AlgorithmThatReadFromCsvAndWriteOnMongo and AlgorithmThatReadFromKafkaAndWriteOnMongo

Well. The above is the problem of class inheritance. The above is the problem of code reuse.

As discussed in the post Dependency, class inheritance is the worst form of dependency between two classes. We already related the concept of dependency between classes and the probability of modifying one against one change in another.

It is normal that if two classes are linked through this type of relation, they must be used together. This fact is the problem of inheritance: Tight coupling.

First of all, do not use class inheritance. Do not use inheritance to reuse some code. Use inheritance so share behavior among components.

Afterward, do not put more than one responsibility inside a class. I don' t dwell on what responsibility is. I already wrote about this topic in Single-Responsibility Principle done right. However, if you let your components to implement more than one use case; If you let two different clients of a component to depend from different slices of it, then the problem is not inheritance, it is in your whole design. A component that implements only one responsibility is very likely to inherit from a hierarchy that contains only one type, which is most of the times an abstract type.

Concerning our previous example, there is a problem of code reuse and a violation of the Single-Responsibility Principle. Each class is doing too much, and it is doing in the wrong way.

Inheritance and encapsulation

If you think about it, there is a big problem with class inheritance: It seems to break encapsulation. A subclass (a class that inherits from another) knows and can virtually access to the internal state of the base class. We are breaking encapsulation. So, we are violating the first principle of object-oriented programming, are not we?

Well, not properly. If a class you inherit from exposes some protected state, it is like that class is exposing two kinds of interfaces.

The public interface lists what the general client may see, whereas the protected interface lists what inheritors may see.

The problem is that maintaining two different interfaces of the same type is very hard. Adding more types of clients to a class means to add dependencies. The more dependencies, the higher the coupling. A high coupling means a higher probability of changes cascades among types.

So, don't use class inheritance. Stop.

If you can't use inheritance, why is this considered one of the essential principles of object-oriented programming? Technically, inheritance is still possible under certain circumstances.

Subtyping or reusing behavior

The first case in which you are allowed to use inheritance is subtyping, or behavior inheritance, or interface inheritance.

Class inheritance defines an object's implementation in terms of another object's implementation. In short, it's a mechanism for code and representation sharing. In contrast, interface inheritance (or subtyping) describes when an object can be used in place of another.

The above sentence is very informative. It screams to the world that you must not override methods of superclasses if you want to reuse behavior.

Following this principle, the only types from which we can inherit are interfaces and abstract classes, avoiding to override any concrete method of the latter.

When inheritance is used carefully (some will say properly), all classes derived from an abstract class will share its interface. [..] Subclasses merely adds or overrides operations and does not hide the operations of the parent class.

Why is this principle so important? Because of polymorphism depends on it. In this way, clients remain unaware of the specific type of objects they use, reducing implementation dependencies drastically.

Program to an interface, not an implementation.

Favor object composition over class inheritance

The only way we have to extend the behavior of a class is to use object composition. Obtain new functionalities by assembling objects to get more complex functionalities. Objects are composed using their well-defined interfaces solely.

This style of reuse is called black-box reuse. No internal details of objects are visible from the outside, producing the lowest possible dependency degree.

Object composition has another effect on system design. Favoring object composition over class inheritance helps you keep each class encapsulated and focused on one task, on a single responsibility.

Our initial example can be rewritten using two new highly specialized types: a Reader, and a Writer.

trait Reader {
  def read(): List[String]
}
trait Writer {
  def write(lines: List[String]): Unit
}
class CsvReader(filePath: String) {
  def read(): List[String] = {
      // Code that reads from a CSV file
  }  
}
class MongoWriter(mongoUri: String) extends Writer {
  def write(lines: List[String]): Unit = {
      // Code that writes to MongoDb
  }  
}
class KafkaReader(broker: String, topic: String) extends Reader {
  def read(): List[String] = {
      // Code that reads from a Kafka topic
  }  
}
class LogWriter(logFile: String) extends Writer {
  def write(lines: List[String]): Unit = {
      // Code that writes to a log file
  }  
}
class AlgorithmThatReadFromCsvAndWriteOnMongo(val filePath: String, val mongoUri: String) {
  def read(): List[String] = {
      // Code that reads from a CSV file
  }
  def write(lines: List[String]): Unit = {
      // Code that writes to MongoDb
  }
}
// Composing the above classes into this class, we can read and write from and 
// to whatever we want.
class Migrator(reader: Reader, writers: List[Writer]) {
  val lines = reader.read()
  writers.foreach(_.write(lines))
}

The Liskov Substitution Principle

It seems that inheritance should not be used in any case. You must not inherit from a concrete class, only from abstract types. Is it correct? Well, there is a specific case, in which the inheritance from concrete classes is allowed.

The Liskov Substitution Principle (LSP) tells us exactly which is this case.

Functions that use pointers or references to base classes must be able to use objects of derived classes without knowing it.

A class can inherit from another concrete class if and only if it does not override the pre e postcondition of the base class. In a derivated class, preconditions must not be stronger than in the base class. In a derivate class, postconditions must be stronger than in the base class.

When redefining a routine [in a derivative], you may only replace its precondition by a weaker one, and its postcondition by a stronger one.

The above is called design by contract. Respecting this principle avoids the redefinition of the extrinsic public behavior of a base class, that is the behavior the clients of a class depend upon.

Returning to our previous example, take the class AlgorithmThatReadFromCsvAndWriteOnMongo and its subclass AlgorithmThatReadFromKafkaAndWriteOnMongo. The LSP tells us that whenever we need a reference to the first, we can use a reference to the second instead. However, in our example, we can't. The first read from a CSV; the second from Kafka.

It's easy to write a test that uses an object of type AlgorithmThatReadFromKafkaAndWriteOnMongo where an
object of type AlgorithmThatReadFromCsvAndWriteOnMongo is requested. It's easy to see it fail.

Why? The reason is that we have violated base class invariants. We tried to reuse code, and not to reuse behavior.

The same, old story.

Conclusion

To sum up: try to avoid the reuse of code. Avoid class inheritance, if possible. Try to reuse behavior. Try to use subtyping. Prefer not to extend a concrete class. Alternatively, if you have to, verify that you fulfill the Liskov Substitution Principle.

If you follow these simple rules, the dependency degree between your classes will stay low, and your architecture will be simpler to maintain and evolve.

Simple. As. F**k.

References

The Secret Life of Objects: Information Hiding

Riccardo Cardin — Mon, 18 Jun 2018 19:37:27 +0000

Originally posted on: Big ball of mud

I've been in the software development world for a while, and if I understood a single thing is that programming is not a simple affair. Also, object-oriented programming is even less accessible. The idea that I had of what an object is after I ended the University is very far from the idea I have now. Last week I came across a blog post Goodbye, Object Oriented Programming. After having read it, I fully understand how easily object-oriented programming can be misunderstood at many levels. I am not saying that I have the last answer to the million dollar question, but I will try to give a different perspective of my understanding of object-oriented programming.

Introduction

Probably, this will be the harder post I have ever written by now. It is not a simple affair to reason to the basis of object-oriented programming. I think that the first thing we should do is to define what an object is.

Once, I have tried to give a definition to objects:

The aim of Object-oriented #programming is not modeling reality using abstract representations of its component, accidentally called "objects". #OOP aims to organize behaviors and data together in structures, minimizing dependencies among them.
— Riccardo Cardin (@riccardo_cardin ) May 3, 2018

Messages are the core

In the beginning, there was procedural programming. Exponents of such programming paradigm are languages like COBOL, C, PASCAL, and more recently, Go. In procedural programming, the building block is represented by the procedure, which is a function (not mathematically speaking) that takes some input arguments and could return some output values. During its evaluation, a procedure can have side effects.

Data can have some primitive form, like int or double, or it can be structured into records. A record is a set of correlated data, like a Rectangle, which contains two primitive height and length of type double. Using C notation, the definition of a rectangle is the following.

struct Rectangle {
   double   height;
   double   length;
};

Despite inputs and outputs, there is no direct link between data (records) and behaviors (procedures). So, if we want to model all the operations available for a Rectangle, we have to create many procedures that take it as an input.

double area(Rectangle r)
{
    // Code that computes the area of a rectangle 
}
void scale(Rectangle r, double factor)
{
    // Code that changes the rectangle r, mutating its components directly
}

As you can see, every procedure insists on the same type of structure, the Rectangle. Every procedure needs as input an instance of the structure on which executes. Moreover, every piece of code that owns an instance of the Rectangle structure can access its member values without control. There is no concept of restriction or authorization.

The above fact makes the procedures' definition very verbose and their maintenance very tricky. Tests become very hard to design and execute, because of the lack of information hiding: everything can modify everything.

The primary goal of object-oriented programming was that of binding the behavior (a.k.a. methods) with the data on which they operate (a.k.a., attributes). As Alan Kay once said:

[..] it is not even about classes. I'm sorry that I long ago coined the term "objects" for this topic because it gets many people to focus on the lesser idea. The big idea is "messaging" [..]

The concept of classes allows us to regain the focus on behavior, and not on methods inputs. You should not even know the internal representation of a class. You only need its interface. In object-oriented programming, the above example becomes the following class definition (I choose to use Scala because of its lack of ceremony).

trait Shape {
  def area: Double
  def scale(factor: Double): Shape
}
case class Rectangle(height: Double, length: Double) extends Shape {
  // Definition of functions declared abstract in Shape trait
}

The example given is very trivial. Starting from elements height, length and procedures scale and area, it was very straight to derive an elegant object-oriented solution. However, is it possible to formalize (and, maybe to automate) the process we just did to define the class Rectangle? Let's try to answer this question.

Information hiding and class definition

We can begin with an unstructured set of procedures.

def scale(height: Double, length: Double, factor: Double): (Double, Double) = {
  (height \* factor, length \* factor)
}
def area(height: Double, length: Double): Double = {
  height * length
}

First of all, we notice that height and length parameters are present in both procedures. We might create a type for each parameter, like Height and Length. However, we immediately understand that the two parameters are always used together in our use cases. There are not procedures that use only one of the two.

So, we decided to create a structure to bind them together. We call such structure Rectangle.

type Rectangle = (Double, Double)

We also understand that a simple structure does not fit our needs. Rectangle internal should not be changed by anything else than the two procedures (forget for a moment that tuples are immutable in Scala). Telling the truth, we are interested only in the two procedures. So, we restrict the access to rectangle information only to the two procedures.

How can we do that? We should bind information of a rectangle with the behaviors associated with it. We need a class.

case class Rectangle(height: Double, length: Double) {
  def scale(factor: Double): Rectangle = Rectangle(height * factor, length)
  val area: Double = height * length
}

Well, taking into consideration only the use cases we have, we could stop here. The solution is already optimal. We hid the information of height and length behind our class; the behavior is the only thing client can access from the outside; clients that want to use a rectangle can interact only with the interface of the class Rectangle.

What if we want to support also shapes like squares and circles? Well, through the use of intefaces, that are types that are pure behavior, object-oriented programming allows our clients to abstract from the concrete implementation of a shape. Then, the above example becomes the following.

trait Shape {
  def scale(factor: Double): Shape
  def area: Double
}
case class Rectangle(height: Double, length: Double) extends Shape {
  // Definition of functions declared abstract in Shape trait
}
case class Square(length: Double) extends Shape {
  // Definition of functions declared abstract in Shape trait
}
case class Circle(ray: Double) extends Shape {
  // Definition of functions declared abstract in Shape trait
}

As Wikipedia reminds us

Information hiding is the principle of segregation of the design decisions in a computer program that is most likely to change, thus protecting other parts of the program from extensive modification if the design decision is changed. The protection involves providing a stable interface which protects the remainder of the program from the implementation (the details that are most likely to change).

Information hiding and dependency degree

As anyone who follows me from some time knows, I am a big fan of dependency degree minimization between classes. I have developed a small theoretical framework that allows calculating the dependency degree of architecture. This framework is based on the number of dependencies a class has with other classes and on the scope of these dependencies, concerning a class life cycle.

I had already used my framework in other circumstances, like when I spoke about the Single-Responsibility Principle.

This time I will try to use it to sketch the process we just analyzed, whose goal is to aggregate information and related behaviors inside the same class, hiding the former to the clients of the class. I will try to answer the question _Why do height and length be collapsed inside one single type (which is incidentally called Rectangle)

Just as a recap, I defined in the post Dependency the degree of dependency between classes A and B as

is the quantity of code (i.e., SLOC) that is shared between
types A and B. is the total number of code (i.e. SLOC) of
B class. Finally, is a factor between 0 and 1 and the wider
the scope between A and B, the greater the factor.

If Height and Lenght had been defined as dedicated types each, then a client C that needed to use a rectangle would always have to use both types. Moreover, the Rectangle type would still have been necessary, to put the methods area and scale. Using this configuration, Rectangle, Height and Length are said to be tightly coupled because they are always used together.

The degree of dependency of class C would be very high, using the above definition. Also, the degree of dependency of class Rectangle would be high too, due to references to Height and Lenght in the methods area and scale.

Probably, many class configurations can reduce the degree of dependency of the above example. However, the minimization of the value can be reached by the solution we gave in the previous paragraph.

So, in some way, we started to trace a new way of design architecture, reducing in some way the art of design, to find an architecture that minimizes a mathematical function on the degree of dependency. Nice.

Conclusion

To sum up, using a toy example (I admit it) we tried to sketch the informal process that should be used to defined types and classes. We started from the lacks we find in procedural programming paradigm, and we ended up trying to give a mathematical formulation of this process. All the ideas we used are tightly related to the basic concept of information hiding, that we confirmed to be one of the most essential concepts in object-oriented programming.

This post is just my opinion. It was not my intention to belittle procedural programming, not to celebrate object-oriented programming.

References

Template Method Pattern Revised

Riccardo Cardin — Fri, 06 Apr 2018 14:12:12 +0000

Originally posted on: Big ball of mud

When I started programming, there was a design pattern among all the others that surprised me for its effectiveness. This pattern was the Template Method pattern. While I proceeded through my developer career, I began to understand that the inconsiderate use of this pattern could lead to a big headache. The problem was that this pattern promotes code reuse through class inheritance. With functional programming became mainstream, this pattern can be revised using lambda expressions, avoiding any inheritance panic.

The original pattern

It's the year 2004. Martin Fowler had just published one of its most popular post Inversion of Control Containers and the Dependency Injection pattern (IoC). The pattern is a concretization of the famous Hollywood Principle, which states

Don't call us, we'll call you

Every Java framework in those years implements that principle: Struts, Spring MVC, Hibernate, and so on. However, the IoC was not a fresh new idea. It took its roots from a well-known design pattern of the Gang of Four, the Template Method Pattern.

The intent of the pattern is the following.

Define the skeleton of an algorithm in an operation, deferring some steps to subclasses. Template Method lets subclasses redefine certain steps of an algorithm without changing the algorithm's structure.

Uhm, it looks promising. I think it is better to make a concrete example. The example is taken directly from the GoF's book.

Consider an application framework that provides Application and Document classes. The Application class is responsible for opening existing documents stored in an external format, such as a file. Whereas, a Document object represents the information in a document once it's read from the file.

So, the objective is to create a structure with these classes that enable to add easily new kinds of documents.

In our example, the open algorithm is made of the following tasks.

Check if the document can be opened
Create an in-memory representation of the document
Eventually, make preliminary operations
Read the document

Translating this into some dirty code, we obtain the following class.

trait Application {
  // Template method
  def openDocument(fileName: String): Try[Document] = {
    Try {
      if (canOpen(fileName)) {
        val document = create(fileName)
        aboutToOpen(document)
        read(document)
        document
      }
    }
  }
  // Hook method
  def aboutToOpen(doc: Document) = { /* Some default implementation */ 34}
  // Primitives operations
  def canOpen(fileName: String): Boolean
  def create(fileName: String): Document
  def read(doc: Document)
}

As you can see, the method openDocument defines a template of the algorithm needed to open a document. For this reason, it is called a template method. Whereas, all other methods are abstract or provide defaults implementation for a task. The formers are called primitive operations; The latter is called hook operation, instead.

Implementing an application that can open CSV file means to extend the Application trait, implementing correctly the methods that were left abstract.

class CsvApplication() extends Application {
  def canOpen(fileName: String): Boolean = { /* Some implementation */ }
  def create(fileName: String): Document = { /* Some implementation */ }
  def read(doc: Document) = { /* Some implementation */ }
}

Drawbacks

One of the main consequence of the original Template Method pattern is code reuse. In this blog, we don't like code reuse. In the Dependency post, we learned that we should avoid code reuse in favor of behaviour reuse. Code reuse leads to an increase of dependency between classes, resulting in architectures, which components are tightly coupled.

Bad. Really, bad.

Moreover, the abuse of the Template Method pattern tends to generate into deep hierarchies of concrete classes. Let's return to our example. One day, your boss asks you to extend the above code to handle more than one type of storage. He wants the software to be able to read both from local filesystem and the cloud, or from distributed filesystems like HDFS.

Easy, you think. We already have a primitive method, create (read is reserved to unmarshal bytes into object-oriented structures), that I can override to allow my program to read from any storage. You are a smart boy.

class CsvApplication() extends Application { /* ... */ }
class CsvOnHDFSApplication() extends Application { 
  /* ... */ 
  override def create(fileName: String): Document = { /* Some implementation */ }
}
class CsvOnCloudApplication() extends Application { 
  /* ... */ 
  override def create(fileName: String): Document = { /* Some implementation */ }
}
// And so on...

At some point, you need to change the canOpen in the CsvApplication. Unfortunately, the new implementation does not fit the behavior in the CsvOnCloudApplication class. Then, you need to override also the canOpen method in this class. One day, you will have to change the implementation of another primitive method in some other class in your hierarchy. And the story will repeat over and over again.

Do you see the problem? You miss the maintainability of your classes. You do not know which behavior is implemented in which class. Every time you modify a class, it is difficult to predict which other classes you need to change.

What can we do at this point? Which alternatives do we have?

Favor object composition over class inheritance

The above motto, coined by the GoF, remembers us one of the principles that should guide us every time we develop something using an object-oriented programming language.

Template Method pattern: the right way

It is not difficult to image how we are going to change the original pattern to use composition, instead of inheritance. We are going to extract all the behaviors enclosed in primitive methods into dedicated classes. So, in our example, we can identify the following types.

// Subtypes of this trait give access to different types of storages
trait Storage {
  def openDocument(fileName: String): Try[Document]
  def canOpen(fileName: String): Boolean
  def create(fileName: String): Document
}
// Different types of file format can be read simply defining 
// subtypes of this trait
trait Reader {
  def read(doc: Document): Document
}

In this way, our original Template type becomes the following.

class Application(storage: Storage, reader: Reader) {
  def openDocument(fileName: String): Try[Document] = {
    Try {
      if (storage.canOpen(fileName)) {
        val document = storage.create(fileName)
        storage.aboutToOpen(document)
        reader.read(document)
      }
    }
  }  
}

Whoa! We do not need an abstract type anymore! We do not need to use class inheritance either! Everything can be resolved using object composition during the instantiation of the Application class.

val hdfsCsvApplication = new Application(new HdfsStorage(), new CsvReader())
val cloudCsvApplication = new Application(new CloudStorage(), new CsvReader())
// And so on...

As a plus, we reduced the dependencies of the overall architecture, avoiding all those annoying subclasses.

The Programmable Template Method

Most of the modern programming languages have functions as first-class citizens. A variant of the Template Method pattern uses lambdas as implementations of primitive methods. I like to call it Programmable Template Method.

As Joshua Bloch wrote in the third edition of Effective Java,

The Template Method pattern [..] is far less attractive. The modern alternative is to provide a static factory or constructor that accepts a function object to achieve the same effect.

The trick is passing functions in place of primitive methods during object instantiation. So, let's turn our original Application trait into a concrete class receiving some functions as input during the construction process.

class Application(
  // Functions used to 'program' the behavior of the application
  canOpen: String => Boolean,
  create: String => Document,
  read: Document => ()
) {
  // Template method
  def openDocument(fileName: String): Try[Document] = {
    Try {
      if (canOpen(fileName)) {
        val document = create(fileName)
        aboutToOpen(document)
        read(document)
        document
      }
    }
  }
  def aboutToOpen(doc: Document) = { /* Some default implementation */ 34}
}

val app = new Application(
    (filename: String) => /* Some implementation */,
    (filename: String) => /* Some implemenation */,
    (doc: Document) => /* Some implementation */
  )

In this way, you have not to declare a new type for each behavior you want to implement, just create a lambda expression, and pass it to the constructor during object creation.

The Scala way

So far, so good. We all know that the Scala language can do better than a simple object composition. Scala has some very powerful constructs that allow us to use some smarter versions of the Template Method pattern.

Mixins

The first construct we are going to use is Scala mixins. Mixins are traits which are used to compose a class. Using mixins, we can add some code to a class without using inheritance. It's a concept very similar to the composition.

Revamping the first Application trait we presented in this post, we can use mixins to implement the Template Method also in this way.

trait Application {
  def openDocument(fileName: String): Try[Document]
  def canOpen(fileName: String): Boolean
  def create(fileName: String): Document
  def aboutToOpen(doc: Document) = { /* Some default implementation */ 34}
  def read(doc: Document)
}
// Hdfs storage implementation
trait HdfsStorage {
  def openDocument(fileName: String): Try[Document] = /* HDFS implementation */
  def canOpen(fileName: String): Boolean = /* HDFS implementation */
  def create(fileName: String): Document = /* HDFS implementation */
}
// Csv reader implementation
trait CsvReader {
  def read(doc: Document): Document = /* CSV implementation */
}
// Creating the application using the proper implementations
val hdfsCsvApplication = new Application with HdfsStorage with CsvReader

As you can see, we can mix any trait during the instantiation process of another trait. We achieved composition using mixins, using a native approach.

Functional programming: currying and partial application

Many of us abandoned the object-oriented path after the functional way enlighted them. Is the Template Method pattern worth also in functional programming? Well, in some way the answer is "yes".

In functional programming, there is a technique called currying. The name "currying" becomes from the mathematician Haskell Curry, who was the first to use this technique.

Currying transforms a multi-argument function so that it can be called as a chain of single-argument functions

For the sake of simplicity, let's take a function that sums two integers.

def sum(a: Int, b: Int): Int = a + b

Then, using currying, we can derive from sum its curryfied form.

def curryfiedSum(a: Int)(b: Int): Int = a + b

If we pass only the first parameter to curryfiedSum, the result will be a new function with only one parameter left. In mathematical jargon, the function was partially applied. Using this approach, we can quickly obtain a new function that sums five to any integer number.

def sumFive(x: Int): Int = curryfiedSum(5)

Well, now that we have given the necessary background, we can reveal which is the link between currying and partial application and the Template Method Pattern. Let's define a function that repeatedly applies a lambda to a list of integer numbers. We can call it aggregate. Using this function, we can identify two new functions, the factorial n! and the sum of all numbers up to n.

def aggregate(neutral: Int, op: (Int, Int) => Int)(n: Int): Int =
  if (n == 0) neutral 
  else op(aggregate(neutral, op)(n-1), n)
// Factorial of integers from n to 0
def factorial = aggregate(1, _ * _)_
// Summing integers from n to 0
def sum = aggregate(0, _ + _)_

As you can see, the function aggregate defines the body of the general algorithm, whereas the first two arguments define the variable parts of the algorithm. Template method and primitive operations: It's the same approach of the Template Method pattern.

Do you see it? Awesome!

Conclusions

Summarizing, we started presenting the original Template Method design pattern, as shown by the GoF. Following the principle prefer composition over inheritance, we highlighted all the problems with the original approach. Then, we brought some procedures that correct the original issue.

Using composition, lambdas and curryfication, we were able to bring the Template Method pattern in the 21st century.

References

Single-Responsibility Principle done right

Riccardo Cardin — Fri, 05 Jan 2018 10:58:00 +0000

Originally posted on: Big ball of mud

I am a big fan of SOLID programming principles by Robert C. Martin. In my opinion, Uncle Bob did a great work when it first defined them in its books. In particular, I thought that the Single-Responsibility Principle was one of the most powerful among these principles, yet one of the most misleading. Its definition does not give any rigorous detail on how to apply it. Every developer has left to his own experiences and knowledge to define what a responsibility is. Well, maybe I found a way to standardize the application of this principle during the development process. Let me explain how.

The Single-Responsibility Principle

As it is used to do for all the big stories, I think it is better to start from the beginning. In 2006, Robert C. Marting, a.k.a. Uncle Bob, collected inside the book Agile Principles, Patterns, And Practices in C# a series of articles that represent the basis of clean programming, through the principles also known as SOLID. Each letter of the word SOLID refers to a programming principle:

S stands for Single-Responsibility Principle
O stands for Open-Closed Principle
L stands for Liskov Substitution Principle
I stands for Interface Segregation Principle
D stands for Dependency Inversion Principle

Despite the resonant names and the clearly marketing intent behind them, in the above principles are described some interesting best practices of object-oriented programming.

The Single-Responsibility principle is one of the most famous of the five. Robert uses a very attractive sentence to define it:

A class should have only one reason to change.

Boom. Concise, attractive, but so ambiguous. To explain the principle, the author uses an example that is summarized in the following class diagram.

In the above example, the class Rectangle is said to have at least two responsibilies: drawing a rectangle on a GUI and calculate the area of such rectangle. Is it really bad? Well, yes. For example, this design forces the ComputationalGeometryApp class to have a dependency on the class GUI.

Moreover, having more than one responsibility means that, every time a change to a requirement linked to the user interface comes, there is a non zero probability that the class ComputationalGeometryApp could be changed too. This is also the link between responsibilities and reasons to change.

The design that completely adheres to the Single-Responsibility Principle is the following.

Arranging the dependencies among classes as depicted in the above class diagram, the geometrical application does not depend on user interface stuff anymore.

The dark side of the Single-Responsibility Principle

Well, probably it is one of my problems, but I ever thought that a principle should be defined in a way that two different people understand it in the same way. There should be no space left for interpretation. A principle should be defined using a quantitave approach, rather than a qualitative approach. Probably, my fault comes from my mathematical extraction.

Given the above definition of the Single-Responsibility Principle, it is clear that there is no mathematical rigor to it.

Every developer, using its own experience can give a different meaning to the word responsibility. The most common misunderstanding regarding responsibilities is which is the right grain to achieve.

Recently, a "famous" blogger in the field of programming, called Yegor Bugayenko, published a post on his blog in which he discusses how the Single-Responsibility Principle is a hoax: SRP is a Hoax. In the post, he gave a wrong interpretation of the conception of responsibility, in my opinion.

He started from a simple type, which aim is to manage objects stored in AWS S3.

class AwsOcket {
    boolean exists() { /* ... */ }
    void read(final OutputStream output) { /* ... */ }
    void write(final InputStream input) { /* ... */ }
}

In his opinion, the above class has more than one responsibility:

Checking the existence of an object in AWS S3
Reading its content
Modifying its content

Uhm. So, he proposes to split the class into three different new types, ExistenceChecker, ContentReader, and ContentWriter. With this new type, in order to read the content and print it to the console, the following code is needed.

if (new ExistenceChecker(ocket.aws()).exists()) {
  new ContentReader(ocket.aws()).read(System.out);
}

As you can see, Yegor experience drives him to defined too fine-grained responsibilities, leading to three types that clearly are not properly cohesive.

Where is the problem with Yegor interpretation? Which is the keystone to the comprehension of the Single-Responsibility Principle? Cohesion.

It's all about Cohesion

Telling the truth, Uncle Bob opens the chapter dedicated to the Single-Responsibility Principle with the following two sentences.

This principle was described in the work of Tom DeMarco and Meilir Page-Jones. They called it cohesion. They defined cohesion as the functional relatedness of the elements of a module.

Wikipedia defines cohesion as

the degree to which the elements inside a module belong together. In one sense, it is a measure of the strength of the relationship between the methods and data of a class and some unifying purpose or concept served by that class.

So, which is the relationship between the Single-Responsibility Principle and cohesion? Cohesion gives us a formal rule to apply when we are in doubt if a type owns more than one responsibility. If a client of a type tends to use always all the functions of that type, then the type is probably highly cohesive. This means that it owns only one responsibility, and hence only one reason for changing.

It turns out that, like the Open-Closed Principle, you cannot say if a class fulfills the Single-Responsibility Principle in isolation. You need to look at its incoming dependencies. In other words, the clients of a class define if it fulfills or not the principle.

Shocking.

Looking back at Yegor example, it is clear that the three classes he created, thinking of adhering to the Single-Responsibility Principle in this way, are loosely cohesive and hence tightly coupled. The classes ExistenceChecker, ContentReader, and ContentWriter will probably always be used together.

Pushing to the limit: Effects on the degree of dependency

In the post Dependency, I defined a mathematical framework to derive a degree of dependency between types. The natural question that arises is: applying the above reasoning, does the degree of dependency of the overall architecture decrease or increase?

Well, first of all, let's recall how we can obtain the total degree of dependency of a type A.

$$
\delta_{tot}^{A} = \frac{1}{n} \displaystyle\sum_{C_j \in {C_1, \dots, C_n}} \delta_{A \to C_j }
$$

In our case, type A is the client of the class AwsOcket. Recalling that the value of $\delta_{A \to C_j }$ ranges between 0 and 1, dividing without any motivation the class AwsOcket into three different types will not increase the overall degree of dependency of client A. In fact, the normalizing factor $\frac{1}{n}$ assure us that refactoring processes will not increase the local degree of dependency.

The overall degree of the entire architecture will instead increase since we have three new types that still depend on AwsOcket.

Does this mean that the view of the Single-Responsibility Principle I gave during the post is wrong? No, it does not. However, it shows us that the mathematical framework is incomplete. Probably, the formula for the degree of dependency should be recursive, in order to take into consideration the addition of new tightly coupled types.

Conclusions

Starting from the definition given by Robert C. Martin of the Single-Responsibility Principle, we showed how simple is to misunderstand it. In order to give some more formal definition, we showed how the principle can be viewed in terms of the concept of cohesion. Finally, we try to give a mathematical proof of what we have done, but we went to the conclusion that the framework that we were using is incomplete.

This post concludes the year 2017. I want to thank all the people that took some of their time to read my post during this year. I will certainly return in 2018. Stay tuned.

Happy new year.

References

A Cameo that is worth an Oscar

Riccardo Cardin — Wed, 29 Nov 2017 16:20:31 +0000

Originally posted on: Big ball of mud

Rarely, during my life as a developer, I found pre-packaged solutions that fit my problem so well. Design patterns are an abstraction of both problems and solutions. So, they often need some kind of customization on the specific problem. While I was developing my concrete instance of Actorbase specification, I came across the Cameo pattern. It enlighted my way and my vision about how to use Actors profitably. Let's see how and why.

The problem: capturing context

Jamie Allen, in his short but worthwhile book Effective Akka, begins the chapter dedicated to Actors patterns with the following words:

One of the most difficult tasks in asynchronous programming is trying to capture context so that the state of the world at the time the task was started can be accurately represented at the time the task finishes.

This is exactly the problem we are going to try to resolve.

Actors often model long-lived asynchronous processes, in which a response in the future corresponds to one or more messages sent earlier. Meanwhile, the context of execution of the Actor could be changed. In the case of an Actor, its context is represented by all the mutable variables owned by the Actor itself. A notable example is the sender variable that stores the sender of the current message being processed by an Actor.

Context handling in Actorbase actors

Let's make a concrete example. In Actorbase there are two types of Actors among the others: StoreFinder and Storekeeper. Each Actor of type StoreFinder represents a distributed map or a collection, but it does not physically store the key-value couples. This information is stored by Storekeeper Actors. So, each StoreFinder owns a distributed set of its key-value couples, which means that owns a set of Storekeeper Actors that stores the information for it.

StoreFinder can route to its Storekeeper many types of messages, which represent CRUD operations on the data stored. The problem here is that if a StoreFinder owns n Storekeeper, to find which value corresponds to a key (if any), it has to send n messages of type Get("key") to each Storekeeper. Once all the Storekeeper answer to the query messages, the StoreFinder can answer to its caller with the requested value.

The sequence diagram below depicts exactly the above scenario.

The number of answers of Storekeeper Actors and the body of their responses represent the execution context of StoreFinder Actor.

Actor's context handling

So, we need to identify a concrete method to handle the execution context of an Actor. The problem is that between the sending of a message and the time when the relative response is received, an Actor processes many other messages.

Naive solution

Using nothing that my ignorance, the first solution I depicted in Actorbase was the following.

class StoreFinder(val name: String) extends Actor {
  def receive: Receive = nonEmptyTable(StoreFinderState(Map()))  
  def nonEmptyTable(state: StoreFinderState): Receive = {
    // Query messages from externl actors
    case Query(key, u) =>
      // Route a Get message to each Storekeeper
      broadcastRouter.route(Get(key, u), self)
      context.become(nonEmptyTable(state.addQuery(key, u, sender())))
    // Some other stuff...
    // Responses from Storekeeper
    case res: Item =>
      context.become(nonEmptyTable(state.copy(queries = item(res, state.queries))))   
  }
  // Handling a response from a Storekeeper. Have they all answer? Is there at least
  // a Storekeeper that answer with a value? How can a StoreFinder store the original 
  // sender?
  private def item(response: Item,
                   queries: Map[Long, QueryReq]): Map[Long, QueryReq] = {
    val   Item(key, opt, id) = response
    val QueryReq(actor, responses) = queries(id)
    val newResponses = opt :: responses
    if (newResponses.length == NumberOfPartitions) {
      // Some code to create the message
      actor ! QueryAck(key, item, id)
      queries - id
    } else {
      queries + (id -> QueryReq(actor, newResponses))
    }
  }
}
// I need a class to maintain the execution context
case class StoreFinderState(queries: Map[Long, QueryReq]) {
  def addQuery(key: String, id: Long, sender: ActorRef): StoreFinderState = {
    // Such a complex data structure!
    copy(queries = queries + (id -> QueryReq(sender, List[Option[(Array[Byte], Long)]]())))
  }
  // Similar code for other CRUD operations
}
sealed case class QueryReq(sender: ActorRef, responses: List[Option[(Array[Byte], Long)]])

A lot of code to handle only a bunch of messages, isn't it? As you can see, to handle the execution context I defined a dedicated class, StoreFinderState. For each Query message identified by a UUID of type Long, this class stores the following information:

The original sender
The list of responses from Storekeeper Actors for the message
The values the Storekeeper answered with

As you can imagine, the handling process of this context is not simple, as a single StoreFinder has to handle all the messages that have not received a final response from all the relative Storekeeper.

We can do much better, trust me.

Asking the future

A first attempt to reach a more elegant and concise solution might be the use of the Ask pattern with Future.

This is a great way to design your actors in that they will not block waiting for responses, allowing them to handle more messages concurrently and increase your application’s performance.

Using the Ask pattern, the code that handles the Query message and its responses will reduce to the following.

case Query(key, u) =>
  val futureQueryAck: Future[QueryAck] = for {
    responses <- Future.sequence(routees map (ask(_, Get(key, u))).mapTo[Item])
  } yield {
    QueryAck(/* Some code to create the QueryAck message from responses */)
  }
  futureQueryAck map (sender ! _)

Whoah! This code is fairly concise with respect to the previous one. In addition, using Future and a syntax that is fairly declarative, we can achieve quite easily the right grade of asynchronous execution that we need.

However, there are a couple of things about it that are not ideal. First of all, it is using futures to ask other actors for responses, which creates a new PromiseActorRef for every message sent behind the scenes. This is a waste of resources.

Annoying.

Furthermore, there is a glaring race condition in this code—can you see it? We’re referencing the “sender” in our map operation on the result from futureQueryAck, which may not be the same ActorRef when the future completes, because the StoreFinder ActorRef may now be handling another message from a different sender at that point!

Even more annoying!

The Extra pattern

The problem here is that we are attempting to take the result of the off-thread operations of retrieving data from multiple sources and return it to whoever sent the original request to the StoreFinder. But, the actor will likely have move on to handling additional messages in its mailbox by the time the above futures complete.

The trick is capturing the execution context of a request in a dedicated inner actor. Let's see how our code will become.

case Query(key, u) => {
  // Capturing the original sender
  val originalSender = sender
  // Handling the execution in a dedicated actor
  context.actorOf(Props(new Actor() {

    // The list of responses from Storekeepers
    var responses: List[Option[(Array[Byte], Long)]] = Nil

    def receive = {
      case Item(key, opt, u) =>
        responses = opt :: responses
        if (responses.length == partitions) {
          // Some code that creates the QueryAck message
          originalSender ! QueryAck(key, item, u)
          context.stop(self)
        }
    }
  }))
}

Much better. We have captured the context for a single request to StoreFinder as the context of a dedicated actor. The original sender of StoreFinder Actor was captured by the constant originalSender and shared with the anonymous Actor using a closure.

It's easy, isn't it? This simple trick is known as the Extra pattern. However, we are searching for a Cameo in our movie.

Finally presenting the Cameo pattern

The Extra pattern is very useful when the code inside the anonymous Actor is very small and trivial. Otherwise, it pollutes the main Actor with details that do not belong to its responsibility (one for all, Actor creation).

It is also similar to lambdas, in that using an anonymous instance gives you less information in stack traces on the JVM, is harder to use with a debugging tool, and is easier to close over state.

Luckily, the solution is quite easy. We can move the anonymous implementation of the Actor into its own type definition.

This results in a type only used for simple interactions between actors, similar to a cameo role in the movies.

Doing so, the code finally becomes the following.

class StoreFinder(val name: String) extends Actor {
  override def receive: Receive = {
    // Omissis...
    case Query(key, u) =>
      val originalSender = sender()
      val handler = context.actorOf(Props(new QueryResponseHandler(originalSender, NumberOfPartitions)))
      broadcastRouter.route(Get(key, u), handler)
  }
  // Omissis...
}

// The actor playing the Cameo role
class QueryResponseHandler(originalSender: ActorRef, partitions: Int) {

  var responses: List[Option[(Array[Byte], Long)]] = Nil

  override def receive: Receive = LoggingReceive {
    case Item(key, opt, u) =>
      responses = opt :: responses
      if (responses.length == partitions) {
        // Some code to make up a QueryAck message
        originalSender ! QueryAck(key, item, u)
        context.stop(self)
      }
  }
}

Much cleaner, such satisfying.

Notice that the router in the StoreFinder tells the routees to answer to the actor that handles the query messages, broadcastRouter.route(Get(key, u), handler). Moreover, remember to capture the sender in a local variable in the main actor, before passing its reference to the inner actor.

Make certain you follow that pattern, since passing the sender ActorRef without first capturing it will expose your handler to the same problem that we saw earlier where the sender ActorRef changed.

Conclusions

So far so good. We started stating that context handling is not so trivial when we speak about Akka Actors. I showed you my first solution to such problem in Actorbase, the database based on the Actor model I am developing. We agreed that we do not like it. So, we moved on and we tried to use Futures. The solution was elegant but suffered from race conditions. In the path through the final solution, we met the Extra pattern, which solved the original problem without any potential drawback. The only problem is that this solution was no clean enough. Finally, we approached the Cameo pattern, and it shined in all its beauty. Simple, clean, elegant.

P.S.: All the code relative to Actorbase can be found on my GitHub.

References

Resolve me, Implicitly

Riccardo Cardin — Tue, 17 Oct 2017 09:40:25 +0000

Originally posted on: Big ball of mud

Reading my posts you can easily find that there is a topic that cares about me a lot: Dependency management in the development process. There is a feature of the Scala programming language that I liked since the beginning. Without any external library, it is possible to successfully implement vary dependency injection mechanisms. In the past, I wrote about the Cake pattern. Now it's time to talk about dependency injection through the use of implicits. Let this race starts!

The problem: dependency injection

Dependency

I have written many times about this topic, then I will not make a long introduction. To summarize, every time a component needs to send a message to another component, a dependency is defined between them. Components may be packages, classes, functions and so on. Messages are always associated with methods or functions calls. Dependency between two components can have many levels of strength. If you want a complete explanation of the dependency concept, have a look at this post: Dependency.
For sake of completeness, let's give an example of the simplest type of dependency: Association.

class A {
  // ...
}
class B(val a: A) {
  // ...
}

An association means that a class is made of references to other classes.

Dependency injection

Other than the coupling between components that derives from dependencies, the problem with dependencies is also the process that a component has to take in place to resolve them. Dependencies resolution is a complex problem. To treat it successfully, the best thing to do is to apply the divide et impera principle.

Dependency resolution can be divided into two different tasks: dependencies declarations and resolution.

First of all, a component should be able to declare its dependencies. A component should be able to scream "Hey, anybody listening, I need these f@cking classes to work!". There are many ways a component may choose to declare its dependencies. The most accreditated by the developers' community is constructor injection.

Using constructor injection, a component declares its dependencies as the parameters of its constructor.

class Connection (private val mainActor: ActorSelection) {
  // Some cool stuff
}

In the above example, the class Connection is screaming to everyone that it needs an instance of an ActorSelection class to work properly.

If you want to know which other types of dependency declarations are available, please have a look at the post Resolving your problems with Dependency Injection.

Now that we know how to declare which kinds of objects a component needs to work, we also need a technique to resolve these object. And it's here that the "injection" part comes into play.

Resolving dependencies in Scala

In the JVM ecosystem, there are a lot of libraries that implement the injection technique. We have many alternatives, such like the following.

Spring framework
Guice
Weld
Dagger 2

There is also a Java Specification Requests dedicated to dependency injection, the JSR 330. As you can see, the JDK needs external libraries to implement the dependency injection mechanism.

But, Scala is different. The Scala programming language has a richer semantic that allows implementing dependency injection mechanisms without the need of any kind of external libraries. As you we will see in a moment, this technique applies both to classes and to functions. That's awesome!

In the past, I wrote about the Cake Pattern, which implements dependency resolution using traits and self-type. This time I want to write about a dependency injection mechanisms that uses two other features of Scala, function currying, and implicits.

Currying

As you know, Scala is also quoted as a functional programming language. Functional programming languages have many nice features. One of these features is function currying.

Let's take for example a function that takes two arguments:

def mul(x: Int, y: Int) = x * y

We can refactor this function into a new one, that takes only one parameter and returns a new function.

def mulOneAtTime(x: Int) = (y: Int) => x * y

To multiply two integers you have to call the last function in the following way.

mulOneAtTime(7)(6) // Returns 42

We said the function mulOneAtTime is equal to the curried function of the original function mul.

In mathematics and computer science, currying is the technique of translating the evaluation of a function that takes multiple arguments (or a tuple of arguments) into evaluating a sequence of functions, each with a single argument.

There is a shortcut for defining such curried function in Scala:

def mulOneAtTime(x: Int)(y: Int) = x * y

The languages save us from defining a lot of intermediate functions, giving us some vanilla-flavored syntactic sugar.

I suppose that you are asking why are we talking about currying instead of dependency injection. Be patient.

Implicits

Implicits are another awesome feature of the Scala programming language. Hated by someone and feared by most, implicits can be applied to resolve a lot of problems, from automatic conversion between two types to the automatic resolution of dependencies. What we are going to explain is how implicit parameters work in Scala.

The parameters of a function (or a method) can be marked with the keyword implicit. In this case, the compiler will automagically look for a value to supply to these parameters. Here is a simple example of a function using an implicit parameter, taken directly from the Scala SDK:

object Future {
  def apply[T](body: => T)(implicit execctx: ExecutionContext): Future[T]
}

// Usage - thanks to Stanislav Spiridonov for the funny example :)
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global

val beast = hell.createBeastFor(credentials)
val f: Future[Option[Blood]] = Future {
  beast.tear(user)
}

In the above example, the parameter execctx of method apply is automatically resolved and provided by the compiler to the program. How does the Scala compiler know how to resolve implicit parameters? Compiler search for an object that has the same type of an implicit parameter and that is declared using the word implicit. In the case of execctx in the object scala.concurrent.ExecutionContext.Implicits is defined the constant global.

implicit lazy val global: ExecutionContext = ???

During implicits resolution, the compiler search for var/val/def that are available in the same scope of the function that as some parameters marked as implicit.

Implicit resolution is also one of the reasons why Scala compiler is so slow. In fact, implicits resolution has not any impact on runtime performances, as it is done completely at compile time. For a more detailed explanation on implicit resolution, have a look at Implicit Parameter Resolution

So far, so good. We just added another piece to our dependency injection puzzle. Now it's time to put all the ingredients together and bake a tasty dependency injection cake.

Dependency Injection using implicits

Until now, we learned how to currying a function; We learned how implicits work in Scala. It's time to put them all together.

Object-oriented programming

As you probably already understood, we can use implicits to instruct the compiler to automatically resolve components dependencies. Let's start from types. We have already learned that dependencies of a class should be declared in its constructor. We learned that for every parameter of a function or method that is marked as implicit, the compiler search for an object of the same type in the appropriate scope.

trait UserService {
  def findById(id: String): User
}
class SimpleUserService(implict repository: UserRepository) extends UserService {
  override def findById(id: String): User = repository.findUser(id)
}

In the above example, we declared a SimpleUserService class, which declares as its unique external dependency an instance of the type UserRepository. The dependency is marked as implicit. To let the compiler properly resolve this dependency, we have to provide an object of type UserRepository marked as implicit in the same scope of the clients of the class SimpleUserService.

We have declared also a UserService trait over the class, such that clients of this class have not to deal with the boilerplate code of implicits.

We have mainly two possibilities to provide this information to the compiler. The first is to use a configuration trait to mix with every class that declares an implicit dependency.

trait Conf {
  implicit val repository = new MongoDbUserRepository
}
object UserController extends Conf
class UserController {
  // I know, this code is not the best :P
  def findById(id: String) = new UserService().findById(id)
}

We chose to mix the trait into the companion object of the class to avoid replication of the instances of the variable repository.

The second approach is to use a package object placed in the same package of the class UserController.

package controllers {
  class UserController {
    // I know, this code is not the best :P
    def findById(id: String) = new UserService().findById(id)
  }
}
package object controllers {
  implicit val repository = new MongoDbUserRepository
}

Using the latter approach, the definition of the object UserController is not polluted by any external and strange extensions. The drawback is that it becomes harder to trace how each implicit parameter is resolved by the compiler.

Functional programming

Easy enough! We have just implemented dependency injection in Scala using implicits! Wow! But, wait: why did we introduce also function curring? We did not use currying by now.

The secret is soon unveiled. The above examples treat dependency injection in object-oriented programming. But, what about functional programming? In functional programming, there is the same requirement for functions to declare and resolve their dependencies. In this world, the only way to declare a dependency is to put it into the signature. In this way, a client of the function can provide the needed dependency at the same time of the function call.

// The example is taken from the book Scala for the Impatient
case class Delimiters(left: String, right: String)
def quote(what: String, delims: Delimiters) = delims.left + what + delims.right

To clearly divide the inputs of the functions and the declaration of dependencies, avoiding polluting of the signature, we can use currying.

def quote(what: String)(delims: Delimiters) = delims.left + what + delims.right

Finally, to let the compiler to resolve automatically the dependencies, we can use implicits, as we did for the object-oriented case. This technique is called implicit parameter.

def quote(what: String)(implicit delims: Delimiters) = 
  delims.left + what + delims.right

Conclusions

"This is the end, my only friend, the end". We just analyzed the techniques we can use to implement natively dependency injection in Scala: Implicits. We analyzed in details the features of the language that helps us to achieve the goal: implicits and function currying. We showed an example both using object-oriented programming and functional programming. Well.

Different from the Cake Pattern, implicits gives us a mechanism to implement dependency injection that is concise and sexy.

Dependencies are resolved at compile-time, you write less code, there is no boilerplate, classes are loosely coupled, everything is extendable yet still typesafe.

On the other end, implicits resolution can become hard to understand an maintain very quickly.

It is common to use package objects with implicits, so there is no way to eyeball that this small import on top of a file feeds several functions calls with implicit variables. Even worse, it is impossible to understand from a code that this particular function call needs implicits.

As for any other technique in programming, we always have to compare pros and cons. In my opinion, if using properly, implicits offers more pros than cons. What do you think? Anybody with some bad story about implicits out of there? Cheers.