DEV Community: Stojan Anastasov

Modelling UI State on Android

Stojan Anastasov — Fri, 29 Jan 2021 00:00:00 +0000

The recommended approach from Google for Android development is holding the UI state in a ViewModel and having the View observe it. To achieve that one can use LiveData, StateFlow, RxJava or a similar tool. But how to model the UI state? Use a data class or a sealed class? Use one observable property or many? I will describe the tradeoffs between the approaches and present a tool to help you decide which one to use. This article is heavily inspired by Types as Sets from the Elm guide.

Photo by Marc-Olivier Jodoin on Unsplash

Types as sets

By Making Data Structure we can make sure the possible values in code exactly match the valid values in real life. Doing that helps to avoid a whole class of bugs related to invalid data. To achieve that, first we need to understand the relationship between Types and Sets.

We can think of Types as sets of values, they contain unique elements and there is no ordering between them. For example:

Nothing - the empty set, it contains no elements
Unit - the singleton set, it contains one element - Unit
Boolean - contains the elements true and false
Int - contains the elements: … -2, -1, 0, 1, 2 …
Float - contains the elements: 0.1, 0.01, 1.0 ….
String - contains the elements: "", "a", "b", "Kotlin", "Android", "Hello world!"…

So when you write: val x: Boolean it means x belongs to the set of Boolean values and can be either true or false.

Cardinality

In Mathematics, Cardinality is the measure of “number of elements” of a Set. For example the set of Boolean contains the elements [true, false] so it has a cardinality = 2.

Let’s take a look at the cardinality of the sets mentioned above:

Nothing - 0
Unit - 1
Boolean - 2
Short - 65535
Int - ∞
Float - ∞
String- ∞

Note : The cardinality of Int and Float is not exactly infinity, it’s 2^32 however that is a huge number.

When building apps, we use built-in types and create custom types using constructs like data classes and sealed classes.

Product Types (*)

One flavor of product types in Kotlin are Pair and Triple. Let’s take a look at their cardinality:

Pair<Unit, Boolean> - cardinality(Unit) * cardinality(Boolean) = 1 * 2 = 2
Pair<Boolean, Boolean> - 2 * 2 = 4

Pair<Unit, Boolean> contains the elements:

Pair(Unit, false)
Pair(Unit, true)

Other examples:

Triple<Unit, Boolean, Boolean> = 1 * 2 * 2 = 4
Pair<Int, Int> = cardinality(Int) * cardinality(Int) = ∞ * ∞ = ∞

Well, that escalated quickly.

When combining types with Pair/Triple their cardinalities multiply (hence the name Product Types).

Pair/Triple are the generic version of data classes with 2 and 3 properties respectively.

data class User(val emailVerified: Boolean, val isAdmin: Boolean) has the same cardinality as Pair<Boolean, Boolean>, 4. The elements are:

User(false, false)
User(false, true)
User(true, false)
User(true, true).

Sum Types (+)

In Kotlin we use sealed classes to implement Sum types. When combining types using sealed classes, the total cardinality is equal to the sum of the cardinality of the members. Some examples are:

sealed class NotificationSetting
object Disabled : NotificationSettings() // an object has one element -> cardinality = 1
data class Enabled(val pushEnabled: Boolean, val emailEnabled: Boolean) : NotificationSettings()

// cardinality = cardinality (Disabled) + cardinality(Enabled)
// cardinality = 1 + (2 * 2)
// cardinality = 1 + 4 = 5

sealed class Location
object Unknown : Location()
data class Somewhere(val lat: Float, val lng: Float) : Location()

// cardinality = cardinality (Unknown) + cardinality(Somewhere)
// cardinality = 1 + (∞ * ∞)
// cardinality = 1 + ∞ = ∞

Nullable Types

Another way to model the Location type is using nullable types data class Location(val lat: Float, val lng: Float) and represent it as: val location: Location?. In this scenario we use null when the location is unknown. These two representations have the same cardinality and we can convert between them without any information loss. A few more examples:

Unit? - cardinality = 2 (1 + cardinality(Unit))
Boolean? - 3 (1 + cardinality(Boolean))

The elements of a nullable type A? are null + the elements of the original type A. The cardinality of a nullable type is 1 + the cardinality of the original type.

Enums

Enums are another way of representing Sum types in Kotlin:

enum class Color { RED, YELLOW, GREEN }

The cardinality of Color is equal to the number of elements, in this case 3. An alternative representation using a sealed class is:

sealed class Color
object Red : Color()
object Yellow : Color()
object Green : Color()

and it has the same cardinality - 3.

Why does it matter

Thinking about Types as Sets and their cardinality helps with data modelling to avoid a whole class of bugs related to Invalid Data. Let’s say we are modelling a traffic light. The possible colors are: red, yellow and green. To represent them in code we could use:

a String where "red", "yellow" and "green" are the valid options and everything else is invalid data. But then the user types "rad" instead of red and we have an issue. Or "yelow" or "RED". Should all functions validate their arguments? Should all functions have tests? The root cause of the issue here is cardinality. A String has cardinality of ∞ while the our problem has 3. There are ∞ - 3 possible invalid values.

a data class data class Color(val isRed: Boolean, val isYellow: Boolean, val isGreen: Boolean) - here Color(true, false, false) represents red. Yet this still leaves room for invalid data e.g. Color(true, true, true). Again you would need checks and tests to ensure values are valid. The cardinality of the data class Color is 8 and it has 8 - 3 = 5 illegal values. It’s much better then the String, but we can still improve it.

an enum - enum class Color { RED, YELLOW, GREEN } - this has a cardinality = 3. It matches exactly the possible valid values of the problem. Illegal values are now impossible, so there is no need for tests that check data validity.

By modelling the data in a way that rules out illegal values, the resulting code will also end up shorter, more clear and easier to test.

Make sure the set of possible values in code match the set of valid values of the problem and a lot of issues disappear.

Exposing State from a ViewModel

The task:

Build an app that calls a traffic light endpoint and shows the color of the traffic light (red, yellow or green). During the network call show a ProgressBar. On success show a View with the color and in case of an error show a TextView with a generic text. Only one view is visible at a time and there is no possibility to retry errors.

class TrafficLightViewModel : ViewModel() {

    val state: LiveData<TrafficLightState> = TODO()
}

I will represent the color as an enum with three values. How should the type TrafficLightState look like?

data class TrafficLightState(
    val isLoading: Boolean, 
    val isError: Boolean, 
    val color: Color?
)

one way is a data class with three properties. Yet this has possible invalid states e.g. TrafficLightState(true, true, Color.RED). Both error and loading are true. Showing both the ProgressBar and the error TextView should not be possible in the UI. We also have a Color which is impossible in case of an error. The cardinality of TrafficLightState is 2 (Boolean) * 2 (Boolean) * 4 (Color?) = 16.

What about using many observable properties?

class TrafficLightViewModel : ViewModel() {

    val loading: LiveData<Boolean> = TODO()

    val error: LiveData<Boolean> = TODO()

    val color: LiveData<Color?> = TODO()
}

This has a cardinality of 2 (Boolean) * 2 (Boolean) * 4 (Color?) = 16. it has the same cardinality as the data class approach and enables illegal values.

Another approach is using a sealed class:

sealed class TrafficLightState
object Loading : TrafficLightState()
object Error : TrafficLightState()
data class Success(val color: Color) : TrafficLightState()

which has a cardinality of 1 (Loading) + 1 (Error) + 3 (Success) = 5 which exactly matches the possible states of the problem:

during the network call: Loading and show the ProgressBar
if the network call fails: Error and show a TextView
on success: Success and show a view with the corresponding color

These are all valid approaches and when designing your UI state. But only one of them matches exactly the possible valid states of the problem. To eliminate bugs, simplify code and reduce the number of tests, use that one.

Conclusion

To model problems in code we use built-in types like: Boolean, Int, String… and in Kotlin we also create custom types using constructs like data class and sealed class. Different language constructs have different effects on the cardinality of the model. Reduce the number of invalid states by picking the right combination. That will results in simpler and more robust code.

Thanks Gaël for the review.

Unit Tests and Concurrency

Stojan Anastasov — Wed, 06 Jan 2021 00:00:00 +0000

Once Retrofit added RxJava support, RxJava became my go-to concurrency framework for writing Android apps. One of the great things about RxJava is the excellent testing support. It includes TestObserver, TestScheduler, RxJavaPlugins so you can switch your schedulers in tests.

A common approach in testing RxJava code is using a JUnit rule that replaces the Scheduler pools with Schedulers.trampoline() before tests are run and resets them to the original thread pools after the tests. This makes the whole Observable chain runs on a single thread, the same thread the test runs on, which means we can write assertions without worrying about concurrency. However the production code usually is not single threaded. IO operations are done on the IO thread pool, views are updated on the main thread and everything else happens on the computation pool. By using different schedulers in the tests and using a different strategy (single threaded) we make those unit tests useless in catching concurrency issues.

A real world scenario

I was working on a side project. The screen consists of a RecyclerView displaying a list of elements. To get the elements I need to perform two different API calls. The first API call returns a list with N elements, then for each item in the list I need to perform the second call. After combining the data I send it to the UI for displaying. Using RxJava this looks like:

// Emits Loading then Content or Problem
private fun requestData(): Observable<ViewState> =
    service.firstApiCall()
        .observeOn(Schedulers.computation())
        .map { it.message }
        .flatMap(this::secondApiCall)
        .map<ViewState> { ViewState.Content(it) }
        .startWith(Single.just(ViewState.Loading))
        .onErrorReturn { ViewState.Problem }
        .toObservable()

// Concurrently executes secondApiCall for each element in list. 
// Transforms the result to ViewEntity, combines everything in a list
private fun secondApiCall(list: List<String>): Single<List<ViewEntity>> =
    Observable.fromIterable(list)
        .concatMapEager { item ->
            service.secondEndpoint(item)
                .observeOn(Schedulers.computation())
                .map { it.message }
                .map { ViewEntity(item, it.toHttpUrl()) }
                .onErrorReturn { DogViewEntity(item, null) }
                .toObservable()
        }
        .toList()

Since the list is 15-20 elements, I am using concatMapEager to execute the API calls for the second endpoint concurrently while maintaining the order of the elements. This worked great and I wrote my unit tests using the real schedulers. RxJava has amazing testing support that enables this. Thanks Simon Vergauwen for the tips on how to do that.

@Test
fun `execute - first call succeeds, one failed call to #2 - one full item, one item with default value`() {
    val items = listOf("Item 1" to "https://item1.jpg", "Item 2" to "invalid url")
    val service = successService(items) // Fake implementation of a retrofit service, always returns success

    val viewModel = ViewModel(service)

    val expected = listOf(
        ViewEntity("Item 1", "https://item1.jpg".toHttpUrl()),
        ViewEntity("Item 2", null)
    )
    viewModel.state
        .test()
        .also { viewModel.execute() }
        .awaitCount(2) // (0) Loading, (1) Content
        .assertValueAt(1, Content(expected))
}

Here, awaitCount(2) makes sure the assertions after it are ran only after it gets at least 2 items.

The Refactor

This worked reasonably well, however I decided to try and optimise it further.

My current code would:

start with a progress bar
do API call #1
do API call #2 N times concurrently
combine the results
show the full list in the UI

What if we display the data as it becomes available.

start with a progress bar
do API call #1
show the data from API call #1 in the UI
do API calls #2 N times concurrently without displaying progress
update the UI N times as the API calls for #2 are completing

I was already using ListAdapter which has built-in diffing support. I can send an updated list and it will do it’s magic to update the changed elements only.

Improving the solution

The original code was written as a single Rx chain. It emits a Loading state, then does the API calls, combines the data and emits a Content state (or an Error state in case of failure). I modified it to: emit a Content state containing the list of elements from API call #1 right after the call is done. The data from call #2 had some default values for the time being. After each invocation of call #2, get the current state, update the element and emit a new Content state. I ran my tests and they failed as expected. I updated the tests to include the intermittent states (the changes caused emitting partial Content states). Ran the tests again, two worked, one failed. Ran again, all green.

The updated test:

@Test
fun `execute - first call succeeds, one failed call to #2 - one full item, one item with default value`() {
    val items = listOf("Item 1" to "https://item1.jpg", "Item 2" to "invalid url")
    val service = successService(items)

    val viewModel = ViewModel(service)

    val expected = listOf(
        ViewEntity("Item 1", "https://item1.jpg".toHttpUrl()),
        ViewEntity("Item 2", null)
    )
    viewModel.state
        .test()
        .also { viewModel.execute() }
        .awaitCount(4) // (0) Loading, (1) Content, (2) Content (3) Content
        .assertValueAt(3, Content(expected))
}

the only changes in the test are: the argument to awaitCount went from 2 to 4 (to account for the intermittent Content values) and the final result is now at index 3.

My refactoring made my tests flaky. Depending of the scheduling and the order of the requests for API call #2 I would get the correct or wrong result. My code had a concurrency issue.

My state lives in a serialized BehaviorSubject. I would update the state by reading it from the subject, create an immutable copy of the updated state, write back the updated state to the Subject. However this is NOT thread safe. Since the state updates happened on the computation pool, multiple threads could do it at the same time. This resulted in inconsistent state, the results of some API calls would get overwritten by the next call.

Conclusion

When testing RxJava code, you do NOT have to replace all schedulers with Trampoline. Doing that will make you miss possible concurrency issues. Using the real schedulers can help you catch those concurrency issues. It is not a perfect solution, but better than nothing. Don’t replace all schedulers with trampoline by default, make it a conscious choice to do so, or not do it.

Thanks to Marcello and Gaël for the review of this post.

Fragments, ViewBinding and memory leaks

Stojan Anastasov — Fri, 30 Oct 2020 00:00:00 +0000

As an Android engineer one of the basic things you need to do is bind the views (written in XML) with Kotlin/Java code. You can do this with the basic primitive -findViewById(), using a library like ButterKnife, using a compiler plugin like Kotlin Android Extensions or starting with Android Studio/AGP 3.6 ViewBinding. There are a few other options out there but in my experience these are the most common.

The Kotlin Android Extensions plugin will be deprecated (except the @Parcelize functionality) in favor of ViewBinding soon. I see this as a positive change, but not everyone shares my opinion. One of the common arguments against ViewBinding (according to a comment in the ticket, a discussion on reddit and a friend of mine) is:

ViewBinding introduces memory leaks in Fragments.

Let’s take a look into the usage example from the official docs:

private var _binding: ResultProfileBinding? = null
// This property is only valid between onCreateView and
// onDestroyView.
private val binding get() = _binding!!

override fun onCreateView(
    inflater: LayoutInflater,
    container: ViewGroup?,
    savedInstanceState: Bundle?
): View? {
    _binding = ResultProfileBinding.inflate(inflater, container, false)
    val view = binding.root
    return view
}

override fun onDestroyView() {
    super.onDestroyView()
    _binding = null
}

The fragment outlives the view, which means that if we forget to clear the binding reference in onDestroyView this will cause a memory leak. Now I do agree that this is error prone, you have to remember to clear the binding reference in each fragment you create. However this problem is not inherent to ViewBinding, but to this kind of usage. The problem here is: a component with a larger scope (the fragment) keeps a reference to a component with a smaller scope (the binding).

Clearing the reference is a workaround, the proper solution is to move the reference to the correct scope:

private val viewModel by viewModels<ProfileViewModel>()

override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
    val binding = ResultProfileBinding.bind(view)

    TODO("Set any (click) listeners")

    viewModel.liveData.observe(viewLifecycleOwner) { data ->
        TODO("Bind the data to views")
    }
}

If you move the binding reference from the Fragment scope to a function scope, you don’t have to worry about clearing it because the function scope is smaller then the view scope. Now this does assume the MVVM pattern for the UI layer.

Another argument against ViewBinding that I have seen is that is more code, you have to type binding.view instead of just view (using kotlin android extensions):

// kotlin-android-extensions
viewModel.liveData.observe(viewLifecycleOwner) { data ->
    textView.text = data.title
    imageView.load(data.imageUrl)
}

// view-binding
val binding = ResultProfileBinding.bind(view)

viewModel.liveData.observe(viewLifecycleOwner) { data ->
    binding.textView.text = data.title
    binding.imageView.load(data.imageUrl)
}

To ease the pain here you can use kotlin’s scoping function with:

// view-binding
val binding = ResultProfileBinding.bind(view)

viewModel.liveData.observe(viewLifecycleOwner) { data ->
    with(binding) {
        textView.text = data.title
        imageView.load(data.imageUrl)
    }
}

and the repetitive binding is gone. The more views you have, the better this scales - you only write with(binding) once.

Conclusion

Using ViewBinding with Fragments can be error prone, but it doesn’t have to be. The solution is moving the reference to the binding from a Fragment scope to a local function scope.

Communicating with your Lifecycle Owner using RxJava

Stojan Anastasov — Tue, 08 Sep 2020 00:00:00 +0000

Google introduced Jetpack, a family of opinionated libraries to make Android development easier a few years ago. One of the core classes in Jetpack is LiveData - an observable, lifecycle aware data holder. The typical use case is having a ViewModel that exposes LiveData as a property, and observing it from your lifecycle owner, a Fragment or an Activity.

A typical usage would look like this:

data class MyState(val value: String)

class MyViewModel : ViewModel {

    private val _state = MutableLiveData<MyState>()
    val state: LiveData<MyState>
    get() = _state
}

class MyFragment : Fragment {

    val viewModel by viewModels<MyViewModel>()

    override fun onViewCreated() {
        viewModel.state.observe(this, Observer(::handleState))
    }

    private fun handleState(state: MySate): Unit = TODO()
}

There are multiple benefits of using LiveData:

Your observer is notified when the data changes
The observer is only notified of changes when it’s active
Observers are notified when they become active again, like entering into foreground etc

Check the LiveData docs for all benefits.

LiveData and Events

In situations like showing a Snackbar/dialog or navigating to a different Activity/Fragment the ViewModel also needs to notify the LifecycleOwner. A plain old LiveData doesn’t work well here because it caches the last item. As a workaround, in the official Android architecture samples there is a SingleLiveEvent implementation of LiveData.

Data Layer

But what about the rest of the app? You can use LiveData in your data layer, in fact Room, the persistence library from Jetpack, support LiveData as the return type natively. However while using LiveData across all the layer in the app is possible, it is less than ideal. The operations are always executed on the Main Thread and it comes with limited number of transformation functions compared to RxJava or Flow.

To fix this problem LiveData comes with adapters for both RxJava and Flow from KotlinX Coroutines. This means developers can use RxJava or Flow in their data layer and convert to LiveData in the ViewModel benefiting from the lifecycle awareness.

State and RxJava

I was working on a sample app last week using the Marvel API, you can check the full code on Github. It uses a fully reactive architecture powered by RxJava. My initial approach was to use LiveData for the ViewModel -> Fragment communication. However I also wanted to benefit from the RxJava excellent testing support, so I decided to try and do what LiveData does using RxJava.

For observing state, RxJava offer BehaviorSubject, a Subject that caches the last value it observer and emits it to each subscribed Observer. That takes care of the caching of the last value and observing changes. For observing on the Main Thread there is RxAndroid. What about the lifecycle part. To take care of that I created a utility class, built atop LifecycleOwner to help me dispose of the subscription at the right lifecycle callback.

class LifecycleDisposable(
    private val disposable: Disposable
) : DefaultLifecycleObserver {

    override fun onDestroy(owner: LifecycleOwner) {
        disposable.dispose()
        super.onDestroy(owner)
    }
}

the code using this:

class RxViewModel : ViewModel() {

    private val _state = BehaviorSubject.create<MyState>()
    val state: Observable<MyState>
    get() = _state
}

class RxFragment : Fragment() {

    val viewModel by viewModels<MyViewModel>()

    fun onViewCreated() {
        viewModel.state
            .subscribe(::handleState)
            .autoDispose() // extension function, check the Github link
    }

    private fun handleState(state: MySate) = TODO()
}

Unlike LiveData, this starts observing in onViewCreated and unsubscribes in onDestroyView. The code would be similar to subscribe in onStart and stop observing in onStop, however in my experience (and I could be wrong) updating view state while the view is not active never caused me problems.

Events and RxJava

Events are a bit different than state. The main requirement is to be delivered exactly once (they are consumable). Another important requirement is to be delivered ONLY when the view is active (between onStart and onStop). My first attempt was using UnicastSubject:

A Subject that queues up events until a single Observer subscribes to it, replays those events to it until the Observer catches up and then switches to relaying events live to this single Observer until this UnicastSubject terminates or the Observer unsubscribes.

however that didn’t work out well. The queuing of events when there was no observers was working great. Upon subscription all events would be delivered. When there was a subscriber events were getting delivered. The problem was when the subscriber unsubscribes and a new subscriber tries to subscribe (onStart and onStop can be triggered multiple times). That led me to UnicastWorkSubject from RxJavaExtensions:

A Subject variant that buffers items and allows one Observer to consume it at a time, but unlike UnicastSubject, once the previous Observer disposes, a new Observer can subscribe and resume consuming the items.

which was exactly what I needed. For the lifecycle part, again I turned to LifecycleObserver and created:

class EffectsObserver<E>(
    private val effects: Observable<E>,
    private val executeEffect: (E) -> Unit
) : DefaultLifecycleObserver {

    private var disposable: Disposable? = null

    override fun onStart(owner: LifecycleOwner) {
        super.onStart(owner)
        disposable = effects.flatMap {
            Observable.fromCallable { executeEffect(it) }
        }.subscribe()
    }

    override fun onStop(owner: LifecycleOwner) {
        disposable!!.dispose()
        super.onStop(owner)
    }
}

the usage looks like:

override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    lifecycle.addObserver(EffectsObserver(viewModel.effects) { effect ->
        when (effect) {
            is NavigateToDetails -> TODO("Navigate")
        }
    })
}

this will make sure events such as navigation and showing a Snackbar are queued when the app is in the background and happen only when the app is in foreground. It helps avoid the dreaded java.lang.IllegalStateException: Can not perform this action after onSaveInstanceState.

Note: when using this with a Fragment it is important to register it in onCreate (called exactly once) and not in onViewCreated (may be called multiple times with fragments on the backstack).

Conclusion

You can emulate the way LiveData works for ViewModel -> LifecycleOwner communication with RxJava. However there are a few edge cases that you need to consider to do it right, like delivering events only when the LifecycleOwner is active.

If you have a different idea to achieve this leave a comment below. Happy coding!

What is a Unit in unit testing

Stojan Anastasov — Thu, 29 Aug 2019 17:33:03 +0000

What is a unit in unit testing? Is it a class, a function, a method, all of the above?

Is it considered a unit test if we are testing a class without mocking all of the collaborators?

Side Effects and Composition

Stojan Anastasov — Tue, 13 Aug 2019 00:00:00 +0000

Photo by Ryan Yoo on Unsplash

Functional Programing is amazing. It limits the things you can do (no nulls, no exceptions, no side effects…) and in return you get some benefits. Some benefits of FP are easy to explain and some not so much. I have been playing with FP for more than a year and only few weeks ago, when I started reading Functional Programming in Scala, I found an amazing example the benefits of pure functions.

The problem

Building a software for a Coffee Shop. For the initial MVP the requirements are:

Sell coffee
Pay with a card

so the first draft of the code looks like this:

import coffee.*

fun buyCoffee(cc: CreditCard): Coffee {
    val cup = Coffee()
    cc.charge(cup.price)
    return cup
}

The buyCoffee function receives a CreditCard as an input and returns a Coffee as an output. It creates a Coffee object, then executes a charge and returns the Coffee object.

Testability

The call cc.charge uses an SDK/API to talk to the credit card company. It involves authorization, network call(s), persisting a record in the DB. The buyCoffee function returns a Coffee and the charging happens on the side hence the terms “side effect”.

The side effect makes buyCoffee hard to test in isolation. It would be nice (and cheaper) to run tests without actually talking to the credit card company and doing an actual charge. One could also argue that a CreditCard should not know how it’s being charged.

The testability problem can be fixed using Dependency Injection.

fun buyCoffee(cc: CreditCard, p: Payments): Coffee {
    val cup = Coffee()
    p.charge(cc, cup.price)
    return cup
}

The logic of charging the credit card is extracted into the Payments object. In production code the real Payments implementation is used and a mock version is injected for tests.

New requirements

After a few weeks in business new requirements arrive. Some people buy more than one coffee so the next version of the software has additional requirements:

Buying N coffees
Single charge per Credit Card

Composition

The problem of buying a single coffee is already solved with buyCoffee. It would be nice to reuse the code for one coffee to buy multiple coffees. The code is complex (could be), tested and debugged. Reusing the existing solution would also save time.

Buying multiple coffees

The first draft of buyCoffees looks like this:

fun buyCoffees(
    cc: CreditCard,
    p: Payments,
    n: Int
): List<Coffee> = List(n) { buyCoffee(cc, p) }

it takes an additional parameter (compared to buyCoffee), the number of coffees to buy. It returns a List<Coffee> as expected. It uses buyCoffee to fill a list with n coffees.

Unfortunately this code charges the credit card multiple times. That is bad both for the customer and the business (each transaction costs money).

Again the problem is the side effect.

p.charge(cc, cup.price) // <-- Side effect

Side Effects and Referential Transparency

I mentioned the word side effect multiple times. It’s time to explain what it is. A side effect is something that breaks referential transparency (RT). Which leads to the question: What is referential transparency?

An expression is called referentially transparent if it can be replaced with its corresponding value without changing the program’s behavior. - Wikipedia

RT Example

Given this version of buyCoffee:

fun buyCoffee(cc: CreditCard, p: Payments): Coffee {
    val cup = Coffee()
    p.charge(cc, cup.price)
    return cup
}

RT means that the coffeeA can be replaced with it’s result a Coffee without altering the program.

val coffeeA: Coffee = buyCoffee(cc, p)  // returns Coffee()

val coffeeB: Coffee = Coffee()

coffeeA and coffeeB are not equivalent. With coffeeA your credit card is charged, with coffeeB you get a free coffee. That means buyCoffee is not referentially transparent.

Solving the problem

I can push the problem to the payment processor. I can create BatchPaymentsProcessor that batches the payments before executing. This opens questions like:

How long does it wait
How many payments do it batch
Is buyCoffee responsible for indicating start/end batch

BatchPaymentsProcessor solves the code reuse problem but brings a whole set of new problems to the table. Can we do better?

Removing the Side Effect

The side effect is preventing composition. If I remove it maybe I can regain composition.

fun buyCoffee(cc: CreditCard): Pair<Coffee, Charge> {
    val cup = Coffee()
    val charge = Charge(cc, cup.price)
    return Pair(cup, charge)
}

The side effect is gone now, so is the Payments object. The return type changed to Pair<Coffee, Charge>. A Charge object is returned in addition to the Coffee. The Charge object is a pure value (a data class) indicating a charge should be made. The Charge also contains the information needed to execute it like CC number and price.

The buyCoffee function is only responsible for creating the Charge which is pure (no side effects). The responsibility of executing it is in another part of the program.

Combining charges

Since Charge is a regular value it can easily be combined.

fun combine(c1: Charge, c2: Charge): Charge =
    if (c1.cc == c2.cc) Charge(c1.cc, c1.price + c2.price)
    else throw IllegalArgumentException(
        "Can't combine charges with different cc"
    )

two charges, given they have the same Credit Card are combined by adding their price.

Buying multiple coffees revisited

I can now implement the buyCoffees function in terms of the pure buyCoffee like this:

fun buyCoffees(
    cc: CreditCard,
    n: Int
): Pair<List<Coffee>, Charge> {
    val purchases: List<Pair<Coffee, Charge>> =
        List(n) { buyCoffee(cc) }
    val (coffees, charges) = purchases.unzip()
    val charge = charges.reduce { c1, c2 -> combine(c1, c2) }
    return Pair(coffees, charge)
}

The return type is Pair<List<Coffee, Charge>> containing a list of purchased coffees and a single charge for all of them. To create this the buyCoffee function is used to populate a list with n coffee items. The unzip function is used to separate the coffees from the charges. Charges are combined using the combine function defined above.

Again the concern of executing the purchase is pushed to another part of the program.

The benefits gained are:

Better testability - the functions are testable without mocks
Composition - the bigger function is implemented in terms of the smaller
Separation of concerns - decision of purchase is separate of purchase execution

Conclusion

By doing a simple thing like separating the decision of making a side effect and executing it the code is much more composable and reusable. This also improves testability as pure functions can be tested without using mocks. Being aware of the way side effects influence composition helps writing composable code.

Functional Hangman in Kotlin with Arrow (part 2)

Stojan Anastasov — Tue, 25 Dec 2018 00:00:00 +0000

Converting a Functional Hangman game from Scala (ZIO) to Kotlin (with Arrow) was a nice exercise. I enjoyed working on it and I learned a lot. When I asked for feedback on the #arrow channel, one of the maintainers, Leandro had an interesting suggestion. Instead of hard-coding the data type IO I should try and make the program polymorphic and use Kind instead. That means writing the code focusing on the domain logic, using abstractions, and deferring the decision for the concrete data type like IO or Single (from RxJava) until the main function.

The journey

I was not familiar with that style of programming so I used this example from the excellent Arrow documentation as a guide.

Writing to the the console

In the previous article I used IO<A> to interact with the console. IO<A> represents an operation that can be executed lazily, fail with an exception (the exception is captured inside IO), run forever or return a single A. Let’s take a look at the original implementation:

fun putStrLn(line: String): IO<Unit> = IO { println(line) }

// Usage in main()
putStrLn("Hello world!").unsafeRunSync()

putStrLn is a function that take a String and return a IO<Unit>. IO takes a lambda that is lazily evaluated at the end of the world, when we call unsafeRunSync(). If we want to achieve the same thing with Single we could use Single.fromCallable wrap our lambda and evaluate it in the main function when we call subscribe().

fun putStrLn(line: String): Single<Unit> = Single.fromCallable {
    println(line)
}

// Usage in main()
putStrLn("Hello World").subscribe()

Here bothIO and Single have something in common. A set of capabilities like: lazy evaluation , exception handling , and running forever or completing with a result of type A. IO and Single do a lot more, but for this use case, we want something as simple as possible that has the same capabilities. There is a type-class in Arrow that can do just that and it’s called MonadDefer(more info). After a few iterations, and feedback from the Arrow team, this is the code I came up with for printing to the console.

fun <F> putStrLn(M: MonadDefer<F>, line: String): Kind<F, Unit> = M.invoke {
    println(line)
}

The putStrLn function is generic with type F, but not any F. This F, whatever it is, need to do certain things like lazy evaluation and error handling. This F thing needs the capabilities of MonadDefer. The return value also needs a type parameter, and in Arrow we can do that by returning Kind<F, SOMETHING>. In the case of printing to command line, that SOMETHING is Unit (no return value). MonadDefer comes with an invoke function that we can use to construct the value of Kind<F, Unit>. We pass a lambda inside which will be lazily evaluated.

Compared to the original implementation we have a few key differences:

a type parameter F
one more parameter of type MonadDefer<F>
the return type is Kind<F, Unit> instead of IO<Unit>

Now we can use this function to print something to the console. In order to do that, we need a data type that has the capabilities of MonadDefer. IO can do that so we can use it. Arrow also ships with SingleK, a wrapper for Single that has the MonadDefer capabilities.

putStrLn(IO.monadDefer(), "Hello World!").fix()
        .unsafeRunSync()

putStrLn(SingleK.monadDefer(), "Hello World!").fix()
        .single.subscribe()

Arrow also ships with ObservableK for Observable, DeferredK for coroutines etc.

Reading from the console

Reading from the console is similar to writing to it. We still need a type parameter F, we still need to return Kind<F, SOMETHING> and we need to perform the operation lazily with success or an error. readLine() returns a nullable String? and if that happens we need to signal an error.

// Original code
fun getStrLn(): IO<String> = IO { readLine() ?: throw IOException("Failed to read input!") }

// Polymorphic code
fun <F> getStrLn(M: MonadDefer<F>): Kind<F, String> = M.invoke {
    readLine() ?: throw IOException("Failed to read input!")
}

Reading from the console #2

I am throwing an IOException to indicate failure. IO would wrap that exception so it isn’t that bad. But there is a better way (thanks Leandro).

I convert the nullable String? to an Option<String>. Then I use fold to check if the Option has a value. If it’s empty I use raiseError to create MonadDefer which when evaluated returns an error. If it’s not empty I create a MonadDefer that returns a String using just.

The key difference here is I am NOT throwing the IOException. Using exceptions can be expensive.

M.defer here means the readLine() happens lazily.

fun <F> getStrLn(M: MonadDefer<F>): Kind<F, String> = M.defer {
    readLine().toOption()
            .fold(
                    { M.raiseError<String>(IOException("Failed to read input!")) },
                    { M.just(it) }
            )
}

The Hangman class

The next step is to make the Hangman class polymorphic. To do that I added a type parameter F and property of the type MonadDefer<F>.

class Hangman<F>(private val M: MonadDefer<F>) {
    ...
}

Choosing a letter

To make the getChoice() function work in a polymorphic way we need a few changes. The return type changes from IO<Char> to Kind<F, Char>. IO.binding becomes M.binding (where M is the property of type MonadDefer<F>).

The getStrLn() and putStrLn() also need M as the parameter.

fun getChoice(): Kind<F, Char> = M.binding {
        putStrLn(M, "Please enter a letter").bind()
        val line = getStrLn(M).bind()
        val char = line.toLowerCase().first().toOption().fold(
                {
                        putStrLn(M, "Please enter a letter")
                                .flatMap { getChoice() }
                },
                { char ->
                        M { char }
                }
        ).bind()
        char
}

Wrapping up

Updating all other functions follows the same pattern. Replace IO<SOMETHING> with Kind<F, SOMETHING>, replace IO.binding with M.binding and pass M as parameter for reading/writing to the console.

You can find the full code here.

The main program

The main program, run with Single, looks like this:

Hangman(SingleK.monadDefer()).hangman.fix().single.subscribe()

or with IO

Hangman(IO.monadDefer()).hangman.fix().unsafeRunSync()

The decision in which type constructor to run is made at the point of execution. Switching from Single to IO or Observable requires only updating the main function.

Incremental improvements

I am still learning FP and I don’t know most of the type-classes in Arrow and what they can do. In my first attempt I used Async instead of MonadDefer. Async extend MonadDefer and adds additional capabilities to it. I asked for feedback on the #arrow channel and they pointed me towards MonadDefer. Together with the Arrow maintainers we made a few more improvements improvements.

To avoid passing M to printStrLn every time I can convert it to an extension function on MonadDefer

fun <F> MonadDefer<F>.putStrLn(line: String): Kind<F, Unit> = invoke {
    println(line)
}

and I can use Implementation by delegation and have the Hangman class implement MonadDefer<F>.

class Hangman<F>(private val M: MonadDefer<F>): MonadDefer<F> by M

This means everywhere inside the Hangman class I can use the methods of MonadDefer like binding and invoke and extension methods like putStrLn.

val hangman: Kind<F, Unit> = binding {
        putStrLn("Welcome to purely functional hangman").bind()
        val name = getName.bind()
        putStrLn("Welcome $name. Let's begin!").bind()
        val word = chooseWord.bind()
        val state = State(name, word = word)
        renderState(state).bind()
        gameLoop(state).bind()
        Unit
    }

You can find the full implementation here.

Conclusion

Working with abstractions like MonadDefer frees the business logic from implementation details like IO or Single. It can also enable easier composition of different modules because the decision for the concrete data type is delayed until the main program.

Functional Hangman in Kotlin with Arrow

Stojan Anastasov — Sun, 11 Nov 2018 00:00:00 +0000

A few days ago I run into this blog post about implementing a console Hangman game using Scala ZIO. The blog post is based on this talk delivered by John De Goes, for the Scala Kyiv meetup, where he codes a console Hangman game using functional programming.

After I saw the post I was curious how the code would look like written in Koltin with arrow so I decided to try and write it. You can find the result here.

I am still learning functional programming so I asked for feedback from the arrow maintainers on the kotlinlang slack workspace. They were very helpful and I made a few improvements based on their feedback.

Key differences

I would like to point out that the programs in Scala and Kotlin are not 100% equivalent. There are both differences in the language and the functional libraries used.

The IO monad

In Scala ZIO IO[E, A] describes an effect that may fail with an E, run forever, or produce a single A. In the Kotlin version I am using IO<A> from Arrow. IO<A> describes an effect that can fail with Throwable or produce a single A.

Both type classes produce an A. The key difference the error. In Scala ZIO you can use any error type or even Nothing to indicate the program never ends. In Arrow the E type is always Throwable so you don’t have to specify it.

Reading and writing from the Console

Scala ZIO comes with built in primitives for interacting with the console. In the Kotlin version I had to implement the readStrLn and putStrLn functions myself.

fun putStrLn(line: String): IO<Unit> = IO { println(line) }

fun getStrLn(): IO<String> = IO { 
    readLine() ?: throw IOException("Failed to read input!") 
}

For comprehensions

For comprehensions are built into the scala language. Unfortunately Kotlin doesn’t have the same feature. But Kotlin has Coroutines so the Arrow team built Comprehensions over coroutines which can be used in a similar way to make the code more readable.

//Scala with For comprehensions
val hangman : IO[IOException, Unit] = for {
    _ <- putStrLn("Welcome to purely functional hangman")
    name <- getName
    _ <- putStrLn(s"Welcome $name. Let's begin!")
    word <- chooseWord
    state = State(name, Set(), word)
    _ <- renderState(state)
    _ <- gameLoop(state)
} yield()

//Kotlin with Arrow (Comprehensions over coroutines)
val hangman: IO<Unit> = IO.monad().binding {
    putStrLn("Welcome to purely functional hangman").bind()
    val name = getName.bind()
    putStrLn("Welcome $name. Let's begin!").bind()
    val word = chooseWord.bind()
    val state = State(name, word = word)
    renderState(state).bind()
    gameLoop(state).bind()
    Unit
}.fix()

Conclusion

Rewriting Functional Hangman from Scala to Kotlin was a nice exercise for learning FP. There are differences between Scala and Kotlin and the IO type class in Scala ZIO and Arrow but the core principles are the same, we write programs by solving small problems and then combine the solutions. We use lazy evaluation and move the side effects to the edge of the system.

Thanks to John who wrote this functional hangman, Abhishek for writing the article and Paco, Raul and Leandro, from the Arrow team, for the feedback about my code.

Setting up GitLab CI for Android Projects

Stojan Anastasov — Sun, 03 Dec 2017 00:00:00 +0000

My first experience with continuous integration was using Bitbucket in combination with Jenkins. I was pretty happy with my setup. Jenkins would run on every commit making sure my code compiles, run android lint and run my unit tests. I also set up continuous deployment using Fabric.

Now, at work, we use GitLab as a code repository. GitLab also offers continuous integration. When we decided to start using continuous integration at work we decided to give GitLab a chance. It was already integrated with GitLab and to use it we just needed to install a runner.

Using CI with GitLab is simple, after you install a runner you need to add a .gitlab-ci.yml file at the root of the repository. GitLab even offers template .gitlab-ci.yml files for various languages and frameworks. The android template is based on this blog post from 2016. It is a great guide but unfortunately today it doesn’t work. Google introduced a few changes in the command line tools.

Installing Android SDK

To install the Android SDK on a CI we need to install the command line tools (scroll to the bottom to Get just the command line tools). The command line tools include the sdkmanager - a command line tool that allows you to view, install, update, and uninstall packages for the Android SDK. So instead of

- wget --quiet --output-document=android-sdk.tgz https://dl.google.com/android/android-sdk_r${ANDROID_SDK_TOOLS}-linux.tgz
  - tar --extract --gzip --file=android-sdk.tgz

we can use

- wget --quiet --output-document=android-sdk.zip https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip
  - unzip -q android-sdk.zip -d android-sdk-linux

to download and install the Android SDK tools.

There is also an improvement in accepting licenses for the Android SDK. After you accept the licenses on your development machine the tools will generate a licenses folder in the Android SDK root directory. You can transfer the licenses from your development machine to your CI server. To accept the licenses we can use:

- mkdir android-sdk-linux/licenses
  - printf "8933bad161af4178b1185d1a37fbf41ea5269c55\nd56f5187479451eabf01fb78af6dfcb131a6481e" > android-sdk-linux/licenses/android-sdk-license
  - printf "84831b9409646a918e30573bab4c9c91346d8abd" > android-sdk-linux/licenses/android-sdk-preview-license

This will create a folder licenses with two files inside. The values written in the files are from my local machine. Depending on which components you use there might be other files on your machine like license for Google TV or Google Glass. Once the licenses are accepted we can install packages:

- android-sdk-linux/tools/bin/sdkmanager --update > update.log
  - android-sdk-linux/tools/bin/sdkmanager "platforms;android-${ANDROID_COMPILE_SDK}" "build-tools;${ANDROID_BUILD_TOOLS}" "extras;google;m2repository" "extras;android;m2repository" > installPlatform.log

A sample project based on the Android Testing codelab with the full .gitlab-ci.yaml file can be found here. The rest of the script for building and unit testing is the same as the original post. The latest x86 emulator requires hardware acceleration to run so I’ll skip the functional tests for now.

Contributing to OSS for Hacktoberfest

Stojan Anastasov — Tue, 10 Oct 2017 00:00:00 +0000

Digital Ocean in partnership with Github are organizing the fourth Hacktoberfest event this October. If you make four pull requests between October 1 and October 31 to any Github hosted repository you get a Hacktoberfest T-shirt. This year I decided to take part and I already made two pull requests.

My first OSS contribution

I created my first pull request to an OSS project 3 years after I registered on Github. I know it can be hard to start contributing. At first I didn’t think my code was good enough, then I couldn’t find the right project with the right issue I could fix.

One day an opportunity presented itself. At work I was using auto-parcel to generate the boilerplate required for Parcebale implementation on Android. It’s a cool library based on auto-value by Google. At the time I was using version 0.3 - the version displayed in the README file on Github. A few days later while creating another value class I got an error. I googled the error and it led me to an issue on the project’s Github page. The issue was closed and a new version, version 0.3.1 was released. The README file was outdated, it still pointed to the previous release containing the bug.

I spent 10 minutes because of a bug that was already fixed. What a waste of time. If I wasted time chances are someone else will also run into the same problem so I decided to do something about it. Opening an issue on Github would be nice but the solution was so simple I decided to fix it myself and make a pull request. I also updated another dependency (android-apt) in the README to the latest version. The next day my pull request was accepted and it felt good.

Hacktoberfest 2017

A few days after Hacktoberfest 2017 started I run into an interested project on Github - TornadoFx a Lightweight JavaFX framework for Kotlin. I like Kotlin and it’s Hacktoberfest so I decided to contribute and maybe get a cool T-Shirt in the process. I noticed the project didn’t have enough unit tests so I decided to write a few. I found a file with some extension methods that were easy to test. They didn’t have tests so a wrote a few and I even found a little bug. I opened a pull request and my changes were merged. In my pull request I asked if more tests would be something they would be interested in. The owner replied:

Thanks for your contribution. Tests are most welcome of course :)

I also opened another pull request fixing the bug I had found.

Getting started with contributing to OSS can be intimidating. Starting small can help gain some confidence. Helping with documentation or tests, like I did, is a good place to start. There are thousands of beginner friendly issues tagged with hacktoberfest waiting for you.

Are you participating in Hacktoberfest? Do you remember your first OSS contribution?

My First Unit Test

Stojan Anastasov — Tue, 26 Sep 2017 19:45:02 +0000

There are different reasons developers write tests. Some of them are:

Validate the system
Feedback
To prevent regression bugs
Code coverage
To enable refactoring
Documenting the system
Orders from manager
And many other reasons

There are also reasons developers don't write tests like:

Don't know how to write tests
Writing tests is too hard
Not enough time to write tests
The code is too simple for tests.

This is the story about my first unit test written to save time.

The app

I was working on a simple Android app. The app consisted of two screens, a list of items and a details screen for creating/updating/deleting items. There was also a register/login screen that are irrelevant for this story. The app worked with a remote database (on parse.com) using the REST API. You can create an account in the app and you can CRUD some items via the API. There was a requirement for the app to work offline and sync changes with the server when it connected to the Internet.

To support offline work, I created a local database to mirror the remote database. To sync the local with the remote database I used an IntentService. The IntentService was responsible for comparing the items in the two databases and determining what should be updated in the local and/or remote database. This was the most complicated part of the app.

Testing

After I completed the database synchronization code I run the app on the emulator to test if it worked.

Testing scenarios:

New item in the local database -> update the remote db
New item in the remote database -> update the local db
Updated item in the local database -> update remote db
Updated item in the remote database -> update local db
Updated (different) item in both databases -> update both db

I was doing manual testing. First I would clear the app data (to get an empty database) and clear the remote database (using the Parse website). Then I created an item in the local db, ran the sync service and checked if it appears on the remote db. The I created an item in the remote db, ran the service and checked if it appears in the local db. Then I created two new items, one in the local db, one in the remote, ran the service, checked if it's OK. Now repeat this for updating items, deleting items… It could take up to 10-15 min to test the whole synchronization logic.

Of course my code didn't work the first time, it had a few bugs and I had to update my code. After a few cycles of changing two lines of code then test for 15 min I got tired. It was taking too long for me to see the effects from changing a few lines of code. There must be a better way. This boring, repetitive process was something a computer could do better.

Testing done right

Fortunately Android Studio version 1.1 (and the corresponding android gradle plugin) added support for unit testing. I extracted the synchronization logic from the IntentService into a Java class responsible for comparing two lists and determining what should I update. Then I wrote some unit tests for my synchronization logic. My only Android dependency in the synchronization code was TextUtils.isEmpty(). As a quick workaround I decided to copy/paste the implementation into my synchronization class. The unit tests run with JVM so no emulator. I run the tests and I got instant feedback, what had taken me 10 to 15 minutes manually was now automated down to just a few seconds. After fixing a few bugs in both the code and the tests (yeah I also wrote bugs in the tests) my app worked as expected.

Writing unit tests for my synchronization logic was simple because the class had no collaborators, didn't touch the UI and had no Android dependencies[1]. Usually writing unit tests is harder, you need to design your code to be testable and it takes time to see the benefits. This low effort - high reward test made me start testing more.

Do you write tests? Any interested testing stories? You can share them in the comments.

[1]except for TextUtils.isEmpty()