DEV Community: Aniket Bhoite

Kotlin Flow: Simple yet Powerful Implementation

Aniket Bhoite — Sat, 02 Sep 2023 17:55:18 +0000

Ever since Flow was first introduced in Kotlin version 1.3.0, it has become an important and highly discussed topic in the Kotlin community. But why?

Flow is a simple yet powerful implementation for asynchronous programming in Kotlin.

Flow is nothing but a coroutine that returns multiple results. It allows developers to write structurally concurrent, asynchronous and non-blocking code in a sequential style. But what do all these technical terms means to new developers, it's hard to grasp all of them at once. It took me some time to understand it.

The reason Flow can do this without being too expensive is because it has a very simple internal implementation. Flow is nothing but two interfaces and their functions calling each other. Let me explain, but first, forget for the moment anything you know about coroutines suspend function and all.

Let's start.

To understand how flow internally works, we only need to understand two main concepts.

Interface and Anonymous Implementation of an Interface
Functional interface, or a Single Abstract Method (SAM) interface in Kotlin

Let’s create two Flow interfaces.

public interface Flow {
    public fun collect(collector: FlowCollector)
}

public interface FlowCollector {
    public fun emit(value: String)
}



val outputStream = object : Flow {
    override fun collect(collector: FlowCollector) {

        collector.emit("first")
        collector.emit("second")

    }
}

outputStream.collect(
    object : FlowCollector {
        override fun emit(value: String) {
            println(value)

            println(value.uppercase())
        }

    }
)


/*

OUTPUT:

first
FIRST
second
SECOND

 */

P.S: For simplicity, we are currently only considering the Flow type as a string.

That's it!! This is flow in its simplest and bare-bones form, and we have only used the first concept so far. As you can see, this is nothing but one function directly calling another one multiple times.

Let's make this a little bit prettier by using a Functional Interface

An interface with only one abstract method is called a Functional Interface or a Single Abstract Method (SAM) interface. The functional interface can have multiple non-abstract members but only one abstract member.

public interface Flow {
    public fun collect(collector: FlowCollector)
}

public fun interface FlowCollector {
    public fun emit(value: String)
}


val outputStream = object : Flow {
    override fun collect(collector: FlowCollector) {

        collector.emit("first")
        collector.emit("second")

    }
}

outputStream.collect { value ->
    println(value)

    println(value.uppercase())
}

// SAME OUTPUT

It's starting to look similar now. The anonymous implementation of the Flow interface is boilerplate code. This can be solved by using a top-level function with the Flow anonymous implementation as a return type.

fun flow(block: FlowCollector.() -> Unit): Flow = object : Flow {
    override fun collect(collector: FlowCollector) {
        collector.block()
    }
}

public interface Flow {
    public fun collect(collector: FlowCollector)
}

public fun interface FlowCollector {
    public fun emit(value: String)
}


val outputStream = flow {

    emit("first")
    emit("second")

}


outputStream.collect { value ->
    println(value)

    println(value.uppercase())
}

//SAME OUTPUT

This is the familiar-looking flow we use under the hood. This doesn’t appear to be as feature-rich as we mentioned above. In theory, any programming language can do this so what makes Kotlin so unique? The answer is Kotlin’s magic function keyword "suspend”.

public interface Flow {
    public suspend fun collect(collector: FlowCollector)
}

public fun interface FlowCollector {
    public suspend fun emit(value: String)
}

Making functions suspendable means they can do much more than simple functions. Making these functions suspend ensures that flow can only be called from other suspend functions or coroutine builders. Try to remember what coroutines can do. Flow incorporates all the features of coroutine. For example:

Structured Concurrency: The Flow Builder block (collect function) runs in the same coroutine scope as the flow collector block by default. So when the scope is stopped both the builder and collector blocks stops executing, saving precious CPU resources. No need to manually close the stream.
Asynchronous, non-blocking code: We can control which code decide which code should run which Dispatcher. This makes it very useful for running IO operations on Background threads without blocking the original/main thread.
Sequential & Readable: Suspend functions wait for the result of other suspend functions in the order until then they remain in a suspending state without blocking the thread.
Back Pressure: The Flow Builder and Collector block runs on the same coroutine so when collector block is processing result, the Builder block is suspended to prevent it from producing unnecessary results that may not be processed by the Builder block. This special feature comes with flow by default. Other libraries have to do fancy workarounds to achieve this.

And many more features like this.

One of the flagship features of Flow is its operator functions. Are they complex to understand? No, because flow is nothing more than simple function calling each other. So, if we create an extension function with a flow as a return type, we can theoretically accomplish anything. Let’s look at the onStart and onCompletion operator functions.

fun Flow.onStart(
    action: FlowCollector.() -> Unit
) = flow {
    action()
    collect(this)
}

fun Flow.onCompletion(
    action: FlowCollector.() -> Unit
) = flow {
    collect(this)
    action()
}


......

outputStream
   .onStart {
        println("onStart")
    }
    .onCompletion {
        println("onComplete")
    }
    .collect { value ->
        println(value)

        println(value.uppercase())
    }


/*

OUTPUT:

onStart
first
FIRST
second
SECOND
onComplete

 */

PS: This is a highly simplified explanation. There are many complex processes happening behind the scenes, but the main logic remains unchanged.

So, as you can see, we have the ability to create any type of flow operator we can imagine using extension functions. Feel free to go crazy!

To satisfy someone's curiosity, this is a Flow interface with a Generic type:

public interface Flow<out T> {
    public suspend fun collect(collector: FlowCollector<T>)
}

public fun interface FlowCollector<in T> {
    public suspend fun emit(value: T)
}

I hope you understand why Flow is a simple yet powerful API implementation.

Resources:

KotlinConf 2019: Asynchronous Data Streams with Kotlin Flow by Roman Elizarov: https://youtu.be/tYcqn48SMT8?si=4uo-G2Ryw4IQPYdS&t=1385
Functional (SAM) interfaces: https://kotlinlang.org/docs/fun-interfaces.html
Kotlin extension function: https://kotlinlang.org/docs/extensions.html
Kotlin Generics: https://kotlinlang.org/docs/generics.html#variance

Using KSP output in app: Part 4

Aniket Bhoite — Mon, 11 Apr 2022 06:07:12 +0000

Now that we have our generated class, Let's try to use it in code.

Checkout other parts in this series:

Android KSP guide for dummies by a Dummy: Part 1 (link)

KSP Gradle setup & Processor's first log: Part 2

Generate the code using KSP : Part 3 (link)

Using KSP output in app: Part 4

First let's look at what is Event Interface and what methods it contains.

package com.aniket.myevent

import android.os.Bundle

interface Event {
    fun getHashMapOfParamsForCustomAnalytics(): HashMap<*, *>?

    fun getBundleOfParamsForFirebase(): Bundle
}

It means all generated Event classes must override both of the above-mentioned methods.

We also need a Utility(helper) class that will use Event interface classes to fire events to respective analytics SDKs.

package com.aniket.myevent

import android.util.Log

object EventUtils {

    fun postEvent(event: Event) {
        sendFirebaseEvent(event)
        sendCustomAnalyticEvent(event)
    }

    private fun sendFirebaseEvent(event: Event) {
        Log.i("Firebase_Event_fire", event.getBundleOfParamsForFirebase().toString())
    }

    private fun sendCustomAnalyticEvent(event: Event) {
        Log.i("Custom_Analytics_Event_fire", event.getHashMapOfParamsForCustomAnalytics().toString())
    }
}

By default IDE is not aware the generated code. So it will mark references to generated symbols unresolvable. To make an IDE to aware about the generated codes we have to add following line in app(or consumer module)/build.gradle so IDE can import correct package names.

Groovy Gradle

// app/build/gradle

buildTypes {
    release {
        ....
        sourceSets {
            main {
                java { srcDirs = ["src", "build/generated/ksp/release/kotlin"]}
            }
        }
    }
    debug {
        ....
        sourceSets {
            main {
                java { srcDirs = ["src", "build/generated/ksp/debug/kotlin"]}
            }
        }
    }
}

Kotlin Gradle

// app/build.gradle.kts

buildTypes {
    getByName("release") {
        ....
        proguardFiles(
                getDefaultProguardFile("proguard-android-optimize.txt"),
                "proguard-rules.pro"
        )
        sourceSets {
            getByName("main") {
                java.srcDir(File("build/generated/ksp/release/kotlin"))
            }
        }
    }
    getByName("debug") {
        ....
        sourceSets {
            getByName("main") {
                java.srcDir(File("build/generated/ksp/debug/kotlin"))
            }
        }
    }
}

Last thing do is actually trigger an Event

bookTicketButton.setOnClickListener {
    EventUtils.postEvent(
        TicketBookClickedParamsEvent(
            TicketBookClickedParams(
                eventName = "TicketBookClicked",
                screenName = "MainActivity",
                ticketNumber = 1223,
                ticketAmount = "1220"
            )
        )
    )
}

Let's run the app and see fired events log

Links
GitHub repo part-4 branch: https://github.com/aniketbhoite/ksp-my-event/tree/part-4

Generate the code using KSP : Part 3

Aniket Bhoite — Mon, 11 Apr 2022 06:06:31 +0000

Good Morning, In this part let's start with understanding how processor gets called and how to use it.

Checkout other parts in this series:

Android KSP guide for dummies by a Dummy: Part 1 (link)

KSP Gradle setup & Processor's first log: Part 2 (link)

Generate the code using KSP : Part 3

Using KSP output in app: Part 4 (link)

Steps

1) Defining MyEvent annotation

First lets define MyEvent annotations class in annotations module

package com.aniket.myevent.annotations

@Target(AnnotationTarget.CLASS)
annotation class MyEvent

2) Processing all Annotated classes in MyEventProcessor

Registered Class MyEventProcessorProvider which implements SymbolProcessorProvider interface. We must override the create method with SymbolProcessorEnvironment variable. In this method we have to initialise our MyEventProcessor class with following variables(check part 2 point 4)

environment.codeGenerator: helps with creating and managing files
environment.logger: for logging to build output.

MyEventProcessor class implements SymbolProcessor interface. We have to override abstract method process(resolver: Resolver). In this method we use resolver to get access to classes, functions or variables that are annotated with specified Annotation.

Following code snippet we get us all class's with our Annotation i.e. MyEvent as KSAnnotated implantation.

val symbols = resolver
    .getSymbolsWithAnnotation(MyEvent::class.qualifiedName!!)
val unableToProcess = symbols.filterNot { it.validate() }

We have to filter symbols because we only want symbols which are instance of KSClassDeclaration and then we will visit each symbol one by one separately


val dependencies = Dependencies(false, *resolver.getAllFiles().toList().toTypedArray())

symbols.filter { it is KSClassDeclaration && it.validate() }
    .forEach { it.accept(MyEventKClassVisitor(dependencies), Unit) }

return unableToProcess.toList()

In KSP we use Visitor Pattern, We will create MyEventKClassVisitor inner class that implements KSVisitorVoid() interface. Here we are only concerned with Classes so we will only override visitClassDeclaration method.

JavaTPoint website has very good easy explanation on visitor pattern, click here to read

private inner class MyEventKClassVisitor(val dependencies: Dependencies) : KSVisitorVoid() {

    override fun visitClassDeclaration(classDeclaration:             
        KSClassDeclaration, data: Unit) {
        ....
    }
}

Before we go any further we need some validations that the classes that are annotated with MyEvent are not Abstract or Interface.

if (classDeclaration.isAbstract()) {
         logger.error(
          "||Class Annotated with MyEvent should kotlin data class",classDeclaration)
         }

if (classDeclaration.classKind != ClassKind.CLASS) {
       logger.error(
            "||Class Annotated with Projections should kotlin data class", classDeclaration)
    }

Next we need some variable strings.

val className = classDeclaration.simpleName.getShortName()
val classPackage = classDeclaration.packageName.asString() + "." + className
val classVariableNameInCamelCase = className.replaceFirst(className[0], className[0].lowercaseChar()) //need this for using in generated code

logger.warn("package $classPackage")

We will also access that Class's primary constructor variables list

val properties = classDeclaration.primaryConstructor?.parameters ?: emptyList()

if (properties.isEmpty())
                logger.error("No variables found in class", classDeclaration)

Tip: Use logger to understand and verify KSP variables.

Now that we are on topic of properties lets add some helper Kotlin Extension function in processor module as below


fun KSValueParameter.isNotKotlinPrimitive(): Boolean {

    return when (type.element?.toString()) {
        "String", "Int", "Short", "Number", "Boolean", "Byte", "Char", "Float", "Double", "Long", "Unit", "Any" -> false
        else -> true
    }
}

fun KSValueParameter.getPrimitiveTypeName(): String {

    return type.element?.toString() ?: throw IllegalAccessException()
}

Now we have access to all things we need for code generation.

As per our requirement we need to generate 3 things

Override Method getBundleOfParamsForFirebase which returns Bundle with key values of event params
Override Method getHashMapOfParamsForCustomAnalytics which returns HashMap
class that implements Event interface and holds above methods

We need to add all parameters in hashmap and bundle in specific syntax.

for Hashmap we need all properties like

hashmap.put("screenName",parmas.screenName)

for bundle

putString("screenName", params.screenName)

We will loop through all the properties and create their hash map and bundle entries in String builder.

val hashmapEntries = StringBuilder()
val bundleEntries = StringBuilder()
for (prop in properties) {
    // Throw Error if param is not primitive
    if (prop.isNotKotlinPrimitive())
        logger.error("|| Event params variables should be Primitive", prop)

    val propName = prop.name?.getShortName() ?: ""
    logger.warn("|| ${prop.name?.getShortName()}")

    hashmapEntries.append(
        """
                put("$propName", $classVariableNameInCamelCase.$propName)

                """.trimMargin()
    )

    val propPrimitiveTypeName = prop.getPrimitiveTypeName()

    bundleEntries.append(
        """
                put$propPrimitiveTypeName("$propName", $classVariableNameInCamelCase.$propName)

                """.trimMargin()
    )
}

Now we just have to generate Event class

private val packageName = "com.aniket.myevent"

val toGenerateFileName = "${classDeclaration.simpleName.getShortName()}Event"

val outputStream: OutputStream = codeGenerator.createNewFile(
    dependencies = dependencies,
    packageName,
    fileName = toGenerateFileName
)



outputStream.write(
    """
    |package $packageName

    |import $classPackage
    |import android.os.Bundle
    |import $packageName.Event

    |class $toGenerateFileName(val $classVariableNameInCamelCase: $className): Event {
    |   override fun getHashMapOfParamsForCustomAnalytics(): HashMap<*, *>? {
    |       val map = HashMap<String, Any>().apply {
    |       $hashmapEntries
    |       }
    |       return map
    |    }
    |    
    |    override fun getBundleOfParamsForFirebase(): Bundle {
    |       val bundle = Bundle().apply {
    |       $bundleEntries
    |       }
    |       return bundle
    |    }
    |}
    """.trimMargin().toByteArray()
)

That's it. Lets look at example of our usecase.

For example with following class we ran or build app

import com.aniket.myevent.annotations.MyEvent

@MyEvent
data class TicketBookToClickedParams(
    val eventName: String,
    val screenName: String,
    val ticketNumber: Int,
    val ticketAmount: String,
)

The generated code of above Annotated class will be as follow at project/app/build/generated/ksp/debug/kotlin/com/aniket/myevent :

import android.os.Bundle

class TicketBookToClickedParamsEvent(val ticketBookToClickedParams: TicketBookToClickedParams) :
    Event {
    override fun getHashMapOfParamsForCustomAnalytics(): HashMap<*, *>? {
        val map = HashMap<String, Any>().apply {
            put("eventName", ticketBookToClickedParams.eventName)
            put("screenName", ticketBookToClickedParams.screenName)
            put("ticketNumber", ticketBookToClickedParams.ticketNumber)
            put("ticketAmount", ticketBookToClickedParams.ticketAmount)

        }
        return map
    }

    override fun getBundleOfParamsForFirebase(): Bundle {
        val bundle = Bundle().apply {
            putString("eventName", ticketBookToClickedParams.eventName)
            putString("screenName", ticketBookToClickedParams.screenName)
            putInt("ticketNumber", ticketBookToClickedParams.ticketNumber)
            putString("ticketAmount", ticketBookToClickedParams.ticketAmount)

        }
        return bundle
    }
}

At this point we are able generate Event class. Now the only thing remaining is Initialising and calling their methods to fire respective Events. We will do that in 4th and last part. Using KSP output in app: Part 4

Links
GitHub repo part-3 branch: https://github.com/aniketbhoite/ksp-my-event/tree/part-3

KSP Gradle setup & Processor's first log: Part 2

Aniket Bhoite — Mon, 11 Apr 2022 06:04:52 +0000

In first part we covered what KSP is and what problem we want solve using KSP. This part will cover setting up KSP in Android project with Groovy Gradle or new Kotlin DSL Gradle o Running KSP processor with first log

Checkout other parts in this series:

Android KSP guide for dummies by a Dummy: Part 1 (link)

KSP Gradle setup & Processor's first log: Part 2

Generate the code using KSP : Part 3 (link)

Using KSP output in app: Part 4 (link)

Steps

1) Adding JVM plugin in Project

In project build gradle

Note: Different Kotlin DSL syntax is added in code block wherever needed

//project/build.gradle

...
plugins {
    ...
    id "org.jetbrains.kotlin.jvm" version "1.6.10" apply false
}


//In kts.
/*
plugins {
    ...
    kotlin("jvm") version "1.6.10" apply false
}
*/

...

If project doesn’t have gradlePluginPortal() in plugin repositories then add it as below:

repositories {
    gradlePluginPortal()
    ...
}

2) Creating Processor Module

Create New module as java library named processor to host symbol processor. Apply java-library, org.jetbrains.kotlin.jvm & kotlin plugins in gradle with KSP's symbol processing api dependency. In processor's build.gradle

//processor/build.gradle

plugins {
    id 'java-library'
    id 'org.jetbrains.kotlin.jvm'
    id 'kotlin'
}

//in Kts
/*
plugins {
    id("java-library")
    id("org.jetbrains.kotlin.jvm")
    id("kotlin")
}
*/

java {
    sourceCompatibility = JavaVersion.VERSION_1_8
    targetCompatibility = JavaVersion.VERSION_1_8
}

dependencies {
    implementation(project(":annotations"))
    implementation("com.google.devtools.ksp:symbol-processing-api:1.6.10-1.0.2")
}

3) Creating Annotations Module

Create New module as java library named annotations to define our custom annotations.

annotations module's build.gradle

//annotations/build.gradle

plugins {
    id 'java-library'
    id 'org.jetbrains.kotlin.jvm'
    id 'kotlin'
}

//in KTS
/*
plugins {
    id("java-library")
    id("org.jetbrains.kotlin.jvm")
    id("kotlin")
}
*/

java {
    sourceCompatibility = JavaVersion.VERSION_1_8
    targetCompatibility = JavaVersion.VERSION_1_8
}

dependencies {
    implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.6.10")
}

4) Creating Processor Classes

Create MyEventProcessor class

package com.aniket.myevent.processor

...

class MyEventProcessor(
    private val codeGenerator: CodeGenerator,
    private val logger: KSPLogger,
    private val options: Map<String, String>
): SymbolProcessor {
    override fun process(resolver: Resolver): List<KSAnnotated> {

        logger.warn("Async Task API was very good API")
        return emptyList()
    }
}

Create MyEventProcessorProvider class

package com.aniket.myevent.processor

...

class MyEventProcessorProvider: SymbolProcessorProvider {
    override fun create(environment: SymbolProcessorEnvironment): SymbolProcessor {
        return MyEventProcessor(
            environment.codeGenerator,
            environment.logger,
            environment.options
        )
    }
}

Register your processor in processor module
processor/src/main/resources/META-INF/services/com.google.devtools.ksp.processing.SymbolProcessorProvider

as follows

com.aniket.myevent.processor.MyEventProcessorProvider

5) Applying KSP modules to app module

Lets use annotation & processor module for KSP

Add KSP plugin in the Project
project's build.gradle

//project/build.gradle

buildscript {
    dependencies {
        ...
        classpath("com.google.devtools.ksp:com.google.devtools.ksp.gradle.plugin:1.6.10-1.0.2")
    }
}

Apply KSP plugin, annotation module and processor module as KSP in app module

//app/build.gradle


plugins {
    ...
    id 'com.google.devtools.ksp'
}

//in KTS
/*
plugins {
    ...
    id("com.google.devtools.ksp")
}
*/

dependencies {
    ...
    implementation(project(":annotations"))
    ksp(project(":processor"))
}

Note: For now we are not writing any code in Annotation module, we will
do that in future parts

If you want to run KSP in any other modules then same steps apply to that modules too

Let's add Event Interface early so we can use that in future parts. You can add this interface in any package but make sure you use same the package name for Event interface in the code generation in future parts.

package com.aniket.myevent

import android.os.Bundle

interface Event {
    fun getHashMapOfParamsForCustomAnalytics(): HashMap<*, *>?

    fun getBundleOfParamsForFirebase(): Bundle
}

Now Run app or build project and we can see our KSP log in build output

Hooray!!!! You just added & ran your first KSP processor. Good job. Take a deep breath and go to sleep, KSP is not that important. GN

In Next part, We will understand how KSP processor works and how to generate Code. Generate the code using KSP : Part 3

Links
GitHub repo part-2 branch: https://github.com/aniketbhoite/ksp-my-event/tree/part-2

Android KSP guide for dummies by a Dummy: Part 1

Aniket Bhoite — Mon, 11 Apr 2022 06:04:26 +0000

KSP, Kotlin Symbol Processing is answer to annotation processing for Kotlin by google. It replaces older Kapt(Kotlin Annotations Processing) which was slower and less flexible to support native features of Kotlin language.

KSP (or Kapt) is useful for meta-programming and writing repetitive code. We analyse code and use code to write code

Problem to solve

To demonstrate let's take a problem to solve using KSP. In an average app there are maybe more than one analytic tools that requires event params in different formats. For example firebase Analytics SDK require params in key-value bundle but our own custom Analytic SDK accepts params in key-value pair HashMap

For that we need to create class with two functions, one returning attributes in Bundle and 2nd in HashMap for every event

    fun getBundleOfParamsForFirebase(): Bundle {
        val bundle = Bundle().apply {
            putString(paramName1, param1)
            putString(paramName2, param2)
        }
        return bundle
    }

    fun getHashMapOfParamsForCustomAnalytics(): HashMap<*, *>? {
        val map = HashMap<String, Any>().apply { 
            put(paramName1, param1)
            put(paramName1, param1)
        }
        return map
    }

This looks simple and doable manually but as number of params and number events increases it becomes cumbersome and error prone

Let's look at a solution, in this we will only create EventParams class with annotation based on which our Event class will get created. for ex.

    @MyEvent
    data class TicketBookClickedParams(
        val eventName: String,
        val screenName: String,
        val ticketNumber: String,
        val ticketAmount: String
    )

KSP processor will recognise all data classes Annotated with @MyEvent annotation and generate Event class for it.

This guide is further divided into parts, feel free to jump around if you like.

Intro
KSP Gradle setup & Processor's first log: Part 2 (link)
Generate the code using KSP : Part 3 (link)
Using KSP output in app: Part 4 (link)

Title & disclaimer

This series is for devs who have never used or created anything using KSP. This will not cover advance & complex use cases. This example will not have any kind of design pattern. We will go with straight forward approach which is not recommended for production apps. That’s why title is guide for dummies by a Dummy(a or the Dummy depending upon whom you ask.)

If anyone finds any mistakes or have any suggestion please feel free to add comment.

Links:
GitHub Repo : https://github.com/aniketbhoite/ksp-my-event
Google KSP Doc: https://github.com/google/ksp
KotlinLang KSP Doc: https://kotlinlang.org/docs/ksp-overview.html