DEV Community: Daniel Kuroski

Building a Webhook payload delivery service in Go

Daniel Kuroski — Mon, 20 Jan 2025 08:51:14 +0000

The full repo can be found here https://github.com/kuroski/go-webhook-delivers

Introduction

I was recently building an application that needed to listen to GitHub Webhooks.

For that, a "Payload URL" must be provided so GitHub can forward its events to this endpoint.

Once deployed and a public endpoint is available, it is a matter of only providing it.

But...

What to do during development?
How can I receive those Webhook events in my localhost?

There is no way I can just input http://localhost:3000 and expect it to work.

To develop this app locally, I needed a way to forward those events to the local environment.

In this tutorial, we will build a Webhook proxy URL.
It will forward events from some service (in this case GitHub) to your local application.

As a disclaimer, yes... There are plenty of ways to address this problem.

You can use services like smee.io (which is already suggested on GitHub tutorials).

Or even use something like ngrok to make the application securely available on its global edge in seconds.

But how cool it is to build this kind of service yourself, right?

What will we build?

This tutorial will consist of two applications

A CLI, which will run on our local machine, establishing a connection with the server and will be listening for events and forwarding them to a target local URL
A server that receives events from GitHub and forwards them to any listener (in this case, the CLI)

First, let's understand the big picture on how things will work.

We have four main players:

GitHub, that will produce the events in which will be dispatched to an endpoint
The server, will receive those events and forwards them to any listening client
The CLI, which will be running on your machine, it will subscribe for events on the server and forwards them
localhost, in this case, we will create a dummy server to receive the forwarded requests from the CLI

The CLI application will establish a connection with the server
GitHub will forward its events to an endpoint of our server
The request will be encoded
The encoded request will be forwarded to the CLI application
The data will be decoded
And finally, a new request will be made using the same parameters the GitHub Webhook request had sent initially
The connection between CLI and the server will close once we stop the CLI execution.

Server-sent events

This "connection" between the CLI and the Server happens in real time, but... How does this communication work?

There are plenty of ways to deal with "real-time". We could be using Polling, HTTP2 Push, Websockets, gRPC, etc.

But for this application we will be relying on SSE (server-sent events).

SSE is a pushing technology that enables pushing notification/message/events from the server to the client(s) via HTTP connection.
It has a simple protocol, and communication happens on a one-way connection, meaning that only the server sends events (unlike Websockets in which clients can also send events to the server), this makes it easier to reason and implement.

Starting out

First, let's create a basic structure for our project, then work out from it.

Create a folder for your project and start a new Go application

mkdir go-webhook-deliveries
cd go-webhook-deliveries
go mod init github.com/<your-username>/go-webhook-deliveries # I will be using `kuroski` as the name of my repo

We will be using a few packages, so let's install them

go get github.com/cenkalti/backoff/v5 github.com/lmittmann/tint github.com/tmaxmax/go-sse github.com/google/go-github/v67

Server

This will be a slim server with only two endpoints.

GET	/channel/{channel}	Starts sending incoming events to the client while logging any errors. CLI will use this endpoint to subscribe for messages.
POST	/channel/{channel}	GitHub will hit this endpoint with the events, the server will then encode the request and publish to its listeners.

It is not required to have a wildcard {channel}, feel free to work with a static endpoint if you don't need to have multiple entries.

First, we have our server main function, here we will be only instantiating its dependencies

// cmd/web/main.go

package main

import (
    "flag"
    "github.com/kuroski/go-webhook-deliveries/internal/server"
)

func main() {
    // grab the server port or default it to 3000 in case nothing is provided
    addr := flag.String("addr", ":3000", "HTTP network address")
    flag.Parse()

    // instantiate and start the server
    srv := server.NewServer(*addr)
    srv.Start()
}

Then, we can create the actual server implementation

// internal/server/server.go
package server

import (
    "context"
    "github.com/kuroski/go-webhook-deliveries/internal/logger"
    "github.com/tmaxmax/go-sse"
    "log/slog"
    "net/http"
    "os"
)

type Server struct {
    sseServer *sse.Server
    server    http.Server
    log       *slog.Logger
    ctx       context.Context
}

func NewServer(addr string) *Server {
    server := &Server{
        log: logger.NewLogger(),
        sseServer: &sse.Server{
            // this callback is called when an SSE session is started
            // here we set a subscription of the client for a given topic
            // if you decide that you don't need or want to handle wildcards, feel free to append the client to a static topic
            OnSession: func(s *sse.Session) (sse.Subscription, bool) {
                channel := s.Req.PathValue("channel")
                return sse.Subscription{
                    Client: s,
                    Topics: append([]string{channel}, sse.DefaultTopic),
                }, true
            },
        },
    }

    server.server = http.Server{
        Addr:     addr,
        Handler:  server.routes(),
        ErrorLog: slog.NewLogLogger(logger.NewLogger().Handler(), slog.LevelError),
    }

    return server
}

func (srv *Server) Start() {
    srv.log.Info("starting server", slog.String("addr", srv.server.Addr))

    err := srv.server.ListenAndServe()
    if err != nil {
        srv.log.Error(err.Error())
        os.Exit(1)
    }
}

Then, we need to define our routes

// internal/server/routes.go

package server

import (
    "net/http"
)

func (srv *Server) routes() http.Handler {
    mux := http.NewServeMux()
    mux.HandleFunc("GET /channel/{channel}", srv.subscribeToChannelHandler)
    mux.HandleFunc("POST /channel/{channel}", srv.forwardRequestToChannelHandler)
    return mux
}

And finally, the actual endpoint handlers

// internal/server/handlers.go

package server

import (
    "encoding/json"
    "fmt"
    "github.com/kuroski/go-webhook-deliveries/internal/model"
    "github.com/tmaxmax/go-sse"
    "io"
    "net/http"
    "time"
)

// this will handle the "GET /channel/{channel}" endpoint 
// it will establish the connection with the client and dispatch messages
// this will trigger the "OnSession" we configured on the server file, and will subscribe the client to a topic (in this case, the channel wildcard)
func (srv *Server) subscribeToChannelHandler(w http.ResponseWriter, r *http.Request) {
    srv.sseServer.ServeHTTP(w, r)
}

// this will handle the "POST /channel/{channel}" endpoint
// GitHub will send its events here, we will encode its data + request metadata and publish it to all channel listeners
func (srv *Server) forwardRequestToChannelHandler(w http.ResponseWriter, r *http.Request) {
    // first, we grab the "{channel}" wildcard value
    channel := r.PathValue("channel")

    // then, we read the Body content
    body, err := io.ReadAll(r.Body)
    defer r.Body.Close()
    if err != nil {
        srv.serverError(w, r, err)
        return
    }

    // we then wrap the received request into our own request struct (you can find its implementation below)
    // here we are wrapping everything needed to "re-create" the request later
    req := model.Request{
        Headers:   r.Header,
        Body:      body,
        Query:     r.URL.RawQuery,
        Timestamp: time.Now(),
    }

    // this is then encoded as JSON
    payload, err := json.Marshal(req)
    if err != nil {
        srv.serverError(w, r, err)
        return
    }

    // and then its content is wrapped over a SSE Message (which is a plain string)
    message := &sse.Message{}
    message.AppendData(string(payload))
    srv.log.Info("Hook received, forwarding the request", "req", req, "body", body)

    // finally, the message is published to all listeners to the channel
    if err = srv.sseServer.Publish(message, channel); err != nil {
        srv.serverError(w, r, err)
    }
    if _, err := fmt.Fprintf(w, "Message forwarded"); err != nil {
        srv.serverError(w, r, err)
    }
}

With that, we have our entire server configured and working.

Feel free to deploy this application if you have a preferred way.

If you don't know how to deploy a Go application, no worries, I will be writing in the appendices section how I did it.

Structs and utility functions

// internal/model/request.go

package model

import (
    "net/http"
    "time"
)

type Request struct {
    Headers   http.Header
    Body      []byte
    Query     string
    Timestamp time.Time
}

// internal/server/helpers.go

package server

import (
    "net/http"
    "runtime/debug"
)

func (srv *Server) serverError(w http.ResponseWriter, r *http.Request, err error) {
    var (
        method = r.Method
        uri    = r.URL.RequestURI()
        trace  = string(debug.Stack())
    )

    srv.log.Error(err.Error(), "method", method, "uri", uri, "trace", trace)
    http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
}

// internal/logger/logger.go

package logger

import (
    "github.com/lmittmann/tint"
    "log/slog"
    "os"
    "time"
)

func NewLogger() *slog.Logger {
    return slog.New(
        tint.NewHandler(os.Stdout, &tint.Options{
            Level:      slog.LevelDebug,
            TimeFormat: time.Kitchen,
            AddSource:  true,
        }),
    )
}

Creating the CLI

We have our server up and running, now we need to create the final piece.

Just like the server...

First, we have our main function, here we will be only instantiating its dependencies

// cmd/cli/main.go

package main

import (
    "bytes"
    "context"
    "encoding/json"
    "flag"
    "github.com/cenkalti/backoff/v5"
    "github.com/kuroski/go-webhook-deliveries/internal/logger"
    "github.com/kuroski/go-webhook-deliveries/internal/model"
    "github.com/tmaxmax/go-sse"
    "net/http"
    "net/url"
    "os"
    "os/signal"
    "syscall"
)

func main() {
    // we need a source URL for listening to the events (this will be the in the end something like "https://my-app.com/channel/my-channel"
    source := flag.String("source", "", "The source server that we will listen for events")
    // and then the target URL, where we want the requests to be forwarded to, something like "http://localhost:3000"
    target := flag.String("target", "", "The target server that the request will be forwarded to")
    flag.Parse()

    log := logger.NewLogger()

    if *source == "" || *target == "" {
        log.Error("Required flags --source and --target must be set")
        flag.Usage()
        os.Exit(1)
        return
    }

    sourceURL, err := url.Parse(*source)
    if err != nil {
        log.Error("--source argument must be a valid url", "source", *source)
    }

    targetURL, err := url.Parse(*target)
    if err != nil {
        log.Error("--target argument must be a valid url", "target", *target)
    }

    ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
    defer cancel()

    // this is the most important part, we must create a SSE Client
    client := sse.DefaultClient

    // and then initialize a connection with the server
    r, _ := http.NewRequestWithContext(ctx, http.MethodGet, sourceURL.String(), http.NoBody)
    conn := client.NewConnection(r)

    // finally, we need only to subscribe for messages sent from the server
    // every time a event occurs, this callback function will be called receiving an encoded "model.Request"
    conn.SubscribeMessages(func(event sse.Event) {
        // for the final steps, we need to Unmarshal the content
        var request model.Request
        if err := json.Unmarshal([]byte(event.Data), &request); err != nil {
            log.Error(err.Error())
            return
        }

        log.Info(
            "Message received",
            "headers",
            request.Headers,
            "query",
            request.Query,
            "timestamp",
            request.Timestamp,
            "body",
            request.Body[:100],
        )

        // and then we re-create the request to our target URL
        targetURL.RawQuery = request.Query
        req, err := http.NewRequest("POST", targetURL.String(), bytes.NewBuffer(request.Body))
        if err != nil {
            log.Error(err.Error())
            return
        }

        req.Header = request.Headers.Clone()
        res, err := client.HTTPClient.Do(req)
        if err != nil {
            log.Error(err.Error())
            return
        }

        log.Info("Request delivered successfully", "response", res)
    })

    // We are making use of "backoff" library to handle some edge cases.
    // After establishing a connection with the server, "tmaxmax/go-sse" library handles automatically the connection.
    // But it only happens after the first event is sent.
    // So, if there is no initial event for a while, we get a timeout and the application exits.
    // To avoid that situation, we use Exponential Backoff, meaning that in case of timeout, we avoid exiting the app, and try to establish a new connection instead after a period of time.
    _, err = backoff.Retry(ctx, func() (any, error) {
        log.Info("connecting sse, waiting for events from", "source", sourceURL.String(), "target", targetURL.String())
        if err := conn.Connect(); err != nil {
            log.Info(err.Error())
            return nil, err
        }

        return nil, nil
    }, backoff.WithBackOff(backoff.NewExponentialBackOff()))
    if err != nil {
        log.Error(err.Error())
    }
}

Testing out

If you finished everything and deployed your server, we can finally test our application.

If you don't know how to deploy a Go application, no worries, I will be writing in the appendices section how I did it.

First, go to your repository, and inside the "Settings > Webhooks" menu, let's create a new webhook

Provide your server URL and feel free to select which events you would like to trigger this webhook
I will be using the default which is triggered every time something is pushed to the project
Then, let's create a basic local server to receive the forwarded event from the CLI

// cmd/test/main.go

package main

import (
    "fmt"
    "github.com/google/go-github/v67/github"
    "net/http"
)

func main() {
    http.HandleFunc("/push-event", func(w http.ResponseWriter, r *http.Request) {
        payload, _ := github.ValidatePayload(r, nil)
        evt, _ := github.ParseWebHook(github.WebHookType(r), payload)
        pushEvt := evt.(*github.PushEvent)
        commit := pushEvt.Commits[0]
        fmt.Println("----- Webhook received", *commit.Message, *commit.Timestamp)
    })
    http.ListenAndServe(":3000", nil)
}

This should be only for testing the forwarded messages, so we will log only the commit message and timestamp.

Then, run the test server

go run ./cmd/test

Finally, let's run the CLI application

go run ./cmd/cli --source=https://my-server.com/channel/my-channel --target=http://localhost:3000

Now, if you push something to your repo, you should see something like this

Conclusion

We have successfully built our own Webhook payload delivery service.

I hope this was a fun little project and that maybe you learned something new.

SSE is awesome, and there are plenty of other things we can do with it.

Thanks for reading and feel free to ask any questions!

Appendices

Deploying the server

As I mentioned before, if you already have a preferred way of deploying your application, go for it.

I will show how I like to deploy my applications.

1. Hosting

We will be hosting our server at DigitalOcean, so make sure you have an account set up and that you are logged in.

First, we will create a new Droplet (A cloud server)

Choose the region that best fits your needs (probably the one closest to where you live)

Then, you can choose Ubuntu as the operating system, with the simplest machine available (A regular SSD: 1 GB / 1 CPU, 25 GB SSD Disk, 1000 GB transfer)

Select an SSH key as our authentication method and configure it

Finally, create the Droplet

After the Droplet is created, you will have access to its IP

Then, make sure everything is working by logging into the VM through SSH

ssh root@<your-ip-address>

Aaand, we are on!

2. Configuring Docker

For the actual deployment, we will need to "Dockerize" our project.

Since this is a simple Go application, we will only need a Dockerfile at the root of the project

# 1. build the server
FROM golang:1.23.1 AS builder
WORKDIR /app
COPY . .
ENV CGO_ENABLED=0
ENV GOOS=linux
RUN go build -o app ./cmd/web/

# 2. run the server
FROM scratch
# This is needed to include CA certificates in the `scratch` image, enabling your app to verify HTTPS requests, as `scratch` is completely empty by default.
# since golang:1.23.1 image is based on Debian or similar distributions, and it comes with CA certificates pre-installed, we can just copy from there
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
WORKDIR /app
COPY --from=builder /app/app .
EXPOSE 80
CMD ["./app", "-addr", ":80"]

3. Deploy with Kamal

For actually deploying the application, I really enjoy using Kamal, and with its newest version, it makes all the process a breeze.

Kamal offers zero-downtime deploys, rolling restarts, asset bridging, remote builds, accessory service management, and everything else you need to deploy and manage your web app in production with Docker. Originally built for Rails apps, Kamal will work with any type of web app that can be containerized.

First, make sure you have Kamal installed, you can achieve that by following their installation guide
In the end, you should be able to run Kamal through the terminal

Then, let's init Kamal into our application

kamal init

Which should give you something like this

Now, you should see a config/deploy.yml file and a .kamal folder

Now, you can give a name for your service and image, fill in your webserver IP address and provide the credentials for your image host

You should have something like this in the end

# Name of your application. Used to uniquely configure containers.
service: my-go-webhook-deliveries

# Name of the container image.
image: kuroski/my-go-webhook-deliveries

# Deploy to these servers.
servers:
  web:
    - 206.189.59.71

# You can configure a hostname later
# This will Enable SSL auto certification via Let's Encrypt and allow for multiple apps on a single web server.
#proxy:
#  ssl: true
#  host: app.example.com
  # Proxy connects to your container on port 80 by default.
  # app_port: 3000

# Credentials for your image host.
# By default, it uses Docker Hub, so, just insert your username there
# https://hub.docker.com/
registry:
  username: kuroski
  password:
    - KAMAL_REGISTRY_PASSWORD

# Configure builder setup.
builder:
  arch: amd64

The last piece of the puzzle is to add the KAMAL_REGISTRY_PASSWORD to your environment variables

Kamal manages environment variables through .kamal/secrets file.

By default, it reads secrets from your computer environment

KAMAL_REGISTRY_PASSWORD=$KAMAL_REGISTRY_PASSWORD

So, generate a new access token in DockerHub with full access permission

And then add it to your profile accordingly (~/.zshr, ~/.bashrc, ~/.config/fish/config.fish, etc...)

I prefer to store my secrets in a password manager like 1Password instead, and Kamal has a helper that reads from several different password managers
In case you have 1Password, you can store your credentials there and access it by having the following command in your .kamal/secrets file instead

SECRETS=$(kamal secrets fetch --adapter 1password --account my.1password.com --from <your-vault>/<name-of-the-vault-item> DOCKER_HUB_ACCESS_TOKEN) # DOCKER_HUB_ACCESS_TOKEN is just an alias, you can name an item the way you want

# then, you can use the "DOCKER_HUB_ACCESS_TOKEN" variable from your 1Password and store its value into the required "KAMAL_REGISTRY_PASSWORD" variable
KAMAL_REGISTRY_PASSWORD=$(kamal secrets extract DOCKER_HUB_ACCESS_TOKEN $SECRETS)

And we need to include a "health check" endpoint, since the proxy will by default hit /up once every second until we hit the deploy timeout, with a 5-second timeout for each request.

// internal/server/routes.go

package server

import (
    "net/http"
)

func (srv *Server) routes() http.Handler {
    mux := http.NewServeMux()
     // we need this "/up" endpoint
    mux.HandleFunc("GET /up", srv.upHandler)
    mux.HandleFunc("GET /channel/{channel}", srv.subscribeToChannelHandler)
    mux.HandleFunc("POST /channel/{channel}", srv.forwardRequestToChannelHandler)
    return mux
}

// internal/server/handlers.go

package server

import (
    "encoding/json"
    "fmt"
    "github.com/kuroski/go-webhook-deliveries/internal/model"
    "github.com/tmaxmax/go-sse"
    "io"
    "net/http"
    "time"
)

func (srv *Server) upHandler(w http.ResponseWriter, r *http.Request) {
    if _, err := fmt.Fprintf(w, "Hello World!!"); err != nil {
        srv.serverError(w, r, err)
    }
}

// ...

Now, if everything is set up correctly, you need only to run kamal setup for the first run, and for future deployments, you can run kamal deploy

kamal setup # Setup all accessories, push the env, and deploy app to servers
kamal deploy # Deploy app to servers

Please make sure you have added the application under Git, otherwise Kamal won't work

When everything is done, you will be finally able to access the application, you can try accessing the health check endpoint http://<your-server-ip>/up

This should be enough for you to experiment with the application we made on this tutorial.

Extras

As a couple of extra points, you can

Add your own domain name and set up SSL
I enjoy using https://www.namecheap.com/ for that.

After you acquire your domain, you can configure it in your DigitalOcean server

And then update your Kamal config file

# ....

# You can configure a hostname later
# This will Enable SSL auto certification via Let's Encrypt and allow for multiple apps on a single web server.
#proxy:
ssl: true
  host: my-domain.com

# .....

And the reeeally cool thing is that configuring subdomains is a breeze, change the host attribute to your subdomain

ssl: true
  host: hello.my-domain.com

And then include it the A record at DigitalOcean

Kamal will also take care of SSL certificates and subdomains 🎉

Add some security goods
Great, we have the server, we have HTTPS, but there are plenty of things that can be made to optimise and secure the server.

For playing around, you might not need anything else, but if you want to give it a go, please check out this awesome repository

https://github.com/guillaumebriday/kamal-ansible-manager
This is an Ansible playbook to automatically optimise and secure your servers for Kamal, for Ubuntu only.

And the author made a great video explaining how things work.

General improvements in the dev environment
You can create a compose.dev.yml to provide some basic setup for your application

x-common_config: &common_config
  image: golang:1.23.1
  working_dir: /app
  networks:
    - app
  volumes:
    - .:/app

services:
  web:
    <<: *common_config
    ports:
      - "3000:3000"
    command: go run ./cmd/web
    develop:
      watch:
        - action: sync+restart
          path: ./cmd/web
          target: /app
        - action: sync+restart
          path: ./internal
          target: /app
    restart: on-failure
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/up"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 10s

  cli:
    <<: *common_config
    depends_on:
      - web
    develop:
      watch:
        - action: sync+restart
          path: ./cmd/cli
          target: /app
    command: go run ./cmd/cli/main.go --source="${DEV_CLI_SOURCE_URL}" --target="${DEV_CLI_TARGET_URL}"
    restart: on-failure:3

networks:
  app:

volumes:
  app:

Then creating a Makefile

include .env

# ==================================================================================== #
# CONSTANTS
# ==================================================================================== #
IMAGE_NAME=my-go-webhook-deliveries
CONTAINER_NAME=my-go-webhook-deliveries

# ==================================================================================== #
# HELPERS
# ==================================================================================== #
## Prints usage information for each target
.PHONY: help
help:
    @echo 'Usage:'
    @sed -n 's/^##//p' ${MAKEFILE_LIST} | column -t -s ':' | sed -e 's/^/ /'
.PHONY: confirm
confirm:
    @echo -n 'Are you sure? [y/N] ' && read ans && [ $${ans:-N} = y ]

# ==================================================================================== #
# DEVELOPMENT
# ==================================================================================== #
## Web Application: Runs the web application without docker
.PHONY: web-run
web-run:
    go run ./cmd/web

## Client: Runs the client CLI
.PHONY: cli-run
cli-run:
    @source_arg=$${source:-${DEV_CLI_SOURCE_URL}}; \
    target_arg=$${target:-${DEV_CLI_TARGET_URL}}; \
    go run ./cmd/cli --source=$$source_arg --target=$$target_arg

## Dev: Runs dev environment with docker compose
.PHONY: dev
dev:
    docker compose -f compose.dev.yml up
    @echo 'Your server is running on localhost:3000'


# ==================================================================================== #
# QUALITY CONTROL
# ==================================================================================== #
## Tidy: Formats .go files and tidies dependencies
.PHONY: tidy
tidy:
    @echo 'Formatting .go files...'
    go fmt ./...
    @echo 'Tidying module dependencies...'
    go mod tidy

## Audit: Runs quality control checks
.PHONY: audit
audit:
    @echo 'Checking module dependencies'
    go mod tidy -diff
    go mod verify
    @echo 'Vetting code...'
    go vet ./...
    staticcheck ./...
    @echo 'Running tests...'
    go test -race -vet=off ./...


# ==================================================================================== #
# PRODUCTION
# ==================================================================================== #
## Deploy the application (skipping Kamal hooks)
.PHONY: deploy
deploy:
    @echo 'Deploying the app...'
    kamal deploy -H
    @echo 'Tidying module dependencies...'
    go mod tidy

This way you can just make dev and have everything up and running

Building a subscription tracker Desktop and iOS app with compose multiplatform — Offline data

Daniel Kuroski — Fri, 20 Dec 2024 08:41:57 +0000

Photo by Jesse Schoff on Unsplash

If you want to check out the code, here's the repository:
https://github.com/kuroski/kmp-expense-tracker

Introduction

In the previous part, we worked on providing feedback to users when using the application.

In this section we will build the base for offline support, making it easier on later steps to use SQLite through SQLDelight.

Offline data + Repositories

Since SQLDelight requires some configuration, we can take an "incremental step" and create an in memory storage implementation first.

We can build all the structure necessary to interact with data, and at the very end, switch it to SQLDelight.

// composeApp/src/commonMain/kotlin/ExpenseStorage.kt

import io.github.oshai.kotlinlogging.KotlinLogging
import kotlinx.coroutines.flow.*

private val logger = KotlinLogging.logger {}

interface ExpenseStorage {
    suspend fun saveExpenses(newExpenses: List<Expense>)

    suspend fun getExpenses(): StateFlow<List<Expense>>
}

class InMemoryExpenseStorage : ExpenseStorage {
    private val storedExpenses = MutableStateFlow(emptyList<Expense>())

    override suspend fun saveExpenses(newExpenses: List<Expense>) {
        logger.debug { "Replacing expenses on storage" }
        storedExpenses.value = newExpenses
    }

    override suspend fun getExpenses(): StateFlow<List<Expense>> = storedExpenses.asStateFlow()
}

All right, we have a InMemoryExpenseStorage implementation, which will be in charge of being our "source of truth" for our expense list.

ℹ️ What is "StateFlow" and why we are using it here??

StateFlow creates a flow of data that can be observed by multiple collectors, and it always holds a state (the latest value).

StateFlow is also a hot observable, which means it starts producing values as soon as it is created.

Next, we can define which storage to use and delegate its creation to Koin

// composeApp/src/commonMain/kotlin/Koin.kt

object Koin {
    val appModule =
        module {
            single<SnackbarHostState> { SnackbarHostState() }
            single<APIClient> { APIClient(Env.NOTION_TOKEN) }
+           single<ExpenseStorage> { InMemoryExpenseStorage() }

            factory { ExpensesScreenViewModel(apiClient = get()) }
        }
}

This way, when starting integrating SQLDeligt, we need to make a change in one place.

And finally, we can create a Repository to be used for all operations related to Expenses.

// composeApp/src/commonMain/kotlin/ExpenseRepository.kt

import api.APIClient
import api.QueryDatabaseRequest
import io.github.oshai.kotlinlogging.KotlinLogging
import io.ktor.utils.io.core.*
import kotlinx.coroutines.flow.StateFlow
import org.koin.core.component.KoinComponent

private val logger = KotlinLogging.logger {}

class ExpenseRepository(
    private val databaseId: String,
    private val apiClient: APIClient,
    private val expenseStorage: ExpenseStorage,
) : KoinComponent, Closeable {
    suspend fun all(forceUpdate: Boolean = false): StateFlow<List<Expense>> {
        // get local expenses from our storage
        val expenses = expenseStorage.getExpenses()

        /**
         * We are moving the request handling from ViewModel to here
         * Now we can either "force" upgrade, handle initial request
         * or even just grab data that is already stored
         */
        if (forceUpdate || expenses.value.isEmpty()) {
            logger.debug { "Refreshing expenses" }
            val response = apiClient.queryDatabaseOrThrow(databaseId, request)
            val newExpenses = response.results.map {
                Expense(
                    id = it.id,
                    name = it.properties.expense.title.firstOrNull()?.plainText ?: "-",
                    icon = it.icon?.emoji,
                    price = it.properties.amount.number,
                )
            }
            expenseStorage.saveExpenses(newExpenses)
        }

        logger.debug { "Loading all expenses" }
        return expenses
    }

    override fun close() {
        apiClient.close()
    }
}

Since now we have to deal with requests and storage handling, I am placing this logic in a "Repository" class to avoid overcomplicating the ViewModel, especially since we will add more operations when reaching other screens.

You can choose to work with interfaces like we did with the ExpenseStorage if you like it, for this case, I have chosen not to do it.

Here we also have to deal with the data that is stored in our app.

From what I have found, there are tons of ways to handle this.

I have chosen to force local data update through the forceUpdate parameter.

Now we have to integrate this repository with our application.
First, we can upgrade our ViewModel to actually use it instead of our APIClient.

// composeApp/src/commonMain/kotlin/ui/screens/expenses/ExpensesScreenViewModel.kt

- class ExpensesScreenViewModel(private val apiClient: APIClient) : StateScreenModel<ExpensesScreenState>(
+ class ExpensesScreenViewModel(private val expenseRepository: ExpenseRepository) : StateScreenModel<ExpensesScreenState>(
    ExpensesScreenState(
        data = RemoteData.NotAsked,
    ),
) {
    init {
        fetchExpenses()
    }

-    fun fetchExpenses() {
+    fun fetchExpenses(forceUpdate: Boolean = false) {
        mutableState.value = mutableState.value.copy(data = RemoteData.Loading)

        screenModelScope.launch {
            try {
-                logger.info { "Fetching expenses" }
-                val database = apiClient.queryDatabaseOrThrow(Env.NOTION_DATABASE_ID)
-                val expenses = database.results.map {
-                    Expense(
-                        id = it.id,
-                        name = it.properties.expense.title.firstOrNull()?.plainText ?: "-",
-                        icon = it.icon?.emoji,
-                        price = it.properties.amount.number,
-                    )
-                }
-                mutableState.value =
-                    ExpensesScreenState(
-                        lastSuccessData = expenses,
-                        data = RemoteData.success(expenses),
-                    )

+               expenseRepository.all(forceUpdate).collect { expenses ->
+                   logger.info { "Expenses list was updated" }
+                   mutableState.value =
+                       ExpensesScreenState(
+                           lastSuccessData = expenses,
+                           data = RemoteData.success(expenses),
+                       )
                }
            } catch (cause: Throwable) {
                logger.error { "Cause ${cause.message}" }
                cause.printStackTrace()
                mutableState.value = mutableState.value.copy(data = RemoteData.failure(cause))
            }
        }
    }
}

We will take advantage of Koin and create a singleton of our Repository.

// composeApp/src/commonMain/kotlin/Koin.kt

// .....
object Koin {
    val appModule =
        module {
            single<SnackbarHostState> { SnackbarHostState() }
            single<APIClient> { APIClient(Env.NOTION_TOKEN) }
            single<ExpenseStorage> { InMemoryExpenseStorage() }
            single { ExpenseRepository(Env.NOTION_DATABASE_ID, get(), get()) }

            factory { ExpensesScreenViewModel(apiClient = get()) }
        }
}

Then we have to upgrade our screen refresh button to force upgrading the list items.

// composeApp/src/commonMain/kotlin/ui/screens/expenses/ExpensesScreen.kt

navigationIcon = {
                        IconButton(
                            enabled = state.data !is RemoteData.Loading,
-                            onClick = { viewModel.fetchExpenses() },
+                            onClick = { viewModel.fetchExpenses(forceUpdate = true) },
                        ) {
                            Icon(Icons.Default.Refresh, contentDescription = null)
                        }
                    },

And that's it.

We have shifted the responsibility of fetching/storing Expenses to a repository, which later we will also use it for other operations.

By running the app, things should look the same as before.

Helper functions to convert API responses to Domain

This is optional, but we can also write some helper functions to help translate API responses into our application internal model (our domain).

// composeApp/src/commonMain/kotlin/api/ExpensePageResponse.kt

//.....

@Serializable
data class ExpensePageResponse(
    val id: ExpenseId,
    val icon: IconProperty? = null,
    val properties: ExpensePageProperties,
)

fun ExpensePageResponse.toDomain(): Expense = Expense(
    id = id,
    name = properties.expense.title.firstOrNull()?.plainText ?: "-",
    icon = icon?.emoji,
    price = properties.amount.number,
)

// composeApp/src/commonMain/kotlin/api/QueryDatabaseResponse.kt

// ....
@Serializable
data class QueryDatabaseResponse(
    val results: List<ExpensePageResponse>,
    @SerialName("next_cursor")
    val nextCursor: String? = null,
    @SerialName("has_more")
    val hasMore: Boolean,
)

fun QueryDatabaseResponse.toDomain(): List<Expense> =
    results.map { it.toDomain() }

Placing those functions here or even as companion objects (factory-function-like) in our Expense model is mostly a design choice.

For this one, I place it "closer" to the API/Network layer, where data is decoded from the API directly.

I find it a bit easier to search for it, and maybe to reason about "your model shouldn't know about the external API model".

Now we can clean up our repository and remove the responsibility of mapping API models to our domain.

// composeApp/src/commonMain/kotlin/ExpenseRepository.kt

// .......
    suspend fun all(forceUpdate: Boolean = false): StateFlow<List<Expense>> {
        val expenses = expenseStorage.getExpenses()

        if (forceUpdate || expenses.value.isEmpty()) {
            logger.debug { "Refreshing expenses" }
            val response = apiClient.queryDatabaseOrThrow(databaseId, request)
-            val newExpenses = response.results.map {
-                Expense(
-                    id = it.id,
-                    name = it.properties.expense.title.firstOrNull()?.plainText ?: "-",
-                    icon = it.icon?.emoji,
-                    price = it.properties.amount.number,
-                )
-            }
-            expenseStorage.saveExpenses(newExpenses)
+            expenseStorage.saveExpenses(response.toDomain())
        }

        logger.debug { "Loading all expenses" }
        return expenses
    }

    override fun close() {
        apiClient.close()
    }
}

In the next part of this series, we will handle editing the expenses.

Thank you so much for reading, any feedback is welcome, and please if you find any incorrect/unclear information, I would be thankful if you try reaching out.

See you all soon.

Building a subscription tracker Desktop and iOS app with compose multiplatform—Providing feedbacks

Daniel Kuroski — Mon, 29 Jul 2024 08:04:25 +0000

Photo by Luzelle Cockburn on Unsplash

If you want to check out the code, here's the repository:
https://github.com/kuroski/kmp-expense-tracker

Introduction

In the previous part, we configured a Notion database and listed its contents in our application.

It is nice that we are getting dynamic results, but we still have some work to do on the list page:

Monetary values aren't properly formatted
The screen is unresponsive, no loading/error state is displayed

In this section, we will address those problems and give some minimal information about its status, let's start!

UI feedbacks

When dealing with plain screens that are fetching data, there are a few possible states we can list:

We haven't requested anything yet
We've requested something, but we haven't got a response yet
We got a response, but it was an error
We got a response, and it was the data we wanted

This can be represented in the following image

There are several ways we can handle feedbacks based on a request state.

One common way to manage this is by using flags or individual variables, like:

data class State<E, T>(val data: T?, val error: E?, val isLoading: Boolean)

Then you can if-else your way in the screen to display the desired state.

This is a common approach in JS land, and there are plenty of libraries that help abstract the logic and make sure the state is consistent, like swr or TanStack Query.

Some drawbacks to this are:

We tend to ignore handling some scenarios

Who needs to provide feedback if something went wrong or indicate progress for async operations anyway, right?
Managing those cases by hand can be pretty verbose, and still... it is possible to achieve inconsistent states
- In the data class example, it is possible to have isLoading = true and an error at the same time
This is a n! issue
- If you are handling data, error and isLoading cases, there are 3! = 6 different possibilities to cover
- If you need to add one scenario (like the Not Asked), then it is 4! = 24 possibilities

A different solution that I particularly enjoy is RemoteData ADT (algebraic data type), which will be used to represent the current request state (or any "promise-like" case).

Where RemoteData shines the most is that it helps make "impossible states impossible", contrary to the first example, it is impossible to have "loading + error" states (for example), or any other invalid case.

I have first heard about it when I was working with Elm [1] through a popular blog post (at the time) How elm slays a UI antipattern.

There are implementations already written for multiple languages [1] [2] [3], but since our app is not so complex, we can create a simpler version ourselves.

Creating a simple RemoteData implementation

// shared/src/utils/RemoteData.kt

package utils

// There are two generics, one that represents the type of the "Error" and a second one that represents the "Success" type
sealed class RemoteData<out E, out A> {
    data object NotAsked : RemoteData<Nothing, Nothing>()

    data object Loading : RemoteData<Nothing, Nothing>()

    data class Success<out E, out A>(val data: A) : RemoteData<E, A>()

    data class Failure<out E, out A>(val error: E) : RemoteData<E, A>()

    companion object {
        // We need to define constructors for "Success" and "Failure" cases given they are `data class` and not `data object`
        fun <A> success(data: A): RemoteData<Nothing, A> = Success(data)

        fun <E> failure(error: E): RemoteData<E, Nothing> = Failure(error)
    }
}

// For operators, we only need `getOrElse`
// This is used to remove boilerplate for cases where you need to access data value directly (which you will see a case in the section below)
// Normally you would find other things like `fold`, `fold3`, `map`, `chain`, etc...
// But for our case, only `getOrElse` is enough
fun <A, E> RemoteData<E, A>.getOrElse(otherData: A): A =
    when (this) {
        is RemoteData.Success -> data
        else -> otherData
    }

Now we need to integrate RemoteData into our application.

Refactoring ViewModel

The first step is to refactor our main model representation.

Previously we had a List<Expense>, which was initially empty, and after we had a result, we populated it.

Now, we will wrap our state data property it in a RemoteData.

In the end, we have a RemoteData<Throwable, List<Expense>>, this means that:

data property is a structure that might

Not have started any request

Be pending (waiting for some request)

Be a successful request that will be a List<Expense>

Be a failed request that is a Throwable

Because we wrap it in a RemoteData, we cannot access List<Expense> without unwrapping it first.

We will also need to set the state accordingly depending if is NotAsked, Pending, Failure<Throwable> or Success<List<Expense>>

// composeApp/src/commonMain/kotlin/ui/screens/expenses/ExpensesScreenViewModel.kt

data class ExpensesScreenState(
-   val data: List<Expense>,
+   val data: RemoteData<Throwable, List<Expense>>,
) {
    val avgExpenses: String
-       get() = data.map { it.price }.average().toString()
+       get() = data.getOrElse(emptyList()).map { it.price }.average().toString()
}

 class ExpensesScreenViewModel(apiClient: APIClient) : StateScreenModel<ExpensesScreenState>(
    ExpensesScreenState(
-       data = listOf(),
+       data = RemoteData.NotAsked,
    ),
) {
    init {
-       screenModelScope.launch {
-           logger.info { "Fetching expenses" }
-           val database = apiClient.queryDatabaseOrThrow(Env.NOTION_DATABASE_ID)
-           val expenses = database.results.map {
-               Expense(
-                   id = it.id,
-                   name = it.properties.expense.title.firstOrNull()?.plainText ?: "-",
-                   icon = it.icon?.emoji,
-                   price = it.properties.amount.number,
-               )
-           }
-           mutableState.value = ExpensesScreenState(
-               data = expenses
-           )
-       }
+       fetchExpenses()
    }

+   fun fetchExpenses() {
+       mutableState.value = mutableState.value.copy(data = RemoteData.Loading)
+
+       screenModelScope.launch {
+           try {
+               logger.info { "Fetching expenses" }
+               val database = apiClient.queryDatabaseOrThrow(Env.NOTION_DATABASE_ID)
+               val expenses = database.results.map {
+                   Expense(
+                       id = it.id,
+                       name = it.properties.expense.title.firstOrNull()?.plainText ?: "-",
+                       icon = it.icon?.emoji,
+                       price = it.properties.amount.number,
+                   )
+               }
+               mutableState.value =
+                   ExpensesScreenState(
+                       data = RemoteData.success(expenses),
+                   )
+           } catch (cause: Throwable) {
+               logger.error { "Cause ${cause.message}" }
+               cause.printStackTrace()
+               mutableState.value = mutableState.value.copy(data = RemoteData.failure(cause))
+           }
+       }
+   }
}

In the end ExpensesScreenViewModel.kt should look like this:

// composeApp/src/commonMain/kotlin/ui/screens/expenses/ExpensesScreenViewModel.kt

package ui.screens.expenses

import Expense
import api.APIClient
import cafe.adriel.voyager.core.model.StateScreenModel
import cafe.adriel.voyager.core.model.screenModelScope
import io.github.oshai.kotlinlogging.KotlinLogging
import kotlinx.coroutines.launch
import utils.Env
import utils.RemoteData
import utils.getOrElse

private val logger = KotlinLogging.logger {}

data class ExpensesScreenState(
    val data: RemoteData<Throwable, List<Expense>>,
) {
    val avgExpenses: String
        get() = data.getOrElse(emptyList()).map { it.price }.average().toString()
}

class ExpensesScreenViewModel(private val apiClient: APIClient) : StateScreenModel<ExpensesScreenState>(
    ExpensesScreenState(
        data = RemoteData.NotAsked,
    ),
) {
    init {
        fetchExpenses()
    }

    fun fetchExpenses() {
        mutableState.value = mutableState.value.copy(data = RemoteData.Loading)

        screenModelScope.launch {
            try {
                logger.info { "Fetching expenses" }
                val database = apiClient.queryDatabaseOrThrow(Env.NOTION_DATABASE_ID)
                val expenses = database.results.map {
                    Expense(
                        id = it.id,
                        name = it.properties.expense.title.firstOrNull()?.plainText ?: "-",
                        icon = it.icon?.emoji,
                        price = it.properties.amount.number,
                    )
                }
                mutableState.value =
                    ExpensesScreenState(
                        data = RemoteData.success(expenses),
                    )
            } catch (cause: Throwable) {
                logger.error { "Cause ${cause.message}" }
                cause.printStackTrace()
                mutableState.value = mutableState.value.copy(data = RemoteData.failure(cause))
            }
        }
    }
}

Great, now we need to upgrade our screen.

Refactoring ExpensesScreen

The same thing as before, now our state.data will be a RemoteData<Throwable, List<Expense>>, and not a List<Expense> directly.

Here we also

Are checking for failures if (state.data is RemoteData.Failure), and logging the error
Added a "Refresh button", allowing to re-fetch the Notion database for new entries
Unwrapping state.data with a when expression, and printing out something based on the current state

package ui.screens.expenses

import Expense
import androidx.compose.foundation.layout.*
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.lazy.items
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Refresh
import androidx.compose.material3.*
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
import cafe.adriel.voyager.core.screen.Screen
import cafe.adriel.voyager.koin.getScreenModel
import io.github.oshai.kotlinlogging.KotlinLogging
import ui.theme.BorderRadius
import ui.theme.IconSize
import ui.theme.Spacing
import ui.theme.Width
import utils.RemoteData

private val logger = KotlinLogging.logger {}

object ExpensesScreen : Screen {
    @Composable
    override fun Content() {
        val viewModel = getScreenModel<ExpensesScreenViewModel>()
        val state by viewModel.state.collectAsState()
        val onExpenseClicked: (Expense) -> Unit = {
            logger.info { "Redirect to edit screen" }
        }

        // [1]
        // here every time `data` changes, we can check for failures and handle its result
        // maybe by showing a toast or by tracking the error
        LaunchedEffect(state.data) {
            val remoteData = state.data
            if (remoteData is RemoteData.Failure) {
                logger.error { remoteData.error.message ?: "Something went wrong" }
            }
        }

        Scaffold(
            topBar = {
                CenterAlignedTopAppBar(
                    // [2]
                    // We can now a button to refresh the list
                    navigationIcon = {
                        IconButton(
                            enabled = state.data !is RemoteData.Loading,
                            onClick = { viewModel.fetchExpenses() },
                        ) {
                            Icon(Icons.Default.Refresh, contentDescription = null)
                        }
                    },
                    title = {
                        Text("My subscriptions", style = MaterialTheme.typography.titleMedium)
                    },
                )
            },
            bottomBar = {
                BottomAppBar(
                    contentPadding = PaddingValues(horizontal = Spacing.Large),
                ) {
                    Row(
                        modifier = Modifier.fillMaxWidth(),
                        verticalAlignment = Alignment.CenterVertically,
                        horizontalArrangement = Arrangement.SpaceBetween,
                    ) {
                        Column {
                            Text(
                                "Average expenses",
                                style = MaterialTheme.typography.bodyLarge,
                            )
                            Text(
                                "Per month".uppercase(),
                                style = MaterialTheme.typography.bodyMedium,
                            )
                        }
                        Text(
                            state.avgExpenses,
                            style = MaterialTheme.typography.labelLarge,
                        )
                    }
                }
            },
        ) { paddingValues ->
            Box(modifier = Modifier.padding(paddingValues)) {
                // [3]
                // We don't need to use if-else conditions
                // We need only to unwrap `state.data` and handle each scenario 
                    // A nice thing is that now we have exaustive chekings!
                when (val remoteData = state.data) {
                    is RemoteData.NotAsked, is RemoteData.Loading -> {
                        Column {
                            Column(
                                modifier = Modifier.fillMaxWidth().padding(Spacing.Small_100),
                                verticalArrangement = Arrangement.spacedBy(Spacing.Small_100),
                                horizontalAlignment = Alignment.CenterHorizontally,
                            ) {
                                CircularProgressIndicator(
                                    modifier = Modifier.width(Width.Medium),
                                )
                            }
                        }
                    }

                    is RemoteData.Failure -> {
                        Column(
                            modifier = Modifier.fillMaxSize(),
                            horizontalAlignment = Alignment.CenterHorizontally,
                            verticalArrangement = Arrangement.spacedBy(
                                Spacing.Small,
                                alignment = Alignment.CenterVertically
                            ),
                        ) {
                            Text("Oops, something went wrong", style = MaterialTheme.typography.titleMedium)
                            Text("Try refreshing")
                            FilledIconButton(
                                onClick = { viewModel.fetchExpenses() },
                            ) {
                                Icon(Icons.Default.Refresh, contentDescription = null)
                            }
                        }
                    }

                    is RemoteData.Success -> {
                        ExpenseList(remoteData.data, onExpenseClicked)
                    }
                }
            }
        }
    }
}

// ....

If you run the application, you should finally have UI feedback!!

Persisting list entries once the first load is complete

You might notice when trying to click on the refresh button that the list is swapped with a spinner.

This might not be ideal in some cases, so... what to do if we want to keep the previously computed list?

Using RemoteData might force you to think some scenarios differently.

As a simple solution to this problem, we can "cache" the last successful list entries.

It can be done directly on the screen, or you can store it as a state in ViewModel.

// composeApp/src/commonMain/kotlin/ui/screens/expenses/ExpensesScreenViewModel.kt

data class ExpensesScreenState(
+   val lastSuccessData: List<Expense> = emptyList(),
    val data: RemoteData<Throwable, List<Expense>>,
) {
    val avgExpenses: String
        get() = data.getOrElse(emptyList()).map { it.price }.average().toString()
}

// ......

fun fetchExpenses() {
        mutableState.value = mutableState.value.copy(data = RemoteData.Loading)

        screenModelScope.launch {
            try {
                logger.info { "Fetching expenses" }
                val database = apiClient.queryDatabaseOrThrow(Env.NOTION_DATABASE_ID)
                val expenses = database.results.map {
                    Expense(
                        id = it.id,
                        name = it.properties.expense.title.firstOrNull()?.plainText ?: "-",
                        icon = it.icon?.emoji,
                        price = it.properties.amount.number,
                    )
                }
                mutableState.value =
                    ExpensesScreenState(
+                       lastSuccessData = expenses,
                        data = RemoteData.success(expenses),
                    )
            } catch (cause: Throwable) {
                logger.error { "Cause ${cause.message}" }
                cause.printStackTrace()
                mutableState.value = mutableState.value.copy(data = RemoteData.failure(cause))
            }
        }
    }

And now you can render them on screen when needed

// composeApp/src/commonMain/kotlin/ui/screens/expenses/ExpensesScreen.kt


                when (val remoteData = state.data) {
                    is RemoteData.NotAsked, is RemoteData.Loading -> {
                        Column {
                            Column(
                                modifier = Modifier.fillMaxWidth().padding(Spacing.Small_100),
                                verticalArrangement = Arrangement.spacedBy(Spacing.Small_100),
                                horizontalAlignment = Alignment.CenterHorizontally,
                            ) {
                                CircularProgressIndicator(
                                    modifier = Modifier.width(Width.Medium),
                                )
                            }
+                           ExpenseList(
+                               state.lastSuccessData,
+                               onExpenseClicked,
+                           )
                        }
                    }

                    is RemoteData.Failure -> {
+                       if (state.lastSuccessData.isNotEmpty()) {
+                           ExpenseList(
+                               state.lastSuccessData,
+                               onExpenseClicked,
+                           )
+                       } else {
                            Column(
                                modifier = Modifier.fillMaxSize(),
                                horizontalAlignment = Alignment.CenterHorizontally,
                                verticalArrangement = Arrangement.spacedBy(
                                    Spacing.Small,
                                    alignment = Alignment.CenterVertically
                                ),
                            ) {
                                Text("Oops, something went wrong", style = MaterialTheme.typography.titleMedium)
                                Text("Try refreshing")
                                FilledIconButton(
                                    onClick = { viewModel.fetchExpenses() },
                                ) {
                                    Icon(Icons.Default.Refresh, contentDescription = null)
                                }
                            }
+                       }
                    }

                    is RemoteData.Success -> {
                        ExpenseList(remoteData.data, onExpenseClicked)
                    }
                }

Adding toasts for errors

As an extra touch, let's add a toast for errors

// composeApp/src/commonMain/kotlin/Koin.kt

import androidx.compose.material3.SnackbarHostState
import api.APIClient
import org.koin.dsl.module
import ui.screens.expenses.ExpensesScreenViewModel
import utils.Env

object Koin {
    val appModule =
        module {
+           single<SnackbarHostState> { SnackbarHostState() }
            single<APIClient> { APIClient(Env.NOTION_TOKEN) }

            factory { ExpensesScreenViewModel(apiClient = get()) }
        }
}

// composeApp/src/commonMain/kotlin/App.kt

import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.material3.*
import androidx.compose.runtime.Composable
import androidx.compose.ui.Modifier
import cafe.adriel.voyager.navigator.Navigator
import cafe.adriel.voyager.transitions.SlideTransition
import org.koin.compose.KoinApplication
import org.koin.compose.koinInject
import ui.screens.expenses.ExpensesScreen
import ui.theme.AppTheme

@Composable
fun App() {
    KoinApplication(
        application = {
            modules(Koin.appModule)
        },
    ) {
        AppTheme {
+           val snackbarHostState = koinInject<SnackbarHostState>()

            Surface(
                modifier = Modifier.fillMaxSize(),
                color = MaterialTheme.colorScheme.background,
            ) {
-               Scaffold {
+               Scaffold(
+                   snackbarHost = {
+                       SnackbarHost(hostState = snackbarHostState)
+                   },
+               ) {
                    Navigator(ExpensesScreen) { navigator ->
                        SlideTransition(navigator)
                    }
                }
            }
        }
    }
}

// composeApp/src/commonMain/kotlin/ui/screens/expenses/ExpensesScreen.kt

// ......

    @Composable
    override fun Content() {
+       val snackbarHostState = koinInject<SnackbarHostState>()
        val viewModel = getScreenModel<ExpensesScreenViewModel>()
        val state by viewModel.state.collectAsState()
        val onExpenseClicked: (Expense) -> Unit = {
            logger.info { "Redirect to edit screen" }
        }

        LaunchedEffect(state.data) {
            val remoteData = state.data
            if (remoteData is RemoteData.Failure) {
-               logger.error { remoteData.error.message ?: "Something went wrong" }
+            
   snackbarHostState.showSnackbar(remoteData.error.message ?: "Something went wrong")
            }
        }

Formatting money

Let's finally address the money formatting.

First, let's add a computed property for formatting our prices.

// composeApp/src/commonMain/kotlin/Model.kt

import kotlinx.serialization.Serializable

+expect fun formatPrice(amount: Int): String

typealias ExpenseId = String

@Serializable
data class Expense(
    val id: ExpenseId,
    val name: String,
    val icon: String?,
    val price: Int,
-)
+) {
+   val formattedPrice: String
+       get() = formatPrice(price)
+}

Since formatting numbers are handled differently depending on the platform, we are using a expect-actual function.

Let's provide the platform-specific implementations.

// composeApp/src/desktopMain/kotlin/Model.jvm.kt

import java.text.NumberFormat
import java.util.Currency

actual fun formatPrice(amount: Int): String =
    (
        NumberFormat.getCurrencyInstance().apply {
            currency = Currency.getInstance("EUR")
        }
    ).format(amount.toFloat() / 100)

// composeApp/src/iosMain/kotlin/Model.ios.kt

import platform.Foundation.NSNumber
import platform.Foundation.NSNumberFormatter
import platform.Foundation.NSNumberFormatterCurrencyStyle

actual fun formatPrice(amount: Int): String {
    val formatter = NSNumberFormatter()
    formatter.minimumFractionDigits = 2u
    formatter.maximumFractionDigits = 2u
    formatter.numberStyle = NSNumberFormatterCurrencyStyle
    formatter.currencyCode = "EUR"
    return formatter.stringFromNumber(NSNumber(amount.toFloat() / 100))!!
}

Then we can use it on our screen and ViewModel

// composeApp/src/commonMain/kotlin/ui/screens/expenses/ExpensesScreenViewModel.kt

data class ExpensesScreenState(
    val lastSuccessData: List<Expense> = emptyList(),
    val data: RemoteData<Throwable, List<Expense>>,
) {
    val avgExpenses: String
-        get() = data.getOrElse(emptyList()).map { it.price }.average().toString()
+        get() = formatPrice(lastSuccessData.map { it.price }.average().toInt())
}

// composeApp/src/commonMain/kotlin/ui/screens/expenses/ExpensesScreen.kt


Text(
-   text = (expense.price).toString(),
+   text = (expense.formattedPrice),
    style = MaterialTheme.typography.bodyLarge.copy(color = MaterialTheme.colorScheme.onSurfaceVariant),
)

Now you should have pretty formatted values.

We are now fetching dynamic data, providing some feedback in our list screen.

In the next part of this series, we will finally store our data locally and make our application work offline.

Thank you so much for reading, any feedback is welcome, and please if you find any incorrect/unclear information, I would be thankful if you try reaching out.

See you all soon.

Building a subscription tracker Desktop and iOS app with compose multiplatform - Configuring Notion

Daniel Kuroski — Tue, 25 Jun 2024 08:46:45 +0000

Photo by Carl Tronders on Unsplash

If you want to check out the code, here's the repository:
https://github.com/kuroski/kmp-expense-tracker

Introduction

In the previous part, we bootstrapped our application and did some basic tweaking, if you followed along, you should have a static screen that lists our expenses.

In this part, we will be adding some "sauce" by making things dynamic with Notion's databases.

Let's start!

Configuring Notion

Creating the table

Notion will be our "database", so make sure to sign up, for the next steps:

Create a new page (there is a link on the sidebar menu for that)
Then type /database and select the Database - inline option
Now, we can edit the database properties
- Rename the column Name to Expense
- Remove Tags column
- Create a new property and name it Amount, make sure it is a Number type, and select your currency in the Number format option

Great, now we have a place to fill in our data.

In case you are having problems adding icons, you can right-click an item, and there is an icons property in the context menu

But, adding icons through this method is tedious 😅

To make things easier, you can create a template for new items, this way you can have an icon set by default, making new entries easier to create.

Click on the New blue button arrow and select New template
- Then you can provide some values, in this case, just provide a random icon with some placeholder text
From the template list, click on the "..." menu of the template you have just created and select the "Set as default" option
Now, every time you create a new entry, the icon will be already there and make your life easier when handling new data

Generating access keys

Great, our database configuration is done, and you can already use this table to manage your expenses with Notion.

To use our database through Notion API, you will need an access key.

Open https://developers.notion.com/
Click on View my integrations menu, which is located at the top-right on the page
After logging in, you will see a "My Integrations" page
Click on the "Create new integration" option
Select your workspace (it should be already pre-selected)
Give it a name
Make sure you gave Read/Update/Insert content capabilities (enter in the Capabilities menu

In the end, you should have something like this

Finally, you can find your API token on the Secrets page, we will be using that to make the requests.

Testing out

First of all, we have created the integration, we still must "connect it" to the database.

To do that:

Open the database we have created previously
Then connect it to the Expenses integration we have created

Great, now that it is connected, we can test it by using Notion's database query endpoint.

Open any HTTP client, like IntelliJ IDEA HTTP Client plugin or Postman
Create a new POST HTTP request
Set the Authorization Bearer Token to be the API token of the integration you created in previous steps
Make sure you have the following headers set as well
- Notion-Version: "2022-06-28"
- Content-Type: "application/json"
And fill in the URL https://api.notion.com/v1/databases/<your-database-id>/query
- To find the database id, just go to the database page (like I showed on the previous step)
- You will be able to see the database ID through the URL https://www.notion.so/<database-id>?v=non-interesting-things-here

If you got stuck, please check Notion's documentation about setting up Authorization

Finally, we have our data 🎉

Integrating Notion API with our app

To make requests to Notion's API, we will work with Ktor to instantiate an HTTP client on our app.

// composeApp/src/commonMain/kotlin/api/APIClient.kt

package api

import io.ktor.client.*
import io.ktor.client.engine.*
import io.ktor.client.plugins.*
import io.ktor.client.plugins.auth.*
import io.ktor.client.plugins.auth.providers.*
import io.ktor.client.plugins.contentnegotiation.*
import io.ktor.client.plugins.logging.*
import io.ktor.client.request.*
import io.ktor.http.*
import io.ktor.serialization.kotlinx.json.*
import io.ktor.utils.io.core.*
import kotlinx.serialization.json.Json

/**
 * Each platform has its own http client engine
 * I will give more details on this later
 */
expect fun clientEngine(): HttpClientEngine

/**
 * Here we have our main APIClient class.
 * 
 * Closeable is a Ktor interface indicating something
 * that must have a "close" method.
 * 
 * We will need that since we are instantiating an httpClient
 * and need to make sure it can be closed to avoid memory leak or further problems.
 */
class APIClient(
    private val token: String,
) : Closeable {
    /**
     * Here we are defining our Ktor http client
     * It has a delegated "lazy" property, to make sure
     * its value is only computed on first access.
     */
    private val httpClient: HttpClient by lazy {
        HttpClient(clientEngine()) {
            /**
             * Notion API requires you to provide a "Notion-Version" header
             */
            defaultRequest {
                header(NOTION_HEADER, NOTION_HEADER_VERSION)
                contentType(ContentType.Application.Json)
            }
            install(Logging) {
                logger = Logger.SIMPLE
                level = LogLevel.ALL
            }
            install(ContentNegotiation) {
                json(
                    Json {
                        ignoreUnknownKeys = true
                        prettyPrint = true
                    },
                )
            }
            install(Auth) {
                bearer {
                    loadTokens {
                        BearerTokens(accessToken = token, refreshToken = token)
                    }
                }
            }
        }
    }

    init {
        require(token.isNotEmpty()) { "Notion API token is required" }
    }

    companion object {
        const val NOTION_HEADER: String = "Notion-Version"
        const val NOTION_HEADER_VERSION: String = "2022-06-28"
        const val API_BASE_URL: String = "https://api.notion.com/v1"
    }

    override fun close() = httpClient.close()
}

Nice, we now must define the expect-actual declaration to getting the client engine.
For more information about it, you can check it here, but to give an overview:

expect-actual declaration allows you to access platform-specific API from KMP modules.
In our case, each platform needs a different http engine; then we can define an "expected" function to be present on the "actual" platform
This way, when we execute clientEngine() function here, the call will be made on the respective platform the code is running in.

In other words, expect defines that we need an HttpClientEngine.
And with actual, KMP wires this up with a platform-specific instance.

No need for the equivalent of compiler directives that you may see in languages like C++ or C#. KMP handles this platform-specific implementation for you.

Let's implement the actual functions on iOS and Desktop modules.

// composeApp/src/desktopMain/kotlin/api/APIClient.jvm.kt

package api

import io.ktor.client.engine.HttpClientEngine
import io.ktor.client.engine.cio.CIO

actual fun clientEngine(): HttpClientEngine = CIO.create()

// composeApp/src/iosMain/kotlin/api/APIClient.ios.kt

package api

import io.ktor.client.engine.HttpClientEngine
import io.ktor.client.engine.darwin.Darwin

actual fun clientEngine(): HttpClientEngine = Darwin.create()

This is, in my opinion, one of the coolest things about KMP.
Mostly, when working with other tooling, integration with other platforms is a pain.
But here, you can work with platform-specific code in a seamless way.

Fleet will even show us some useful information about it

Now we can finally make requests, let's first create a helper function to query Notion databases.

// composeApp/src/commonMain/kotlin/api/APIClient.kt

/**
 * You can think of this like a branded type.
 *
 * This is not required for our use case, feel free to just handle `DatabaseId` as plain `String` if you wish, but I thought it would be interesting to bring this up.
 *
 * More details below.
 */
@Serializable
@JvmInline
value class DatabaseId(private val value: String) {
    override fun toString(): String {
        return value
    }
}

// ...

class APIClient(
    private val token: String,
) : Closeable {
    // ...

    suspend fun queryDatabaseOrThrow(
        databaseId: DatabaseId,
        query: QueryDatabaseRequest = QueryDatabaseRequest(),
    ): QueryDatabaseResponse =
        httpClient
            .post("$API_BASE_URL/databases/$databaseId/query") {
                setBody(query)
            }
            .body<QueryDatabaseResponse>()

    override fun close() = httpClient.close()
}

Since we used DatabaseId as an inline value class, remember to change the ExpenseId as well.

// composeApp/src/commonMain/kotlin/Model.kt

- typealias ExpenseId = String
+ @Serializable
+ @JvmInline
+ value class ExpenseId(private val value: String) {
+     override fun toString(): String {
+         return value
+     }
+ }

ℹ️ Why have DatabaseId and ExpenseId as inline value class??

I thought it could be interesting to add DatabaseId as a branded type, as an example of how you can narrow your application types.

This is not particularly noticeable in our use case since we are dealing with fewer entities, but I think it could be useful to have an example in case you wish to expand this concept.

First of all, let's understand what branded types are:

They allow us to create distinct types based on an existing underlying type.
This helps improve type safety, makes your code more explicit, and safe.

Let me illustrate better with an example.

Let's say we have two functions, one to search users by their ID and another one for expenses’ ID.

fun searchUser(id: String) {
    // ...
}

fun searchExpense(id: String) {
    // ...
}

This is an innocent example, but normally we may have more functions to search several more entities.

The main issue here is that those functions accept any String parameter as the id, meaning scenarios like the following might occur.

// somewhere in the code
val user: User = //....
val expense: Expense = // .....

searchExpense(expense.id) // success, this is a valid operation

searchExpense("Oops") // this is valid, but it will break, since "Oops" does not exist as id (I hope)
searchExpense(user.id) // this is valid as well, but we are going to return a expense that has the user id

At first glance, it seems like an obvious error, but along the years, I faced several problems that occurred due to cases like this.

We can improve this by adding branded types.

@JvmInline
value class UserId(private val value: String)

@JvmInline
value class ExpenseId(private val value: String)

data class User(val id: UserId, val name: String)

fun searchUser(id: UserId) {
    // ...
}

fun searchExpense(id: ExpenseId) {
    // ...
}

init {
    val authenticatedUser = getAuthenticatedUser() // returns a `User`
    searchUser(authenticatedUser.id) // ok
    searchExpense(authenticatedUser.id) // error
}

By simply creating a UserId and ExpenseId branded type, we now cannot send incorrect parameters.

Since we are using the Kotlin serialization library, we must map out our requests and responses.

When building client side applications, personally, I prefer to keep API requests/responses on their own data classes and not try re-using them inside my applications.

ℹ️ Why having this "intermediate" model and not just directly map API results into our app model??

In the client, I think API shouldn't be generally shaped for a specific screen nor directly used as your app domain.

I think it is nicer if we have a layer that would make sure the API result is what we expect.

Furthermore, we can convert that data to the actual internal modeling of our app (even if the data is similar or equal).

Having this separation is useful because

Your endpoint shouldn't be necessarily shaped FOR a specific screen
- I had several cases where one endpoint was built for screen X, but later on it was also used on screens Y and Z
- Every screen has its own requirements, the endpoint and|or screen are likely to change
If the endpoint changes, you don't have to refactor your screen, do whatever you need, and pass down properties to follow the defined screen contract

I chose to use Notion for this purpose

their API is far from ideal
it is verbose
you don't have control over it
it was built to be generic, and for our case it makes little sense to try creating a "generic" model to comport Notion's API
they might change it, and you only have to adapt one thing from the application

This same rule goes to screens

If you have an endpoint that returns an "Expense" with its details, it doesn't mean you should use this model to shape the "Edit expense" screen
Form field values are nullable
The form might touch different entities other than the expense
The shape of the form generally is different from your app modeling, a name field in the Expense class will always exist, but a name in the form is nullable
Trying to generalise this might give you a hard time instead of having this separation

Given that, we will have the following flow

Receive Notion API response
Serialize it into a "Response" data class
Convert the response data class into our actual App state

First, let's handle the request encoding and response decoding.

// composeApp/src/commonMain/kotlin/api/QueryDatabaseRequest.kt

package api

import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable

/**
 * https://developers.notion.com/reference/post-database-query
 */
@Serializable
data class QueryDatabaseRequest(
    @SerialName("start_cursor")
    val startCursor: String? = null,
    @SerialName("page_size")
    val pageSize: Int? = 100,
) {
    init {
        pageSize?.let {
            require(it in 1..100) { "Illegal property, pageSize must be between 1 and 100" }
        }
    }
}

// composeApp/src/commonMain/kotlin/api/QueryDatabaseResponse.kt

package api

import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable

@Serializable
data class QueryDatabaseResponse(
    // https://developers.notion.com/reference/page
    val results: List<ExpensePageResponse>,
    @SerialName("next_cursor")
    val nextCursor: String? = null,
    @SerialName("has_more")
    val hasMore: Boolean,
)

The Notion database results property returns a list of pages.
Later on we will have to handle page request/response for the new/edit screens, so we will already create a ExpensePageResponse data class.

// composeApp/src/commonMain/kotlin/api/ExpensePageResponse.kt

package api

import ExpenseId
import api.model.ExpensePageProperties
import api.model.IconProperty
import kotlinx.serialization.Serializable

@Serializable
data class ExpensePageResponse(
    val id: ExpenseId,
    val icon: IconProperty? = null,
    val properties: ExpensePageProperties,
)

And now comes the tricky part.

Notion Databases entries are pages, which are generic.

In our case, they contain:

id
icon that is an emoji object
and a map of properties.

Since we are requesting our Expenses database, we know there are only two properties

Expense which is a title property
Amount which is a number property

// composeApp/src/commonMain/kotlin/api/model/IconProperty.kt

package api.model

import kotlinx.serialization.Serializable

/**
 * https://developers.notion.com/reference/emoji-object
 */
@Serializable
data class IconProperty(
    val emoji: String? = null,
    val type: String? = "emoji",
)

// composeApp/src/commonMain/kotlin/api/model/ExpensePageProperties.kt

package api.model

import api.serializers.MoneySerializer
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable

@Serializable
data class ExpensePageProperties(
    @SerialName("Expense")
    val expense: TitleProperty,
    @SerialName("Amount")
    val amount: NumberProperty,
)

@Serializable
class TitleProperty(val id: String, val title: List<Value>) {
    @Serializable
    data class Value(
        @SerialName("plain_text")
        val plainText: String,
    )
}

@Serializable
data class NumberProperty(
    @Serializable(with = MoneySerializer::class)
    val number: Int,
)

To finish up, number property from Notion uses float numbers, which is not what we want to work inside our app.

To address that, we can create a custom serializer.

// composeApp/src/commonMain/kotlin/api/serializers/MoneySerializer.kt

package api.serializers

import kotlinx.serialization.KSerializer
import kotlinx.serialization.descriptors.PrimitiveKind
import kotlinx.serialization.descriptors.PrimitiveSerialDescriptor
import kotlinx.serialization.descriptors.SerialDescriptor
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder

object MoneySerializer : KSerializer<Int> {
    override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("Money", PrimitiveKind.INT)

    override fun serialize(
        encoder: Encoder,
        value: Int,
    ) = encoder.encodeFloat(value.toFloat() / 100)

    override fun deserialize(decoder: Decoder): Int = (decoder.decodeFloat() * 100).toInt()
}

Unfortunately, because of Notion limitation, there are no DB Integers, so we might still face rounding issues.

But even though, I chose to do the conversion to at least work with integers inside the app.

Regarding modeling our API, we are done for now, we will have to revisit those files later on.

Injecting our client with Koin

Now it's time to use our APIClient, we could instantiate one in our ViewModel and move on, which is totally fine for our case.

Still, we will have more places where APIClient is needed, so we would need to instantiate it in all those places as well.

We also have to provide the authentication token for the APIClient, which would require us to manually pass this parameter whenever necessary.

This is where Koin can help us out, from their website, Koin is a

The pragmatic Kotlin & Kotlin Multiplatform Dependency Injection framework

ℹ️ What is this dependency injection (DI)??

Well... DI is nothing more than moving the responsibility of creating something to somewhere else, just like function params.

For example, let's say we have a function called every time we want to save a user.

fun saveUser(user: User) {
    val service = MyApiClient(
        apiKey = "abc", 
        logger = KotlinLogging.logger {}
    )
    service.saveUser(user)
    // ... do some processing
}

This is simple enough, but saveUser function must know how to build MyApiClient instance, which can be something non-trivial and maybe requiring us to pass none relevant parameters to this function.

fun saveUser(user: User, apiKey: String, logger: KLogger) {
    val service = MyApiClient(
        apiKey = apiKey,
        logger = logger
    )
    service.saveUser(user)
    // ... do some processing
}

In this case, apiKey and logger are something that doesn't seem to belong to saveUser function.

imagine that maybe saveUser might make use of more services, which would require us to send more parameters
Or other functions might also want to use MyApiClient, which would require us to inject apiKey and logger everywhere

DI is just moving this responsibility to somewhere else

fun saveUser(user: User, service: MyApiClient) {
    service.saveUser(user)
    // ... do some processing
}

💥 you just did it, no need of "fancy jargon's" and concepts.

One other example is in cases where you want to write some test for this function, now we can even expand this.

interface MyApiClient {
    fun saveUser(user: User): Unit
}

// --

fun saveUser(user: User, service: MyApiClient) {
    service.saveUser(user)
    // ... do some processing
}

// -- APP
class MyKtorApiClient(apiKey: String, logger: KLogger): MyApiClient {
  // ....
}

val myClient = MyKtorApiClient(
    apiKey = "abc",
    logger = KotlinLogging.logger {}
)
saveUser(
    user = User(), 
    service = myClient
)

// -- TEST

class MyInMemoryApiClient: MyApiClient {
  // ....
}

val myClient = MyInMemoryApiClient()
saveUser(
    user = User(), 
    service = myClient
)

There are libraries that help us out do fancier stuff with that, but in general... that's it!

Refactor `ExpensesScreenViewModel`

First of all, ExpensesScreenViewModel will receive an APIClient.

// composeApp/src/commonMain/kotlin/ui/screens/expenses/ExpensesScreenViewModel.kt

- class ExpensesScreenViewModel : StateScreenModel<ExpensesScreenState>(
+ class ExpensesScreenViewModel(apiClient: APIClient) : StateScreenModel<ExpensesScreenState>(
    ExpensesScreenState(
        data = listOf(),
    ),
) {

Handle environment variables

We can't hardcode our Notion API Token and Database id in the application, we need to provide them through a safer way.

There are a few ways we can handle environment variables, I like dotenv files, and we already injected them in our composeApp/build.gradle.kts through buildkonfig.

The neat part is that BuildKonfig generates a file at compile time with those variables that were configured there, it happens automatically when the kotlin compile task is triggered.

In case you wish to manually generate it, run the task ./gradlew generateBuildKonfig.

// composeApp/build/buildkonfig/commonMain/org/expense/tracker/BuildKonfig.kt

package org.expense.tracker

import kotlin.String

internal object BuildKonfig {
  public val NOTION_TOKEN: String = "MY_TOKEN"

  public val NOTION_DATABASE_ID: String = "MY_DB_ID"
}

A great benefit is that it will work for any platform.

Another approach would be to work with expect-actual functions, and handle that on each platform specifically.

I like creating my own utility function to get environment variables (despite having access to BuildKonfig).

I am doing this mostly because I am also learning about this technology, and I found little content over how people handle env variables.

At least creating this kind of utility layer would allow me to tweak how I fetch my env vars and change at one place in case I find a different approach over BuildKonfig.

// composeApp/src/commonMain/kotlin/utils/Env.kt

package utils

import api.DatabaseId
import org.expense.tracker.BuildKonfig

object Env {
    val NOTION_TOKEN: String
        get() {
            val notionToken = BuildKonfig.NOTION_TOKEN
            require(notionToken.isNotBlank()) { "You must provide a NOTION_TOKEN env variable" }
            return notionToken
        }

    val NOTION_DATABASE_ID: DatabaseId // hey, this is our branded type =D
        get() {
            val notionDatabaseId = BuildKonfig.NOTION_DATABASE_ID
            require(notionDatabaseId.isNotBlank()) { "You must provide a NOTION_DATABASE_ID env variable" }
            return notionDatabaseId
        }
}

Don't forget to add your dotenv files and configure gitignore.

Never ever commit them!

// .gitignore

!*.xcworkspace/contents.xcworkspacedata
**/xcshareddata/WorkspaceSettings.xcsettings
+.env

# .env.sample
NOTION_TOKEN=
NOTION_DATABASE_ID=

# .env
NOTION_TOKEN=secret_YOUR_SECRET
NOTION_DATABASE_ID=your_DB_ID

Then, we need to configure Koin itself.

// composeApp/src/commonMain/kotlin/Koin.kt

import api.APIClient
import org.koin.dsl.module
import ui.screens.expenses.ExpensesScreenViewModel
import utils.Env

object Koin {
    val appModule =
        module {
            /**
             * Here we are creating a Koin module and asking
             * > Hey Koin, when someone asks for you an `ApiClient`, please provide the return of this function, and make it a singleton.
             */
            single<APIClient> { APIClient(Env.NOTION_TOKEN) }

            /**
             * Our list screen ViewModel won't be a singleton, we will always re-create it once the user navigates to the screen
             * So in this case we are asking Koin to instantiate a `ExpensesScreenViewModel` every time someone asks Koin for it.
             * The interesting bit here is that `apiClient` parameter will be resolved from the singleton we defined.
             * Koin relies on the type and identifies which thing to inject.
             */
            factory { ExpensesScreenViewModel(apiClient = get()) }
        }
}

To make it clear...

all the dependencies are resolved at compile time and based on the typing

Making an analogy if you are used to something like PHP, what we did would be like this in a Laravel app:

<?php

namespace App\Providers;

// ...imports

class MyServiceProvider extends ServiceProvider
{
    public function register(): void    
    {
        $this->app->singleton(
            APIClient::class, 
            fn (Application $app) => new APIClient(env('NOTION_TOKEN')
        );

        // Adds a singleton `Connection`
        $this->app->bind(
            ExpensesScreenViewModel::class, 
            fn (ContainerInterface $container) => new ExpensesScreenViewModel($container->get(APIClient::class));
        );
    }
}

And now we have to integrate Koin with our app

// composeApp/src/commonMain/kotlin/App.kt

@Composable
fun App() {
+    KoinApplication(
+        application = {
+            modules(Koin.appModule)
+        },
+    ) {
        AppTheme {
            Surface(
                modifier = Modifier.fillMaxSize(),
                color = MaterialTheme.colorScheme.background,
            ) {
                Scaffold {
                    Navigator(ExpensesScreen) { navigator ->
                        SlideTransition(navigator)
                    }
                }
            }
        }
    }
+}


// composeApp/src/commonMain/kotlin/ui/screens/expenses/ExpensesScreen.kt

- import cafe.adriel.voyager.core.model.rememberScreenModel
+ import cafe.adriel.voyager.koin.getScreenModel

object ExpensesScreen : Screen {
    @Composable
    override fun Content() {
-        val viewModel = rememberScreenModel { ExpensesScreenViewModel() }    
+        val viewModel = getScreenModel<ExpensesScreenViewModel>()
        val state by viewModel.state.collectAsState()

And finally, we can refactor ExpensesScreenViewModel to actually use the APIClient

package ui.screens.expenses

import Expense
import api.APIClient
import cafe.adriel.voyager.core.model.StateScreenModel
import cafe.adriel.voyager.core.model.screenModelScope
import io.github.oshai.kotlinlogging.KotlinLogging
import kotlinx.coroutines.launch
import utils.Env

private val logger = KotlinLogging.logger {}

data class ExpensesScreenState(
    val data: List<Expense>,
) {
    val avgExpenses: String
        get() = data.map { it.price }.average().toString()
}

class ExpensesScreenViewModel(apiClient: APIClient) : StateScreenModel<ExpensesScreenState>(
    ExpensesScreenState(
        data = listOf(),
    ),
) {
    init {
        screenModelScope.launch {
            logger.info { "Fetching expenses" }
            val database = apiClient.queryDatabaseOrThrow(Env.NOTION_DATABASE_ID)
            val expenses = database.results.map {
                Expense(
                    id = it.id,
                    name = it.properties.expense.title.firstOrNull()?.plainText ?: "-",
                    icon = it.icon?.emoji,
                    price = it.properties.amount.number,
                )
            }
            mutableState.value = ExpensesScreenState(
                data = expenses
            )
        }
    }
}

If you run the app, we finally have data coming from Notion.

We are finally fetching dynamic data, but there are quite a few things left to do in our list screen.

In the next part of this series, we will handle user feedback by handling loading/error/success states and tweaking the UI a bit to display monetary values properly.

Thank you so much for reading, any feedback is welcome, and please if you find any incorrect/unclear information, I would be thankful if you try reaching out.

See you all soon.

Building a subscription tracker Desktop and iOS app with compose multiplatform - Setup

Daniel Kuroski — Mon, 27 May 2024 07:09:53 +0000

Cover photo by Andrew Schultz on Unsplash

If you want to check out the code, here's the repository:
https://github.com/kuroski/kmp-expense-tracker

Introduction

I made a significant shift in my career from mainly working with JS to fully committing to Kotlin some time ago.

It was quite a task to familiarize myself with this vast ecosystem without coming from the "Java world".

However, Kotlin has become my favorite programming language to work with to this date.

This series is aimed mainly for developers that are moving away from the JS ecosystem and those who want a more detailed explanation of Java/Kotlin tooling, while building something more than a "Hello World" application.

More explanations are included in collapsed sections, always prefixed with a ℹ️

What are we building?

You probably are paying for services like Netflix, Disney+, Spotify, a VPN service, and may other subscriptions.

Or even have monthly expenses like your rent, maybe an insurance, or you might be sending some money to a savings account.

In this series, we'll build a subscription tracker app for Desktop and iOS from scratch using Kotlin with Compose multiplatform.

With it, you can keep track of what services you are using, to keep an eye on the budget and to identify which services you may no longer need.

By the end, you'll have a fully functional Desktop and iOS app that communicates with an external API, works offline, while learning some concepts of Kotlin and its ecosystem if you're not familiar with them.

Please note that this is not a full-fledged app like Bobby, but it should be enough for something functional and useful.

We'll build this application entirely with Kotlin, compose multiplatform, and the data will be stored in a Notion database.

ℹ️ What is Kotlin, Kotlin Multiplatform (KMP) and Compose Multiplatform?

Kotlin is a programming language designed to be concise, safe, interoperable with Java (and other languages) and facilitates code reuse across multiple platforms.

Kotlin Multiplatform (KMP) is an Open-source technology by JetBrains for flexible multiplatform development

In the Android development world, there is a "UI framework" called Jetpack Compose.

Compose multiplatform is a framework based on Jetpack Compose, but that works for multiple platforms

Setup

Before anything, please make sure you have your environment properly configured, and for that, follow Kotlin Multiplatform Development documentation on "Set up an environment".

Since we are not building an Android app, you don't actually need to install Android Studio, so it is completely fine if by the end of the documentation you have a kdoctor result like

Finally, let's begin with our project.

The easiest way to create a project is to navigate to Kotlin Multiplatform Wizard https://kmp.jetbrains.com/

You can select as many platforms as you want, but for this tutorial, we will build an iOS and Desktop app
For the wizard field values, I am using
- Project Name ExpenseTracker
- Project ID org.expense.tracker
- Make sure you have selected iOS and Desktop options through the checkbox
- And since we want to write everything with Kotlin and share as may components as we want, make sure on selecting Share UI (with Compose Multiplatform UI framework)
Click the "Download" button, and a new project will be generated for you
This will download a project with some boilerplate, feel free to open it with your favorite editor

To this date, there is a list of Recommended IDEs to use for development, in this tutorial, I am going to use Fleet given it is free to use for hobby and educational projects.

This is a Gradle project, and if you editor (in my case, Fleet), it will automatically handle the importation, download dependencies, and provide me a nice UI to run the project.

When you open the project, you'll see a "Gradle project" import happening.

Wait for the tasks to finish (it might take a while), and we're good to go.

ℹ️ What is Gradle?

From their website:

Gradle is a modern automation tool that helps build and manage projects written in Java, Kotlin, Scala, and other JVM-based languages.

Basically, we use it to handle project settings, dependency management, tasks execution, and builds.

You can think of it as a combination of npm and webpack/vite/gulp/etc.

A basic Gradle project structure consists of

.
├── build.gradle
├── gradle
│   └── wrapper
│       ├── gradle-wrapper.jar
│       └── gradle-wrapper.properties
├── gradlew
├── gradlew.bat
└── settings.gradle

build.gradle is the main file that specifies a Gradle project, in here we define project metadata, dependencies, tasks, and plugins
settings.gradle file is used to define the project structure by specifying which modules should be included when building the project
gradle-wrapper.jar, gradle-wrapper.properties, gradlew and gradlew.bat belong to Gradle Wrapper, which allows you to run Gradle without its manual installation (their "binaries")

To give a glimpse on the tool, you can open the project folder we just downloaded on terminal, and we can play a bit with some tasks.

List all available tasks

# ./gradlew.bat in case you are using Windows
./gradlew tasks --all

Run your app

# ./gradlew.bat in case you are using Windows
./gradlew composeApp:run

Build the project (this is what Fleet is doing for us behind the scenes)

# ./gradlew.bat in case you are using Windows
./gradlew build

In case you are using an IDE, usually things are handled for you automatically + there is usually some UI to help you execute those tasks.

For Fleet, you have the run button

Project walkthrough

You can refer to JetBrains Get started with Compose Multiplatform — tutorial for a basic walkthrough.

It contains a good general overview and explanation about the project initial setup.

The only difference is that we are building an iOS + desktop app instead of iOS + Android, so we will have a desktopMain module instead of an androidMain.

I am including some explanations below of some topics that are not covered there.

ℹ️ I am not familiar with compose, what should I do?

As I mentioned before, compose is a modern toolkit for building native Android UI, and compose multiplatform extends this beyond Android land.

For this article most things should look "self-explanatory", and you can assume a lot of the components we will be using.

But in case you are having a hard time following along, I recommend you going through the Jetpack Compose for Android Developers pathways.

Compose can be straightforward at first glance. However, when reaching the "good stuff", like how to manage state, requests handling, structuring your app, there is quite some mindset change if you are coming from JS, and it might feel a bit weird.

In either way, please make sure you at least read it through Jetpack Compose for Android Developers pathways at some point in time if this is the first time you are working with this tool.

ℹ️ Walkthrough "build.gradle.kts"

/**
 * this is very important, the rootProject name will be
 * directly translated to imports of some resources, for example, in
 * `composeApp/src/commonMain/kotlin/App.kt` we are importing
 * - import expensetrackerapp.composeapp.generated.resources.Res
 * - import expensetrackerapp.composeapp.generated.resources.compose_multiplatform
 *
 * Which then is used to import a resource (our image).
 * The `ExpenseTrackerApp` is translated into `expensetrackerapp`
 */
rootProject.name = "ExpenseTrackerApp"

/**
 * This feature makes it safer and more idiomatic to access 'elements' of a Gradle project object from Kotlin build scripts.
 */
enableFeaturePreview("TYPESAFE_PROJECT_ACCESSORS")

/**
 * Here we are defining which repositories
 * Gradle will look into when trying to resolve a plugin dependency
 */
pluginManagement {
    repositories {
        maven("https://maven.pkg.jetbrains.space/public/p/compose/dev")
        google()
        gradlePluginPortal()
        mavenCentral()
    }
}

/**
 * Same thing as `pluginManagement` section, but those are
 * for plain project dependencies
 */
dependencyResolutionManagement {
    repositories {
        google()
        mavenCentral()
        maven("https://maven.pkg.jetbrains.space/public/p/compose/dev")
    }
}

/**
 * Like it was explained on the Gradle section of this tutorial.
 * > `settings.gradle.kts` file is used to define the project structure by specifying which modules should be included when building the project
 *
 * And here we are including `composeApp` modile in the root project.
 */
include(":composeApp")

ℹ️ Walkthrough "composeApp/build.gradle.kts"

The previous build.gradle.kts was our root project build configuration, this one is specific to composeApp project.

import org.jetbrains.compose.desktop.application.dsl.TargetFormat

/**
 * Gradle plugins definition, think of them like webpack/vite/gulp plugins
 * they will extend our project and add new capabilities, new tasks, etc.
 *
 * Here, kotlinMultiplatform and jetbrainsCompose are used to add capabilities for
 * Kotlin multiplatform projects and Jetpack Compose UI, respectively.
 */
plugins {
    alias(libs.plugins.kotlinMultiplatform)

    alias(libs.plugins.jetbrainsCompose)
}

kotlin {
    /**
     * Here, we are specifying an additional JVM target named "desktop",
     * and we have to do the same for other platforms
     */
    jvm("desktop")

    /**
     * This section defines targets for different iOS architectures.
     * For each target, a static framework named ComposeApp is created.
     * When building the app, the compiled code will be accessible through Swift code, since the base name is "ComposeApp", you can see in `iosApp/iosApp/ContentView.swift` that we have an `import ComposeApp`, and we are using `MainViewControllerKt.MainViewController()` which is generated from `composeApp/src/iosMain/kotlin/MainViewController.kt` 
     *
     * The name is not required to be the same as our project name, but this will be the name of the generated binary artifact.
     * If you change it, please go through `iosApp` folder and make sure
     * you update the `import ComposeApp` statement to wathever you have named here.
     */
    listOf(
        iosX64(),
        iosArm64(),
        iosSimulatorArm64()
    ).forEach { iosTarget ->
        iosTarget.binaries.framework {
            baseName = "ComposeApp"
            isStatic = true
        }
    }

    /**
     * In here, we defined the dependencies for different source sets (modules).
     */
    sourceSets {
        /**
         * This is a [delegated property](https://kotlinlang.org/docs/delegated-properties.html) and we are retrieving a named configuration object named "desktopMain" (this is coming from kotlin multiplatform plugin), we will use it to add specific dependencies to the desktop/JVM target of the project.
         * It might feel odd that we don't have to do that to "commonMain" for example, but this is a special property coming from the plugin that groups desktop targets (`linuxX64`, `mingwX64`, and `macosX64`).
         * 
         * Instead of adding 
         * - linuxX64.dependencies { implementation("a") }
         * - mingwX64.dependencies { implementation("a") }
         * - macosX64.dependencies { implementation("a") }
         * 
         * We can just
         * - val desktopMain by getting
         * - desktopMain.dependencies { implementation("a") }
         */
        val desktopMain by getting

        /**
         * Every dependency in `commonMain` are going to be available on every platform,
         * if the third party dependency you are adding is not compatible with some platform
         * you are targeting, you will get a compile time error.
         * In here, we are just adding the core compose libraries.
         */
        commonMain.dependencies {
            implementation(compose.runtime)
            implementation(compose.foundation)
            implementation(compose.material)
            implementation(compose.ui)
            implementation(compose.components.resources)
            implementation(compose.components.uiToolingPreview)
        }

        /**
         * Here we have the desktop specific dependencies definitions
         * In case of compose multiplatform, there are things that only
         * desktop targets can do (e.g.: context menu)
         */
        desktopMain.dependencies {
            implementation(compose.desktop.currentOs)
        }
    }
}

/**
 * Remember when we have added the "alias(libs.plugins.jetbrainsCompose)" plugin?
 * We can access some compose configuration for specific platforms.
 * In here, we are configuring the Desktop application.
 *
 * We are defining the `mainClass` to be `MainKt`, since the desktop app
 * entry point is located at `composeApp/src/desktopMain/kotlin/main.kt`
 * this file will be compiled, and the main class will be named as `MainKt`.
 * If you change your entry point name, make sure to upgrade it here.
 *
 * Here we also define the targets for our final build, in this case, we will
 * be building for Mac, Windows and Linux (Debian distros).
 */
compose.desktop {
    application {
        mainClass = "MainKt"

        nativeDistributions {
            targetFormats(TargetFormat.Dmg, TargetFormat.Msi, TargetFormat.Deb)
            packageName = "org.expense.tracker"
            packageVersion = "1.0.0"
        }
    }
}

Adding dependencies

We will need a few dependencies to help us build our app.

Luckily, the project already comes configured with Gradle version catalog which makes things easier.

ℹ️ What is version catalog?

From Gradle documentation

A version catalog is a list of dependencies, represented as dependency coordinates, that a user can pick from when declaring dependencies in a build script.

In a Gradle project, we can create a gradle/libs.versions.toml file and define in one place the dependencies and their version meta information.

We can even create groups of dependencies and refer to them as one alias in our build.gradle.kts.

For example:

[versions]
my-lib-version = "1.1.0"
another-lib-version = "3.1.0"
my-plugin-version = "2.1.0"

[libraries]
my-lib-core = { module = "org.my.lib:core", version.ref = "my-lib-version" }
my-lib-test = { module = "org.my.lib:test", version.ref = "my-lib-version" }
my-lib-bla = { module = "org.my.lib:bla", version.ref = "my-lib-version" }
another-lib = { module = "org.another.lib:bla", version.ref = "another-lib-version" }

[bundles]
my-lib = ["my-lib-core", "my-lib-test", "my-lib-bla"]

[plugins]
myPlugin = { id = "org.my.plugin", version.ref = "my-plugin-version" }

Then each dependency can be referred as

// composeApp/build.gradle.kts

plugins {
    alias(libs.plugins.myPlugin)
}

dependencies {
    implementation(libs.another.lib)
    // this will inject "my-lib-core", "my-lib-test" and "my-lib-bla"
    implementation(libs.bundles.my.lib)
}

There is so much more towards version catalogs, feel free to read more about them here.

Let's change our version catalog to the following

[versions]
compose = "1.6.2"
compose-plugin = "1.6.1"
junit = "4.13.2"
kotlin = "1.9.23"
# custom
logback = "1.5.3"
koin = "3.5.3"
koin-compose = "1.1.2"
ktor = "2.3.10"
kotlin-logging = "6.0.4"
voyager = "1.0.0"
coroutines-swing = "1.8.0"
stately-common = "2.0.7"
buildkonfig = "0.15.1"
dotenvgradle = "4.0.0"

[libraries]
kotlin-test = { module = "org.jetbrains.kotlin:kotlin-test", version.ref = "kotlin" }
kotlin-test-junit = { module = "org.jetbrains.kotlin:kotlin-test-junit", version.ref = "kotlin" }
junit = { group = "junit", name = "junit", version.ref = "junit" }
# custom
koin-core = { module = "io.insert-koin:koin-core", version.ref = "koin" }
koin-compose = { module = "io.insert-koin:koin-compose", version.ref = "koin-compose" }
ktor-client-core = { module = "io.ktor:ktor-client-core", version.ref = "ktor" }
ktor-client-cio = { module = "io.ktor:ktor-client-cio", version.ref = "ktor" }
ktor-client-darwin = { module = "io.ktor:ktor-client-darwin", version.ref = "ktor" }
ktor-client-logging = { module = "io.ktor:ktor-client-logging", version.ref = "ktor" }
ktor-client-auth = { module = "io.ktor:ktor-client-auth", version.ref = "ktor" }
ktor-client-content-negotiation = { module = "io.ktor:ktor-client-content-negotiation", version.ref = "ktor" }
ktor-serialization-kotlinx-json = { module = "io.ktor:ktor-serialization-kotlinx-json", version.ref = "ktor" }
logback-classic = { module = "ch.qos.logback:logback-classic", version.ref = "logback" }
kotlin-logging = { module = "io.github.oshai:kotlin-logging", version.ref = "kotlin-logging" }
voyager-navigator = { module = "cafe.adriel.voyager:voyager-navigator", version.ref = "voyager" }
voyager-screenModel = { module = "cafe.adriel.voyager:voyager-screenmodel", version.ref = "voyager" }
voyager-transitions = { module = "cafe.adriel.voyager:voyager-transitions", version.ref = "voyager" }
voyager-koin = { module = "cafe.adriel.voyager:voyager-koin", version.ref = "voyager" }
kotlinx-coroutines-swing = { module = "org.jetbrains.kotlinx:kotlinx-coroutines-swing", version.ref = "coroutines-swing" }
stately-common = { module = "co.touchlab:stately-common", version.ref = "stately-common" }
buildkonfig-gradle-plugin = { module = "com.codingfeline.buildkonfig:buildkonfig-gradle-plugin", version.ref = "buildkonfig" }

[plugins]
jetbrainsCompose = { id = "org.jetbrains.compose", version.ref = "compose-plugin" }
kotlinMultiplatform = { id = "org.jetbrains.kotlin.multiplatform", version.ref = "kotlin" }
kotlinSerialization = { id = "org.jetbrains.kotlin.plugin.serialization", version.ref = "kotlin" }
buildKonfig = { id = "com.codingfeline.buildkonfig", version.ref = "buildkonfig" }
dotenvgradle = { id = "co.uzzu.dotenv.gradle", version.ref = "dotenvgradle" }

There are a few dependencies needed to run our project, so aside from the ones that already come with the boilerplate, we have:

Koin to manage dependency injection (more explanation about that later)
Ktor client to manage HTTP requests
Logback we will use mostly with Ktor, this dependency is not required, but it is nice to see logs of requests + it will get rid of some annoying warnings while running the project
BuildKonfig + dotenv-gradle we are going to use those plugins to inject our environment variables
Voyager this library help us handle screen navigation, Voyager works in multi platforms, and it has integration with libraries like Koin, think of it something like react-router or TanStack Router
kotlinx-coroutines-swing and stately-common are here since I will show how to write specific code to Desktop and iOS, and we will need those libraries to use a custom Desktop feature, the context menu.

And now, it's a matter of including the dependencies on the project.

// build.gradle.kts

plugins {
    // this is necessary to avoid the plugins to be loaded multiple times
    // in each subproject's classloader
    alias(libs.plugins.jetbrainsCompose) apply false
    alias(libs.plugins.kotlinMultiplatform) apply false
    alias(libs.plugins.buildKonfig) apply false

    // dotenvgradle must be loaded at the project root
    alias(libs.plugins.dotenvgradle)
}

// In case you get an error from buildkonfig, just uncomment the lines below
//buildscript {
//    dependencies {
//        classpath(libs.buildkonfig.gradle.plugin)
//    }
//}

// composeApp/build.gradle.kts

plugins {
    alias(libs.plugins.kotlinMultiplatform)
    alias(libs.plugins.jetbrainsCompose)
    alias(libs.plugins.kotlinSerialization)
    alias(libs.plugins.buildKonfig)
}

// ...

sourceSets {
        val desktopMain by getting

        commonMain.dependencies {
            implementation(compose.runtime)
            implementation(compose.foundation)
            implementation(compose.material3)
            implementation(compose.ui)
            implementation(compose.components.resources)
            implementation(compose.components.uiToolingPreview)
            implementation(compose.animation)
            implementation(compose.materialIconsExtended)
            implementation(libs.ktor.client.core)
            implementation(libs.ktor.client.cio)
            implementation(libs.ktor.client.logging)
            implementation(libs.ktor.client.auth)
            implementation(libs.ktor.client.content.negotiation)
            implementation(libs.ktor.serialization.kotlinx.json)
            implementation(libs.koin.core)
            implementation(libs.koin.compose)
            implementation(libs.logback.classic)
            implementation(libs.kotlin.logging)
            implementation(libs.voyager.navigator)
            implementation(libs.voyager.screenModel)
            implementation(libs.voyager.transitions)
            implementation(libs.voyager.koin)
        }
        desktopMain.dependencies {
            implementation(compose.desktop.currentOs)
            implementation(libs.dotenv.kotlin)
            implementation(libs.kotlinx.coroutines.swing)
        }
        iosMain.dependencies {
            implementation(libs.ktor.client.darwin)
            implementation(libs.stately.common)
        }
    }

// ...
// ...

compose.desktop {
    application {
        mainClass = "MainKt"

        nativeDistributions {
            targetFormats(TargetFormat.Dmg, TargetFormat.Msi, TargetFormat.Deb)
            packageName = "org.expense.tracker"
            packageVersion = "1.0.0"
        }
    }
}

/**
 * BuildKonfig that will export environment variables to our application
 */
buildkonfig {
    packageName = "org.expense.tracker"

    defaultConfigs {
        buildConfigField(STRING, "NOTION_TOKEN", env.fetchOrNull("NOTION_TOKEN") ?: "")
        buildConfigField(STRING, "NOTION_DATABASE_ID", env.fetchOrNull("NOTION_DATABASE_ID") ?: "")
    }
}

Please note that:

We are going to use material 3, so remember to change the dependency from compose.material to compose.material3
With compose plugin installed, you have access to extra functionalities, we are including here compose.animation and compose.materialIconsExtended
dotenv-gradle actually generates autocompletion for your dotenv files, but since we are not going to need it for now, we are using fetchOrNull with a hard-coded string

Taking advantage we are here, feel free to include a language setting opt in ExperimentalMaterial3Api, since for some functionalities of our app we are relying on experimental APIs.

This is entirely optional, and you can add the "Experimental annotations" directly in code when Fleet suggests it, but in case you want to avoid typing annotations all over the place, we can add this following option.

// composeApp/build.gradle.kts

// ....
        iosMain.dependencies {
            implementation(libs.ktor.client.darwin)
            implementation(libs.stately.common)
        }

            all {
                languageSettings.optIn("androidx.compose.material3.ExperimentalMaterial3Api")
        }
} // end of `sourceSets`

Modeling our data

We are building a subscription tracking app, and in our case only one data class is needed.

// composeApp/src/commonMain/kotlin/Model.kt

typealias ExpenseId = String

@Serializable
data class Expense(
    /**
     * You can type id with "String" directly, but it is nice
     * to add some thin "layer of type safety" here
     */
    val id: ExpenseId, 
    val name: String,
    val icon: String?,
    val price: Int,
)

Since we are dealing with money, please make sure to at least work with integer values.

ℹ️ Why not use float values for handling money values??

If we work with floats (like € 9.10), normally this value would be stored in the same way in the database.

Whenever we need to show the price, we would need to format it like:

val eurFormatter = NumberFormat
        .getCurrencyInstance()
        .apply {
            currency = Currency.getInstance("EUR")
        }
val price = eurFormatter.format(9.1)
println(price) // €9.10

So, the result would be shown with two digits.

In most cases, this should be fine, as long as we are just working with one currency only, and not doing a lot of calculations with those fields.

The main problem is that we might encounter rounding problems, meaning we might have incorrect calculations by 0.01 if the wrong roundings add up [1].

A better way to handle money values as float/decimal fields like 9.13, we can create an integer field with the value 913.

Meaning, we can do simple conversions when manipulating this value:

913 / 100 = 9.13
9.13 * 100 = 913

Please be aware that there are a few exceptions:

Not all world currencies have exact two digits
There are countries that have no decimal digits at all
You might want/need to store more than two decimal digits

Saving money values as integers works quite well.

In case you need to deal with one of those exceptions, there's always some library to help you out dealing with those extra cases.

We won't need that in our case, but in case you need, you might want to check

Configuring routing

Compose multiplatform is the "UI toolkit", we need something extra to manage our routes.

Fortunately, there is a cool library called Voyager, from their website

Voyager is a multiplatform navigation library built for, and seamlessly integrated with, Jetpack Compose.

Feel free to delete
- composeApp/src/commonMain/kotlin/Greeting.kt
- composeApp/src/commonMain/kotlin/Platform.kt
- composeApp/src/desktopMain/kotlin/Platform.jvm.kt
- composeApp/src/iosMain/kotlin/Platform.ios.kt
And let's add a basic application setup

// composeApp/src/commonMain/kotlin/App.kt

import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.material3.*
import androidx.compose.runtime.Composable
import androidx.compose.ui.Modifier
import cafe.adriel.voyager.core.screen.Screen
import cafe.adriel.voyager.navigator.Navigator
import cafe.adriel.voyager.transitions.SlideTransition

@Composable
fun App() {
    MaterialTheme {
        Surface(
            modifier = Modifier.fillMaxSize(),
            color = MaterialTheme.colorScheme.background,
        ) {
            Scaffold {
                Navigator(HelloWorldScreen) { navigator ->
                    SlideTransition(navigator)
                }
            }
        }
    }
}

object HelloWorldScreen : Screen {

    @Composable
    override fun Content() {
        Column {
            Text("Hello World!")
            Button(onClick = {}) {
                Text("Click here!")
            }
        }
    }
}

And that's it.

Voyager expects you to create a

data class (if you need to send params)
class (if no param is required)
or even object (useful for tabs)

They must extend from the Screen interface, which provides a Content method where you can put your composables (your components).

If you run the app, you should see something like this.

Configuring material theme

Since we are using compose multiplatform, this means we "inherit" almost everything from Jetpack Compose (Android UI toolkit), which naturally uses Material theming.

If you are not a fan of Material, there are definite ways to style compose to make it look exactly the way you want.

As some examples

compose-cupertino which provides compose multiplatform components for iOS
Jewel which provides IntelliJ look and feels in Compose for Desktop

You can also translate almost anything from Jetpack Compose (even docs and tutorials) to build something interesting with Compose multiplatform.

For this app, we will keep with Material and do some basic theming.

Customising a Material theme can be made though Material theme builder app, which generates the files below (include them in the project).

// shared/src/ui/theme/Color.kt

package ui.theme

import androidx.compose.ui.graphics.Color

val md_theme_light_primary = Color(0xFFA1401A)
val md_theme_light_onPrimary = Color(0xFFFFFFFF)
val md_theme_light_primaryContainer = Color(0xFFFFDBCF)
val md_theme_light_onPrimaryContainer = Color(0xFF390C00)
val md_theme_light_secondary = Color(0xFFAA2E5C)
val md_theme_light_onSecondary = Color(0xFFFFFFFF)
val md_theme_light_secondaryContainer = Color(0xFFFFD9E1)
val md_theme_light_onSecondaryContainer = Color(0xFF3F001A)
val md_theme_light_tertiary = Color(0xFF7146B5)
val md_theme_light_onTertiary = Color(0xFFFFFFFF)
val md_theme_light_tertiaryContainer = Color(0xFFECDCFF)
val md_theme_light_onTertiaryContainer = Color(0xFF270057)
val md_theme_light_error = Color(0xFFBA1A1A)
val md_theme_light_errorContainer = Color(0xFFFFDAD6)
val md_theme_light_onError = Color(0xFFFFFFFF)
val md_theme_light_onErrorContainer = Color(0xFF410002)
val md_theme_light_background = Color(0xFFFAFCFF)
val md_theme_light_onBackground = Color(0xFF001F2A)
val md_theme_light_surface = Color(0xFFFAFCFF)
val md_theme_light_onSurface = Color(0xFF001F2A)
val md_theme_light_surfaceVariant = Color(0xFFF5DED7)
val md_theme_light_onSurfaceVariant = Color(0xFF53433F)
val md_theme_light_outline = Color(0xFF85736E)
val md_theme_light_inverseOnSurface = Color(0xFFE1F4FF)
val md_theme_light_inverseSurface = Color(0xFF003547)
val md_theme_light_inversePrimary = Color(0xFFFFB59C)
val md_theme_light_shadow = Color(0xFF000000)
val md_theme_light_surfaceTint = Color(0xFFA1401A)
val md_theme_light_outlineVariant = Color(0xFFD8C2BB)
val md_theme_light_scrim = Color(0xFF000000)

val md_theme_dark_primary = Color(0xFFFFB59C)
val md_theme_dark_onPrimary = Color(0xFF5C1900)
val md_theme_dark_primaryContainer = Color(0xFF812903)
val md_theme_dark_onPrimaryContainer = Color(0xFFFFDBCF)
val md_theme_dark_secondary = Color(0xFFFFB1C4)
val md_theme_dark_onSecondary = Color(0xFF65002E)
val md_theme_dark_secondaryContainer = Color(0xFF8A1244)
val md_theme_dark_onSecondaryContainer = Color(0xFFFFD9E1)
val md_theme_dark_tertiary = Color(0xFFD6BAFF)
val md_theme_dark_onTertiary = Color(0xFF410984)
val md_theme_dark_tertiaryContainer = Color(0xFF592B9B)
val md_theme_dark_onTertiaryContainer = Color(0xFFECDCFF)
val md_theme_dark_error = Color(0xFFFFB4AB)
val md_theme_dark_errorContainer = Color(0xFF93000A)
val md_theme_dark_onError = Color(0xFF690005)
val md_theme_dark_onErrorContainer = Color(0xFFFFDAD6)
val md_theme_dark_background = Color(0xFF001F2A)
val md_theme_dark_onBackground = Color(0xFFBFE9FF)
val md_theme_dark_surface = Color(0xFF001F2A)
val md_theme_dark_onSurface = Color(0xFFBFE9FF)
val md_theme_dark_surfaceVariant = Color(0xFF53433F)
val md_theme_dark_onSurfaceVariant = Color(0xFFD8C2BB)
val md_theme_dark_outline = Color(0xFFA08D87)
val md_theme_dark_inverseOnSurface = Color(0xFF001F2A)
val md_theme_dark_inverseSurface = Color(0xFFBFE9FF)
val md_theme_dark_inversePrimary = Color(0xFFA1401A)
val md_theme_dark_shadow = Color(0xFF000000)
val md_theme_dark_surfaceTint = Color(0xFFFFB59C)
val md_theme_dark_outlineVariant = Color(0xFF53433F)
val md_theme_dark_scrim = Color(0xFF000000)

val seed = Color(0xFFFF865B)

// shared/src/ui/theme/Theme.kt

package ui.theme

import androidx.compose.foundation.isSystemInDarkTheme
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.darkColorScheme
import androidx.compose.material3.lightColorScheme
import androidx.compose.runtime.Composable
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp

private val LightColors =
    lightColorScheme(
        primary = md_theme_light_primary,
        onPrimary = md_theme_light_onPrimary,
        primaryContainer = md_theme_light_primaryContainer,
        onPrimaryContainer = md_theme_light_onPrimaryContainer,
        secondary = md_theme_light_secondary,
        onSecondary = md_theme_light_onSecondary,
        secondaryContainer = md_theme_light_secondaryContainer,
        onSecondaryContainer = md_theme_light_onSecondaryContainer,
        tertiary = md_theme_light_tertiary,
        onTertiary = md_theme_light_onTertiary,
        tertiaryContainer = md_theme_light_tertiaryContainer,
        onTertiaryContainer = md_theme_light_onTertiaryContainer,
        error = md_theme_light_error,
        errorContainer = md_theme_light_errorContainer,
        onError = md_theme_light_onError,
        onErrorContainer = md_theme_light_onErrorContainer,
        background = md_theme_light_background,
        onBackground = md_theme_light_onBackground,
        surface = md_theme_light_surface,
        onSurface = md_theme_light_onSurface,
        surfaceVariant = md_theme_light_surfaceVariant,
        onSurfaceVariant = md_theme_light_onSurfaceVariant,
        outline = md_theme_light_outline,
        inverseOnSurface = md_theme_light_inverseOnSurface,
        inverseSurface = md_theme_light_inverseSurface,
        inversePrimary = md_theme_light_inversePrimary,
        surfaceTint = md_theme_light_surfaceTint,
        outlineVariant = md_theme_light_outlineVariant,
        scrim = md_theme_light_scrim,
    )

private val DarkColors =
    darkColorScheme(
        primary = md_theme_dark_primary,
        onPrimary = md_theme_dark_onPrimary,
        primaryContainer = md_theme_dark_primaryContainer,
        onPrimaryContainer = md_theme_dark_onPrimaryContainer,
        secondary = md_theme_dark_secondary,
        onSecondary = md_theme_dark_onSecondary,
        secondaryContainer = md_theme_dark_secondaryContainer,
        onSecondaryContainer = md_theme_dark_onSecondaryContainer,
        tertiary = md_theme_dark_tertiary,
        onTertiary = md_theme_dark_onTertiary,
        tertiaryContainer = md_theme_dark_tertiaryContainer,
        onTertiaryContainer = md_theme_dark_onTertiaryContainer,
        error = md_theme_dark_error,
        errorContainer = md_theme_dark_errorContainer,
        onError = md_theme_dark_onError,
        onErrorContainer = md_theme_dark_onErrorContainer,
        background = md_theme_dark_background,
        onBackground = md_theme_dark_onBackground,
        surface = md_theme_dark_surface,
        onSurface = md_theme_dark_onSurface,
        surfaceVariant = md_theme_dark_surfaceVariant,
        onSurfaceVariant = md_theme_dark_onSurfaceVariant,
        outline = md_theme_dark_outline,
        inverseOnSurface = md_theme_dark_inverseOnSurface,
        inverseSurface = md_theme_dark_inverseSurface,
        inversePrimary = md_theme_dark_inversePrimary,
        surfaceTint = md_theme_dark_surfaceTint,
        outlineVariant = md_theme_dark_outlineVariant,
        scrim = md_theme_dark_scrim,
    )

@Composable
fun AppTheme(
    useDarkTheme: Boolean = isSystemInDarkTheme(),
    content:
    @Composable()
        () -> Unit,
) {
    val colors =
        if (!useDarkTheme) {
            LightColors
        } else {
            DarkColors
        }

    MaterialTheme(
        colorScheme = colors,
        content = content,
    )
}

Since we are here, we can also add some UI constants to avoid adding arbitrary numbers to spacings, fonts, etc.


// shared/src/ui/theme/Theme.kt

// .....

object Spacing {
    val Small = 4.dp
    val Small_100 = 8.dp
    val Medium = 12.dp
    val Medium_100 = 16.dp
    val Large = 20.dp
    val Large_100 = 24.dp
    val ExtraLarge = 32.dp
}

object BorderRadius {
    val small = 4.dp
}

object Width {
    val Small = 20.dp
    val Medium = 32.dp
}

object IconSize {
    val Medium = 24.sp
}

@Composable
fun AppTheme(

// .....

To apply the custom theme, we just need to swap MaterialTheme with our new AppTheme composable

// composeApp/src/commonMain/kotlin/App.kt

@Composable
fun App() {
-    MaterialTheme {
+    AppTheme {
        Surface(
            modifier = Modifier.fillMaxSize(),
            color = MaterialTheme.colorScheme.background,
        ) {
            Scaffold {
                Navigator(HelloWorldScreen) { navigator ->
                    SlideTransition(navigator)
                }
            }
        }
    }
}

By running the app, you should see our custom theme.

Expenses list screen

Great, we have our model and theming in place, let's create the first screen.

In the end, it should look like this.

// composeApp/src/commonMain/kotlin/ui/screens/expenses/ExpensesScreenViewModel.kt

package ui.screens.expenses

import Expense
import cafe.adriel.voyager.core.model.StateScreenModel
import cafe.adriel.voyager.core.model.screenModelScope
import io.github.oshai.kotlinlogging.KotlinLogging
import kotlinx.coroutines.delay
import kotlinx.coroutines.launch

private val logger = KotlinLogging.logger {}

/**
 * Base state definition for our screen
 */
data class ExpensesScreenState(
    val data: List<Expense>,
) {
    /**
     * Computed property to get the avg price of the expenses
     */
    val avgExpenses: String
        get() = data.map { it.price }.average().toString()
}

/**
 * View model of our screen
 * More about ViewModels below
 */
class ExpensesScreenViewModel : StateScreenModel<ExpensesScreenState>(
    ExpensesScreenState(
        data = listOf(),
    ),
) {
    init {
        /**
         * Simulating the "API request" by adding some latency
         * and fake data
         */
        screenModelScope.launch {
            logger.info { "Fetching expenses" }
            delay(3000)
            mutableState.value = ExpensesScreenState(
                data = listOf(
                    Expense(
                        id = "1",
                        name = "Rent",
                        icon = "🏠",
                        price = 102573,
                    ),
                    Expense(
                        id = "2",
                        name = "Apple one",
                        icon = "🍎",
                        price = 2595,
                    ),
                    Expense(
                        id = "3",
                        name = "Netflix",
                        icon = "📺",
                        price = 1299,
                    ),
                )
            )
        }
    }
}

// composeApp/src/commonMain/kotlin/ui/screens/expenses/ExpensesScreen.kt

package ui.screens.expenses

import Expense
import androidx.compose.foundation.layout.*
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.lazy.items
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.*
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
import cafe.adriel.voyager.core.model.rememberScreenModel
import cafe.adriel.voyager.core.screen.Screen
import io.github.oshai.kotlinlogging.KotlinLogging
import ui.theme.BorderRadius
import ui.theme.IconSize
import ui.theme.Spacing

private val logger = KotlinLogging.logger {}

/**
 * Voyager screen, since there are no params
 * we can define it as a plain `object`
 */
object ExpensesScreen : Screen {
    @Composable
    override fun Content() {
        /**
         * Instantiating our ViewModel
         * https://voyager.adriel.cafe/screenmodel
         */
        val viewModel = rememberScreenModel { ExpensesScreenViewModel() }

        /**
         * More about this below, but for now, differently than JS
         * we handle values over time with Kotlin coroutine `Flow's` (in this case, `StateFlow`)
         * you can think of it as something similar to `Observables` in reactive programming
         */
        val state by viewModel.state.collectAsState()
        val onExpenseClicked: (Expense) -> Unit = {
            logger.info { "Redirect to edit screen" }
        }

        Scaffold(
            topBar = {
                CenterAlignedTopAppBar(
                    title = {
                        Text("My subscriptions", style = MaterialTheme.typography.titleMedium)
                    },
                )
            },
            bottomBar = {
                BottomAppBar(
                    contentPadding = PaddingValues(horizontal = Spacing.Large),
                ) {
                    Row(
                        modifier = Modifier.fillMaxWidth(),
                        verticalAlignment = Alignment.CenterVertically,
                        horizontalArrangement = Arrangement.SpaceBetween,
                    ) {
                        Column {
                            Text(
                                "Average expenses",
                                style = MaterialTheme.typography.bodyLarge,
                            )
                            Text(
                                "Per month".uppercase(),
                                style = MaterialTheme.typography.bodyMedium,
                            )
                        }
                        Text(
                            state.avgExpenses,
                            style = MaterialTheme.typography.labelLarge,
                        )
                    }
                }
            },
        ) { paddingValues ->
            Box(modifier = Modifier.padding(paddingValues)) {
                ExpenseList(state.data, onExpenseClicked)
            }
        }
    }
}

@Composable
private fun ExpenseList(
    expenses: List<Expense>,
    onClick: (expense: Expense) -> Unit,
) {
    LazyColumn(
        verticalArrangement = Arrangement.spacedBy(Spacing.Small_100),
    ) {
        items(
            items = expenses,
            key = { it.id },
        ) { expense ->
            ExpenseListItem(
                expense = expense,
                onClick = {
                    logger.info { "Clicked on ${expense.name}" }
                    onClick(expense)
                },
            )
        }

        item {
            Spacer(Modifier.height(Spacing.Medium))
        }
    }
}

@Composable
private fun ExpenseListItem(
    expense: Expense,
    onClick: () -> Unit = {},
) {
    Surface(
        modifier =
        Modifier
            .fillMaxWidth()
            .padding(horizontal = Spacing.Medium)
            .defaultMinSize(minHeight = 56.dp),
        onClick = onClick,
        shape = RoundedCornerShape(BorderRadius.small),
        color = MaterialTheme.colorScheme.surfaceVariant,
    ) {
        Row(
            modifier =
            Modifier
                .padding(
                    horizontal = Spacing.Medium_100,
                    vertical = Spacing.Small_100,
                ),
            verticalAlignment = Alignment.CenterVertically,
            horizontalArrangement = Arrangement.spacedBy(Spacing.Large),
        ) {
            Text(
                text = expense.icon ?: "",
                fontSize = IconSize.Medium,
                modifier = Modifier.defaultMinSize(minWidth = 24.dp),
            )
            Text(
                text = expense.name,
                style = MaterialTheme.typography.bodyLarge.copy(color = MaterialTheme.colorScheme.onSurfaceVariant),
                modifier = Modifier.weight(1f),
            )
            Text(
                text = (expense.price).toString(),
                style = MaterialTheme.typography.bodyLarge.copy(color = MaterialTheme.colorScheme.onSurfaceVariant),
            )
        }
    }
}

First of all, we have a ViewModel, which for now contains some basic states for our screen
Next we have a set of plain boring composables

ℹ️ What is this "ViewModel"??

From the documentation definition, ViewModel

Is a business logic or screen level state holder
It allows you to persist UI state
It can be reused on other screens

Since ViewModel class is specific to the Android platform (at the time I am writing this series), Voyager library provides a ScreenModel class, which can be used for the same purposes.

There are plenty of different approaches on how you can handle state on a screen, you can even go with a simpler approach and keep your state local, use some LaunchedEffect and might be good to go (at least for the kind of application we are building).

But in case things start to get messy, you can rely on ViewModel's as a way to organise your application and separate better the responsibilities.

ℹ️ What is this "StateFlow"??

From the documentation

StateFlow is a state-holder observable flow that emits the current and new state updates to its collectors.

When working with compose, there are multiple ways to handle state.

We can make use of something that might "feel" more familiar, like the State and MutableState classes to manage simple things.

But generally, state is handled with "observable-like" structures.

Reactivity works differently here compared to JS, there's no Promises nor Proxies, so it might not be something so familiar at first sight.

To handle async operations, we work with coroutines.

For example, we have to make API requests to Notion API, so we have to deal with something that can emit multiple values sequentially like Flow's.

The differences between a regular Flow and StateFlow are:

Flow is a cold stream (starts producing data only when observed)
StateFlow is a hot flow (produces data regardless of subscribers) variant that represents state-holding observable

All right, by running the application you should be presented with a "working" list screen.

Still, there are quite a lot of issues here:

We are mocking our data
There is no feedback to the user, we are not handling Loading/Error/Success states
We have to display the pricing properly

In the next part of this series, we will configure and integrate Notion in our application.

Thank you so much for reading, any feedback is welcome, and please if you find any incorrect/unclear information, I would be thankful if you try reaching out.

See you all soon.

Writing tests for Next.js, next-auth, Prisma using Playwright and msw

Daniel Kuroski — Mon, 15 Aug 2022 08:37:00 +0000

Cover photo by Catherine Heath on Unsplash

If you prefer to just dig into the code, you can find it here:

kuroski / github-issue-viewer

A Github issue viewer apps

Welcome!

This is a Github issue viewer project in which I'm just experimenting on testing my application with certain tools.

application.mov

To read more about this, just check at

https://dev.to/kuroski/writing-integration-tests-for-nextjs-next-auth-prisma-using-playwright-and-msw-1eo1-temp-slug-6804418

View on GitHub

If you would like to read more articles like this one, you can find them here.

Introduction

This article is something I wish I had back then when I was first trying to set up the testing environment for my side project.

It is a "Github issue viewer" app, so you should be able to login into your Github account and see a list of issues for your account.

Feel free to check at: https://github-issue-viewer.up.railway.app

I've chosen to build this app with a combination of several tools, but probably the most important ones are:

If you want to see more in-depth the tech stack, check it out here

I could use something simpler, but my main goal is to

validate how I would integrate those specific tools + how I would test my application.

I saw a lot of resources on how you would test a Next.js application, but I was not fully happy with how I would do that with the things I'm using here.

This is also a bit different compared to testing a pure React/Vue/X application, where they are way simpler to write.

The challenges

Here I have to:

Handle NextAuth.js OAuth state in a nicer way compared to other solutions I've found
Handle Github API requests
Handle Prisma integration

We have a full-blown application with a client and a server-side, authentication is taking place (not just on the client), and our app also makes requests to a third-party API FROM the server side.

Just like we have in testing-library

The more your tests resemble the way your software is used,
the more confidence they can give you.

And that is exactly what I wanted to achieve.

Configuring the testing environment

Let's finally dive into the code.

I won't go into details on how the application works and how to set up everything, my main goal here is to show how to set up the testing environment.

The main idea here is:

We would have a Docker file, which we can use to spin up our database
A mock server would be needed to intercept Github API requests and avoid calling/bloating Github API (We do not want to mock internal API calls)
We have to somehow boot up our application when executing Playwright (our testing framework of choice), by doing that we also have to
- Make sure a user is in our database
- We have to authenticate our user
- We have to mock Github API calls with our mock server

Docker

Let's start by creating a docker-compose.yml file, like this one:

# https://github.com/kuroski/github-issue-viewer/blob/main/docker-compose.yml

version: "3.9"

services:
  # in case you prefer going with mysql
  db:
    image: mysql:oracle
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: secret
      MYSQL_PASSWORD: secret
      MYSQL_DATABASE: githubIssueViewer
    volumes:
      - ./scripts/init.sql:/docker-entrypoint-initdb.d/init-script.sql
      - mysql:/var/lib/mysql
    ports:
      - "3306:3306"
    networks:
      - mysql

  # in case you prefer going with pg
  db-pg:
    image: postgres
    restart: always
    environment:
      POSTGRES_PASSWORD: secret
    volumes:
      - postgres:/data/postgres
    ports:
      - "5432:5432"
    networks:
      - postgres

networks:
  mysql:
    driver: bridge
  postgres:
    driver: bridge

volumes:
  postgres:
  pgadmin:
  mysql:

By running docker-compose up you'll have a database ready to use for our dev environment and our local tests 🎉.

Just don't forget to properly configure Prisma to use the correct database provider and add the database URL in your environment variables:

# in case you want to use PG
DATABASE_URL=postgresql://postgres:secret@127.0.0.1:5432

# in case you want to use mysql
DATABASE_URL=mysql://root:secret@127.0.0.1:3306/githubIssueViewer

Playwright

For this project, I chose to use Playwright, since I wanted to be more familiar with it.

For Playwright, you should have a configuration file in your project root:

// https://github.com/kuroski/github-issue-viewer/blob/main/playwright.config.ts

import type { PlaywrightTestConfig } from "@playwright/test"
import { devices } from '@playwright/test'

// Reference: https://playwright.dev/docs/test-configuration
const config: PlaywrightTestConfig = {
  // ...

  // We will set up our required credentials here
  globalSetup: './e2e/setup/globalSetup.ts',

  use: {
    // ...

    // This is the file we will use to setup and retrieve our application state (in our case, the authentication cookies)
    storageState: './e2e/setup/storageState.json'
  },

  projects: [
    {
      name: 'Desktop Chrome',
      use: {
        ...devices['Desktop Chrome'],
      },
    },
  ],
}
export default config

Then we need to configure globalSetup.ts which we will make sure our testing environment will have all the pre-required data in place.

// https://github.com/kuroski/github-issue-viewer/blob/main/e2e/setup/globalSetup.ts

import { chromium } from '@playwright/test';
import path from 'node:path'

import prisma from '@/lib/prisma';

async function globalSetup() {
  const storagePath = path.resolve(__dirname, 'storageState.json')

  const date = new Date()

  // This is a dummy random session token
  const sessionToken = '04456e41-ec3b-4edf-92c1-48c14e57cacd2'

  // 1. We make sure a test user exists in our local database, `upsert` will make sure we only have this user in our database
  await prisma.user.upsert({
    where: {
      email: 'e2e@e2e.com'
    },
    create: {
      name: 'e2e',
      email: 'e2e@e2e.com',
      // 2. We need a session which is used by NextAuth and represents this `e2e@e2e.com` user login session
      sessions: {
        create: {
          // 2.1. Here we are just making sure the expiration is for a future date, to avoid NextAuth to invalidate our session during the tests
          expires: new Date(date.getFullYear(), date.getMonth() + 1, 0),
          sessionToken
        }
      },
      // 3. Here we are binding our user with a "Github fake account", this is needed since we are using OAuth, we don't have to worry about this data since we are gonna intercept and mock the direct Github API calls
      accounts: {
        create: {
          type: 'oauth',
          provider: 'github',
          providerAccountId: '2222222',
          access_token: 'ggg_zZl1pWIvKkf3UDynZ09zLvuyZsm1yC0YoRPt',
          token_type: 'bearer',
          scope: 'read:org,read:user,repo,user:email'
        }
      }
    },
    update: {},
  })

  // 4. Finally we need to set up the authentication cookie into our test browser state
  // This will guarantee you will have an authenticated user once you boot up your tests
  const browser = await chromium.launch();
  const context = await browser.newContext({ storageState: storagePath });
  // 4.1. This cookie is what `NextAuth` will look after to validate if our user is authenticated
  // Please note that the `value` of the cookie **must be the same** as the `sessionToken` we added in `step 2.` 
  await context.addCookies([
    {
      name: 'next-auth.session-token',
      value: sessionToken,
      domain: 'localhost',
      path: '/',
      httpOnly: true,
      sameSite: 'Lax',
      expires: 1661406204
    }
  ])
  await context.storageState({ path: storagePath });
  await browser.close();
}

export default globalSetup;

Mock server

For this part, I've chosen to go with msw, because I wanted to validate this library since I usually used mirage.js.

msw also has a @msw/data library to help out with the data modeling of our tests.

The first thing would be creating our API endpoint handlers:

// https://github.com/kuroski/github-issue-viewer/blob/main/e2e/mocks/handlers.ts

import { faker } from '@faker-js/faker';
import { factory, manyOf, nullable, oneOf, primaryKey } from '@mswjs/data';
import type { ENTITY_TYPE, PRIMARY_KEY } from "@mswjs/data/lib/glossary";
import { rest } from 'msw';

// 1. We define our factory functions, so every time we request those Github endpoints, we will have some "generated" data instead of calling the Github API
export const db = factory({
  label: {
    id: primaryKey(faker.datatype.number),
    url: faker.internet.url,
    name: faker.random.words,
    description: nullable(faker.hacker.phrase),
    color: faker.internet.color,
  },
  // ...
})
type DB = typeof db;

// 2. We will need this type for later, it will be used to enable us to extract the factory function property types, for example, `FactoryValue<'label'>`
export type FactoryValue<Key extends keyof DB> = Omit<
  ReturnType<DB[Key]['create']>,
  typeof ENTITY_TYPE | typeof PRIMARY_KEY
>;

// 3. Finally, we will have our request handlers
// They are the gonna determine whether an outgoing request should be mocked, and specifies its mocked response
export const issuesHandler = () =>
  rest.get('https://api.github.com/user/issues', (_req, res, ctx) => {
    // 3.1. For this case, I chose to just create one issue for each situation I wanted to test, and we need to make sure the are not re-created between our tests
    if (db.issue.count() === 0) {
      db.issue.create({
        state: 'open',
        repository: db.repository.create(),
        pull_request: db.pullRequest.create(),
        assignees: [...Array(3)].map(db.assignee.create),
        labels: [],
      })
      db.issue.create({
        state: 'open',
        repository: db.repository.create(),
        pull_request: null,
        assignees: [],
        labels: [],
      })
      db.issue.create({
        state: 'closed',
        closed_at: faker.date.recent().getTime(),
        repository: db.repository.create(),
        pull_request: db.pullRequest.create(),
        assignees: [...Array(3)].map(db.assignee.create),
        labels: [],
      })
      db.issue.create({
        state: 'closed',
        closed_at: faker.date.recent().getTime(),
        repository: db.repository.create(),
        pull_request: null,
        assignees: [],
        labels: [],
      })
    }

    return res(
      ctx.status(200),
      ctx.json(db.issue.getAll())
    )
  })

const handlers = [
  issuesHandler(),
  rest.get('https://api.github.com/user/orgs', (_req, res, ctx) => res(
    ctx.status(200),
    ctx.json([...Array(5)].map(db.org.create))
  )),
  rest.get('https://api.github.com/user/repos', (_req, res, ctx) => res(
    ctx.status(200),
    ctx.json([...Array(5)].map(db.repo.create))
  )),
]

export default handlers

Then finally we just need to set up our server

// https://github.com/kuroski/github-issue-viewer/blob/main/e2e/mocks/mockServer.ts

import { setupServer } from "msw/lib/node"

import handlers from "@/e2e/mocks/handlers"

function bootstrap() {
  // 1. The server will make use of the handlers we created in the previous section
  return setupServer(...handlers)
}

export default bootstrap

Connecting everything and testing the application

This is the tricky part of the process, we have two main problems here:

How to boot our application during our tests + connect it with our mock server?
How to make the mock server run against the server layer (not the browser)?

If you wish you can skip the explanation and go directly to the next section to see the code.

I have found two amazing articles that explain this with more in-depth concepts:

I've tried a few different approaches here, the one worth mentioning is:

Running a dev server with our tests turn on msw while building it

You would need to add a custom webServer to playwright
That will spin up a custom Next.js server before our tests
You can start the mock server during that phase
Then you can just write your tests

The main problem with this approach is that we can't change mocked responses after Next.js startup.

Which will lead us to the actual solution I chose to go for.

The solution

To make it work, first, we need to create a custom Next.js server

// https://github.com/kuroski/github-issue-viewer/blob/main/e2e/mocks/server.ts

import Koa from "koa"
import { SetupServerApi } from "msw/lib/node";
import next from "next";
import { parse } from "node:url";

import bootstrapMockServer from '@/e2e/mocks/mockServer';

// 1. Create a new Koa server
const server = new Koa()

// 2. Create a new Next.js instance
const app = next({ dev: process.env.CI ? false : true })
const handle = app.getRequestHandler()
const port = process.env.PORT || 3000;

async function bootstrap(): Promise<SetupServerApi> {
  return new Promise(async (resolve) => {
    try {
      await app.prepare()

      // 3. Create a new mock server after Next.js instance is ready (The order matters)
      const mockServer = bootstrapMockServer()

      // 4. From this part on we can already intercept HTTP requests coming from the Next.js server
      mockServer.listen({ onUnhandledRequest: 'warn' })

      // 5. Create a middleware function in our Koa server, that will intercept every request and we handle that request with our Next.js app
      server.use(ctx => {
        const parsedUrl = parse(ctx.req.url!, true);
        return handle(ctx.req, ctx.res, parsedUrl)
      });

      // 6. Finally we just need to listen to our Koa server
      // We are using `0` here since we might have several tests running in parallel, by doing that, Node will choose a random available port for us
      const s = server.listen(0)

      // 7. We need to wait until the server is ready
      // We are doing this since ports are automatically generated, this way we can provide the port to the service that instantiate the server
      s.on('listening', () => {
        const port = (<AddressInfo>s.address()).port
        console.log(`> Ready on localhost:${port} - env ${process.env.NODE_ENV}`);
        resolve({
          mockServer,
          baseURL: `http://localhost:${port}`
        })
      });
    } catch (e) {
      console.error(e);
      process.exit(1);
    }
  })

}

export default bootstrap

This is just a custom Next.js server, just as they provide in their documentation, the two main differences here are:

I'm using Koa instead of express
I'm adding our mock server in the middle of the code

Then you just have to boot up the server in the correct testing phase.

This is an important part! We have to generate our servers in automatic ports, otherwise, we might face a collision if a server is already instantiated in that same port in a different process.

Our first test

// https://github.com/kuroski/github-issue-viewer/blob/main/e2e/issues.spec.ts

import { drop } from '@mswjs/data';
import { Page } from '@playwright/test';
import { SetupServerApi } from 'msw/lib/node';

import { db, FactoryValue } from '@/e2e/mocks/handlers';
import server from '@/e2e/mocks/server'
import { expect, test } from "@/e2e/test";
import { dateTimeFormat } from "@/lib/utils";

test.describe("Github issues app", () => {
  let props: { mockServer: SetupServerApi, baseURL: string }

  // 1. First, we make sure mockServer is properly instantiated and that we also store the baseURL that our server is returning
  // We can instantiate it in other stages, but doing that in beforeAll is a bit easier for this example + we only create one instance per suite
  test.beforeAll(async () => {
    props = await server()
  })

  // 2. We must always reset mockServer handlers and drop the in-memory database after each test 
  test.afterEach(() => {
    props.mockServer.resetHandlers()
    drop(db)
  })

  // 3. I like to have a "build" function which I'm using to extract some common login from my tests
  // This function will handle the page navigation/will wait for all required requests to be done
  const build = async (page: Page) => {
    const date = dateTimeFormat({ day: 'numeric', month: 'short', year: 'numeric' })

    // 3.1 don't forget to prefix urls with `baseURL`, since our server address is automatically generated
    await page.goto(`${props.baseURL}/?state=all&type&visibility`);

    await Promise.all([
      page.waitForResponse('**/api/trpc/github.issues.list*'),
    ])

    const openedIssues = db.issue.findMany({
      where: {
        state: {
          equals: 'open'
        }
      }
    })

    const closedIssues = db.issue.findMany({
      where: {
        state: {
          equals: 'closed'
        }
      }
    })

    // 3.2 Then we return a list of common helpers, like our database entries, our locators, etc... For me, this make our tests look cleaner
    // All that "configuration" part is extracted into this section, which I can ignore and I can read my tests in a more "human" way
    // I like the fact my "helpers" live next to my tests + try not to extract everything into this place (or different files), this function just has some "minor" things, but if I have to scroll up and down (or change files) to understand my test flow, then I think is not very cool since it kinda distracts you from actually understanding what you are testing
    return {
      openedIssues,
      closedIssues,
      issuesResponse: () => page.waitForResponse('**/api/trpc/github.issues.list*'),
      locators: {
        openedIssuesCountButton: () => page.locator(`button:has-text("Open ${openedIssues.length}")`),
        closedIssuesCountButton: () => page.locator(`button:has-text("Closed ${closedIssues.length}")`),
        issue: (issue: FactoryValue<'issue'>) => {
          const issueRow = page.locator(`data-testid=issue-${issue.id}`)
          const icon = {
            'open': 'issue-open-icon',
            'closed': 'issue-closed-icon',
          }[issue.state]
          const subtitle = {
            'open': `${issue.repository!.full_name} #${issue.number} opened on ${date.format(issue.created_at)} by ${issue.user.login}`,
            'closed': `${issue.repository!.full_name} #${issue.number} by ${issue.user.login} was closed on ${date.format(issue.closed_at)}`,
          }[issue.state]

          return {
            title: () => issueRow.locator('h3', { hasText: issue.title }).locator(`a[href="${issue.html_url}"]`),
            icon: () => issueRow.locator(`data-testid=${icon}`),
            subtitle: () => issueRow.locator('p', { hasText: subtitle }),
            prLink: () => issueRow.locator(`data-testid=issue-pull-request-${issue.id}`),
            comments: () => issueRow.locator(`a[href="${issue.html_url}"]`, { hasText: String(issue.comments) }),
            assignee: (assignee: FactoryValue<'assignee'>) => issueRow.locator(`a[href="${assignee.html_url}"]`).locator(`img[src="${assignee.avatar_url}"][alt~="${assignee.login}"]`),
          }
        }
      }
    }
  }

  // 4. Finally, we have our first tests
  test("a user can see a list of issues", async ({ page }) => {
    // 4.1 Because of the `build` function, we just have to call it and use our locators to assert what we want
    const {
      openedIssues,
      closedIssues,
      locators
    } = await build(page)

    // Then I can just follow through to my assertions =D
    await expect(locators.openedIssuesCountButton()).toBeVisible()
    await expect(locators.closedIssuesCountButton()).toBeVisible()

    for (const issue of [...openedIssues, ...closedIssues]) {
      const issueLocators = locators.issue(issue)
      await expect(issueLocators.title()).toBeVisible()
      await expect(issueLocators.icon()).toBeVisible()
      await expect(issueLocators.subtitle()).toBeVisible()


      if (issue.pull_request) {
        await expect(issueLocators.prLink()).toBeVisible()
      } else {
        await expect(issueLocators.prLink()).not.toBeVisible()
      }


      if (issue.comments > 0) {
        await expect(issueLocators.comments()).toBeVisible()
      } else {
        await expect(issueLocators.comments()).not.toBeVisible()
      }

      for (const assignee of issue.assignees) {
        await expect(issueLocators.assignee(assignee)).toBeVisible()
      }
    }
  });

  test("a user can filter issues", async ({ page }) => {
    // ......
  });
});

We can override our factory function's behaviors by changing things before calling the build function

test("a user can see a list of issues", async ({ page }) => {
    props.mockServer.use(
      issuesHandler(...),
      // or manually
      rest.get('https://api.github.com/user/orgs', (_req, res, ctx) => res(
        ctx.status(500),
        ctx.json({ error: "OOps" })
      ))
    )

    const { locators } = await build(page)
})

We can also evolve how we use our factory functions and create some cool APIs to make our tests read better, check this pseudo-code on an idea of a composable API

test("a user can see a list of issues", async ({ page }) => {
    await pipe(
      props.mockServer, // I want to start with my mockServer
      withClosedIssues(5), // I want it with 5 closed issues
      withIssue({
        title: "Custom issue"
      }) // I want also a random issue with a specific title
    )

    // if you are not a fan of pipping + using composable functions, you can create a different structure

    const mock = new MockServer(props.mockServer)
      .withClosedIssues(5)
      .withIssue({
        title: "Custom issue"
      })

    await mock.build()

    const { locators } = await build(page)
})

Configure Github Actions

Finally the last topic =D
We made it! The tests are running, and everything is green and working, but how would we add to our CI environment?

Well, is pretty simple:

# https://github.com/kuroski/github-issue-viewer/blob/main/.github/workflows/ci.yml

name: CI

on:
  push:
    branches:
      - main

# 1. First, we add the required environment variables to our file (don't forget to include everything you need from your .env file)
# You can also provide them within your repo actions secret page
env:
  DATABASE_URL: postgresql://postgres:postgres@localhost:5432/githubIssueViewer
  GITHUB_ID: "123"
  GITHUB_SECRET: "123"
  PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1 # Skip downloading during yarn install
  PLAYWRIGHT_BROWSERS_PATH: 0 # Places binaries to node_modules/@playwright/test

jobs:
  playwright:
    name: "Playwright Tests"
    runs-on: ubuntu-latest
    services:
      # 2. Include a postgres service, this will spin up a database for you in your CI environment (check the script below if you want to use mysql)
      postgres:
        image: postgres
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 5432:5432
    steps:
      # 3. Checkout our repo code
      - uses: actions/checkout@v3

      # 4. Setup node with the latest lts version
      - uses: actions/setup-node@v3
        with:
          node-version: "lts/*"

      # 5. I'm using pnpm, so here I'm making sure it's properly installed
      - name: Install pnpm
        uses: pnpm/action-setup@v2.2.2
        id: pnpm-install
        with:
          version: 7.7.0
          run_install: false

      # 6. Setup the CI cache directory for pnpm
      - name: Get pnpm store directory
        id: pnpm-cache
        run: |
          echo "::set-output name=pnpm_cache_dir::$(pnpm store path)"

      # 7. Setup the cache itself
      - name: Setup pnpm cache
        uses: actions/cache@v3
        with:
          path: ${{ steps.pnpm-cache.outputs.pnpm_cache_dir }}
          key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
          restore-keys: |
            ${{ runner.os }}-pnpm-store-

      # 8. Install our dependencies and build the project
      # As an important note, because of caching we need to make sure we are properly installing Playwright
      # The `pnpm install:playwright` script is just an alias for `playwright install --with-deps`
      - name: Install dependencies
        run: pnpm install && pnpm install:playwright
      - name: Build application
        run: pnpm build

      # 9. We need to make sure our database tables are properly set up
      - name: Migrate database
        run: pnpx prisma db push --preview-feature

      # 10. Finally, we run our tests ;D
      - name: Run your tests
        run: pnpx playwright test

There we go, you have your tests running in Github actions
https://github.com/kuroski/github-issue-viewer/actions/runs/2818168183

In case you are using mysql:

Just replace the DATABASE_URL: mysql://root:secret@127.0.0.1:3306/githubIssueViewer env variable
Remove the PG service
Add an extra step before setting up node

 - name: Set up MySQL
   uses: mirromutth/mysql-action@v1.1
   with:
     mysql database: githubIssueViewer
     mysql user: root
     mysql root password: secret
     mysql password: secret

Conclusion

We did it! Thanks for reading this guide =D!
I hope this setup might help you out, also, feel free to reach me if you have any other suggestions + or if you think I could have implemented it differently.

DEV Community: Daniel Kuroski

Building a Webhook payload delivery service in Go

Introduction

What will we build?

Server-sent events

Starting out

Server

Structs and utility functions

Creating the CLI

Testing out

Conclusion

Appendices

Deploying the server

1. Hosting

2. Configuring Docker

3. Deploy with Kamal

Extras

Building a subscription tracker Desktop and iOS app with compose multiplatform — Offline data

Introduction

Offline data + Repositories

Helper functions to convert API responses to Domain

Building a subscription tracker Desktop and iOS app with compose multiplatform—Providing feedbacks

Introduction

UI feedbacks

Creating a simple RemoteData implementation

Refactoring ViewModel

Refactoring ExpensesScreen

Persisting list entries once the first load is complete

Adding toasts for errors

Formatting money

Building a subscription tracker Desktop and iOS app with compose multiplatform - Configuring Notion

Introduction

Configuring Notion

Creating the table

Generating access keys

Testing out

Integrating Notion API with our app

Injecting our client with Koin

Refactor ExpensesScreenViewModel

Handle environment variables

Building a subscription tracker Desktop and iOS app with compose multiplatform - Setup

Introduction

What are we building?

Setup

Project walkthrough

Adding dependencies

Modeling our data

Configuring routing

Configuring material theme

Expenses list screen

Writing tests for Next.js, next-auth, Prisma using Playwright and msw

kuroski / github-issue-viewer

A Github issue viewer apps

Welcome!

Introduction

The challenges

Configuring the testing environment

Docker

Playwright

Mock server

Connecting everything and testing the application

The solution

Our first test

Configure Github Actions

Conclusion

Refactor `ExpensesScreenViewModel`