DEV Community: Jakub Zalas

Object-oriented event sourcing

Jakub Zalas — Thu, 16 Nov 2023 21:24:38 +0000

In the last post, we’ve shown how to represent state in a domain model. We mostly kept behaviours separate from data in a true functional programming fashion.

This time, we’re going to take a little detour and refactor our solution towards an object-oriented (OOP) style. As it turns out, the finite state machine we implemented previously enables us to take advantage of polymorphic behaviour via dynamic dispatch.

Don't worry though, we're not going to go full-stateful but we'll stick to immutability.

There are two behaviours we're going to move in this exercise:

applyEvent() - used for transitioning state.
execute() - used for command execution.

Transitioning state

State transitions

Last time, we implemented state transitions in a regular function called applyEvent(). It takes advantage of a sealed type hierarchy and transitions the current state to a new state based on an event.

fun applyEvent(
  game: Game,
  event: GameEvent
): Game = when (game) {
  is NotStartedGame -> when (event) {
    is GameStarted -> StartedGame(event.secret, 0, event.totalAttempts, event.availablePegs)
    else -> game
  }

  is StartedGame -> when (event) {
    is GameStarted -> game
    is GuessMade -> game.copy(attempts = game.attempts + 1)
    is GameWon -> WonGame(secret = game.secret, attempts = game.attempts, totalAttempts = game.totalAttempts)
    is GameLost -> LostGame(secret = game.secret, totalAttempts = game.totalAttempts)
  }

  is WonGame -> game
  is LostGame -> game
}

State transitions (implementation)

The advantage is we have all the transition logic in one place. Each state is a data class or object that implements a sealed interface. Thanks to the sealed interface the compiler will remind us to update missing branches in case we add a new state.

sealed interface Game

data object NotStartedGame : Game

data class StartedGame(
  val secret: Code,
  val attempts: Int,
  val totalAttempts: Int,
  val availablePegs: Set<Code.Peg>
) : Game

data object WonGame : Game

data object LostGame : Game

State objects and classes

Let's group the behaviour with data.

First, we'll need to define an applyEvent() method on the Game interface. The signature will be similar to our function's signature, except the game will no longer be a parameter.

sealed interface Game {
  fun applyEvent(event: GameEvent): Game   
}

Moving state transitions to the Game

Now, the compiler will ask us to implement the method on each state data class/object. Our IDE will help us here. We will also need to update the tests to use the new method instead of the old function.

Next, we will need to dissect the applyEvent() function and move logic to an appropriate class/object.

Refactoring steps involve calling the original function, and inlining it, to finally simplify and remove dead branches.

Dissecting state transitions

Starting with NotStartedGame, it responds to the GameStarted event. Any other event is ignored.

data object NotStartedGame : Game {
  override fun applyEvent(event: GameEvent): Game = when (event) {
    is GameStarted -> StartedGame(event.secret, 0, event.totalAttempts, event.availablePegs)
    else -> this
  }
}

Moving state transitions to NotStartedGame

StartedGame ignores GameStarted, increments attempts on GuessMade, and transitions to WonGame or LostGame in response to GameWon and GameLost respectively.

data class StartedGame(
  val secret: Code,
  val attempts: Int,
  val totalAttempts: Int,
  val availablePegs: Set<Code.Peg>
) : Game {

  override fun applyEvent(event: GameEvent): Game = when (event) {
    is GameStarted -> this
    is GuessMade -> this.copy(attempts = this.attempts + 1)
    is GameWon -> WonGame
    is GameLost -> LostGame
  }
}

Moving state transitions to StartedGame

WonGame and LostGame are terminal states so they're free to ignore any events sent their way.

data object WonGame : Game {
  override fun applyEvent(event: GameEvent): Game = this
}

data object LostGame : Game {
  override fun applyEvent(event: GameEvent): Game = this
}

Moving state transitions to WonGame and LostGame

What we end up with is behaviour for each state that's close to its type definition and data.

Executing commands

Moving on to command execution, the steps involved are quite similar.

First, we need to define a new method on the Game interface. Similarly to the previous case, the game argument goes away.

sealed interface Game {
    fun applyEvent(event: GameEvent): Game
    fun execute(command: GameCommand): Either<GameError, NonEmptyList<GameEvent>>

Moving command execution to the Game

Again, the compiler will ask us to implement missing methods and we will be able to use similar refactoring steps to move logic as before.

NotStartedGame only responds to the JoinGame command.

data object NotStartedGame : Game {
  // …

  override fun execute(command: GameCommand): Either<GameError, NonEmptyList<GameEvent>> = when (command) {
    is JoinGame -> nonEmptyListOf(
      GameStarted(command.gameId, command.secret, command.totalAttempts, command.availablePegs)
    ).right()

    else -> GameNotStarted(command.gameId).left()
  }
}

Moving command execution to NotStartedGame

StartedGame enables us to make guesses. In the previous solution, we were not handling the case of someone trying to start an already-started game. We'd simply start a new game again.

This is indicated by TODO() in the code below. Our solution here would be to either return an error or allow this to happen silently by returning no new events. The latter would require us to change the signature to allow for returning an empty list of events.

data class StartedGame(
  private val secret: Code,
  private val attempts: Int,
  private val totalAttempts: Int,
  private val availablePegs: Set<Code.Peg>
) : Game {
  // …

  override fun execute(command: GameCommand): Either<GameError, NonEmptyList<GameEvent>> = when (command) {
    is MakeGuess -> validGuess(command).map { guess ->
      GuessMade(command.gameId, Guess(command.guess, feedbackOn(guess)))
    }.withOutcome()

    else -> TODO()
  }

  // …
}

Moving command execution to StartedGame

Notice we no longer need to validate if the game is started since we're already in the StartedGame state.

Furthermore, we can lower the visibility of all the properties to be private. Nothing outside depends on them. In fact, nothing outside depends on any specific implementation of Game.

Many of our extension functions can be now promoted to private instance methods. The full class is hidden below.

Full StartedGame example


data class StartedGame(
  private val secret: Code,
  private val attempts: Int,
  private val totalAttempts: Int,
  private val availablePegs: Set<Code.Peg>
) : Game {
  private val secretLength: Int = secret.length

  private val secretPegs: List<Code.Peg> = secret.pegs

  override fun applyEvent(event: GameEvent): Game = when (event) {
    is GameStarted -> this
    is GuessMade -> this.copy(attempts = this.attempts + 1)
    is GameWon -> WonGame
    is GameLost -> LostGame
  }

  override fun execute(command: GameCommand): Either<GameError, NonEmptyList<GameEvent>> = when (command) {
    is MakeGuess -> validGuess(command).map { guess ->
      GuessMade(command.gameId, Guess(command.guess, feedbackOn(guess)))
    }.withOutcome()

    else -> TODO()
  }

  private fun validGuess(command: MakeGuess): Either<GameError, Code> {
    if (isGuessTooShort(command.guess)) {
      return GuessTooShort(command.gameId, command.guess, secretLength).left()
    }
    if (isGuessTooLong(command.guess)) {
      return GuessTooLong(command.gameId, command.guess, secretLength).left()
    }
    if (!isGuessValid(command.guess)) {
      return InvalidPegInGuess(command.gameId, command.guess, availablePegs).left()
    }
    return command.guess.right()
  }

  private fun isGuessTooShort(guess: Code): Boolean =
    guess.length < secretLength

  private fun isGuessTooLong(guess: Code): Boolean =
    guess.length > secretLength

  private fun isGuessValid(guess: Code): Boolean =
    availablePegs.containsAll(guess.pegs)

  private fun feedbackOn(guess: Code): Feedback =
    feedbackPegsOn(guess)
      .let { (exactHits, colourHits) ->
        Feedback(outcomeFor(exactHits), exactHits + colourHits)
      }

  private fun feedbackPegsOn(guess: Code) =
    exactHits(guess).map { BLACK } to colourHits(guess).map { WHITE }

  private fun outcomeFor(exactHits: List<Feedback.Peg>) = when {
    exactHits.size == this.secretLength -> WON
    this.attempts + 1 == this.totalAttempts -> LOST
    else -> IN_PROGRESS
  }

  private fun exactHits(guess: Code): List<Code.Peg> = this.secretPegs
    .zip(guess.pegs)
    .filter { (secretColour, guessColour) -> secretColour == guessColour }
    .unzip()
    .second

  private fun colourHits(guess: Code): List<Code.Peg> = this.secretPegs
    .zip(guess.pegs)
    .filter { (secretColour, guessColour) -> secretColour != guessColour }
    .unzip()
    .let { (secret, guess) ->
      guess.fold(secret to emptyList<Code.Peg>()) { (secretPegs, colourHits), guessPeg ->
        secretPegs.remove(guessPeg)?.let { it to colourHits + guessPeg } ?: (secretPegs to colourHits)
      }.second
    }

    private fun Either<GameError, GuessMade>.withOutcome(): Either<GameError, NonEmptyList<GameEvent>> =
      map { event ->
        nonEmptyListOf<GameEvent>(event) +
          when (event.guess.feedback.outcome) {
            WON -> listOf(GameWon(event.gameId))
            LOST -> listOf(GameLost(event.gameId))
            else -> emptyList()
          }
        }
      }

Moving command execution to StartedGame (full example)

Finally, our terminal states do not expect any further commands and respond with errors.

data object WonGame : Game {
  // …

  override fun execute(command: GameCommand): Either<GameError, NonEmptyList<GameEvent>> =
    GameAlreadyWon(command.gameId).left()
}

data object LostGame : Game {
  // …

  override fun execute(command: GameCommand): Either<GameError, NonEmptyList<GameEvent>> =
    GameAlreadyLost(command.gameId).left()
  }
}

Moving command execution to WonGame and LostGame

As before, we end up with behaviour for each state close to its type definition and data. We could make all the properties private since nothing outside of our public methods needs to access them. It wouldn't be terrible to keep them open though, since they're immutable.

Usage

Let's consider how usage has changed since we've moved behaviours to state classes.

Previously, we were building the state by folding past events and applying them over and over to the state transitioned by each iteration.

val game = events.fold(notStartedGame(), ::applyEvent)

Building the state (before)

That hasn't changed, except the function we now need to call is defined on the state object itself. The dynamic dispatch mechanism will take care of delegating the call to the right implementation of Game.

val game = events.fold(notStartedGame(), Game::applyEvent)

Building the state (after)

We now execute a command by sending the message to the game instance.

game.execute(command)

Executing commands

To perform the refactoring we had to update tests shortly after adding a new method to the Game interface. Here's one of the test cases that was updated to use the OOP way that demonstrates the new usage.

@Test
fun `it makes a guess`() {
  val command = MakeGuess(gameId, Code("Purple", "Purple", "Purple", "Purple"))
  val game = gameOf(
    GameStarted(gameId, secret, totalAttempts, availablePegs)
  )

  game.execute(command) shouldSucceedWith listOf(
    GuessMade(
      gameId,
      Guess(
        Code("Purple", "Purple", "Purple", "Purple"),
        Feedback(IN_PROGRESS)
      )
    )
  )
}

private fun gameOf(vararg events: GameEvent): Game = 
  events.toList().fold(notStartedGame(), Game::applyEvent)

An example test case

Summary

We experimented with refactoring our functional solution to be more object-oriented. We ended up with a domain model that's closer to what we're used to when practising OOP since all the data is now private and behaviours are implemented as instance methods. In some way, it's still functional, but it does take advantage of some object-oriented properties like encapsulation or polymorphism.

The functional solution makes it easier to add new behaviours without modifying existing types.

The object-oriented solution makes it easier to add new types without modifying existing behaviours.

Next time, we'll be leaving the realm of functional purity to take a look at how to feed events into the domain model and persist events that the model generates.

Resources

Examples gist

Deriving state from events

Jakub Zalas — Fri, 10 Nov 2023 19:51:01 +0000

Now that we have discussed a functional approach to event sourcing and have chewed over an example implementation, let's consider the state in an event-sourced domain model.

In event sourcing, the state is derived from events that have happened in the past. In a classic approach, it's the state that's persisted. In an event-sourced system, it's the events that are persisted.

In Event Sourcing, the source of truth is the events, and the state of the application is derived from these events.
A Beginner’s Guide to Event Sourcing

Events as state

The most direct thing we can do is to filter events on the fly for the facts that we need to make decisions.

Filtering events for information

Indeed, this is exactly what we've done in the last article. We defined a bunch of extension properties and functions that work on the event list.

typealias Game = List<GameEvent>

private val Game.secret: Code?
  get() = filterIsInstance<GameStarted>().firstOrNull()?.secret

private val Game.secretLength: Int
  get() = secret?.length ?: 0

private val Game.secretPegs: List<Code.Peg>
  get() = secret?.pegs ?: emptyList()

private val Game.attempts: Int
  get() = filterIsInstance<GuessMade>().size

private val Game.totalAttempts: Int
  get() = filterIsInstance<GameStarted>().firstOrNull()?.totalAttempts ?: 0

private val Game.availablePegs: Set<Code.Peg>
  get() = filterIsInstance<GameStarted>().firstOrNull()?.availablePegs ?: emptySet()

private fun Game.isWon(): Boolean =
  filterIsInstance<GameWon>().isNotEmpty()

private fun Game.isLost(): Boolean =
  filterIsInstance<GameLost>().isNotEmpty()

private fun Game.isStarted(): Boolean =
  filterIsInstance<GameStarted>().isNotEmpty()

private fun Game.isGuessTooShort(guess: Code): Boolean =
  guess.length < secretLength

private fun Game.isGuessTooLong(guess: Code): Boolean =
  guess.length > secretLength

private fun Game.isGuessValid(guess: Code): Boolean =
  availablePegs.containsAll(guess.pegs)

Example of filtering events for information

Notice that, since we have no guarantees that certain events are in the stream, we often need to handle missing events by providing default values (like in Game.availablePegs) or returning null (like in Game.secret).

Other than that, the usage is pretty similar to what we'd have if we had a proper class representing the state.

private fun startedNotFinishedGame(
  command: MakeGuess,
  game: Game
): Either<GameError, Game> {
  if (!game.isStarted()) {
    return GameNotStarted(command.gameId).left()
  }
  if (game.isWon()) {
    return GameAlreadyWon(command.gameId).left()
  }
  if (game.isLost()) {
    return GameAlreadyLost(command.gameId).left()
  }
  return game.right()
}

Extension functions in action

This is a good way forward with short streams and a relatively small number of query operations. It's also a good way to figure out what kind of properties and functions we might need for our state implementation. Once we learn what's required, we can always refactor to a stream aggregation.

Event stream aggregation

Event stream aggregation is a process that builds the state from a stream of events. It starts with an initial value and applies events one by one in order. Each application of an event returns an updated state. In other words, stream aggregation is a projection of events to a state.

Event stream aggregation

What then gets passed to our domain functions is an instance of the built state rather than the stream of events. Here's an example for our Game.

data class Game(
  val secret: Code,
  val attempts: Int,
  val totalAttempts: Int,
  val availablePegs: Set<Code.Peg>,
  val outcome: Feedback.Outcome
) {
  val secretLength: Int = secret.length

  val secretPegs: List<Code.Peg> = secret.pegs

  fun isWon(): Boolean = outcome == WON

  fun isLost(): Boolean = outcome == LOST

  fun isGuessTooShort(guess: Code): Boolean =
    guess.length < secretLength

  fun isGuessTooLong(guess: Code): Boolean =
    guess.length > secretLength

  fun isGuessValid(guess: Code): Boolean =
    availablePegs.containsAll(guess.pegs)
}

Example state derived from events

Notice that some of the nullable values or default cases have disappeared. That should help us simplify some of the code previously needed to handle such cases. Other than that, if we managed to keep the same public interface while converting extension properties and functions to instance equivalents, hardly any other code should be required to change.

Deriving the state will usually follow the same pattern. It's a left-fold of previous events.

fun notStartedGame(): Game? = null

val state = events.fold(notStartedGame(), ::applyEvent)

Deriving state from events

What's going to be different with each model implementation is the applyEvent() function. Its job is to take the state we have so far and calculate the new state by applying the next event.

fun applyEvent(
  game: Game?,
  event: GameEvent
): Game? = when (event) {
  is GameStarted -> Game(event.secret, 0, event.totalAttempts, event.availablePegs, IN_PROGRESS)
  is GuessMade -> game?.copy(attempts = game.attempts + 1)
  is GameWon -> game?.copy(outcome = WON)
  is GameLost -> game?.copy(outcome = LOST)
}

Transitioning the state by applying an event

As we can see in the above example, the GameStarted event is used to initialise the Game state as it should be the first event that happened. All the other events only modify the previous Game state by changing a subset of its properties.

As long as the first event in the stream is GameStarted, our applyEvent() function will always return a non-null value. We ignore an event if that's not the case. That's what those ?. safe calls will take care of. It's not our job to validate anything at this point. Any validation is done when we execute the command. If the event was generated it must be possible to apply it to the state.

To make this work, we will need to update our tests to build the state and pass it to tested behaviours instead of events.

@Test
fun `it makes a guess`() {
  val command = MakeGuess(gameId, Code("Purple", "Purple", "Purple", "Purple"))
  val game = gameOf(
    GameStarted(gameId, secret, totalAttempts, availablePegs)
  )

  execute(command, game) shouldSucceedWith listOf(
    GuessMade(
      gameId,
      Guess(
        Code("Purple", "Purple", "Purple", "Purple"),
        Feedback(IN_PROGRESS)
      )
    )
  )
}

private fun gameOf(vararg events: GameEvent): Game = 
  events.toList().fold(notStartedGame(), ::applyEvent)

Building state in tests

Initial state

The initial state is passed as the initial value to the fold function. In previous examples, we used null for the initial state. In many cases like ours, it's perfectly fine. In other cases though, there might be a valid initial representation of a state that's not null. Let's now consider how our game implementation would look like with a non-nullable initial state.

Instead of having null for the initial state, we will now have it explicitly modelled.

sealed interface Game

data object NotStartedGame : Game

data class StartedGame(
  val secret: Code,
  val attempts: Int,
  val totalAttempts: Int,
  val availablePegs: Set<Code.Peg>,
  val outcome: Feedback.Outcome
) : Game {
  val secretLength: Int = secret.length

  val secretPegs: List<Code.Peg> = secret.pegs

  fun isWon(): Boolean = outcome == WON

  fun isLost(): Boolean = outcome == LOST

  fun isGuessTooShort(guess: Code): Boolean =
    guess.length < secretLength

  fun isGuessTooLong(guess: Code): Boolean =
    guess.length > secretLength

  fun isGuessValid(guess: Code): Boolean =
    availablePegs.containsAll(guess.pegs)
}

fun notStartedGame(): Game = NotStartedGame

Explicitly modelled initial state

We now have a sealed interface for the Game with two implementations - NotStartedGame and StartedGame.

The advantage of this approach is its clearness. Furthermore, if there's a natural initial state for the thing we're modelling, it could have some properties initialised. That's not the case in our example. We could possibly initialise attempts or outcome, but all the other properties are provided when the game is started.

While applying events, we can now make a clear distinction between the two cases of not-started and started games. It becomes a bit more verbose, but also more explicit. We can easily see the scenarios when the game is not updated in response to an event. Also, safe calls are gone since now the game cannot be null.

fun applyEvent(
  game: Game,
  event: GameEvent
): Game = when (game) {
  is NotStartedGame -> when (event) {
    is GameStarted -> StartedGame(event.secret, 0, event.totalAttempts, event.availablePegs, IN_PROGRESS)
    else -> game
  }
  is StartedGame -> when (event) {
    is GameStarted -> game
    is GuessMade -> game.copy(attempts = game.attempts + 1)
    is GameWon -> game.copy(outcome = WON)
    is GameLost -> game.copy(outcome = LOST)
  }
}

Applying events with non-nullable initial state

Checks for the state become more explicit too.

private fun startedNotFinishedGame(
  command: MakeGuess,
  game: Game
) : Either<GameError, StartedGame> {
  if (game !is StartedGame) {
    return GameNotStarted(command.gameId).left()
  }
  // ...
  return game.right()
}

An example check for a started game

Notice that in the above example, we take a Game but we return a StartedGame. Courtesy of Kotlin smart casts.

Once we checked if the game was started we can rely on this fact and pass it down to other functions.

startedNotFinishedGame(command, game)
  .flatMap { startedGame: StartedGame ->
    // ...
  }

Smart casts in action

At the end of the day, it's very similar to using a nullable state. Except that our initial state has a name.

The question we should ask ourselves is whether having a non-nullable initial state is so much different to using null that it warrants introducing additional types. Probably not, if we consider our game example. However, it might be a useful technique for complex models.

Finite state machine

Once we realise that our previous example actually models a state machine with two states, we can expand it to model all possible states. Let's continue with our game example to explore how it changes the model.

States of the Mastermind game

As we can see on the above diagram, the game can be in one of four states: NotStartedGame, StartedGame, WonGame, and LostGame. NotStartedGame is our initial state, while WonGame and LostGame are terminal.

Here's how Mastermind game states can be implemented in Kotlin.

sealed interface Game

data object NotStartedGame : Game

data class StartedGame(
  val secret: Code,
  val attempts: Int,
  val totalAttempts: Int,
  val availablePegs: Set<Code.Peg>
) : Game {
  val secretLength: Int = secret.length

  val secretPegs: List<Code.Peg> = secret.pegs

  fun isGuessTooShort(guess: Code): Boolean =
    guess.length < secretLength

  fun isGuessTooLong(guess: Code): Boolean =
    guess.length > secretLength

  fun isGuessValid(guess: Code): Boolean =
    availablePegs.containsAll(guess.pegs)
}

data class WonGame(
  val secret: Code,
  val attempts: Int,
  val totalAttempts: Int,
) : Game

data class LostGame(
  val secret: Code,
  val totalAttempts: Int,
) : Game

States of the Mastermind game (implementation)

While applying events we now need to handle the two new states - WonGame and LostGame in response to GameWon and GameLost events respectively.

fun applyEvent(
  game: Game,
  event: GameEvent
): Game = when (game) {
  is NotStartedGame -> when (event) {
    is GameStarted -> StartedGame(event.secret, 0, event.totalAttempts, event.availablePegs)
    else -> game
  }

  is StartedGame -> when (event) {
    is GameStarted -> game
    is GuessMade -> game.copy(attempts = game.attempts + 1)
    is GameWon -> WonGame(secret = game.secret, attempts = game.attempts, totalAttempts = game.totalAttempts)
    is GameLost -> LostGame(secret = game.secret, totalAttempts = game.totalAttempts)
  }

  is WonGame -> game
  is LostGame -> game
}

Finite state machine implementation

WonGame and LostGame states are terminal so we won't be applying any events once the game reaches one of these two states.

A nice side-benefit of all this is that we could remove some functions from the StartedGame, like isWon() or isLost(). Instead, we can now use types to check the state of the game. The outcome property could also be removed from StartedGame for the same reason.

private fun startedNotFinishedGame(
  command: MakeGuess,
  game: Game
): Either<GameError, StartedGame> = when (game) {
  is NotStartedGame -> GameNotStarted(command.gameId).left()
  is WonGame -> GameAlreadyWon(command.gameId).left()
  is LostGame -> GameAlreadyLost(command.gameId).left()
  is StartedGame -> game.right()
}

Checking the state the game is in

This is even more explicit than the previous model.

Command handlers

What’s left to consider is how to build the state to be used during command execution so it could participate in decision making.

What we'll get from an event store is a stream (or a list) of events. If the command handler expects an instance of state we will need to build it first.

Adapting the type of command handler

Here's a generic creational function that builds an event list-based handler from a state-based handler.

fun <COMMAND : Any, EVENT : Any, ERROR : Any, STATE> handlerFor(
  execute: (COMMAND, STATE) -> Either<ERROR, NonEmptyList<EVENT>>,
  applyEvent: (STATE, EVENT) -> STATE,
  initialState: () -> STATE,
): (COMMAND, List<EVENT>) -> Either<ERROR, NonEmptyList<EVENT>> =
  { command, events -> 
    execute(command, events.fold(initialState(), applyEvent)) 
  }

Function for adapting the type of command handler

To create the new handler we will need to pass the one we want to adapt (execute), a function for applying an event to the state (applyEvent) and the initial state factory.

val handler = handlerFor(::execute, ::applyEvent) { null }

Adapting the type of command handler (nullable state)

To create a handler for a non-nullable initial state we will need to change the initial state factory.

val handler = handlerFor(::execute, ::applyEvent) { NotStartedGame }

Adapting the type of command handler (non-nullable state)

Such an adapted handler can be invoked with events directly. Events will then be used to rebuild the aggregated state that is in turn passed to the model's command handler (our execute function).

handler(command, events)

Invoking a handler

Summary

We considered several ways to implement state to be used in decision-making in an event-sourced domain model. We're equipped with a varied range of tools suitable for different sizes and maturity of problems, from deriving the state with stream extension functions to employing finite state machines.

Next time, we'll take a small detour to experiment with converting our solution to be more object-oriented.

Resources

Functional event sourcing example in Kotlin

Jakub Zalas — Fri, 03 Nov 2023 14:17:00 +0000

Last time we discussed a functional modeling approach to event sourcing. Let's now consider an implementation example in Kotlin.

The problem we'll be working with is the Mastermind game. If you're unfamiliar with the game, you can learn about it from Wikipedia or this GitHub repository.

As before, we're still only focusing on the domain model.

Domain command handler

We're interested in the behaviour and types of the execute() function that we discussed in the previous post. Here's how the function signature will look like for our game:

fun execute(command: GameCommand, game: Game)
  : Either<GameError, NonEmptyList<GameEvent>> = TODO()

Domain command handler

Note: The solution presented here has been developed with TDD. However, our goal now is to show the final solution rather than how we have arrived at it with refactoring. This is an important note, as I wouldn't like to introduce confusion that what we do here is TDD. TDD is better shown in videos than in the written word.

Furthermore, since we've started the series with the model, it might appear as if we're working inside-out. In practice, it's usually the other way around. We work outside-in, starting at the boundaries of the system.

Event model

Let's start with the output of an event modeling workshop to visualise what needs to be done.

Event Modeling is a visual analysis technique that focuses heavily on identifying meaningful events happening in a business process.
Yves Goeleven, What is Event Modeling? (with example)

Diagrams below reveal the commands (blue), events (orange), and error cases (red) the Mastermind game might need. We can also see the views (green), but we're not concerned with them on this occasion.

Playing the game means we join a new game and make guesses.

Playing the game

Eventually, we might make a guess that matches the secret and win the game.

Winning the game

Or, we might run out of available attempts and lose the game.

Losing the game

There are also several scenarios where our guess might be rejected.

Error scenarios

Commands

In the event modeling workshop, we have identified two commands: JoinGame and MakeGuess.

Mastermind game commands

Commands are implemented as data classes with immutable properties. What's common between the two commands is a game identifier.

sealed interface GameCommand {
    val gameId: GameId

    data class JoinGame(
        override val gameId: GameId,
        val secret: Code,
        val totalAttempts: Int,
        val availablePegs: Set<Code.Peg>
    ) : GameCommand

    data class MakeGuess(
      override val gameId: GameId,
      val guess: Code
    ) : GameCommand
}

Mastermind game commands implementation

All the commands that are handled by the same aggregate implement the same sealed interface. Later, we'll use the same pattern with events and errors.

Note: Isn't it curious how we continue to use the term "aggregate" while there's no single entity that represents the aggregate? I like to think that, conceptually, the aggregate is still there. It's represented by commands, events, and the behaviour of the execute() function.

The key benefit of sealed types comes into play with the when expression. It lets us rely on exhaustive matching:

fun execute(command: GameCommand, game: Game) =
  when(command) {
    is JoinGame -> TODO()
    is MakeGuess -> TODO()
  }

Exhaustive matching example

Let's notice there's no else branch in the above example. Since our type is sealed we can be sure we covered all the possible cases. The compiler will tell us otherwise.

If we ever add a new command implementation, the compiler will point out all the places we matched on the command that need to be updated with a new branch.

Events

In the workshop, we have also identified four events: GameStarted, GuessMade, GameWon, and GameLost. GameStarted will only be published in the beginning, while GameWon and GameLost are our terminal events.

Mastermind game events

Similarly to commands, we use a sealed interface to implement all the events.

sealed interface GameEvent {
    val gameId: GameId

    data class GameStarted(
        override val gameId: GameId,
        val secret: Code,
        val totalAttempts: Int,
        val availablePegs: Set<Code.Peg>
    ) : GameEvent

    data class GuessMade(
        override val gameId: GameId,
        val guess: Guess
    ) : GameEvent

    data class GameWon(override val gameId: GameId) : GameEvent

    data class GameLost(override val gameId: GameId) : GameEvent
}

Mastermind game events implementation

Errors

To be honest, not all of these error cases were identified in the modeling workshop. Many were discovered in the implementation phase and added to the diagram later.

Mastermind game errors

Yet again, sealed interfaces come in handy. This time the hierarchy is a bit more nested. This way we'll be able to perform a more fine-grained matching later on when it comes to handling errors and converting them to responses.

sealed interface GameError {
  val gameId: GameId

  sealed interface GameFinishedError : GameError {
    data class GameAlreadyWon(
      override val gameId: GameId
    ) : GameFinishedError

    data class GameAlreadyLost(
      override val gameId: GameId
    ) : GameFinishedError
  }

  sealed interface GuessError : GameError {
    data class GameNotStarted(
      override val gameId: GameId
    ) : GuessError

    data class GuessTooShort(
      override val gameId: GameId,
      val guess: Code,
      val requiredLength: Int
    ) : GuessError

    data class GuessTooLong(
      override val gameId: GameId,
      val guess: Code,
      val requiredLength: Int
    ) : GuessError

    data class InvalidPegInGuess(
      override val gameId: GameId,
      val guess: Code,
      val availablePegs: Set<Code.Peg>
    ) : GuessError
  }
}

Mastermind game errors implementation

Other domain types

Notice how we tend to avoid primitive types in our commands, events, and errors.

Other domain types supporting commands, events, and errors

Rich types help us to express our domain better and prevent us from making simple mistakes (i.e. is this string a game ID or a player ID?).

@JvmInline
value class GameId(val value: String)

data class Code(val pegs: List<Peg>) {
  constructor(vararg pegs: String) : this(pegs.map(::Peg))

  data class Peg(val name: String)

  val length: Int get() = pegs.size
}

data class Guess(val code: Code, val feedback: Feedback)

data class Feedback(val outcome: Outcome, val pegs: List<Peg>) {
  constructor(outcome: Outcome, vararg pegs: Peg) :
    this(outcome, pegs.toList())

  enum class Peg { BLACK, WHITE }

  enum class Outcome { IN_PROGRESS, WON, LOST }
}

Implementation of remaining domain types

Behaviours

With the right workshop, we can discover a lot of types and behaviours upfront. It's nearly impossible to think of everything upfront though. We fear not, as when it comes to the implementation, the tests we write tend to guide us and fill in the gaps.

Joining the game

Joining the game should result in the GameStarted event.

Joining the game

Test cases for domain model behaviours could not be simpler. We need to arrange fixtures, execute the tested behaviour, and verify the outcome. The good old Arrange/Act/Assert, a.k.a Given/When/Then. Meszaros calls it a four-phase test, but in our case, the fourth phase will be implicitly done by the test framework and there's nothing left to tear down.

Four-phase test

Since we planned for our domain model to be purely functional, it's easy to stick to this structure to keep the tests short and readable.

In the test case below, the test fixture (test context) is defined as properties of the test class. The test case creates the command, executes it, and verifies we got the expected events in response. This is how all our tests for the domain model will look like.

class GameExamples {
  private val gameId = anyGameId()
  private val secret = Code("Red", "Green", "Blue", "Yellow")
  private val totalAttempts = 12
  private val availablePegs = setOfPegs("Red", "Green", "Blue", "Yellow", "Purple", "Pink")

  @Test
  fun `it starts the game`() {
    val command = JoinGame(gameId, secret, totalAttempts, availablePegs)

    execute(command) shouldSucceedWith listOf(
      GameStarted(
        gameId,
        secret,
        totalAttempts,
        availablePegs
      )
    )
  }
}

Joining the game test case

The actual result of execute() is of type Either<GameError, NonEmptyList<GameEvent>>, so when verifying the outcome we need to confirm whether we received the left or the right side of it. Left is by convention used for the error case, while Right is for success.

Either a GameError or a list of GameEvents

We created the shouldSucceedWith() and shouldFailWith() extension functions in the hope of removing some noise from the tests.

infix fun <A, B> Either<A, B>.shouldSucceedWith(expected: B) =
  assertEquals(expected.right(), this, "${expected.right()} is $this")

infix fun <A, B> Either<A, B>.shouldFailWith(expected: A) =
  assertEquals(expected.left(), this, "${expected.left()} is $this")

Custom assertions

Read more about Either and typed errors in Arrow's documentation.

Since for this first case, there are no error scenarios or any complex calculations, we only need to return a list with the GameStarted event inside. Matching the command with when() gives us access to command properties for the matched type thanks to smart casts.

typealias Game = List<GameEvent>

fun notStartedGame(): Game = emptyList()

fun execute(
  command: GameCommand,
  game: Game = notStartedGame()
): Either<GameError, NonEmptyList<GameEvent>> = when (command) {
  is JoinGame -> joinGame(command),
  is MakeGuess -> TODO()
}

private fun joinGame(command: JoinGame) =
  either<Nothing, NonEmptyList<GameStarted>> {
    nonEmptyListOf(
      GameStarted(
        command.gameId,
        command.secret,
        command.totalAttempts,
        command.availablePegs
      )
    )
  }
}

Starting the game

Notice the either {} builder above. It makes sure the value we return from its block is wrapped in an instance of Either, in this case, the Right instance. either { nonEmptyListOf(GameStarted())) } is an equivalent of nonEmptyListOf(GameStarted())).right().

Making a guess

Making a valid guess should result in the GuessMade event.

Making a guess

This time, we additionally need to prepare the state of the game, since MakeGuess is never executed as the first command. The game must've been started first.

@Test
fun `it makes a guess`() {
  val command = MakeGuess(gameId, Code("Purple", "Purple", "Purple", "Purple"))
  val game = gameOf(
    GameStarted(gameId, secret, totalAttempts, availablePegs)
  )

  execute(command, game) shouldSucceedWith listOf(
    GuessMade(
      gameId,
      Guess(
        Code("Purple", "Purple", "Purple", "Purple"),
        Feedback(IN_PROGRESS)
      )
    )
  )
}

Making a guess test case

The state is created based on past events. We delegated this task to the gameOf() function. Since for now, we treat the list of events as the state, we only need to return the list. Later on, we'll see how to convert it to an actual state object.

fun gameOf(vararg events: GameEvent): Game = listOf(*events)

Creating the game state

Now we need to return the GuessMade event for the second when() branch.

fun execute(
  command: GameCommand,
  game: Game = notStartedGame()
): Either<GameError, NonEmptyList<GameEvent>> = when (command) {
  is JoinGame -> joinGame(command)
  is MakeGuess -> makeGuess(command, game)
}

private fun makeGuess(command: MakeGuess, game: Game) =
  either<GameError, NotEmptyList<GuessMade>> {
    nonEmptyListOf(
      GuessMade(
        command.gameId,
        Guess(command.guess, game.feedbackOn(command.guess))
      )
    )
  }

private fun Game.feedbackOn(guess: Code) = Feedback(IN_PROGRESS)

Making a guess

Receiving feedback

Feedback should be given on each valid guess with the GuessMade event.

Receiving feedback on the guess

Here's one of the test cases for receiving feedback.

fun `it gives feedback on the guess`() { 
  val secret = Code("Red", "Green", "Blue", "Yellow")
  val guess = Code("Red", "Purple", "Blue", "Purple")
  val game = gameOf(
    GameStarted(gameId, secret, totalAttempts, availablePegs)
  )

  execute(MakeGuess(gameId, guess), game) shouldSucceedWith listOf(
    GuessMade(gameId, Guess(
      guess,
      Feedback(IN_PROGRESS, BLACK, BLACK))
    )
  )
}

Receiving feedback test case

We'll need a few more similar test cases to develop the logic for giving feedback. They're hidden below if you're interested.

Receiving feedback test cases

@TestFactory
fun `it gives feedback on the guess`() = guessExamples { (secret: Code, guess: Code, feedback: Feedback) ->
  val game = gameOf(
    GameStarted(gameId, secret, totalAttempts, availablePegs)
  )

  execute(MakeGuess(gameId, guess), game) shouldSucceedWith listOf(
    GuessMade(gameId, Guess(guess, feedback))
  )
}

private fun guessExamples(block: (Triple<Code, Code, Feedback>) -> Unit) = mapOf(
  "it gives a black peg for each code peg on the correct position" to Triple(
    Code("Red", "Green", "Blue", "Yellow"),
    Code("Red", "Purple", "Blue", "Purple"),
    Feedback(IN_PROGRESS, BLACK, BLACK)
  ),
  "it gives no black peg for code peg duplicated on a wrong position" to Triple(
    Code("Red", "Green", "Blue", "Yellow"),
    Code("Red", "Red", "Purple", "Purple"),
    Feedback(IN_PROGRESS, BLACK)
  ),
  "it gives a white peg for code peg that is part of the code but is placed on a wrong position" to Triple(
    Code("Red", "Green", "Blue", "Yellow"),
    Code("Purple", "Red", "Purple", "Purple"),
    Feedback(IN_PROGRESS, WHITE)
  ),
  "it gives no white peg for code peg duplicated on a wrong position" to Triple(
    Code("Red", "Green", "Blue", "Yellow"),
    Code("Purple", "Red", "Red", "Purple"),
    Feedback(IN_PROGRESS, WHITE)
  ),
  "it gives a white peg for each code peg on a wrong position" to Triple(
    Code("Red", "Green", "Blue", "Red"),
    Code("Purple", "Red", "Red", "Purple"),
    Feedback(IN_PROGRESS, WHITE, WHITE)
  )
).dynamicTestsFor(block)

fun <T : Any> Map<String, T>.dynamicTestsFor(block: (T) -> Unit) =
  map { (message, example: T) -> DynamicTest.dynamicTest(message) { block(example) } }

All receiving feedback test cases

Eventually, we arrived at the solution below.

private fun Game.feedbackOn(guess: Code): Feedback =
  feedbackPegsOn(guess)
    .let { (exactHits, colourHits) ->
      Feedback(IN_PROGRESS, exactHits + colourHits)
    }

private fun Game.feedbackPegsOn(guess: Code) =
  exactHits(guess).map { BLACK } to colourHits(guess).map { WHITE }

Calculating feedback on the guess

We've hidden Game.exactHits() and Game.colourHits() below as they're not necessarily relevant. The point is we identify pegs on the correct and incorrect positions and convert them to BLACK and WHITE pegs respectively.

Exact hits and colour hits

private fun Game.exactHits(guess: Code): List<Code.Peg> = 
  this.secretPegs
    .zip(guess.pegs)
    .filter { (secretColour, guessColour) -> secretColour == guessColour }
    .unzip()
    .second

private fun Game.colourHits(guess: Code): List<Code.Peg> =
  this.secretPegs
    .zip(guess.pegs)
    .filter { (secretColour, guessColour) -> secretColour != guessColour }
    .unzip()
    .let { (secret, guess) ->
        guess.fold(secret to emptyList<Code.Peg>()) { (secretPegs, colourHits), guessPeg ->
            secretPegs.remove(guessPeg)?.let { it to colourHits + guessPeg } ?: (secretPegs to colourHits)
        }.second
    }

/**
 * Removes an element from the list and returns the new list, or null if the element wasn't found.
 */
private fun <T> List<T>.remove(item: T): List<T>? = indexOf(item).let { index ->
    if (index != -1) filterIndexed { i, _ -> i != index }
    else null
}

Detailed calculations of exact and colour hits

What's more interesting is how we accessed the state of the game - namely the secret code and its pegs. The secret is available with the first GameStarted event. Since we have access to the whole event history for the given game, we could filter it for the information we need. We achieved it with extension functions on the Game type (which is actually a List<GameEvent> for now).

private val Game.secret: Code?
    get() = filterIsInstance<GameStarted>().firstOrNull()?.secret

private val Game.secretPegs: List<Code.Peg>
    get() = secret?.pegs ?: emptyList()

Deriving state properties

Enforcing invariants

invariants: rules that have to be protected at all times
"Learning Domain-Driven Design" by Vladik Khononov.

Rejecting a command

Commands that might put our game in an inconsistent state should be rejected. For example, we can't make a guess for a game that hasn't started or has finished. Also, guesses with a code that's of a different length to the secret are incomplete. Finally, we can't make guesses with pegs that are not part of the game.

An invariant is a state that must stay transactionally consistent throughout the Entity life cycle.
"Implementing Domain-Driven Design" by Vaughn Vernon

Game not started

The game should not be played if it has not been started.

@Test
fun `the game cannot be played if it was not started`() {
  val code = Code("Red", "Purple", "Red", "Purple")
  val game = notStartedGame()

  execute(MakeGuess(gameId, code), game) shouldFailWith
    GameNotStarted(gameId)
}

Game not started test case

Our approach to validate an argument is to pass it to a validation function that returns the validated value or an error. That's the job for startedNotFinishedGame() below. Once the game is returned from the validation function, we can be sure that it was started. We can then pass it to the next function with map(). In case of an error, map will short-circuit by returning the error, and the next function won't be called.

private fun makeGuess(command: MakeGuess, game: Game) =
  startedNotFinishedGame(command, game).map { startedGame ->
    nonEmptyListOf(
      GuessMade(
        command.gameId,
        Guess(command.guess, startedGame.feedbackOn(command.guess))
      )
    )
  }

private fun startedNotFinishedGame(command: MakeGuess, game: Game): Either<GameError, Game> {
  if (!game.isStarted()) {
    return GameNotStarted(command.gameId).left()
  }
  return game.right()
}

Validating if the game is started

private fun Game.isStarted(): Boolean =
    filterIsInstance<GameStarted>().isNotEmpty()

Deriving state properties for validation

Guess is invalid

The guess code needs to be of the same length as the secret. Furthermore, it should only contain pegs that the game has been started with.

@Test
fun `the guess length cannot be shorter than the secret`() {
  val secret = Code("Red", "Green", "Blue", "Yellow")
  val code = Code("Purple", "Purple", "Purple")
  val game = gameOf(GameStarted(gameId, secret, 12, availablePegs))

  execute(MakeGuess(gameId, code), game) shouldFailWith 
    GuessTooShort(gameId, code, secret.length)
}

@Test
fun `the guess length cannot be longer than the secret`() {
  val secret = Code("Red", "Green", "Blue", "Yellow")
  val code = Code("Purple", "Purple", "Purple", "Purple", "Purple")
  val game = gameOf(GameStarted(gameId, secret, 12, availablePegs))

  execute(MakeGuess(gameId, code), game) shouldFailWith 
    GuessTooLong(gameId, code, secret.length)
}

@Test
fun `it rejects pegs that the game was not started with`() {
  val secret = Code("Red", "Green", "Blue", "Blue")
  val availablePegs = setOfPegs("Red", "Green", "Blue")
  val game = gameOf(GameStarted(gameId, secret, 12, availablePegs))
  val guess = Code("Red", "Green", "Blue", "Yellow")

  execute(MakeGuess(gameId, guess), game) shouldFailWith
    InvalidPegInGuess(gameId, guess, availablePegs)
}

Guess validation test cases

Here, we use the same trick with a validation function to validate the guess. Notice we have to use flatMap() instead of map() in the outer call. Otherwise an Either result would be placed inside another Either.

private fun makeGuess(command: MakeGuess, game: Game) =
  startedNotFinishedGame(command, game).flatMap { startedGame ->
    validGuess(command, startedGame).map { guess ->
      nonEmptyListOf(
        GuessMade(
          command.gameId,
          Guess(command.guess, startedGame.feedbackOn(guess))
        )
      )
    }
  }

private fun validGuess(command: MakeGuess, game: Game): Either<GameError, Code> {
  if (game.isGuessTooShort(command.guess)) {
    return GuessTooShort(command.gameId, command.guess, game.secretLength).left()
  }
  if (game.isGuessTooLong(command.guess)) {
    return GuessTooLong(command.gameId, command.guess, game.secretLength).left()
  }
  if (!game.isGuessValid(command.guess)) {
    return InvalidPegInGuess(command.gameId, command.guess, game.availablePegs).left()
  }
  return command.guess.right()
}

Guess validation

private fun Game.isGuessTooShort(guess: Code): Boolean =
  guess.length < this.secretLength

private fun Game.isGuessTooLong(guess: Code): Boolean =
  guess.length > this.secretLength

private fun Game.isGuessValid(guess: Code): Boolean =
  availablePegs.containsAll(guess.pegs)

Deriving state properties for guess validation

Winning the game

Winning the game means publishing the GameWon event in addition to the usual GuessMade.

Winning the game

@Test
fun `the game is won if the secret is guessed`() {
  val game = gameOf(GameStarted(gameId, secret, totalAttempts, availablePegs))

  execute(MakeGuess(gameId, secret), game) shouldSucceedWith listOf(
    GuessMade(gameId, Guess(secret, Feedback(WON, BLACK, BLACK, BLACK, BLACK))),
    GameWon(gameId)
  )
}

Winning the game test case

We now need to decide if the game is won based on whether the number of guessed pegs is equal to the secret length.

private fun Game.outcomeFor(exactHits: List<Feedback.Peg>) = when {
  exactHits.size == this.secretLength -> WON
  else -> IN_PROGRESS
}

Winning the game implementation

Again, that requires a bit of state.

private val Game.secretLength: Int
  get() = secret?.length ?: 0

Deriving state properties

We've taken care of deciding whether the game is won. We also need to publish an additional event. To do this, we make makeGuess() return a single event. Next, we create the withOutcome() function that converts the result of makeGuess() to a list and optionally appends the GameWon event.

fun execute(
    command: GameCommand,
    game: Game = notStartedGame()
): Either<GameError, NonEmptyList<GameEvent>> =
  when (command) {
    is JoinGame -> joinGame(command)
    is MakeGuess -> makeGuess(command, game)
                      .withOutcome()
  }

private fun makeGuess(command: MakeGuess, game: Game) =
  startedNotFinishedGame(command, game).flatMap { startedGame ->
    validGuess(command, startedGame).map { guess ->
      GuessMade(command.gameId, Guess(command.guess, startedGame.feedbackOn(guess)))
    }
  }

private fun Either<GameError, GuessMade>.withOutcome(): Either<GameError, NonEmptyList<GameEvent>> =
  map { event ->
    nonEmptyListOf<GameEvent>(event) +
      when (event.guess.feedback.outcome) {
        WON -> listOf(GameWon(event.gameId))
        LOST -> listOf(GameLost(event.gameId))
        else -> emptyList()
      }
  }

Determining the game-terminating events

Once the game is won it shouldn't be played anymore.

@Test
fun `the game can no longer be played once it's won`() {
  val game = gameOf(GameStarted(gameId, secret, totalAttempts, availablePegs))

  val update = execute(MakeGuess(gameId, secret), game)
  val updatedGame = game.updated(update)

  execute(MakeGuess(gameId, secret), updatedGame) shouldFailWith 
    GameAlreadyWon(gameId)
}

private fun Game.updated(update: Either<GameError, Game>): Game = this + update.getOrElse { emptyList() }

Playing a won game test case

This requires an additional validation rule.

private fun startedNotFinishedGame(command: MakeGuess, game: Game): Either<GameError, Game> {
  if (!game.isStarted()) {
    return GameNotStarted(command.gameId).left()
  }
  if (game.isWon()) {
    return GameAlreadyWon(command.gameId).left()
  }
  return game.right()
}

Validation for a won game

The game is won if there has been a GameWon event published.

private fun Game.isWon(): Boolean =
  filterIsInstance<GameWon>().isNotEmpty()

Deriving state properties

Losing the game

Losing the game means publishing the GameLost event in addition to the usual GuessMade.

Losing the game

@Test
fun `the game is lost if the secret is not guessed within the number of attempts`() {
  val secret = Code("Red", "Green", "Blue", "Yellow")
  val wrongCode = Code("Purple", "Purple", "Purple", "Purple")
  val game = gameOf(
    GameStarted(gameId, secret, 3, availablePegs),
    GuessMade(gameId, Guess(wrongCode, Feedback(IN_PROGRESS))),
    GuessMade(gameId, Guess(wrongCode, Feedback(IN_PROGRESS))),
  )

  execute(MakeGuess(gameId, wrongCode), game) shouldSucceedWith listOf(
    GuessMade(gameId, Guess(wrongCode, Feedback(LOST))),
    GameLost(gameId)
  )
}

Losing the game test case

We now need to decide if the game is lost based on whether we have run out of attempts.

private fun Game.outcomeFor(exactHits: List<Feedback.Peg>) = when {
  exactHits.size == this.secretLength -> WON
  this.attempts + 1 == this.totalAttempts -> LOST
  else -> IN_PROGRESS
}

Outcome calculation for a lost game

The following state-accessing functions turned out to be handy for this task.

private val Game.attempts: Int
  get() = filterIsInstance<GuessMade>().size

private val Game.totalAttempts: Int
  get() = filterIsInstance<GameStarted>().firstOrNull()?.totalAttempts ?: 0

Deriving state properties

One last rule is that we cannot continue playing a lost game.

@Test
fun `the game can no longer be played once it's lost`() {
  val secret = Code("Red", "Green", "Blue", "Yellow")
  val wrongCode = Code("Purple", "Purple", "Purple", "Purple")
  val game = gameOf(GameStarted(gameId, secret, 1, availablePegs))

  val update = execute(MakeGuess(gameId, wrongCode), game)
  val updatedGame = game.updated(update)

  execute(MakeGuess(gameId, secret), updatedGame) shouldFailWith
    GameAlreadyLost(gameId)
}

Playing a lost game test case

One more condition in the validation function will do the job.

private fun startedNotFinishedGame(command: MakeGuess, game: Game): Either<GameError, Game> {
  if (!game.isStarted()) {
    return GameNotStarted(command.gameId).left()
  }
  if (game.isWon()) {
    return GameAlreadyWon(command.gameId).left()
  }
  if (game.isLost()) {
    return GameAlreadyLost(command.gameId).left()
  }
  return game.right()
}

Validation for a lost game

The game is lost if there has been a GameLost event published.

private fun Game.isLost(): Boolean =
  filterIsInstance<GameLost>().isNotEmpty()

Deriving state properties

Summary

We explained the implementation details of an event-sourced functional domain model in Kotlin. In the process, we attempted to show how straightforward testing of such a model can be and how it doesn't require any dedicated testing techniques or tools. The model itself, on the other hand, remains a rich core of business logic.

Hopefully, dear reader, you found it useful.

If you prefer to see a full example in one place, here's the gist: https://gist.github.com/jakzal/3f0ee968fe8073ee81e328ecbd59fe8b

Next time, we'll continue the example by replacing extension functions that derive state with an actual state data class.

References

Mastermind game rules
What is Event Modeling? (with example) by Yves Goeleven
CQRS by Martin Fowler
Learning Domain-Driven Design by Vladik Khononov.
Implementing Domain-Driven Design by Vaughn Vernon
Functional Programming Ideas for the Curious Kotliner by Alejandro Serrano Mena
Typed errors in Arrow documentation
Mastermind game example gist

Functional event sourcing

Jakub Zalas — Thu, 19 Oct 2023 22:01:03 +0000

In the previous post, we looked at the benefits of a domain model implemented in a purely functional code.

This time, we’ll consider how it might work in practice by applying the event sourcing pattern to a functional domain model.

As it turns out the two go very well together.

Event sourcing

The idea behind event sourcing is to:

Capture all changes to an application state as a sequence of events.
Martin Fowler’s Event Sourcing entry

Sequence of events

Commands in, events out

At the fundamental level, we only need one function for each aggregate root to handle its behaviours. The function will calculate a sequence of events in response to a command. Command is an intention to trigger a behaviour.

Command in, events out

Let's call this function Execute since its job is to execute the command.

typealias Execute<COMMAND, EVENT> 
    = (COMMAND) -> NonEmptyList<EVENT>

Execute function type alias

Note: I chose Execute as the name for the function following one of the naming strategies for aggregate methods found in "Learning Domain-Driven Design" by Vladik Khononov.

History is not forgotten

As we execute commands for any given aggregate, events will be appended to a journal (event store). Past events will be needed to handle future commands for that aggregate.

For example, a game cannot be played before it is started or after it is lost. Or, once an order has shipped it can no longer be amended.

We will look into the history to make decisions.

Therefore, the Execute function will also need the past events. That list will be empty when the first command for a given aggregate is executed.

Event history takes part in decision making

Here's the updated type definition of Execute:

typealias Execute<COMMAND, EVENT> 
    = (COMMAND, List<EVENT>) -> NonEmptyList<EVENT>

Execute function type alias

Protecting invariants

That's going well, but what about error cases?

As we agreed in the last post, we want to be explicit about domain errors. That's why we'll make them one of the possible outcomes:

Errors become one of the possible outcomes

The Execute function will need to either return an error or a non-empty list of events:

typealias Execute<COMMAND, ERROR, EVENT> 
    = (COMMAND, List<EVENT>) -> Either<ERROR, NonEmptyList<EVENT>>

Execute function type alias

Note: We use Arrow for typed errors, specifically the Either and NonEmptyList types. However, any type-safe error-handling library would work. Result4k is a nice alternative to Arrow.

The matter of state

As we defined it so far, the Execute function is the minimum we need to implement an event-sourced domain model.

Here's an example of a further unidentified game pending the implementation:

typealias Game = List<GameEvent>

fun execute(command: GameCommand, game: Game)
  : Either<GameError, NonEmptyList<GameEvent>> = when(command) {
  is JoinGame -> TODO()
  is MakeGuess -> TODO()
}

Let's assume we don't want to allow to make guesses once the game is won. The game is considered won when the GameWon event is published. In such a case, any MakeGuess command should be rejected with an error.

As we suggested before, we will need to look at the history of events to make decisions. Having the list of events passed to the Execute function we can scan it for facts:

events.filterIsInstance<GameWon>().isNotEmpty()

Furthermore, we can name any concept by defining an extension function for it:

private fun Game.isWon(): Boolean = 
  filterIsInstance<GameWon>().isNotEmpty()

Remember the Game here is actually a List<GameEvent>.

Thanks to extension functions and our previously defined type alias for the Game (as a list of events), it becomes very natural to use:

if (game.isFinished()) {
  return GameAlreadyFinished(gameId).left()
}

// Note: `.left()` wraps the result in an `Either.Left` 
// that represents the error branch.

Just as if we had the Game class defined.

It is just a list of events though.

It will work rather well for short streams of events. However, scanning long streams might get expensive, especially if we make decisions based on a lot of factors. For such scenarios, we can introduce an optimisation in the form of a computed state (projection).

The entity's state can be derived from its history of events.

The difference between using a computed state and scanning events history is that when we reconstruct the state object every event is applied only once to update the state. Then, we can access it however many times we need. In the case of event scanning, the scan is performed every time we need to access a bit of information.

Note: We can go a long way without having to introduce a computed state if we treat the history of events as the state and scan it for information as shown above.

I often use this technique as an intermediate step. It helps me to learn what the properties of the state object should look like. It sometimes turns out to be a different view than I thought of before the implementation.

Execute with state

Let's redefine the Execute function one last time to account for the possibility of the state being any kind of type (not just an events list).

Final form of the Execute function

Here's the final revision of the Execute function type alias:

typealias Execute<COMMAND, STATE, ERROR, EVENT> 
    = (COMMAND, STATE) -> Either<ERROR, NonEmptyList<EVENT>>

Final definition of the Execute function type alias

This way we can still use a sequence of events as a state:

fun execute(command: GameCommand, game: Game)
  : Either<GameError, NonEmptyList<GameEvent>> = TODO()

typealias Game = NonEmptyList<GameEvent>

private fun Game.isWon(): Boolean = 
  filterIsInstance<GameWon>().isNotEmpty()

but we can also decide to implement it as a dedicated data class:

fun execute(command: GameCommand, game: Game?)
  : Either<GameError, NonEmptyList<GameEvent>> = TODO()

data class Game(val isWon: Boolean)

The state is made nullable as it won't exist when the first command is executed.

Alternatively, we could model the initial state as a special type. The choice depends on the use case. Sometimes it's natural to have a dedicated initial state object; while in other cases it makes more sense for the initial state to be null (as no natural initial state exists). As we know, in Kotlin we don't need to be afraid of nulls.

State reconstruction

We will need a way to reconstruct the state with events we got from the event store so that we can pass it to the Execute function.

Instead of scanning through the history of events to access every bit of information, we will instead build the state by applying events to it one by one.

Current state is a left fold of previous behaviours.
Greg Young in many of his talks

To calculate the state, we will start with null (or other initial state), and fold events with an applyEvent function.

val state = events.fold(null, ::applyEvent)

Left fold of previous events

The applyEvent function, or simply Apply, should take the state, an event, and return a new state.

Apply calculates a new state for the given state and event

In other words, it will calculate the new state based on the previous state and an event.

Here's the type alias for the Apply function:

typealias Apply<STATE, EVENT> = (STATE, EVENT) -> STATE

Apply function type alias

Here’s how it could look like for a further unidentified game:

fun applyEvent(game: Game?, event: GameEvent): Game? = when(event) {
    is GameStarted -> Game(event.secret, Outcome.UNRESOLVED)
    is GuessMade -> TODO()
    is GameLost -> game?.copy(outcome = Outcome.LOST)
    is GameWon -> game?.copy(outcome = Outcome.WON)
}

Folding events over the state means that the Apply function is called on the initial state and the first event to calculate a new state, which is in turn passed to the next invocation of Apply with the next event. This process is repeated until we run out of events and return the final version of the state.

Fold of events illustrated

All together now

The command will usually come from a user in one way or another (sometimes from other systems, in reaction to events or other commands etc).

Events will be loaded from an event store.

These two are not compatible parameters for the Execute function if we decide to use a computed state.

We will need to convert:

To:

The conversion of one type to another isn’t too complicated. We can implement a creational function that takes one version of Execute and returns the desired one:

fun <COMMAND : Any, EVENT : Any, ERROR : Any, STATE> handlerFor(
    execute: Execute<COMMAND, STATE, ERROR, EVENT>,
    applyEvent: Apply<STATE, EVENT>,
    initialState: () -> STATE,
): Execute<COMMAND, List<EVENT>, ERROR, EVENT> =
  { command, events -> execute(
      command,
      events.fold(initialState(), applyEvent)
  )}

Apart from Execute we will also need to pass Apply to fold events and build the state.

Additionally, it’s just too easy to support any initial state, not just nulls. That’s the job for the initialState function.

Changing the type of `Execute` to match received parameters

Inspirations

Before I started a project with this approach I did a lot of reading on functional programming techniques for event sourcing. One notable read I remember is Functional event sourcing decider by Jeremie Chassaing. An attentive reader will notice that what I described here comes very close to Jeremie's decider pattern.

I didn't set out to implement the decider pattern, but it kind of happened anyway. Kind of. Not really. Maybe?

Depends on how far you'll allow us to diverge from the pattern.

The decider pattern is made of the following parts:

class Decider<COMMAND: Any, EVENT: Any, STATE: Any>(
  val decide: (COMMAND, STATE) -> List<EVENT>,
  val evolve: (STATE, EVENT) -> STATE,
  val initialState: STATE,
  val isTerminal: (STATE) -> Boolean
)

The approach we've taken is:

class Handler<COMMAND : Any, EVENT : Any, ERROR : Any, STATE: Any?>(
  val execute: (COMMAND, STATE) -> Either<ERROR, NonEmptyList<EVENT>>,
  val applyEvent: (STATE, EVENT) -> STATE,
  val initialState: () -> STATE,
)

Apart from naming conventions (Execute/Apply vs Decide/Evolve), there are a few other differences:

I accept null as a valid initial state.
I’m in favour of explicit domain errors rather than using exceptions.
I treat evolve and initial state parts as optional.
I have not found the need for the terminal state (yet?).

Now go and read Jeremie’s article or listen to one of his talks.

Summary

We have applied the event sourcing pattern in a functional domain model.

Next time, we will discuss an example implementation before moving on to the outer layers of the application.

Resources

Functional domain model

Jakub Zalas — Fri, 13 Oct 2023 08:40:24 +0000

A functional domain model is made of pure functions and immutable types. As Domain Driven Design teaches us, it should be expressed in the language shared by everyone involved in the project.

Pure functions

A pure function always returns the same result given the same arguments.

Pure function

Here are the fundamental properties of a pure function:

A function returns exactly the same result every time it's called with the same set of arguments. In other words a function has no state, nor can it access any external state. Every time you call it, it behaves like a newborn baby with blank memory and no knowledge of the external world.

A function has no side effects. Calling a function once is the same as calling it twice and discarding the result of the first call.
Bartosz Milewski, School of Haskell

Pure functions are calculations.

Here's an example of a pure function:

fun execute(command: Command): Event = when(command) {
    is SayHello -> 
        MessagePublished("Hello ${command.recipientName}.")
    is SayGoodbye -> 
        MessagePublished("See you later ${command.recipientName}!")
}

Pure function example

It doesn’t matter how many times we call the execute function above, as long as we call it with the same command we will always get the same message back.

assert(
  execute(SayHello(“Alice”)) == execute(SayHello(“Alice”))
)
assert(
  execute(SayHello(“Alice”)) != execute(SayHello(“Bob”))
)

Pure function properties

Immutable types

An immutable data structure cannot be changed after its instance was created.

Here's an example of an immutable type:

data class MessagePublished(val message: String) : Event

Immutable type example

Once initialised, its properties cannot be changed:

val event = MessagePublished("Hello!")

// Won't compile: "Val cannot be reassigned"
event.message = "New message"

Since we used val to store the reference to the object, it cannot be reassigned either:

val event = MessagePublished("Hello!")

// Won't compile: "Val cannot be reassigned"
event = MessagePublished("Bye")

Instead of modifying an immutable instance we need to create a new one:

val event = MessagePublished("Hello!")

val newEvent = event.copy(message = "Bye!")
// or:
// val newEvent = MessagePublished("Bye")

assert(MessagePublished("Hello!") == event)
assert(MessagePublished("Bye!") == newEvent)

Modifying immutable data

Testing

Nothing gets close to the joy of writing tests for pure functions that work with immutable data structures.

We pass the input and then verify the output.

The good old Arrange-Act-Assert, a.k.a. Given-When-Then, is so evident.

@Test
fun `it says hello`() {
    val command = SayHello("John")

    val event = execute(command)

    assertEquals(MessagePublished("Hello John."), event)
}

@Test
fun `it says goodbye`() {
    val command = SayGoodbye("Sue")

    val event = execute(command)

    assertEquals(MessagePublished("See you later Sue!"), event)
}

Test examples

This kind of tests are easiest to write, a joy to read, and the fastest to execute.

Reasoning scope

When considered in a narrow scope of a single function call, neither mutable types nor functions with side effects have a big impact on our reasoning or testing. They're manageable with a bit of discipline.

Function accessing a mutable state

However, it gets more complicated in the wider context of the application.

Multiple functions accessing the same mutable state

If mutable data or its reference is shared, it could be changed from (m)any part(s) of the application. Similarly, if a function has side effects, its effects can be seen elsewhere. The order of calls starts to matter as well.

Immutable types and pure functions narrow down the scope of change. State changes are local to the place that triggered them and need to be explicitly passed up or down.

That helps with reasoning, as in our head we can now confidently replace the function call with its result (see referential transparency and equational reasoning).

Referential transparency illustrated

Error handling

One last thing to consider in a functional domain model is our attitude to error cases.

A common approach is to use exceptions.

Unfortunately, exceptions cripple our reasoning ability, similarly to mutable types or side-effect functions, as they break referential transparency. Exceptions are in a way a form of non-local GOTO since they allow to jump up the stack out of the normal execution flow. These properties go against our functional model.

An exception seen breaking an application

Therefore, it's good to make a distinction between domain errors, and infrastructure or panic errors (see types of errors). The first are expected while the latter are mostly exceptions.

Unchecked exceptions are not part of the function's signature, so without looking into the function's body we won't know what kind of errors to expect. The caller could be totally oblivious and ignore exceptions altogether.

fun execute(command: Command): Event
    = throw DisallowedCommand()

Exception example

Exceptions are implicit and unexpected (unless checked).

Errors, on the other hand, need to be explicitly defined as part of the function's signature.

fun execute(command: Command): Either<Error, Event>
    = Left(DisallowedCommand())

Explicit error result example

Explicit error result

The caller needs to either deal with the error or pass it up to the caller.

val response = game
  .execute(SayGoodbye("Bob"))
  .map(::toResponse)
  .getOrElse(::toErrorResponse)

Dealing with errors example

Rich domain model

Domain Driven Design teaches us to express the domain model in the language shared by everyone involved in the project.

If we make the domain types and functions rich in domain vocabulary, there's no translation needed in discussions with the business. In fact, using the same vocabulary we put into the code to talk to the stakeholders is a form of validating the domain model. They’ll catch any discrepancies in the language we use.

Furthermore, if we make illegal states unrepresentable, types become self-documenting. Limited options encoded in a rich model make for an improved ability to reason about it, and therefore maintain or change it.

Summary

The domain model is arguably the most important layer in an application. It should also be where complexity is tackled. After all, that's where business decisions live and that's what makes our application distinctive.

Making the domain model functional helps to reduce the complexity and concentrate it inside the model. Any impure operations will be pushed to the boundaries of the system. What's pushed outside might not be necessarily trivial, but it will be mostly business-decision-free code that deals with side effects.

I/O Sandwich

Hopefully, we'll end up with a code base that's less prone to errors, easier to understand and maintain, and effortless to test.

I find that a functional domain model plays really well with event sourcing. That's what we're going to look at in the next post.

Resources

Collaboration and contract tests

Jakub Zalas — Tue, 16 Aug 2022 14:40:00 +0000

A collaboration test is an example of interactions between the actor that's the focus of the test and its collaborators. For speed, reliability, and isolation, collaborators are replaced with test doubles.

A contract test confirms assumptions made in test doubles.

Writing tests for pure functions or methods with no side effects is straightforward.

Given the input, when the behaviour is invoked, let me verify the expected outcome matches the actual outcome.

No matter how many times we invoke the behaviour, the outcome will be the same as long as the input doesn't change.

Behaviours that depend on external services are more challenging. An example of an external service might be any I/O operation, like reading to or writing from a database or a filesystem, or a network call to a REST API.

The external service will make us split the focus of the test between two separate concerns - business logic and infrastructure (I/O). Eventually, it will be hard to tell one from the other.

The external service will make our test less reliable. Sometimes it will respond slowly. Sometimes it will go down. Sometimes it will fail to respond with the expected outcome.

The external service will make it harder to reproduce some scenarios in a test.

Finally, the external service will make our test slower than it would be without the external call. With tools available today we're talking about tens or hundreds of ms vs a few ms per test, but it adds up rather quickly. Especially if the service is used in many places.

All of this will not only impede my TDD flow but will also make the build server cause me a lot of pain.

My remedy for the problems above is to separate the business logic from the infrastructure concerns.

Collaboration tests

We first need to replace the direct I/O operation(s) in the actor with a collaborator that will take over the I/O operation(s).

The collaborator will be bound by a contract. We will test-drive the interactions between the actor and its collaborators in a collaboration test.

The contract will in most cases be an abstraction like an interface or a function type. Something that can have multiple implementations.

Having the contract in place, we will be able to replace the real implementation in a test with a test double (a fake for example).

Test doubles make our tests predictable since the behaviour is pre-programmed. We can explicitly say in the test what should be the outcome of a method call on the test double (stub). We can also verify which methods on the test double were called (mock).

The pre-programmed behaviour also enables us to reproduce any scenario.

Test doubles make our tests fast since it's all pure code with no I/O involved.

Last but not least, test doubles let us focus our tests on collaboration.

Contract tests

Once we pushed the I/O operation(s) out to the boundaries it's time to write tests to cover interactions with the external service.

Mocking the database or network calls is not something that gives us confidence in integration with an external service. We will need an integration test for that.

A contract test is an integration test that confirms assumptions made in test doubles by exercising the external service.

In the contract test, we will actually call the external service (as much as it is possible). That means we will perform database queries that will hit the database, or we will make HTTP requests that will hit the REST endpoint.

In collaboration tests, we assumed the contract is implemented as expected. In contract tests, we will verify if that is the case.

If a collaboration test relied on a repository to return an entity for a given ID, we will need at least one contract test that verifies the entity is returned from the database.

If a collaboration test relied on the repository to save the entity, there will be at least one contract test to confirm the entity is persisted in the database.

Contract tests will also cover any exceptional and erroneous scenarios. What happens if the connection isn't available? What if the entity isn't found?

Note that the contract isn't limited to the method or function signatures. It also covers rules of the behaviour. For example, given we added something to the repository, we need to be able to take it out.

Final thoughts

If we choose to test-drive the contract from a client's perspective (the collaboration test), it's more likely it will be a rather simple contract. Otherwise, the collaboration test would be a pain to write and maintain.

A simple contract will also make the implementation and its contract tests rather straightforward and less prone to errors.

By pushing side effects and infrastructure concerns to the boundaries of our system we will greatly limit the number of tests that depend on I/O and require the infrastructure. This is critical to having a fast and reliable test suite.

Credits

I've been using this approach for years and I don't recall where I've learned it from. I guess it was a series of lessons over the years. An evolution.

However, I do remember I have learned the vocabulary of collaboration and contract tests from J. B. Rainsberger. Their blog is a great TDD resource. The "Integrated tests are a scam" series is especially worth mentioning in the context of contract and collaboration tests.

On granularity of tests focused on behaviour

Jakub Zalas — Tue, 17 May 2022 20:50:06 +0000

Behaviour

A function is one of the smallest units of behaviour. Given an input, it returns the output.

This means that given the context (input), the function's behaviour can be verified by calling the function and looking at the outcome (output).

Notice that the most straightforward outcome to verify is the function's return value, but it's not the only outcome we can look for. Functions can have side effects, modify state, interact with collaborators, etc.

Note: I'm going to talk about functions as a unit of behaviour for simplicity, but most of it applies to objects as well. After all, objects can be thought of as functions with some context encapsulated.

Composition

Functions can be composed to build more complex behaviours out of the simpler ones.

Composition works all the way up (or down).

It's a simplification, but an application can be thought of as a composition of behaviours. At the top, there's an entry point that the end-user will call. The entry point will in turn call lower levels, that will call even lower levels, that will call [...], all the way to the bottom.

Altitude and granularity

In my tests, I can choose to verify the behaviour on any altitude with any granularity.

By choosing one of the higher and coarser-grained levels, I'm able to verify the behaviour closer to the end user's experience. The cost is less pressure put on the design.

By going with finer-grained behaviours, and writing tests on many levels, I'm able to verify the behaviour of components that make up the system and how they interact. The gain is a better feedback on my design.

The coarser the behaviour I choose to test, the more freedom I get in arranging lower levels. Since it requires more mental effort to keep myself disciplined and take steps small enough to remain in control, it's not necessarily a good kind of freedom. Furthermore, at a certain altitude, it will be harder to verify behaviours buried at lower levels.

The finer the behaviour I choose to test, the more organised I am and the smaller steps I take. Tests will also give me better feedback on my design while being less business-focused, perhaps. I will sometimes need to refactor them or even replace if I decided to change some components or the way they interact.

Slicing

In order to write tests for finer-grained behaviours, I need to replace some dependencies with test doubles (i.e. fakes, stubs, mocks, etc.). I do that, among other reasons, to have a better control over the context in which the behaviour is executed. Introducing test doubles naturally cuts the behaviour to be finer-grained, and this way reduces the scope of the test.

However, I do not need to replace all dependencies. I do not need to replace direct dependencies, either. It's up to me to decide how coarse the unit I'm writing a test for is.

We get to choose when and where we break the dependency chain for testing.

-- Micheal "GeePaw" Hill

Conclusion

So which way do I prefer?

If I had to choose one, I'd go for the tests for finer-grained behaviours.

I tend to write both kinds though, to make sure both external and internal quality is taken care of.

I like to exercise my acceptance tests for coarser-grained behaviours through one of the higher levels. I often use them to create a "walking skeleton". This kind of tests won't execute every possible path. I won't write a lot of them, either. This kind of tests will exercise big chunks of behaviours to confirm and document the business goals. These tests will also make sure everything is correctly wired together. These tests merely supplement the tests that exercise the finer-grained behaviours.

I like to write a lot more tests for finer-grained behaviours. These are smaller, faster to run and write, and more focused. This kind of tests will exercise very small chunks of behaviour. These tests will help me to take smaller steps, keep me disciplined, and give me feedback on the quality of my design.

Why do I practice TDD?

Jakub Zalas — Thu, 05 May 2022 12:38:40 +0000

Why do I practice TDD you ask?

There’s a number of paths I could’ve taken (or combinations of the four).

1) Write no automated tests.
2) Write tests before writing code (in small iterations).
3) Write tests at the same time as code (in small iterations).
4) Write tests after writing code.

Writing no tests

On a project that’s worth practicing DDDesign on, having no tests is not an option. Continuous refactoring towards deeper insights is the foundation of DDD and it’s too scary to refactor with no tests.

As a result, refactoring won’t happen without tests and code will rot. Over time our code and our understanding of the domain will either diverge, or we’ll stop looking for insights to avoid change.

Tests are critical to embracing change. Tests give me the confidence and a peace of mind.

Writing tests first

Note that it's not about writing ALL the tests before writing any code. In fact, it's about writing as little test code as possible to see a failure. Then make it pass. Then improve the design. Then write the next test case. Wash. Rinse. Repeat.

Writing tests first encourages me to take small steps in an iterative and incremental fashion. It’s a very organised way to develop that helps me focus. It keeps me disciplined and productive.

Without this workflow I tend to get distracted, loose the discipline, or start taking steps that are too big.

Big steps mean my code is longer in a “broken” state and it’s harder to get it back to “workable” state. It can go really messy really quick.

Discipline and short cycles encouraged by TDD make me very productive indeed.

Writing tests first leads to a better design. TDD helps me to create modular, loosely coupled and cohesive components. It helps me to come up with simple abstractions and good separation of concerns.

It doesn’t mean there’s no upfront design. Whiteboard is my friend. However, TDD helps me to validate the design I previously thought of and improve on it.

Obviously, as I gain experience, I will intuitively know what good design looks like and very often get it right on the whiteboard. It doesn’t mean TDD stops being useful. On the contrary, the process keeps validating my design when I’m not looking or have a bad day.

Occasionally, especially when working in an unfamiliar territory, I will fall off track. It’s liberating to know TDD will have my back. It’s like double-entry bookkeeping (to quote Uncle Bob).

Writing tests later

Writing tests after the code would give me some of TDD benefits. Feedback on testability and quality is still there, even if it’s late. If code I wrote turns out to be hard to test I need to go back and revisit the design. That’s counter-productive.

Writing tests after the code tends to produce some amount of unnecesary code.

Writing tests and code together?

As for writing the code and the test at the same time, it’s kind of what Test && Commit || Revert (TCR) is about. It’s a technique that’s suppose to encourage us to take even smaller steps, as there’s always the risk of loosing the code and the test we’ve just written.

I’m not really convinced by TCR enough to incorporate it into my regular workflow. I’m afraid it would bring the stress back to my development process. However, it’s a technique that might teach me something I’m sure.

Why do you practice TDD? Why don't you?

This article was originally posted on Twitter and Linkedin.

How to publish Maven packages to a single GitHub repository

Jakub Zalas — Mon, 08 Feb 2021 14:50:06 +0000

A working example of this setup can be found in vlingo XOOM repositories: https://github.com/vlingo/xoom-platform/packages

Publishing to the current repository

Before we look at publishing to an external repository, let's review how straight forward it is to publish to the current one:

name: Build

on:
  release:
    types: [created]

jobs:
  publish:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      - name: Set up JDK 1.8
        uses: actions/setup-java@v1
        with:
          java-version: 1.8

      - name: Deploy to GitHub
        run: mvn --batch-mode -DuseGitHubPackages=true deploy
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

GitHub packages repository will be automatically configured for us in Maven by the actions/setup-java action. All's left to do is to pass the GITHUB_TOKEN secret as an environment variable to the mvn deploy command.

Note: GITHUB_TOKEN secret is available on all GitHub action builds out of the box. The token's scope is limited to the current repository.

In the above example we also enabled the useGitHubPackages flag in order to load the Maven profile that configures distribution mangement:

<!-- pom.xml -->
<project ...>
  <!-- ... -->

  <profiles>
    <profile>
      <id>github</id>
      <activation>
        <property>
          <name>useGitHubPackages</name>
          <value>true</value>
        </property>
      </activation>
      <distributionManagement>
        <repository>
          <id>github</id>
          <name>GitHub Packages</name>
          <url>https://maven.pkg.github.com/vlingo/xoom-common</url>
        </repository>
      </distributionManagement>
    </profile>
  </profiles>
</project>

The above example works for publishing vlingo/xoom-common from within the same repository.

Publishing to a shared repository

The setup from previous section works great if we only have a single repository to publish from. Once we start publishing from multiple GitHub repositories, we'll end up with one Maven repository per GitHub repository.

This is not always desired, especially if we develop a set of libraries that are usually installed together. In such a scenario, our clients would be forced to include in their pom.xml one Maven repository per library we provide.

The solution is to choose a dedicated repository for publishing our Maven packages and use it in distribution management for all the other repositories.

Maven configuration

For example, having libraries like xoom-common, xoom-actors, xoom-http, we could choose to publish all of them to a single repository xoom-platform:

<!-- pom.xml -->
<project ...>
  <!-- ... -->

  <profiles>
    <profile>
      <id>github</id>
      <activation>
        <property>
          <name>useGitHubPackages</name>
          <value>true</value>
        </property>
      </activation>
      <distributionManagement>
        <repository>
          <id>github</id>
          <name>GitHub Packages</name>
          <url>https://maven.pkg.github.com/vlingo/xoom-platform</url>
        </repository>
      </distributionManagement>
    </profile>
  </profiles>
</project>

GitHub action

The GitHub action will remain very similar to the one we looked at in the beggining with one notable difference - the GITHUB_TOKEN:

name: Build

on:
  release:
    types: [created]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      - name: Set up JDK 1.8
        uses: actions/setup-java@v1
        with:
          java-version: 1.8

      - name: Deploy to GitHub
        run: mvn --batch-mode -DuseGitHubPackages=true deploy
        env:
          GITHUB_TOKEN: ${{ secrets.DEPLOY_GITHUB_TOKEN }}

We can no longer use the token provided by default as its scope is limited to the current repository.

Instead, we'll need to create a Personal Access Token and pass it to the action as a secret (DEPLOY_GITHUB_TOKEN in the example above).

Creating a GitHub Personal Access Token

To make it work across repositories we'll need to define a new Personal Access Token in:

Settings > Developer Settings > Personal Access Tokens.

Select the write:packages scope and all the repo scopes should be automatically selected for us.

Defining a secret

Next, the token needs to be defined as a secret in our organisation or each of the repositories we need to publish packages from.

Give it a name (i.e. DEPLOY_GITHUB_TOKEN) and set its value to the Personal Access Token created in the previous step.

Repository secrets are defined in repository Settings > Secrets. There's a similar section for the organisation secrets.

How to generate an aggregated code coverage report for all Gradle subprojects

Jakub Zalas — Fri, 10 Jul 2020 11:20:12 +0000

When using Gradle's multi-project builds, test and coverage reports are generated separately in each build directory of every sub-project.

To make Azure Pipelines happy I needed to generate an index file for all coverage-reports:

// build.gradle.kts

plugins {
    base
    kotlin("jvm") version "1.3.72" apply false
}

allprojects {
    group = "com.kaffeinelabs"
    version = "1.0.0-SNAPSHOT"
    apply(plugin = "jacoco")
}
tasks.register<JacocoReport>("jacocoRootReport") {
    subprojects {
        this@subprojects.plugins.withType<JacocoPlugin>().configureEach {
            this@subprojects.tasks.matching {
                it.extensions.findByType<JacocoTaskExtension>() != null }
           .configureEach {
                sourceSets(this@subprojects.the<SourceSetContainer>().named("main").get())
                executionData(this)
            }
        }
    }

    reports {
        xml.isEnabled = true
        html.isEnabled = true
    }
}

Jacoco configuration for sub-projects should be done as per usual.

To generate the report the jacocoRootReport task needs to be added to the build:

./gradlew test jacocoRootReport

If you also need to aggregate test reports, I covered it in another article:

How to generate an aggregated test report for all Gradle subprojects

How to generate an aggregated test report for all Gradle subprojects

Jakub Zalas — Wed, 08 Jul 2020 06:53:29 +0000

Note: This article is outdated. The Test Report Aggregation Plugin can be used to achieve the same result with less effort.

With the plugin it all boils down to the example below.

With gradle:

plugins {
    id 'test-report-aggregation'
}

tasks.named('check') {
    dependsOn tasks.named('testAggregateTestReport', TestReport)
}

With Kotlin:

plugins {
    id("test-report-aggregation")
}

tasks.check {
    dependsOn(tasks.named<TestReport>("testAggregateTestReport")) 
}

Read more about the plugin in the official documentation: https://docs.gradle.org/current/userguide/test_report_aggregation_plugin.html

When using Gradle's multi-project builds, test reports are generated separately in each build directory of every sub-project.

To make my CI server's life easier I often like to aggregate reports in a single build directory:

// build.gradle.kts

plugins {
    base
    kotlin("jvm") version "1.3.72" apply false
}

allprojects {
    group = "com.kaffeinelabs"
    version = "1.0.0-SNAPSHOT"
}

val testReport = tasks.register<TestReport>("testReport") {
    destinationDir = file("$buildDir/reports/tests/test")
    reportOn(subprojects.mapNotNull {
        it.tasks.findByPath("test") 
    })

subprojects {
    tasks.withType<Test> {
        useJUnitPlatform()
        finalizedBy(testReport)
        testLogging {
            events("passed", "skipped", "failed")
        }
    }
}

If you're interested in how to aggregate code coverage reports, I covered it in another article:

How to generate an aggregated code coverage report for all Gradle subprojects

How to run Spring Boot with a dev profile and Gradle

Jakub Zalas — Tue, 16 Jun 2020 18:53:34 +0000

I like to use the dev profile in my Spring Boot projects to store the application configuration for development. Spring Boot will conviniently load the configuration dedicated to the profile we've selected when starting the app (i.e. application-dev.yaml).

To use the dev profile, all I need to do with maven is to pass -Dspring.profiles.active=dev property to my spring-boot:run target.

Gradle's bootRun task supports passing --args='--spring.profiles.active=dev' with the Spring Boot Gradle plugin.

There's an alternative too. All we need is a task that would set the property, before spring boot app starts.

Here's an example from build.gradle.kts:

tasks.register("bootRunDev") {
    group = "application"
    description = "Runs the Spring Boot application with the dev profile"
    doFirst {
        tasks.bootRun.configure {
            systemProperty("spring.profiles.active", "dev")
        }
    }
    finalizedBy("bootRun")
}

Usage:

$ ./gradlew bootRunDev

> Task :bootRun
  ,---.    ,-----. ,--.   ,--. ,------. 
 /  O  \  '  .--./ |   `.'   | |  .---' 
|  .-.  | |  |     |  |'.'|  | |  `--,  
|  | |  | '  '--'\ |  |   |  | |  `---. 
`--' `--'  `-----' `--'   `--' `------'
:: Spring Boot ::  (v2.2.5.RELEASE)

2020-06-11 09:52:53.382  INFO 25367 --- [  restartedMain] c.k.demo.ApplicationKt       : Starting ApplicationKt on MacBook-Pro.local with PID 25367 (/demo/build/classes/kotlin/main started by kuba in /demo)
2020-06-11 09:52:53.385  INFO 25367 --- [  restartedMain] c.h.demo.ApplicationKt       : The following profiles are active: dev