DEV Community: Marco Sabatini

Event storming, and then what?

Marco Sabatini — Thu, 11 Jul 2024 12:28:29 +0000

References

Overview

Some time ago, I read a fascinating book titled Introducing EventStorming: An Act of Deliberate Collective Learning by Alberto Brandolini.
This book, filled with concrete examples, discusses the event-storming technique for modeling the business processes underlying a digital product to be implemented through software.

What truly captivated me and sparked my curiosity about this methodology is its ability to model software architecture and reveal organizational limitations within a company. In this article, I will demonstrate with a concrete example how event-storming can effectively bridge the gap between business and software development.

Mars Rover Kata

The exercise I used as a reference is the mars rover Kata. I used Python to solve it.

Its requirements involve implementing software for a rover to receive commands for movement. The rover can move forward, rotate left and right, and has constraints related to the surface it can traverse and potential obstacles it might encounter.

Before starting to implement the software, I conducted an Event Storming session. Of course, this came with significant limitations: I was the only participant and not particularly experienced with the technique and this session was just to implement the software (there are several levels of event-storming abstraction we should consider).
I used basic elements to model a business process, including commands, events, aggregates, policies, and projections. The definitions of these components provided in the book are particularly enlightening:

Command: A decision made by the user.
Aggregate: Information necessary for making decisions.
Event: A state transition mapped somewhere.
Projections: Tools to support the decision-making process in the user's brain.
Policy: Triggers that respond whenever something happens

Regarding the flow, we can see how these components interact with each other in the picture below:

Everything will start with a command that triggers actions on a specific aggregate. This will generate an event and by listening to the event we can continue to trigger commands using Policies or create a view using Projections.

Using these building blocks, I attempted to model the Mars Rover design Event Storming.

Event Storming

Below, we can see the first iteration of the process.

First, I thought the rover needed to be powered on and set with a starting point and direction. The aggregate that came to mind here is the Mars Rover itself.
Once somebody starts it, it will be in "Started" mode and ready to move. Next, the rover can receive commands to turn right, turn left, and move forward. Depending on the presence of obstacles, the rover can either continue moving or encounter obstacles.
According to the exercise, in case of an obstacle, the rover should "shut down." Thus, I used a policy to react to the "ObstacleFound" event with a command instructing the rover to shut down.

From the first iterations, I noticed how intuitive it was to think in terms of Commands, Events, Aggregates, and Policies. I also used Projections to create datasets for future analysis, which I will discuss during the implementation phase.

From a modeling perspective, this technique is extremely useful. I could present the process to any product owner or domain expert or even implement it directly with them (as described in the book).
I am confident that in a very short time, we could create a dictionary of common terms understandable to all stakeholders involved (both business and software). Now, let's move on to the implementation phase.

Implementation

You can find the solution code here.

Upon opening the project, you'll find a package named ddd. In this package, I have included the basic elements described earlier:

class Command:
    pass

class Event:
    pass

@dataclasses.dataclass(frozen=True)
class AggregateId:
    value: str

    @classmethod
    def new(cls):
        return cls(value=str(uuid4()))

@dataclasses.dataclass
class Aggregate:
    id: AggregateId
    version: int

class CommandHandler:
    def handle(self, command: Command) -> Event:
        pass

class Policy:
    def apply(self, event) -> Command:
        pass

class Projection:
    def project(self, event):
        pass

I reused the same names to create a common base linking the modeling and implementation parts. I also implemented an in-memory repository responsible for loading and saving an Aggregate object and a very simple in-memory command dispatcher.

The command dispatcher receives a series of commands as input and applies command handlers, policies, and projections based on its construction.
For this exercise, the implementation is in-memory, but you could consider implementing it with remote queues.

Here is our command dispatcher:

class InMemoryCommandDispatcher:
    def __init__(self,
                 command_handlers: Dict[Type[Command], CommandHandler],
                 projections: Dict[Type[Event], List[Projection]],
                 policies: Dict[Type[Event], List[Policy]]):
        self.command_handlers = command_handlers
        self.projections = projections
        self.policies = policies

        self.commands: List[Command] = []

    def submit(self, commands: List[Command]):
        for c in commands:
            self.commands.append(c)

    def run(self):
        while self.commands:
            command = self.commands.pop(0)
            print(f"[COMMAND] {command}")

            event: Event = self.command_handlers[type(command)].handle(command)
            print(f"[EVENT] {event} generated")

            event_policies = self.policies.get(type(event), [])
            for policy in event_policies:
                new_command = policy.apply(event)
                if new_command:
                    self.commands.append(new_command)

            for projection in self.projections.get(type(event), []):
                projection.project(event)

At this point, I have all the necessary components to build the model defined during the Event Storming session. Therefore, I implemented the commands, events, aggregate, and command handlers, as shown in the function defined to build the process.


def create_command_dispatcher(mars_rover_repo: MarsRoverRepository,
                              mars_rover_storage: List[MarsRoverId],
                              path_projection_storage: List[Dict],
                              obstacles_projection_storage: List[Dict]) -> InMemoryCommandDispatcher:
    turn_right_command_handler = TurnRightCommandHandler(repo=mars_rover_repo)
    turn_left_command_handler = TurnLeftCommandHandler(repo=mars_rover_repo)
    move_command_handler = MoveCommandHandler(repo=mars_rover_repo)
    start_command_handler = StartMarsRoverCommandHandler(repo=mars_rover_repo)
    turn_off_command_handler = TurnOffCommandHandler(repo=mars_rover_repo)
    notify_obstacle_command_handler = NotifyObstacleCommandHandler()

    rover_path_projection = MarsRoverPathProjection(repo=mars_rover_repo, storage=path_projection_storage)
    rover_start_projection = MarsRoverStartProjection(repo=mars_rover_repo,
                                                      paths_storage=path_projection_storage,
                                                      mars_rover_storage=mars_rover_storage)
    rover_obstacles_projection = MarsRoverObstaclesProjection(storage=obstacles_projection_storage)

    obstacle_found_policy = NotifyObstacleFoundPolicy()
    turn_off_policy = TurnOffPolicy()

    return (InMemoryCommandDispatcherBuilder()
            .with_command_handler(TurnRight, turn_right_command_handler)
            .with_command_handler(TurnLeft, turn_left_command_handler)
            .with_command_handler(Move, move_command_handler)
            .with_command_handler(StartMarsRover, start_command_handler)
            .with_command_handler(TurnOff, turn_off_command_handler)
            .with_command_handler(NotifyObstacle, notify_obstacle_command_handler)
            .with_projection(MarsRoverStarted, rover_start_projection)
            .with_projection(MarsRoverMoved, rover_path_projection)
            .with_projection(ObstacleFound, rover_obstacles_projection)
            .with_policy(ObstacleFound, obstacle_found_policy)
            .with_policy(ObstacleFound, turn_off_policy)
            .build())

As you can see, this function creates the dispatcher by configuring the process modeled during the Event Storming session. Specifically, it associates commands with their respective command handlers, as well as policies and projections. It is straightforward to understand the actions associated with commands and events.

Command Handlers

Let's take a look at how I manage a command using a command handler for controlling the Rover's movement.

class MoveCommandHandler(CommandHandler):
    def __init__(self, repo: MarsRoverRepository):
        self.repo = repo

    def handle(self, command: Move) -> MarsRoverMoved:
        mars_rover: MarsRover = self.repo.get_by_id(command.id)
        event = mars_rover.move()
        self.repo.save(mars_rover)
        return event

The class uses the repository to load the MarsRover Aggregate into memory and then calls the "move()" function to change its state.
Following this, the event is emitted and the aggregate's state is saved

Domain

Regarding the domain part, the figure below shows the objects used to model it.

Let me show also the MarsRover Aggregate methods contract:

@dataclasses.dataclass
class MarsRover(Aggregate):
    actual_point: Point
    direction: Direction
    world: World
    status: MarsRoverStatus

    def start(self) -> MarsRoverStarted:
        ...

    def turn_off(self) -> MarsRoverTurnedOff:
        ...

    def turn_right(self) -> MarsRoverMoved | MarsRoverTurnedOff:
        ...

    def turn_left(self) -> MarsRoverMoved| MarsRoverTurnedOff:
        ...

    def move(self) -> MarsRoverMoved | ObstacleHit |  MarsRoverTurnedOff:
        ...

It contains all the possible actions the Rover can do and the logic to change its internal state, starting from the actual point and direction to the grid/world in which the rover is moving.

Services

To orchestrate everything, I implemented a service.
Below you can see the service being used in the end-to-end tests developed:

class TestE2E(unittest.TestCase):
    def test_execute_some_commands(self):
        repo = MarsRoverRepository()
        paths = []
        obstacles = []
        mars_rover_ids = []

        runner = (
            MarsRoverRunner(repository=repo,
                            path_projection_storage=paths,
                            obstacles_projection_storage=obstacles,
                            mars_rover_projection_storage=mars_rover_ids)
            .with_initial_point(x=0, y=0)
            .with_initial_direction(direction=Direction.NORTH)
            .with_world(world_dimension=(4, 4),
                        obstacles=[])
        )
        runner.start()

        id = mars_rover_ids[0]

        runner.run(id, "RMLMM")

        actual: MarsRover = repo.get_by_id(MarsRoverId(id))
        self.assertEqual("1:2:N", actual.coordinate())
        self.assertEqual("MOVING", actual.status.value)

        expected_path = ["0:0:N", "0:0:E", "1:0:E", "1:0:N", "1:1:N", "1:2:N"]
        self._assert_paths(expected=expected_path, actual=paths)

        self.assertListEqual([], obstacles)

In this test, I set the starting point, the grid on which the Rover moves, and its initial direction.
After that, the Rover is started calling runner.start() and receives commands in string format calling runner.run(id, "RMLMM"). I need to pass the rover id just to retrieve it using the repository. I can retrieve the RoverId from a projection created when we started it before.

Regarding assertions, I used the datasets generated by the paths projections, which are also developed with an in-memory version to facilitate interactions.

Below there are the logs of Commands and Events generated during the flow:

_e2e.py::TestE2E::test_execute_some_commands 
[COMMAND] StartMarsRover(initial_point=Point(x=0, y=0), initial_direction=<Direction.NORTH: 'N'>, world=World(dimension=(4, 4), obstacles=Obstacles(points=[])))
[EVENT] MarsRoverStarted(id=MarsRoverId(value='aea5e69c-4f08-40db-bf44-097b5ae36380'))
[COMMAND] TurnRight(id=MarsRoverId(value='aea5e69c-4f08-40db-bf44-097b5ae36380'))
[EVENT] MarsRoverMoved(id=MarsRoverId(value='aea5e69c-4f08-40db-bf44-097b5ae36380'))
[COMMAND] Move(id=MarsRoverId(value='aea5e69c-4f08-40db-bf44-097b5ae36380'))
[EVENT] MarsRoverMoved(id=MarsRoverId(value='aea5e69c-4f08-40db-bf44-097b5ae36380'))
[COMMAND] TurnLeft(id=MarsRoverId(value='aea5e69c-4f08-40db-bf44-097b5ae36380'))
[EVENT] MarsRoverMoved(id=MarsRoverId(value='aea5e69c-4f08-40db-bf44-097b5ae36380'))
[COMMAND] Move(id=MarsRoverId(value='aea5e69c-4f08-40db-bf44-097b5ae36380'))
[EVENT] MarsRoverMoved(id=MarsRoverId(value='aea5e69c-4f08-40db-bf44-097b5ae36380'))
[COMMAND] Move(id=MarsRoverId(value='aea5e69c-4f08-40db-bf44-097b5ae36380'))
[EVENT] MarsRoverMoved(id=MarsRoverId(value='aea5e69c-4f08-40db-bf44-097b5ae36380'))

Basically the rover is getting as input this commands "RMLMM" and they are translated during the flow execution.

Path Projection

I used a paths projection in order to store the coordinates every time the rover is moved.
As we can see in the before session we are able to assert the entire Rover path:

expected_path = ["0:0:N", "0:0:E", "1:0:E", "1:0:N", "1:1:N", "1:2:N"]
self._assert_paths(expected=expected_path, actual=paths)

Therefore, for this exercise I implemented an in memory projection that is able to store path data in a list of dictionary:

class MarsRoverPathProjection(Projection):
    def __init__(self, repo: MarsRoverRepository, storage: List[Dict]):
        self.repo = repo
        self.storage = storage

    def project(self, event: MarsRoverMoved):
        mars_rover: MarsRover = self.repo.get_by_id(event.id)

        raw = {"id": mars_rover.id.value, "actual_point": mars_rover.coordinate()}

        self.storage.append(raw)

This class takes as input the event MarsRoverMoved because is triggered on this, loads the aggregate and is able to create a specific read model.
In this case the read model contains the paths traveled by the rover. The good thing of this design is that the projection is totally decoupled by the aggregate state change and could be easy bind on a specific event that represents the Aggregate state change.

Policy

At this point, when the Rover encounters an obstacle, it must automatically shut down.
To achieve this, I implemented a shutdown Policy. This Policy takes the ObstacleFound event as input and generates a shutdown event, which is then handled by its own command handler, as we saw earlier.
Below you can see the TurnOffPolicy implementation:

class TurnOffPolicy(Policy):
    def apply(self, event: ObstacleFound) -> Command:
        return TurnOff(id=event.id)

Additional requirement

I tried to push the design further by considering what would happen if I had an additional requirement, such as sending a notification when the Rover encounters an obstacle.
First, I reviewed the initial design event storming model and I integrated the notification feature, resulting in this new version:

As you can see from the diagram, all I had to do was define a new Policy based on the ObstacleFound event, which was responsible for creating the command to notify the obstacle found.

class NotifyObstacleFoundPolicy(Policy):
    def apply(self, event: ObstacleFound) -> Command:
        return NotifyObstacle(message=f"Rover {event.id.value} hit obstacle")

After the creation of the command we will have the command handler that will manage the notification flow.
Regarding the Policy this is initialized within the builder of the command dispatcher, which effectively configures the process.
Here, I simply added the policy without needing to change or refactor existing code.
The test below is just to show how we can make an end to end test setting up obstacles:

 def test_hit_obstacle(self):
        repo = MarsRoverRepository()
        paths = []
        obstacles = []
        mars_rover_ids = []

        runner = (
            MarsRoverRunner(repository=repo,
                            path_projection_storage=paths,
                            obstacles_projection_storage=obstacles,
                            mars_rover_projection_storage=mars_rover_ids)
            .with_initial_point(x=0, y=0)
            .with_initial_direction(direction=Direction.NORTH)
            .with_world(world_dimension=(4, 4),
                        obstacles=[(2, 2)])
        )

        runner.start()

        id = mars_rover_ids[0]

        runner.run(id, "RMMLMMMMMM")

        obstacles_found = [o["obstacle"] for o in obstacles]
        self.assertEqual([(2, 2)], obstacles_found)

It's easy setup obstacles and assert they are found checking the in memory obstacle projection storage. I created an obstacle projection in order to collect where is the obstacle found by the rover during the travel. It was easy because I had ObstacleFound event and I just needed to listen for this event and project it.

Below there are the logs of the Commands and Events generated during the test. As you can see during the travel the rover found an obstacle. After that it was turned off and the obstacle was notified.

test/test_e2e.py::TestE2E::test_hit_obstacle 
[COMMAND] StartMarsRover(initial_point=Point(x=0, y=0), initial_direction=<Direction.NORTH: 'N'>, world=World(dimension=(4, 4), obstacles=Obstacles(points=[Point(x=2, y=2)])))
[EVENT] MarsRoverStarted(id=MarsRoverId(value='649d914f-967c-41ae-b3ca-ffede9ca9e7d'))
[COMMAND] TurnRight(id=MarsRoverId(value='649d914f-967c-41ae-b3ca-ffede9ca9e7d'))
[EVENT] MarsRoverMoved(id=MarsRoverId(value='649d914f-967c-41ae-b3ca-ffede9ca9e7d'))
[COMMAND] Move(id=MarsRoverId(value='649d914f-967c-41ae-b3ca-ffede9ca9e7d'))
[EVENT] MarsRoverMoved(id=MarsRoverId(value='649d914f-967c-41ae-b3ca-ffede9ca9e7d'))
[COMMAND] Move(id=MarsRoverId(value='649d914f-967c-41ae-b3ca-ffede9ca9e7d'))
[EVENT] MarsRoverMoved(id=MarsRoverId(value='649d914f-967c-41ae-b3ca-ffede9ca9e7d'))
[COMMAND] TurnLeft(id=MarsRoverId(value='649d914f-967c-41ae-b3ca-ffede9ca9e7d'))
[EVENT] MarsRoverMoved(id=MarsRoverId(value='649d914f-967c-41ae-b3ca-ffede9ca9e7d'))
[COMMAND] Move(id=MarsRoverId(value='649d914f-967c-41ae-b3ca-ffede9ca9e7d'))
[EVENT] MarsRoverMoved(id=MarsRoverId(value='649d914f-967c-41ae-b3ca-ffede9ca9e7d'))
[COMMAND] Move(id=MarsRoverId(value='649d914f-967c-41ae-b3ca-ffede9ca9e7d'))
[EVENT] ObstacleFound(id=MarsRoverId(value='649d914f-967c-41ae-b3ca-ffede9ca9e7d'), coordinate=(2, 2))
[COMMAND] NotifyObstacle(message='Rover 649d914f-967c-41ae-b3ca-ffede9ca9e7d hit obstacle')
Rover 649d914f-967c-41ae-b3ca-ffede9ca9e7d hit obstacle
[EVENT] ObstacleNotified()
[COMMAND] TurnOff(id=MarsRoverId(value='649d914f-967c-41ae-b3ca-ffede9ca9e7d'))
[EVENT] MarsRoverTurnedOff(id=MarsRoverId(value='649d914f-967c-41ae-b3ca-ffede9ca9e7d'))

Evolutionary Architecture

Regarding the notification, I did not proceed further in the exercise. This is a typical example of what happens every day in our work: we create something that may have follow-up actions if it proves valuable.

Interestingly, the model already decouples this new concept (the notification), which we could potentially develop into a dedicated product line, with its own aggregate. This could become an external system that reads the ObstacleFound event from a queue and generates the notifications.
If we need to scale up because the notification system requires product enhancements, we could create a dedicated team to handle this domain.

This example is just to illustrate how such a design approach not only helps evolve our architecture but also guides it to support product development and organizational changes.

Some time ago, we had Nicola Moretto speak at our meetup about product development consistency.

He showed us this slide, which I find very significant. He explained that Architecture, Business, and Organization should go hand in hand with product needs and must be easily modifiable to manage product increments.

How often do we encounter these situations?

Implement features on legacy architectures.
Implement features but can't do it end-to-end because we depend on other teams.
Implement features whose business value is uncertain.

To address these types of situations, the architect must act as the point of contact between the business and the organization needed to support it, by implementing an architecture that is most easily adaptable to the problem at hand.

My First Year as a Senior Software Architect in a Startup: Embracing the Journey

Marco Sabatini — Wed, 06 Sep 2023 13:35:51 +0000

Let's Start
Why Choose a Startup?
Mindset
Individual Contributor
Uncertainty
Engineering Practice
Predictable Delivery & Rapid prototyping
The end or the beginning?
References

Let's Start

Stepping into the world of startups as a senior software architect after years of consultancy and big product company experiences was a deliberate choice I made to broaden my horizons and fuel my passion for innovation.
In this article, I will delve into the reasons behind my decision to join a startup, emphasizing the importance of a growth mindset and some engineering practices in the dynamic startup environment.

Why Choose a Startup?

After immersing myself in the world of consultancy and big companies, I wanted a new challenge that would enable me to have a direct impact on a company's development. Startups offer an environment where groundbreaking ideas are supported, and where the ability to innovate and experiment is highly valued. By joining a startup, I sought to tap into a culture that encourages agility, creative problem-solving, and rapid iteration while being part of a tight-knit team working towards a shared vision.

Mindset?

The Right Mindset? Growth!
Thriving in the startup ecosystem requires cultivating a growth mindset—a mindset that embraces continuous learning, adaptability, and resilience. We operate in a dynamic and ever-evolving landscape, where Change is constant and Challenges are abundant. As a senior software architect, I quickly realized that embracing a growth mindset was essential for staying ahead of the curve. This mindset fueled my curiosity, motivated me to explore and learn emerging technologies and architectural patterns, and enabled me to adapt swiftly to new requirements and business goals.

DeNexus Inc is a data company that wants to solve very complex problems leveraging data (tons of them).
I had no prior knowledge of DeNexus's domain, which is: quantified operational technology (OT) cyber risk. I have previous experience with OT (Operational Technology) environments, specifically in the context of discovering and managing facility assets and inventory. However, the terminology and domain concepts I encountered in my previous company were somewhat distinct.
Therefore, I had to educate myself by reading various books (see references) and seeking guidance from my more experienced colleagues. Luckily, the team consists of individuals with twenty years of OT cybersecurity and risk computation expertise.
They provided invaluable assistance in helping me understand the company's purpose and core business. For instance one of my first challenges was to revisit my studies in statistics from university in order to understand better the cyber risk quantification metrics to calculate and communicate to customers. This may appear complex, but for someone who is curious and eager to learn new things, it can be an enjoyable endeavor with a worthwhile return on investment.
As a result, after initially acquiring new knowledge, I am now able to contribute by suggesting improvements and overseeing development initiatives from an e2e perspective. This helps the company because you can scale and minimize integration and coordination between various teams.

I also quickly faced problems managing big data; starting from ingesting, cleaning, and anonymizing. The relentless pursuit of growth as an individual contributor translated into value for the company, as it drove innovation and ensured the scalability and sustainability of our software solutions.

Individual Contributor

The initial team members in a startup play a pivotal role in driving its success. One of the key points is taking ownership of decisions (in my case technical and product), collaborating closely with cross-functional teams, and actively seeking opportunities to contribute beyond the role, trying to understand the business and also the domain in which the company acts.
For this reason, is important to wear multiple hats. Bridge gaps, align technical solutions with business objectives, and provide guidance to the engineering team. So if you don't want to put your hands in the jam is better to stay away :)

In a startup, the number of people is not very high, so you have to try to carry out initiatives until delivery. The scenario in which DeNexus operates is complex, as mentioned before, so we need people with different skills (engineers, cyber security, data analysts, data scientists, and compliance experts).
The engineering aspect plays a crucial role because we have to put the puzzle together and deliver something valuable to our clients. Therefore, we can't just be coders; we also need to understand when and where to intervene to find trade-offs with other team members. For example, I have had some discussions with our team of mathematical data scientists regarding finding trade-offs on the number of iterations required for our models to be executed . A mathematician will provide an optimality that can be proven mathematically, but this could mean billions of iterations, which may not be computationally feasible from an engineering perspective. So, I started by better understanding the problem and then, together with the entire team, finding a solution that would allow us to deliver value within acceptable timeframes for our customers.

Uncertainty

In the fast-paced and ever-evolving world of startups, uncertainty is a constant companion. The ability to navigate this uncertainty effectively is crucial for success.
One approach that proves invaluable in such an environment is adopting an incremental approach to problem-solving. This method emphasizes breaking down complex challenges into smaller, manageable tasks and continuously iterating and adapting based on feedback and new information.

Startups often operate in uncharted territories, disrupting industries or introducing innovative products and services. With limited resources and incomplete information, uncertainties arise at every step.

In this kind of environment, speed is of the essence. An incremental approach encourages a rapid feedback loop, enabling continuous learning and adaptation. By taking small, incremental steps, I saw we could gather feedback early and often from early adopter customers, the customer success team, and stakeholders. This feedback becomes the foundation for refining and enhancing our products, services, and strategies. Each iteration allowed us to gather new data, validate hypotheses, and adjust our course based on real-world insights.

Engineering practice

In the volatile and uncertain world of a digital startup, engineering practices play a crucial role in ensuring stability, adaptability, and scalability.

We immediately started with the practice of Continuous integration, test-driven development (TDD), testing, and a DevOps approach. These helped us not only to tackle uncertainty but also contribute to managing technical debt and facilitating growth.

Continuous Integration & Testing:

In a startup environment, where speed is one of the key points, continuous integration (CI) helps identify potential problems early, minimizing risks associated with merging conflicting code and reducing the time spent on debugging. By ensuring that code changes are continuously integrated and tested, we can maintain a stable and deployable codebase, even in the face of uncertainty. This is also very useful if you have to prototype something to quickly put in a productive environment.

Previously in DeNexus, we had a release process with some manual steps, which caused us some issues because it's normal to occasionally miss a manual step. But since we started writing, automating our tests, and automatically deploying our software, what I have noticed is that people are much more confident in making changes, trying new things quickly, and adding new features.

My two cents: test everything is like taking out insurance, it helps you when you will need it.

Test-Driven Development (TDD):

I noticed that in the startup environment, TDD can become a valuable asset by enabling faster feedback and accelerating the development process. By writing tests up front, you can validate assumptions, identify potential flaws or gaps in logic, and iteratively refine the design of your code that is fundamental to create something easily changeable.

I also noticed that it has also the side effect to help a lot in documenting the software (you have the test and you can run them) and let people work together.

This is another key point. DeNexus is a fully remote company so we have to coordinate our work very well (USA, Switzerland, Spain, London). So, imagine a complex problem to solve with all these locations and multidisciplinary teams :)

DevOps Approach:

DevOps, in the term of emphasize the collaboration and integration of development and operations teams.
By automating infrastructure provisioning, configuration management, and deployment pipelines, the team can rapidly iterate and adapt their software. It also helps to rapidly experiment with new services or entire solutions that could be integrated in order to help develop the product.

Internally in DeNexus, we have organized ourselves into chapters. We have our architectural chapter, which also includes the SRE (Site Reliability Engineering) team members (everyone can participate in it). Our goal is to establish collaboration with SRE to enable product engineering teams to be self-sufficient in infrastructure provisioning. That's why we use infrastructure as code (specifically with Terraform), which is ultimately managed by the product engineering teams. This helps us to easily create environments for experimenting or making QA and performance tests.

For instance, I had the opportunity to set up AWS services, such as SNS (Simple Notification Service) and SQS (Simple Queue Service), with the assistance of SRE. I also automated testing in our CI (Continuous Integration) process. Every time the code is pushed into the source code repository (git) the testing pipeline automatically create infrastructure (executing terraform scripts) execute test and cleanup the environment at the end.

I also have the independence to experiment within the DEV environment or troubleshoot by reproducing any encountered errors.

Non-Overengineering:

Startups often face resource constraints, and an excessive focus on over-engineering can hinder their ability to scale. By adopting a non-over-engineering approach (make it simple not easy), you can strike a balance between building robust systems and maintaining flexibility for growth.
Non-over-engineering involves making conscious decisions about the level of complexity and sophistication of the solutions implemented, considering both the current and anticipated future needs.
This approach helps manage technical debt by preventing the accumulation of unnecessary complexity and maintaining a manageable codebase. It allows it to remain agile and adaptable, enabling the company to scale efficiently and respond to changing market dynamics without being burdened by excessive technical debt.

In DeNexus we are trying to apply the KISS principle (Keep it simple, stupid). This means that in any discussion about product increments or technical development decisions, we should always ask ourselves: do we need this right now?
We need to eliminate all unnecessary things that we don't use.
How many times have we discussed adding a column to a database that isn't used, or how much unused code do we still keep in the codebase just in case we might need it?
To avoid over-engineering, discipline is key, and we need to pay careful attention because we could fall into the opposite trap of not doing anything because nothing is needed. So, we need to be mindful of the trade-off to follow.

Remember to clean your kitchen every time you have a party :)

This helps to have a clean and clear environment to evolve. Making software is a learning process and sometimes we need to quickly make experiments or try new technologies in order to perform some product increments. This could mess up our code base, so we need to cleanup every time we have this kind of iterations.

Having a clean and clear environment to manage allow you to make changes easily, for instance
we are reevaluating our architecture to have a more responsive environment where we can capture changes in the state of our data. For this reason, we are transitioning to an event-based architecture. We are implementing this change in parallel with the development of product features, without the need to block either one.

Predictable Delivery & Rapid prototyping

Engineering practices play a pivotal role in achieving predictable delivery and enabling rapid prototyping. These practices provide startups with the necessary framework to build and iterate on their products efficiently, ultimately increasing their chances of success.

Continuous Integration and Testing help to avoid regression or too many bugs to manage in the near future. Test Driven Development (TDD), DevOps approach, and Non-over-engineering allow you to create a highly changeable codebase, this means having something that you can safely change continuously in order to create a prototype to try with your early-stage customer. Most of the time in the startup environment the team starts delivering a lot at the beginning and later on once it has accumulated a lot of technical debt it slows down rapidly. The key point here is to understand that some practices help the growth of the company itself and not the opposite. If you listen to something like: we have to go faster so avoid to do some of these practices is not because these practices slow down the development but because the people in charge of the development are not skilled to apply them.

Rapid prototyping is a fundamental practice in the startup environment, allowing for quick experimentation, validation of ideas, and gathering of user feedback.

The end or the beginning?

Based on my experience I can say that the success of building a company relies on the combination of individuals with a growth mindset, technical expertise and a bit of domain knowledge.
The vision provided by the founder sets the direction and purpose of the company, but are the people involved that propels the company towards achieving that vision. This is like running a marathon, not a 100-meter sprint. So, it's necessary to pace yourself and try to make mistakes in the shortest time possible in order to learn and adjust your aim.

If you are thinking of embarking on this journey don't forget this:

"It’s a long, hard and dirty job, but someone has to do it"

References

Personal:

Technical:

Startup project with FastAPI

Marco Sabatini — Wed, 23 Mar 2022 09:35:33 +0000

Overview
Pet project
Local Run
Scope
Testing
Considerations
References
Github project link

Overview

For the series "startup project with..." I'm going to approach python web development and I've tried to use FastAPI web framework.
I was very surprised by it's easy of use and I wanted to go in deeper about it; above all it's testing features.

Pet project

In order to test functionality I implemented this pet project. Basically it is a simple server with one rest api.

Local Run

After checkout the code you can simply execute "script/ci" on your terminal application. The script will execute these steps:

build python app in docker container
code checks (like unused code)
test execution

If you want to try the API in a local host container you can just execute these commands:

docker-compose up api  
curl http://localhost:8080/items/123\?q\="test"

The second one should provide this response:

{"item_id":123,"q":"test","use_case":"Production Code"}

Scope

Regarding the code design I just wanted to achieve two points:

encapsulate controller api on a class
inject collaborator in order to easily test API

Regarding the point 1) I have:

class AppController:
    def __init__(self, app: FastAPI, use_case: UseCase):
        @app.get("/items/{item_id}")
        def read_item(item_id: int, q: Optional[str] = None):
            return {
                "item_id": item_id, "q": q, "use_case": use_case.run()

The class AppController could be built with a FastAPI and a UseCase objects.
The first one is something related FastAPI configuration and run the second one is my business logic.

Having this class I can configure my application using a startup() method where wiring the code I need for the App:

def startup(use_case: UseCase = ProductionUseCase()):
    app = FastAPI()
    AppController(app, use_case)
    return app

Finally I can run my application using docker container. This is the container definition:

FROM python:3.10.3-slim-bullseye

WORKDIR /code
COPY ./requirements.txt /code/requirements.txt

RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt
COPY ./app /code/app
CMD ["uvicorn", "app.main:startup", "--host", "0.0.0.0", "--port", "8080", "--workers", "4"]

Testing

Off course I can also use startup() method for injecting dependencies in order to easily test the application.
These are two tests for the controller:

 def test_e2e_production_code(self):
        client = TestClient(startup())
        response = client.get("/items/987?q=this%20is%20the%20query")
        self.assertEqual(200, response.status_code)
        self.assertEqual(
            {"item_id": 987, "q": "this is the query", "use_case": "Production Code"},
            response.json()
        )

class TestableUseCase(UseCase):
    def run(self):
        return "Test Code"

 def test_e2e_testable_code(self):
        client = TestClient(startup(TestableUseCase()))
        response = client.get("/items/987?q=this%20is%20the%20query")
        self.assertEqual(200, response.status_code)
        self.assertEqual(
            {"item_id": 987, "q": "this is the query", "use_case": "Test Code"},
            response.json()
        )

The first test is using production code dependencies, building the client with production use case:

client = TestClient(startup())

Sometimes, during the development, we cannot use real connection or we just want to decompose test (unit, integration ...) using stubbed collaborators (ex. contract test).
For this reason we could inject a TestableCollaborator like in the second test (test_e2e_testable_code):

client = TestClient(startup(TestableUseCase()))

Considerations

From design point of view FastAPI is super useful framework to use to build our microservices. Fits very well with container logic and cloud development. It's also very easy to build a test suite for the application we're going to develop.
Of course you have to pay attention to not introduce complexity with some feature it provides like dependencies or DBMS access.

References

Github project link

My Personal Blog

Test Containers and Clean Architecture

Marco Sabatini — Thu, 25 Nov 2021 22:53:00 +0000

References

Overview

Separating the infrastructure from the domain is a smart move to define clear system boundaries, especially when dealing with components responsible for interacting with the external world, such as databases, external APIs, or web services. In this exploration, we'll delve into the TestContainers technique for constructing a robust test suite that aligns with these system boundaries.

According to the TestContainers documentation, this tool simplifies various types of tests, making them more manageable:

Data access layer integration tests: TestContainers allows us to use a containerized instance of databases like MySQL, PostgreSQL, or any other DBMS. This makes it easy to test data access layer code, ensuring compatibility without the need for intricate setups on developers' machines. The assurance that tests always begin with a known database state is a significant benefit. Additionally, any containerizable database type can be utilized.

Application integration tests: This involves running your application in a transient test mode with dependencies such as databases, message queues, or web servers. TestContainers facilitates this process, making it efficient and reliable.

UI/Acceptance tests: For automated UI tests, TestContainers provides containerized web browsers compatible with Selenium. Each test can spawn a fresh browser instance, eliminating concerns about browser state, plugin variations, or automated browser upgrades. Moreover, TestContainers records video sessions for each test, aiding in debugging and analysis.

Now, our approach will be a bit different. Instead of using the TestContainers framework directly, we'll leverage Docker along with some bash scripts for configuration. Let's dive into this pet code adventure!

Kata

Regarding my previous post, we want to write the DB access layer for this function:

loadEmployees: () -> Either<Error, Employees>

And hey, why not use Docker for testing the infrastructure? I've got these integration tests hanging around for the FileLoadEmployee function:

@Test
internal fun loadSomeEmployees()

@Test
internal fun employeeNotValidEmail()

No need to go into the nitty-gritty details of those tests—they're pretty straightforward.
Here's the plan: I want to repurpose them for the DB implementation, but let's not go down the road of duplicating code. My trick? Creating an abstraction for these tests that can serve both the File and DB implementations.

abstract class LoadEmployeeTest {

    @Test
    internal fun loadEmployee() {
        val loadEmployees = instance()

        assertThat(loadEmployees()).isEqualTo(
            Either.Right(
                Employees(
                    listOf(
                        Employee(
                            "Marco", "Sabatini", DateOfBirth(5, 3, 1983),
                            EmailAddress("address@email.com")
                        )
                    )
                )
            )
        )
    }

    @Test
    internal fun employeeNotValid() {
        val loadEmployees = wrongInstance()

        assertThat(loadEmployees()).isEqualTo(Either.Left(Error("Error For input string: \"wrong\"")))
    }

    abstract fun instance(): () -> Either<Error, Employees>
    abstract fun wrongInstance(): () -> Either<Error, Employees>
}

So, here's the deal: I'll whip up two abstract methods—instance() and wrongInstance(). These are the brainiacs responsible for conjuring up the function instances that handle loading employees. They'll serve up the goods for both the happy path and those pesky corner cases.

Now, for the File access layer:

override fun instance(): () -> Either<Error, Employees> =
        loadEmployeeFrom("./target/test-classes/employees.txt")

    override fun wrongInstance(): () -> Either<Error, Employees> =
        loadEmployeeFrom("./target/test-classes/employeesNotValid.txt")

    @Test
    internal fun fileNotFound() {

        val loadEmployeeFromFile = loadEmployeeFrom("NOT_EXIXSTING_FILE")

        Assertions.assertThat(loadEmployeeFromFile())
            .isEqualTo(Either.Left(Error("File NOT_EXIXSTING_FILE doesn't exist")))
    }

For DB access layer:

  @AfterEach
    fun cleanupTest() {
        execute("DELETE from employees")
    }

    override fun instance(): () -> Either<Error, Employees> {
        execute("INSERT INTO employees VALUES ('Marco', 'Sabatini','05/03/1983','address@email.com')")

        return loadEmployeeWith()
    }

    override fun wrongInstance(): () -> Either<Error, Employees> {
        execute("INSERT INTO employees VALUES ('', '','wrong','')")

        return loadEmployeeWith()
    }

    private fun execute(sql: String) {
        val stmt = connection().createStatement()
        stmt!!.executeUpdate(sql)
        stmt.close()
    }

You know, this trick I'm pulling off is called a contract test. It's like our secret sauce when we need to lay down the law for specific behavior in our code. Especially handy when we've got different implementations floating around.

So, what's cooking here? I'm defining the loadEmployees behavior between my domain code and the infrastructure code. It's like setting the rules of engagement. And the cool part? I'm testing different implementations to make sure they play nice. It's the go-to move if you want to throw in an in-memory implementation of an external 'port' and speed up those acceptance tests. Quick and snappy!

Docker && Docker Compose

Now it's time to move to the infrastructure part. I need a DBMS MySQL instance and a JVM maven runtime where my Kotlin test code can run. These containers have to communicate with each other.
This is the docker-compose file configuration that I used:

services:
  mysql:
    image: mysql
    container_name: mysql_server
    ports:
      - "3306:3306"
    environment:
      - MYSQL_ROOT_PASSWORD=pwd
    networks:
      - db-network

   db_client:
    build:
      context: ./containers/client
    networks:
      - db-network
    environment:
      - WAIT_HOSTS=mysql:3306
      - WAIT_HOSTS_TIMEOUT=300
      - WAIT_SLEEP_INTERVAL=30
      - WAIT_HOST_CONNECT_TIMEOUT=30

  maven:
    image: maven
    container_name: builder
    volumes:
      - ${PWD}:/tmp
    networks:
      - db-network

networks:
  db-network:

MySQL: DBMS
db_client: is a MySQL client command line interface. It is responsible for creating schema and could be used also for other purposes (ex. load demo test data etc.). It uses an entrypoint docker to create a schema or load demo data and this plugin helps to wait for MySQL instance to be started.

case "$@" in
  schema)
    /wait && mysql --host=mysql -uroot -ppwd < employees.sql
    echo "EMPLOYEES schema created!"
  ;;
  demo)
    /wait && mysql --host=mysql -uroot -ppwd < demo.sql
    echo "Could load demo data!"
  ;;
  *)
    exec "$@"
  ;;
esac

Here you can see docker container configuration details.

maven: is the container where my Kotlin test code is executed.

All the CI pipeline is orchestrated from this bash script:

#!/bin/bash
docker-compose up -d mysql
docker-compose build db_client
docker-compose run db_client schema
docker-compose run maven mvn --quiet -f /tmp clean install
docker-compose run maven mvn -f /tmp surefire-report:report -DshowSuccess=false
docker-compose down --remove-orphans

You can build the project directly executing it on your computer or we can use it to create a CI pipeline.

Setting CI pipeline

I have all the pieces to build my CI on github using GH actions.
Under .github/ folder I have this configuration:

name: docker-compose-actions-workflow
on: push
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: CI
        run: script/ci
      - name: DEMO
        run: script/demo

So, here's the lowdown: when we hit that push button, it kicks off a build that runs the CI and demo scripts. Just a smooth ride through the GitHub Actions tab, where you can catch all the action and check out the build outputs.

[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.kata.testcontainers.infrastructure.DBLoadEmployeeTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.561 s - in com.kata.testcontainers.infrastructure.DBLoadEmployeeTest
[INFO] Running com.kata.testcontainers.infrastructure.FileLoadEmployeeTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 s - in com.kata.testcontainers.infrastructure.FileLoadEmployeeTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0
containers ---

Demo output:

######## Employees ########
First Name: Marco
Last Name: Sabatini
Email: EmailAddress(value=address@email.com)
BirthDay: DateOfBirth(day=5, month=3, year=1983)
....

Keep in mind that I could add validation, performance, or quality gate steps.

Considerations

Having the infrastructure right there in the project codebase? It's like having a backstage pass to the entire ecosystem. No more long-distance vibes between dev and ops. Now, in my world, I can toss my container onto any cloud service—ECS on AWS, anyone? And the best part? I'm pretty darn confident the behavior will be just as groovy as what I see during CI or testing demos.

Now, here's the real talk from an organizational angle. Sure, having the infrastructure mingling with the code means having folks who can sling code, pick the right tools, and craft the necessary infrastructure magic. We're not just code jockeys; we're like the wizards who know their way around systems.

Clean Architecture and Either Monad

Marco Sabatini — Mon, 22 Nov 2021 14:27:17 +0000

References

Kata

I've been coding the birthday greeting in kotlin.
This exercise is very useful if you want to learn more about architect your code separating domain from infrastructure code.
I also had the purpose of making some practice with functional programming, in particular I wanted to use some functional data structure like monads.

Start

I started with some doubts: what kind of monad do I have to use? IO monad? Either monad? Both?
I can't find a response. I wanted something that made my code easily readable; I decided to use Either monad because my aim was to make the method signatures explicit.

I used the arrow implementation of the data structure. Docs is very clear and full of examples.

Either monad is like:

// This is the data structure
Either<ErrorType, ResponseType>

// Success case
Either.Right(ResponseType())

// Error case
Either.Left(ErrorType("This is the Error!"))

It helps to manage cases like a service that may result in a connection issue, or any unexpected JSON response.

Kata

Basically, my software will be a composition of functions.
The exercise asks for a software that is able to find employees whose birthday is today and send them emails.
I need to load employees from a data source (a file for this exercise), filter them, and send email.

My software could get an error when it's loading employees or sending Email.
It's curious to notice that when we filter employees (is your birthday?) nothing could go wrong because is something related our domain logic (pure function).

loadEmployees: () -> Either<MyError, Employees>

filterEmployees: (Employees) -> BirthdayEmployees

sendBirthdayNotificationTo: (BirthdayEmployees) -> Either<MyError, Unit>

These are the functions that I will compose into my use case. It's very clear what will be the purpose of code and when I could get an error.
I just need to inject the implementations of the functions using dependency injection.
As a result, I have a function responsibly to execute the composition (tadaaa!).

fun sendGreetingsWith(
    loadEmployees: () -> Either<MyError, Employees>,
    filterEmployees: (Employees) -> BirthdayEmployees,
    sendBirthdayNotificationTo: (BirthdayEmployees) -> Either<MyError, Unit>
): () -> Either<MyError, Unit> = {

    loadEmployees()
        .map(filterEmployees)
        .flatMap(sendBirthdayNotificationTo)
}

I can specify the types of errors I can get during execution, modelling them using sealed classes:

sealed class MyError {
    data class LoadEmployeesError(val msg: String) : MyError()
    data class SendMailError(val msg: String) : MyError()
}

Moreover in this situation the code is quite obvious and we don't need any comment to understand what is the purpose.

Testing

It's very curious to see how easily is testing after separating infrastructure from the domain.
For example: in this acceptance test I could inject the 'load employee logic' using in memory implementation and running the test without having additional configurational stuff for the infrastructure part.

    @Test
    internal fun oneEmployeeIsBornTodayAnotherIsNotBornToday() {

        val inMemoryLoadEmployee: () -> Either<Nothing, Employees> = {
            Right(
                Employees(
                    listOf(
                        employeeBirthday(dateOfBirth(TODAY), "TODAY_EMPLOYEE"),
                        employeeBirthday(dateOfBirth(TOMORROW), "TOMORROW_EMPLOYEE")
                    )
                )
            )
        }

        val sendGreetings: () -> Either<MyError, Unit> =
            sendGreetingsWith(
                inMemoryLoadEmployee,
                todayBirthdayEmployees,
                sendMailWith("localhost",9999)
            )

        val result = sendGreetings()

        val message = mailServer.receivedEmail.next() as SmtpMessage
        assertThat(message.body).isEqualTo("Happy birthday, dear TODAY_EMPLOYEE!")

        assertThat(result).isEqualTo(Right(Unit))
    }

Just inject the in memory behaviour and execute the test.

Do I need to test infrastructure?

Yes of course!
After separating domain logic from the infrastructure logic, It's very easy testing infrastructure code in isolated mode using an integration test like these:

    @Test
    internal fun loadEmployeeFromFile() {

        val loadEmployeeFromFile = loadEmployeeFrom("./target/test-classes/employees.txt")

        assertThat(loadEmployeeFromFile()).isEqualTo(Right(Employees(listOf(Employee("Marco","Sabatini", DateOfBirth(5,3,1983),
            EmailAddress("address@email.com")
        )))))
    }

    @Test
    internal fun employeeNotValid() {

        val loadEmployeeFromFile = loadEmployeeFrom("./target/test-classes/employeesNotValid.txt")

        assertThat(loadEmployeeFromFile()).isEqualTo(Left(MyError.LoadEmployeesError("Error For input string: \"address@email.com\"")))
    }

    @Test
    internal fun fileNotFound() {

        val loadEmployeeFromFile = loadEmployeeFrom("NOT_EXIXSTING_FILE")

        assertThat(loadEmployeeFromFile()).isEqualTo(Left(MyError.LoadEmployeesError("File NOT_EXIXSTING_FILE doesn't exist")))
    }

Imagine that we have to load data from database. We just could write the jdbc implementation of loadEmployees and we could use a docker container for making integration test and voila' (see on my post).
In this way all these integration tests will become a tests container or if we have to integrate with external services (like API) we could write just contract tests.

End

From a design point of view I have a use case responsible to orchestrate the flow.
We can see three main responsibilities:

Load data
Execute some domain logic
Send email messages

The software is quite generic and using dependency injection is very easy loading data from another data source or sending another kind of message like for example a mobile notification, just implementing these functions:

loadEmployees: () -> Either<MyError, Employees>

sendBirthdayNotificationTo: (BirthdayEmployees) -> Either<MyError, Unit>

This is the power of separating domain logic from infrastructure.

Regarding functional programming, we can say 'Either' monad make my code more readable even if I can achieve the same result using Object Oriented programming with exceptions.

I would think about monads like a meta-language on top of our programming language, able to create something understandable for developers (a sort of dev ubiquitous language).
If we had developed the exercise using 'Either' monad with another language probably we'll have the same result.

P.S.

This is an exercise. I could resolve it in a lot of different ways.
Don't focus on technical stuff but on design and testing!

Functional programming building blocks 2nd round

Marco Sabatini — Mon, 24 Aug 2020 13:25:56 +0000

References

OCP principle

In my previous post, I talked about functional programming's fundamental building blocks and introduced the Open-Closed Principle (OCP). Today, I want to explore software modularization and how we can implement the OCP using functional programming techniques.

The Open-Closed Principle (OCP) may sound simple, but it can be challenging to implement effectively: "Software should be open for extension but closed for modification." Achieving this principle is crucial for maintaining agility and adaptability in your development process.

Let's revisit the context of our Video Store Kata. We've already implemented the ability to print receipts in plain text. However, a new requirement has arisen: we need to print receipts in HTML format without altering our existing codebase. This is where the Open-Closed Principle comes into play.

To fulfill this requirement, we'll start by examining our receipt module:

class PrintableMovie {
    title: string;
    priceRepresentation: string;

    constructor(title: string, priceRepresentation: string) {
        this.title = title;
        this.priceRepresentation = priceRepresentation;
    }
}

const printableMovieWith =
    (calculateMoviePrice: (r: Rental) => number) =>
        (r: Rental) => new PrintableMovie(r.mc.title, calculateMoviePrice(r).toPrecision(2));

export const printableMovie: (r: Rental) => PrintableMovie =
    printableMovieWith(calculateMoviePrice);

This module is designed to be generic. It defines a PrintableMovie data type for items that need to be printed. It also includes two functions:

1) printableMovie transforms a Rental into a PrintableMovie.
2) printableMovieWith takes a price calculation function as input and prints the price with a precision of two digits.

This serves as a contract between the pricing module and the receipt module. Defining this contract using functions allows us to test pricing and receipt modules as separate black boxes.

With this foundation, we can now generalize the print receipt function:

export const genericReceipt =
    (header: (user: string) => string,
     body: (rentals: Rental[]) => string,
     footer: (rentals: Rental[]) => string,
     rentalPoint: (rentals: Rental[]) => string) =>

        (user:string, rentals:Rental[]) =>
            header(user) +
            body(rentals) + "\n" +
            footer(rentals) + "\n" +
            rentalPoint(rentals)

Now, we can implement both the plain text and HTML templates.

For plain text:

const textMovieReceipt = (m: PrintableMovie): string =>
     `- ${m.title} ${m.priceRepresentation}`

const textMoviesReceiptWith = (
    movieReceiptFunc: (x: Rental) => string) =>
     (rentals: Rental[]) => rentals.map(r => movieReceiptFunc(r)).join("\n")

const textFooterReceiptWith = (
    totalPrice: (rentals: Rental[]) => number) =>
     (rentals: Rental[]) => `Total ${totalPrice(rentals).toPrecision(2)}`

const textFooterRentalPointReceiptWith = (
    calculateRentalPoint: (rentals: Rental[]) => number) =>
     (rentals: Rental[]) => `Total Rental points ${calculateRentalPoint(rentals)}`

//WIRING HERE
const textFooterRentalPointReceipt =
    textFooterRentalPointReceiptWith(calculateRentalPoints);

const textFooterReceipt: (rentals: Rental[]) => string =
    textFooterReceiptWith(calculateTotalMoviesPrice);

const textMoviesReceipt: (rentals: Rental[]) => string =
    textMoviesReceiptWith(compose(
        printableMovie,
        textMovieReceipt))

const textHeader = (user: string) => `Hello ${user} this is your receipt\n`;

//WIRING THE PRINT FUNCTION WITH PLAIN TEXT BEHAVIOUR
export const printTextReceipt: (user: string, rentals: Rental[]) => string =
    genericReceipt(
        textHeader,
        textMoviesReceipt,
        textFooterReceipt,
        textFooterRentalPointReceipt)

And for HTML:

const htmlMovieReceipt = (m: PrintableMovie): string =>
    `<li>${m.title} ${m.priceRepresentation}</li>`

const htmlMoviesReceiptWith = (
    htmlMovieReceipt: (x: Rental) => string) =>
    (rentals: Rental[]) => `<ul>\n${rentals.map(r => htmlMovieReceipt(r)).join("\n")}\n</ul>`

const htmlFooterReceiptWith = (
    calculateMoviesTotalPrice: (rentals: Rental[]) => number) =>
    (rentals: Rental[]) => `<br>You owed ${calculateMoviesTotalPrice(rentals).toPrecision(2)}`

const htmlFooterRentalPointReceiptWith = (
    calculateRentalPoint: (rentals: Rental[]) => number) =>
    (rentals: Rental[]) => `<br>You earned ${calculateRentalPoint(rentals)} frequent renter points\n</body>\n</html>`

//WIRING HERE
const htmlFooterRentalPointReceipt: (rentals: Rental[]) => string =
    htmlFooterRentalPointReceiptWith(calculateRentalPoints);

const htmlFooterReceipt: (rentals: Rental[]) => string =
    htmlFooterReceiptWith(calculateTotalMoviesPrice);

const htmlMoviesReceipt: (rentals: Rental[]) => string =
    htmlMoviesReceiptWith(compose(
        printableMovie,
        htmlMovieReceipt))

const htmlHeader = (user: string) =>
    `<!DOCTYPE html>\n` +
    `<html>\n` +
    `<head>\n` +
    `<title>Video store - statement for ${user}</title>\n` +
    `</head>\n` +
    `<body>\n` +
    `<h1>Rental Record for ${user}</h1>\n`

//WIRING THE PRINT FUNCTION WITH HTML TEXT BEHAVIOUR
export const printHtmlReceipt: (user: string, rentals: Rental[]) => string =
    genericReceipt(
        htmlHeader,
        htmlMoviesReceipt,
        htmlFooterReceipt,
        htmlFooterRentalPointReceipt)

The code structure remains consistent for both templates. We've successfully achieved the Open-Closed Principle – our code is open for extension (to add new templating formats) but closed for modification, which ensures robustness.

A key takeaway is the importance of an emerging design. My initial version was different from the current one, and I had to refactor my code before implementing the new HTML receipt feature. Continuous refactoring is essential for maintaining a resilient architecture.

Functional programming building blocks

Marco Sabatini — Sun, 16 Aug 2020 14:18:10 +0000

References

Overview

Functional programming is fascinating, but how can you start practicing it? If you're a developer who has been pondering this question, I'd like to share my personal journey and experiences.

I firmly believe in the "learning by doing" approach. That's why I decided to use a coding exercise known as a kata to practice functional programming. Specifically, I chose the Martin Fowler kata for the video store, not the refactoring version but the one starting from scratch. This approach provided me with a clean slate to work with, enabling me to focus on the application's domain rather than getting bogged down in technical details. I opted for TypeScript to leverage the functional capabilities offered by the language.

The chosen kata is rather straightforward. My aim was to concentrate on the application's domain logic rather than dealing with technical intricacies like database persistence or external HTTP service integration. The primary objective of this kata was to create a system capable of renting different types of movies and generating receipts in various formats, such as plain text and HTML.

Test first

My journey began by writing a test suite to calculate the price for specific movie types:

it('rent new Release Movie for one day', () => {
        expect(moviePriceFor(new Rental(1, newReleaseConfiguration("UNUSED")))).toEqual(3.0)
});
it('rent Children Movie for four day', () => {
        expect(moviePriceFor(new Rental(4, childrenConfiguration("UNUSED")))).toEqual(3.0)
});

While writing these tests, I discovered key concepts like rental, movie types, additional price calculations for extra days, and individual movie price calculations.

I implemented the following production code to pass these tests:

const additionalCostFor = (rental: Rental): MoviePrices => {
  let additionalCost = 0.0;
  if (rental.rentalDays > rental.mc.minRentDays) {
    const additionalDays = rental.rentalDays - rental.mc.minRentDays
    additionalCost = rental.mc.additionaCostPerDay * additionalDays;
  }
  return new MoviePrices(additionalCost, rental.mc.price);
}

const priceFor = (moviePrices: MoviePrices): number => {
    return (moviePrices.movieBasePrice + moviePrices.additionalCost).toPrecision(5) 
};

The first function calculates the additional cost, while the second function adds the base price and rounds it to five decimal places.

At this point, I realized that I had the essential building blocks needed to compose a function that calculates the total price for a single movie type.

Let's go and apply composition!

Composition

Next, I decided to implement a compose function, but of course, I wrote a test first:

it('compose two function', () => {

  let f = (x: string): string => `f(${x})`
  let g = (x: string): string => `g(${x})`

  let gfx: (x: string) => string = compose(f, g)

  expect(gfx("value")).toEqual("g(f(value))")
});

In this test, I defined two functions, 'f' and 'g', which take an input parameter and return a string with that parameter interpolated. By composing them, I achieved string concatenation.

The production code for the compose function looks like this:

export const compose = <A,B,C>(
  f: (x: A) => B,
  g: (y: B) => C):
  (x: A) => C => {

    return (x) => g(f(x))
};

Using TypeScript generics, I created a versatile compose function that can be used for any pair of functions where the output type of one matches the input type of the other.

With this compose function in place, I was able to compose the additionalCostFor and priceFor functions like so:

const additionalCostFor = (rental: Rental): MoviePrices => {...}

const priceFor = (moviePrices: MoviePrices): number => {...}

const moviePriceFor: (x: Rental) => number = compose(additionalCostFor, priceFor)

Thanks to the type system, I didn't even need to write a test for this specific composition because it naturally emerged, and the compiler confirmed that the functions could be composed successfully.

Try to compose again!

Leveraging Curry

By creating these basic building blocks, I could easily compose them to create more complex functions. This approach encourages clear and isolated responsibilities, leading to excellent cohesion and loose coupling.

For the total price calculation, I reused the calculation for individual movies by currying the function and applying it using map and reduce:

const additionalCostFor = (rental: Rental): MoviePrices => {...}

const priceFor = (moviePrices: MoviePrices): number => {...}

const moviePriceFor: (x: Rental) => number = compose(additionalCostFor, priceFor)

export const totalPrice = (moviePriceFor:(r:Rental) => number):(rentals:Rental[])=> number =>{
  return (rentals) => rentals.map(r=>moviePriceFor(r)).reduce((x,y)=>x+y);
}

Currying allowed me to partially apply the function and return a configured function, making composition even more powerful.

Modularization of Software

To maintain clean and modular code, I exported the total price calculation function from the pricing module. This function was used by modules responsible for printing receipts in HTML and plain text formats.

By doing so, I defined a clear public interface between the modules. I also had the flexibility to mock this function to facilitate testing of the printing modules (HTML and plain text).

Final Thoughts

In functional programming, functions are the fundamental building blocks. Each function can be thought of as a Lego brick, and pure functions are inherently isolated. Unlike encapsulation, where an object tries to hide information, pure functions can only do what they declare in their interface or signature, making them "honest."

This paradigm shift encourages problem-solving by breaking them down into small, isolated functions and then reassembling them at the application's entry point. While this approach may seem counterintuitive initially, it fundamentally changes how you think about building software.

If you'd like to explore this topic further, check out the second round of my journey.

DEV Community: Marco Sabatini

Event storming, and then what?

References

Overview

Mars Rover Kata

Event Storming

Implementation

Command Handlers

Domain

Services

Path Projection

Policy

Additional requirement

Evolutionary Architecture

My First Year as a Senior Software Architect in a Startup: Embracing the Journey

Table of contents

Let's Start

Why Choose a Startup?

Mindset?

Individual Contributor

Uncertainty

Engineering practice

Continuous Integration & Testing:

Test-Driven Development (TDD):

DevOps Approach:

Non-Overengineering:

Predictable Delivery & Rapid prototyping

The end or the beginning?

References

Startup project with FastAPI

Table of contents

Overview

Pet project

Local Run

Scope

Testing

Considerations

References

Test Containers and Clean Architecture

References

Overview

Kata

Docker && Docker Compose

Setting CI pipeline

Considerations

Clean Architecture and Either Monad

References

Kata

Start

Kata

Testing

Do I need to test infrastructure?

End

P.S.

Functional programming building blocks 2nd round

References

OCP principle

Functional programming building blocks

References

Overview

Test first

Composition

Leveraging Curry

Modularization of Software

Final Thoughts

Next