DEV Community: Andrzej Krzywda

Authentication events and commands

Andrzej Krzywda — Mon, 22 Aug 2022 19:15:52 +0000

The last few days I didn't write too much new code on the Ecommerce project.

However, that's the beauty of open source! Regardless of me not being active, there were 2 new contributions, how lovely!

One of the new features which is being worked on is displaying a list of Shipments. At the moment of writing this, it's not yet merged.

I will focus on the Authentication feature.

This new code (a new bounded context) was implemented by Paweł Strzałkowski (thanks Paweł!) and this was his first contribution.

Overall the goal is to allow clients to properly log in.

Obviously, we could go with Devise and have it done in 10 minutes. However, we all know how it ends. It ends with a User class and its 100 attributes accumulated over time.

Given that the whole project is fully DDD, CQRS and Event Sourcing - it was part of the goal to have authentication implemented in this fashion.

The idea of Authentication as a separate Bounded Context first came to me from the "Implementing Domain Driven Design" book by Vaughn Vernon. It's commonly known as the red book.

Vaughn called it Identity & Access. We're going for Authentication for now.

So far, we're implementing BCs as new directories and separate Ruby gems. They each have their own tests, their own Makefile, their own build system, their own mutation coverage.

OK, let's start with the input to the Authentication module - the commands:

At first, there's no account, so we need to register one:

module Authentication
  class RegisterAccount < Infra::Command
    attribute :account_id, Infra::Types::UUID
    alias aggregate_id account_id
  end
end

As you see, it doesn't really have username or password, it comes as separate commands:

module Authentication
  class SetLogin < Infra::Command
    attribute :account_id, Infra::Types::UUID
    attribute :login, Infra::Types::String
    alias aggregate_id account_id
  end
end

module Authentication
  class SetPasswordHash < Infra::Command
    attribute :account_id, Infra::Types::UUID
    attribute :password_hash, Infra::Types::String
    alias aggregate_id account_id
  end
end

In the end we can connect such an account to a Customer:

module Authentication
  class ConnectAccountToClient < Infra::Command
    attribute :account_id, Infra::Types::UUID
    attribute :client_id, Infra::Types::UUID
    alias aggregate_id account_id
  end
end

Such a design is very flexible. We can have clients without an account. Also, we can have clients with multiple accounts.

We also have a similar set of events, also very granular, allowing for flexibility:

module Authentication
  class AccountRegistered < Infra::Event
    attribute :account_id, Infra::Types::UUID
  end

  class LoginSet < Infra::Event
    attribute :account_id, Infra::Types::UUID
    attribute :login, Infra::Types::String
  end

  class PasswordHashSet < Infra::Event
    attribute :account_id, Infra::Types::UUID
    attribute :password_hash, Infra::Types::String
  end

  class AccountConnectedToClient < Infra::Event
    attribute :account_id, Infra::Types::UUID
    attribute :client_id, Infra::Types::UUID
  end
end

How does it happen that a command is issued and then the module issues an event?

We have a layer of command handlers:

module Authentication
  class RegisterAccountHandler
    def initialize(event_store)
      @repository = Infra::AggregateRootRepository.new(event_store)
    end

    def call(command)
      @repository.with_aggregate(Account, command.aggregate_id) do |account|
        account.register
      end
    end
  end

Here is the part where the infra part is injected - the event_store. We need for retrieving the events, so that the object (here: Account) can be event sourced (loaded from events).

It all happens in the AggregateRootRepository.new so we don't have to care about this.

The object that is loaded is called an aggregate.
We call a register method on it.

Let's take a look at the whole aggregate class:

module Authentication
  class Account
    include AggregateRoot

    AlreadyRegistered = Class.new(StandardError)

    def initialize(id)
      @id = id
    end

    def register
      raise AlreadyRegistered if @registered

      apply AccountRegistered.new(data: { account_id: @id })
    end

    def set_login(login)
      apply LoginSet.new(data: { account_id: @id, login: login })
    end

    def set_password_hash(password_hash)
      apply PasswordHashSet.new(data: { account_id: @id, password_hash: password_hash })
    end

    def connect_client(client_id)
      apply AccountConnectedToClient.new(data: { account_id: @id, client_id: client_id })
    end

    on AccountRegistered do |event|
      @registered = true
    end

    on LoginSet do |event|
      @login = event.data[:login]
    end

    on PasswordHashSet do |event|
      @password_hash = event.data[:password_hash]
    end

    on AccountConnectedToClient do |event|
      @client_id = event.data[:client_id]
    end
  end
end

The pattern which is used here, is called AggregateRoot - it mixed the business logic with the messaging (events) part. It's all simplified with our AggregateRoot module which is injected.

As you can see, so far there's not much actual logic. The only rule is to not allow this Account to be registered more than once.

Keeping all this data in one aggregate is a one possible design. It's simple and very familiar to CRUD designs.

With the current requirements, it's good enough. Maybe in the future we can go with even smaller aggregates - once more fancy requirements come.

The domain level is always surrounded with application layer. In a way the app layer protects the domain layer. It knows how to use it.

Certain requirements can be kept at the app layer.

For example, looking at this code, we may ask:

How are we protected against connecting the same account to multiple clients?

Let's keep in mind that the feature is still in progress and the requirements are created by us. It's a bit vague now.

At the moment, this code is only used in database seeds, from the app layer:

[
  ["BigCorp Ltd", "bigcorp", "12345"],
  ["MegaTron Gmbh", "megatron", "qwerty"],
  ["Arkency", 'arkency', 'qwe123']
].each do |name, login, password|
  account_id = SecureRandom.uuid
  customer_id = SecureRandom.uuid
  password_hash = Digest::SHA256.hexdigest(password)

  command_bus.call(
    Crm::RegisterCustomer.new(customer_id: customer_id, name: name)
  )

  command_bus.call(
    Authentication::RegisterAccount.new(account_id: account_id)
  )

  command_bus.call(
    Authentication::SetLogin.new(account_id: account_id, login: login)
  )

  command_bus.call(
    Authentication::SetPasswordHash.new(account_id: account_id, password_hash: password_hash)
  )

  command_bus.call(
    Authentication::ConnectAccountToClient.new(account_id: account_id, client_id: customer_id)
  )
end

There's no UI for that, so the accounts are not really used in any way.

Still, a nice step forward in this Authentication area. It opens the door to implementing the UI and the app layer (controllers, read models).

As you can see, apart from the Customer reference (maybe we should rename it to User to stick to the Ubiquitous Language of Authentication?) - it's mostly generic code. I hope that this BC together with read models which will be created can become a reusable library which could be a good alternative to Devise in those projects which want to go the DDD/CQRS path.

Exploring a generic read model API

Andrzej Krzywda — Mon, 15 Aug 2022 23:57:29 +0000

Arkency Ecommerce follows the CQRS pattern and creates separate modules for "reads".

We call them read models.

The simplest read models consist of:

one or more database table
some configuration code which maps events to columns in the table

Today, I was about to create a read model. A public offer of products presented to the client.

We already have a Products read model, used on the admin/sales panel.

At first they seem to be almost the same and it's tempting to just reuse it.

This is an often temptation in CQRS. This temptation is obvious in CRUD systems. There's one way of retrieving products, how complex can it be, right?

However, even in our simple feature set it's already visible that they contain different columns.

They serve different needs.

What am I really saving by reusing it?

The only risk is if we start having some calculations to show price (very likely, but is it part of read models?), then we can forget about some of them.

Anyway.

I looked at the current read model:

module Products
  class Product < ApplicationRecord
    self.table_name = "products"
  end

  class Configuration
    def call(cqrs)
      cqrs.subscribe(
        -> (event) { register_product(event) },
        [ProductCatalog::ProductRegistered]
      )
      cqrs.subscribe(
        ->(event) { change_stock_level(event) },
        [Inventory::StockLevelChanged]
      )
      cqrs.subscribe(
        -> (event) { set_price(event) },
        [Pricing::PriceSet])
      cqrs.subscribe(
        -> (event) { set_vat_rate(event) },
        [Taxes::VatRateSet])
    end

    private

    def register_product(event)
      Product.create(id: event.data.fetch(:product_id), name: event.data.fetch(:name))
    end

    def set_price(event)
      find(event.data.fetch(:product_id)).update_attribute(:price, event.data.fetch(:price))
    end

    def set_vat_rate(event)
      find(event.data.fetch(:product_id)).update_attribute(:vat_rate_code, event.data.fetch(:vat_rate).fetch(:code))
    end

    def change_stock_level(event)
      find(event.data.fetch(:product_id)).update_attribute(:stock_level, event.data.fetch(:stock_level))
    end

    def find(product_id)
      Product.where(id: product_id).first
    end
  end
end

I'm not about you, but it screams to me:

THIS CODE IS MOSTLY DATA

It's basically mapping declarations, with some subtleties. All on top of ActiveRecord.

Just to show you the db schema:

  create_table "products", id: :uuid, default: -> { "gen_random_uuid()" }, force: :cascade do |t|
    t.string "name"
    t.decimal "price", precision: 8, scale: 2
    t.integer "stock_level"
    t.datetime "registered_at", precision: nil
    t.string "vat_rate_code"
  end

(and I'm adding to my TODO to check if the registered_at column is somehow magically used or it can be deleted)

Then I looked at how it's tested.

While we maintain almost 100% mutation coverage at the domain level, we don't share a similar quality at the app level (controllers, read models).

I noticed that this read model doesn't have unit tests. It's tested only via integration test, if anything. That's sad. I need to improve on this.

But then a second thought.

If I make some mechanical refactorings (the ones which can be done without full test coverage, then this code will become mostly declarations or some kind of DSL API.

If it really happens, then I should the resulting "framework" which will be extracted instead of the usages.

The issue with testing declarative code is that it's usually a "typotest", repeating what's in the production code.

Long story short, this is my first attempt to make the code more declarative:

module Products
  class Product < ApplicationRecord
    self.table_name = "products"
  end

  class Configuration
    def initialize(cqrs)
      @cqrs = cqrs
    end

    def call
      @cqrs.subscribe(-> (event) { register_product(event) }, [ProductCatalog::ProductRegistered])
      copy(Inventory::StockLevelChanged, :stock_level)
      copy(Pricing::PriceSet,            :price)
      copy_nested_to_column(Taxes::VatRateSet, :vat_rate, :code, :vat_rate_code)
    end

While those 2 lines are quite elegant to me:

copy(Inventory::StockLevelChanged, :stock_level)
copy(Pricing::PriceSet,            :price)

They look nice, because they (by accident) follow the convention of matching the event attribute with the column name.

The other lines still need some love.

First, creating a record here requires a name, it's not just empty constructor. It was hard for me to find some generalization of this code.

The last line was the edge case. It happens to have nested data in the event. Also it doesn't match the column name.

copy_nested_to_column(Taxes::VatRateSet, :vat_rate, :code, :vat_rate_code)

and here is the remaining supporting code to make it all work:

    private

    def copy(event, attribute)
      @cqrs.subscribe(-> (event) { copy_event_attribute_to_column(event, attribute, attribute) }, [event])
    end

    def copy_nested_to_column(event, top_event_attribute, nested_attribute, column)
      @cqrs.subscribe(
        -> (event) { copy_nested_event_attribute_to_column(event, top_event_attribute, nested_attribute, column) }, [event])
    end

    def register_product(event)
      Product.create(id: event.data.fetch(:product_id), name: event.data.fetch(:name))
    end

    def copy_event_attribute_to_column(event, event_attribute, column)
      product(event).update_attribute(column, event.data.fetch(event_attribute))
    end

    def copy_nested_event_attribute_to_column(event, top_event_attribute, nested_attribute, column)
      product(event).update_attribute(column, event.data.fetch(top_event_attribute).fetch(nested_attribute))
    end

    def product(event)
      find(event.data.fetch(:product_id))
    end

    def find(product_id)
      Product.where(id: product_id).first
    end

This code should still become more generic.
We need to un-hardcode the ActiveRecord class name Product.

And just to remind you. My initial goal was to create a new read model. What I'm doing here is a preparatory refactoring. After this, I expect my new read model to be implemented in only few lines of declarations.

We will see how it goes :)

Here is the commit with those changes:
https://github.com/RailsEventStore/ecommerce/commit/9e42950fc7eb34257938b0501fa6e50733d0d568

Thanks for reading ❤️

Ecommerce - the modularisation ideas

Andrzej Krzywda — Thu, 11 Aug 2022 14:33:00 +0000

At the Ecommerce project we have a main split between the app layer and the domain layer.

We aim for the domains to be reusable.
The applications are specific, not meant to be reusable, at least that's not the main goal.

I'd like to share my current thinking snapshot of what I think could help us achieve nice modularity and reusability.

What makes a domain?

commands as the only input layer
events as the only output layer
aggregates as internal mechanism which maintains the state transitions
- they could be objects or functions

In theory, that's all.

In practice, you sometimes need to have something more. Sometimes you will have a read model which exists only for the purpose of this BC (bounded context) to read from. For example, Pricing BC may need to maintains its own internal Pricing Catalog, which to read prices from.

Given the above, we want to keep the commands and events as granular as possible. We want to allow all kind of reusability on top of this.

Some apps may want to have nicer/bigger commands or events. That's great.

They should use the EventSummaryBuilder pattern to build from the small events and provide bigger events.

Similar thing with commands, they may want to build bigger commands - that's great, let's just show how those big commands map onto the existing small commands.

Those builders, at least that's my current thinking, are not part of the BC. They are part of the app layer, because they are a tweak for the application. In a way they are the facade which could/should be maintain by the application, as the domain code consumer.

We can provide them as something reusable, but that's different "blocks".

BCs alone are useful, but they don't create the Feature.

A feature is a set of building blocks which together form a functional and working unit at the application layer combined with the domain/BC.

Let's look at Invoicing.

There's Invoicing BC with its events and commands.

There's a process manager which builds the Invoicing state via reacting to some Pricing events. This is application layer.

There's obviously a way to display the invoice for the customer.
That's a read model.

There's also a way to display the invoice to the admin, usually different than the one for clients. That's another read model.

Those read models have their own storages - they should own it. The migrations should belong to the read models.
The html/json renderers can also belong to read models.

There's this gray area too.
How useful can the BCs be without the accompanying BCs?

Can we imagine the Invoicing BC to work fine without the current Pricing?

I think it should be possible.
One hypothetical scenario is to create a typical Invoicing application, without any ecommerce surroundings.

That's overall the general idea on how things could work.

Things may change, once we have more apps and more reusability which will show which parts of this vision are fragile.

Thoughts?

Html templates as Ruby code

Andrzej Krzywda — Wed, 10 Aug 2022 01:24:33 +0000

At the Arkency Ecommerce project, we went for the server rendered views strategy. Instead of dealing with JavaScript frameworks we have it all under the Rails umbrella.

Given the project is very "business" oriented, there's less need to have a fancy JS UI.

One typical view was our Client Panel login screen:

<div class="max-w-6xl mx-auto py-6 sm:px-6 lg:px-8">
  <%= form_tag("login", method: :post) do %>
    <div>
      <label for="client" class="block font-bold">
        Client
      </label>
      <%= select_tag(:client_id, options_from_collection_for_select(@clients, :uid, :name), id: "client", class: "mt-1 focus:ring-blue-500 focus:border-blue-500 block shadow-sm sm:text-sm border-gray-300 rounded-md") %>
      <%= button_tag('Login', class: "mt-2 bg-blue-500 hover:bg-blue-700 text-white font-bold py-2 px-4 rounded") %>
    </div>
  <% end %>
</div>

It's a usual Rails view called ERB, where it's html with some special Ruby snippets wrapped with <%= %>.

Such views are fine and every Rails dev knows how to modularise it into partials, how to use helpers etc.

However, over time Rails views tend to accumulate some "logic". It's hard to avoid some if statements. The views are more Ruby than HTML.

That's why I decided to try another approach in the Client Panel part of our application.

Let's try to write those views in Ruby and make it generate HTML.

There are obvious cons to this approach - it's no longer HTML so it's harder to get frontend people to tweak it.
Still, the pros might be worth experimenting.

class Login < Arbre::Component
  def self.build(view_context)
    new(Arbre::Context.new(nil, view_context)).build
  end

  def build(attributes = {})
    super(attributes)
    clients = ClientOrders::Client.all
    div class: "max-w-6xl mx-auto py-6 sm:px-6 lg:px-8" do
      safe_join([
        text_node(form_tag("login", method: :post)),
        div do
          safe_join([
            select_tag(:client_id, options_from_collection_for_select(clients, :uid, :name), id: "client", class: "mt-1 focus:ring-blue-500 focus:border-blue-500 block shadow-sm sm:text-sm border-gray-300 rounded-md"),
            button_tag('Login', class: "mt-2 bg-blue-500 hover:bg-blue-700 text-white font-bold py-2 px-4 rounded")
          ])
        end
      ])
    end
  end
end

It's different, yet similar.
I'm using the [arbre](https://activeadmin.github.io/arbre/) library here. There are others, but this one was already battle-tested in one of our projects.

There are some gotchas here. While the Ruby code looks almost like a direct translation from HTML, there are differences.

Note the safe_join calls. It's needed whenever we combine more than one "block" of html.

Another one is text_node which is often needed for the Rails helpers. Otherwise, the tags might not be rendered.

Now the controller looks like this:

module Client
  class ClientsController < ApplicationController
    layout "client_panel"

    def index
      render html: Login.build(view_context), layout: true
    end

It's too early to say if it's worth it. I will turn some more views in this area and try to make the views more complex. Then I will see if it's really easier to deal with the view logic in pure Ruby or ERB approach is better.

One hope which I have with this evolution is that now I can actually test the code as any other Ruby class. Also, it's now possible to measure the test coverage.

What is your idea about this approach?

Implementing Authentication in tiny steps

Andrzej Krzywda — Thu, 04 Aug 2022 13:07:00 +0000

When we started the project which we currently call "Arkency Ecommerce", under the RailsEventStore umbrella - it actually started as an Ordering Management System (OMS), more than actual ecommerce.

I always thought that OMS is an obvious part of ecommerce platforms. Recently I learnt it's not always the case, and even Shopify recommend looking at some other OMS systems as addition to their ecommerce.

Interesting.

Anyway, when our project grew, we were focused on the "Sales Panel" or "OMS panel". The user was probably a sales person. There was no authentication for that. We can assume that it's for internal use of the company hidden in the intranet.

After some time, we have created the "Client View". At first it was without authentication too. We only added a "select list" to choose which client you are "logged in" as.

However, even then, we just used the URL param to remember which client we're logged in as. Also, the functionality was and still is very limited. It was just a list of orders.

There was also a detailed order view, but I just removed it. The reason I removed it, was because it didn't have tests and it didn't use its own read model, just borrowed from the OSM OrdersList read model. It doesn't sound like a big deal, but it actually is. We don't want changes to OSM to interfere with the Client Panel.

One of my recent changes was to extend the authentication with cookies storage, while still not having passwords yet.

I like the concept of tiny steps or as I like to call it "start from the middle".

We started without authentication, but we focused on how it would look like. After "seeing it"/"feeling it", we could decide what next.

That's the luxury of open source codebases, not limited to deadlines and clients.

Small steps allow for two perspectives.

The code perspective

We could start with some code hacks (coupling of read models) and buy some time to think how it should be reflected in the architecture. Only when we feel what smells we can come up with better architecture.
In this case, it was a realisation that the "client panel" is like a separate app. At the application layer it should be reflected in some way. I strongly believe in 2 levels of architectural split in typical web apps.
One level is app layer. The second one is domain layer.

Often people confuse those two and try to merge them. Whatever is the app split, becomes the domain split. This is unfortunate as it's almost not possible to do such one-dimension split.

The UI perspective

Playing with the UI allows to rethink where we are with the features and what are the next stories to implement. Using the UI opens me in thinking what are the next tiny steps to add.
Note that the authentication is not the most important feature of this area. It's crucial once we go live, but before that we can hide it behind a feature flag if we want. We all know how to build authentication. It's not the risky area. But we don't know what to put in the Client Panel exactly for it to be usable. This is more risky.

The current codebase

module Client
  class ClientsController < ApplicationController
    layout "client_panel"

    def index
      @clients = ClientOrders::Client.all
      render "clients/index"
    end

    def login
      cookies[:client_id] = params[:client_id]
      redirect_to client_orders_path
    end

    def logout
      cookies.delete(:client_id)
      redirect_to clients_path
    end
  end
end

module Client
  class OrdersController < ApplicationController

    layout 'client_panel'

    def index
      render html: ClientOrders::OrdersList.build(view_context, cookies[:client_id]), layout: true
    end

  end
end

As you see, it's only setting the cookies, without any user/password checks.

This will be next story to implement, to allow for using passwords for logging in. What's interesting in this "tiny steps" approach is that we can start without even allowing for choosing the password. From the users perspective we can start with the idea of generating the passwords for the client, creating their accounts and only then (later) allowing them to choose/change passwords. That's like 3 other "stories" to implement.

As for the code, some of you may find this bit interesting:

render html: ClientOrders::OrdersList.build(view_context, cookies[:client_id]), layout: true

I went for the Rails view implementation with an unusual approach - writing the templated in Ruby instead of ERB.

But that's a topic for another blogpost.

Thanks!

Unblocking my blogging - recent progress at Arkency Ecommerce project

Andrzej Krzywda — Tue, 02 Aug 2022 10:30:54 +0000

Recently I've been struggling with a blockade in writing posts. It's not the first time nor the last one, so I learnt to accept it.

In the past what helped me is just documenting what I'm doing as blogposts. When you document you focus on what you did. It's almost like a written/async standup. When I try to teach or express opinions as a blogpost things are more difficult for me.

I'm lucky to be able to work on an open source project. Which means I work in public. It also means I can freely share the progress.

I will probably write some posts here, then move to the Arkency blog.

So as an attempt to unblock myself, over the next days I'll share a few small things I did recently on the Arkency Ecommerce project.

The project is an attempt to implement an ecommerce/order-management-system in Rails but fully using Domain Driven Design, Event Sourcing and Command Query Responsibility Segregation. If you don't know those buzzwords, don't worry. It's just difficult names, for simple things. Feel free to ask if something is not clear.

One recent small issue was open source related.

Someone asked "what is the license for this project". The problem was that in the past, this repo was just a typical Rails app with its typical structure. The LICENSE file resides in the main directory.

However, once we went further with the idea of separating the domain code from the Rails app, we changed the directory structure.

We created rails_application subdirectory and moved the rails app over there. Then in the main directory we created ecommerce directory which contains the business modules (aka bounded contexts).

The LICENSE file was moved together with the Rails app. This means that it was no longer following the convention of keeping the LICENSE file in the main directory.

✗ tree -L 1     
.
├── Makefile
├── README.md
├── ecommerce
├── hanami_application
├── infra
├── math
└── rails_application

BTW, we deliberately have the MIT license, so that people can do what they want with this codebase. This project has many educational goals. It's a bit of a challenge too, because of this. When we experiment with new ideas, some people may feel like this is what is now "recommended" and copy the experiments too. I guess that's the risk of doing things in public.

As a fix for the problem, I moved the LICENSE back to the main directory.

I was seriously considering writing a test for that. This is an important convention to keep the license in the main directory. We're usually very brave with refactorings here. This means we're at risk of regressions of such trivial things too.

What do you think?

OK, I think that's enough for an "unblocking" blogpost. I know it was a trivial example. Hopefully this unblocks me and more interesting posts are comming soon 😎

What is event-driven? Order#value example

Andrzej Krzywda — Tue, 06 Jul 2021 09:44:09 +0000

Some time ago, I have mentioned to you that there is this sample DDD/CQRS application.

https://github.com/RailsEventStore/cqrs-es-sample-with-res

We've (Arkency people + Arkademy subscribers) been working on it and it evolved into something new.

It's no longer just a sample application. It's now part of a bigger project called:

Arkency Ecommerce

Ecommerce codebase which makes developers happy.

The goal is exactly this, create such a codebase that will make developers working on it (extending, integration, customising) truly happy.

I honestly don't know if this will be a framework, or a set of libraries or a customisable scaffold or a code generator.

Yet.

What I do know is that the current popular ecommerce platforms are targeting business people (nothing wrong with that), but not always care about programmers (sad).

You either need to use some overcomplicated codebase and patch it so that it serve your custom needs or you need to make a thousand of API calls because integrating 23 different SaaS providers is apparently the way to go in 2021.

Welcome to the integration hell.

What's the alternative?

A simple, event-driven codebase, consisting of clear and mostly generic bounded contexts like: pricing, payments, ordering, inventory, catalog, crm.

A number of preset read models available either via API or via Hotwire/Stimulus approach.

A codebase which you can fork and then customize by implementing the process managers on top of it.

That's the goal of Arkency Ecommerce.

Modulith, not microservices.
Events, no coupling.
Event sourcing, no ORM.
Mutation testing coverage, no dead code or untested areas.

We want to have an easy ecommerce product line.

Are we there? Nope.

Is it useful already? Yes.

To my surprise, there are already 2 companies which are forking this repo to start their efforts on top of that. They do want to contribute back, which is awesome.

Let me explain the modularity goal.

Almost every programmer would say that modularisation is a good thing.

Having modules means having smaller scope. It means easier testing. It also means less bugs in the end.

It's not so easy to find the right boundary for the modules. Some modules are too small, some are too big. Some modules couple certain things together.
Let's say the modules are more or less "right".

What is the challenge now?

Composability

How to connect modules so that together they create working system?

Here's what I did with Arkency Ecommerce.

I split the business logic into business modules following the patter of Domain-Driven Design - Bounded Contexts.

Then I went for the Read/Write split, following the CQRS approach.

This resulted in a number of smaller modules:

Ordering
Payments
Pricing
ProductCatalog
CRM

plus some UI modules:

Orders
Products
Customers

How do they talk to each other? How do they compose together?

via EVENTS
and COMMANDS

Each module can be told what to do using commands.
Each module publishes events as a result of executing commands.

Events and commands are the way to compose modules together.

Let's look at a simple concept of Order's total value.

There are several modules involved:

Pricing needs to calculate the value
Payments needs to know how much to charge
Orders (the UI modules) needs to display it

    cqrs.subscribe(
      -> (event) { cqrs.run(Pricing::CalculateTotalValue.new(order_id: event.data.fetch(:order_id)))},
      [Ordering::OrderSubmitted])

This line of code connects Ordering with Pricing without them knowing about each other.

    cqrs.subscribe(
      -> (event) { cqrs.run(
        Payments::SetPaymentAmount.new(order_id: event.data.fetch(:order_id), amount: event.data.fetch(:amount)))},
      [Pricing::OrderTotalValueCalculated])

This line connects Pricing with Payments.

Again, without them knowing about each other.

This, my friend, is the beauty of event-driven architectures.

This is the reason I felt in love in event-driven DDD.

Such modularity, such isolation, such independence of modules - this all create robust software.

That's the foundation for Arkency Ecommerce.

You can see the event-driven flow in more details in this 5 minutes video on Arkency Youtube:

Here is the whole modules/events setup:

class Configuration
  def call(event_store, command_bus)
    cqrs = Cqrs.new(event_store, command_bus)

    Orders::Configuration.new(cqrs).call
    Ordering::Configuration.new(cqrs).call
    Pricing::Configuration.new(cqrs).call
    Payments::Configuration.new(cqrs).call
    ProductCatalog::Configuration.new(cqrs).call
    Crm::Configuration.new(cqrs).call

    cqrs.subscribe(PaymentProcess.new, [
      Ordering::OrderSubmitted,
      Ordering::OrderExpired,
      Ordering::OrderPaid,
      Payments::PaymentAuthorized,
      Payments::PaymentReleased,
    ])

    cqrs.subscribe(OrderConfirmation.new, [
      Payments::PaymentAuthorized,
      Payments::PaymentCaptured
    ])

    cqrs.subscribe(ProductCatalog::AssignPriceToProduct.new, [Pricing::PriceSet])

    cqrs.subscribe(
      -> (event) { cqrs.run(Pricing::CalculateTotalValue.new(order_id: event.data.fetch(:order_id)))},
      [Ordering::OrderSubmitted])

    cqrs.subscribe(
      -> (event) { cqrs.run(
        Payments::SetPaymentAmount.new(order_id: event.data.fetch(:order_id), amount: event.data.fetch(:amount)))},
      [Pricing::OrderTotalValueCalculated])
  end
end

Let me know if any of the concepts here sounded interesting to you - I'd be happy to explain more.

Querying data from process managers or sagas

Andrzej Krzywda — Mon, 26 Apr 2021 09:32:53 +0000

Today I received such a question:

Hello! Is there ever a time where it’s appropriate to query the read model/projections from inside a saga/process manager? For example:
I have an application/inquiry form that can be submitted by leads, and sometimes they submit it more than once. I don’t want to create multiple records for a lead if I know I already have them (say by normalized email address). My initial thought was to create a saga around the application submitted event and optionally create the lead if they don’t already exist.

This is a question I often receive recently. Here is my answer:

In general, the answer is yes, sometimes it’s hard to avoid such “queries”.
However, there are some things I’d be careful with:

Try to avoid querying an exisitng read model that is also used for a UI purpose. This comes with a risk that a small change on the UI can result on breaking the process manager (in the worst case have a test which covers such integrations)
Watch out for too big projections/read-models - certain data (a list of all leads?) can be huge one day.
Choose the peristence method wisely - for bigger sets of data, projecting from events on the fly can be slow. In that case having a read model backed by a db table is OK
Sometimes having this need points to a potential design problem (the process manager being too big, similarly as with aggregates) - I don’t think this is your case, but just a word of warning

5 days 5 blogposts - the summary of the Arkademy.dev blogging challenge

Andrzej Krzywda — Sun, 25 Apr 2021 09:09:46 +0000

5 days 5 blogposts was an idea for a challenge that appeared in my head some time ago. I never remember where ideas come from - was it my idea, someone told me about it? I don't really care - I'm fascinated by ideas regardless of the author.

Anyway, I disgress here. Which is a common problem in blogging, btw. You sit down and want to write about something and then, BAM, 100 new connections appears and you want to blog about all of them in one blogpost.

I believe in blogging as making the world a better place.

I see blogging as a way of documenting what you know, sharing what your learnt. Sometimes, just 3 sentences can "click" in someone's mind.

I don't buy the counter-argument - "there are too many blogposts already".

Nope, people are unique. The way I think (and blog) about something is unique too. There are certain people with whom my writing resonates.

I don't blog to make millions of people happy, I blog to those 20 people who get inspired/educated from my writing.

The 5 days 5 blogposts challenge was this idea that we can encourage each other to unblock in blogging. The sense of community and the bond is important here too. That's why it all became easier to organise, once Arkademy has started.

Arkademy is this community of programmers, both very very senior and also very very junior. It includes many Ruby programmers, but is not limited to any special technology. We are a welcoming community and the goal is to create a university-like environment of learning/supporting each other.

One part of Arkademy is courses (including 2 blogging courses), The other part is communication - also via Discord.

When Arkademy started - it was possible to start the blogging challenge. The community is already big enough that targeting 5-10 people was a realisting goal. At the same time it's still not too crowdy and it's easy to recognize people by their nicknames.

The first thing was a blogging conference. This consisted of 4 talks and was like a preludium to the actual challenge. The Arkency bloggers + Mira (English expert) shared their tips during online talks.

It's worth wathing this talk if you're worried about your English.

After the conference, we have started the challenge.

During the first day, I was a bit worried - what if no one joins?

It was a great relief to see the first "submissions". We've used one "Blogging" Discord channel. My goal was to review and motivate in the first days of the challenge. Somehow it happened that I joined the actual challenge too. It was nice to see also that Paweł from Arkency has joined the challenge too.

You can see all/most blogposts grouped here: http://5days5blogposts.arkademy.dev/

Most of the people who joined the challenge are people who either never blogged or tried but got stuck and didn't blog for a long time.

It was so nice to see how they unblock in writing. It was also very motivating how they overcome all the possible problems:

I hate my writing
noone is going to read this
I'm tired, I blogged already yesterday
I have nothing to write about

But somehom most of us stayed till the end.

As you can see, the format of the blogposts was varying a lot.

Some of my lessons here:

It's hard to write a tutorial-like high-quality blogpost every day.
Perfectionism is a hard opponent
Showing code makes blogging much easier
Sharing opinons may be controversial but is easy to write as you stay true to yourself
Maintaing the persona of who you write for helps
Keeping a backlog of blogging ideas helps people
Trying to use my strengths (somehow simulated by for example Gallup talents) is a good idea, as it helps me with focus
People like reading my opinions
A little clickbait in the title can help

What helped me the most, though was the "team effort".

I'm an indivudual but I love being part of the team where we can support each other. That's why the success of the Arkency blog (listed as top-10 ruby blogs in the world) - we help each other, we trust each other, we embrace our differences.

During the challenge, at first I wasn't sure if the "team effect" could help that much. We didn't really know each other that well.

In the end, it's the group that helped me finish the challenge. In the moments where I wanted to give up it was motivating to see Discord notifications that Mikołaj, Anna, Piotr, Manuel and others have posted their blogposts.

My Gallup's Maximizer (the talent to see a true potential in other people) is very happy. I've seen people transitioning from not bloggging at all to be almost addicted to blogging after only 5 days.

This is the kind of challenge I dreamt of.

Thank you to all the people who started the Arkademy Blogging Challenge. You're great.

I'm already motivated to prepare and organize other such challenges at Arkademy.

Some ideas:

work on a fresh Rails app for 5 days
implementing typical ecommerce challenge as a programming kata
designing a good architecture for existing systems - architectural kata
React component a day
Improving test coverage via mutation testing

But that's a topic for another blogpost.

Thanks for reading!

Fixing the node-sass problem in Rails - node downgrade helps

Andrzej Krzywda — Fri, 23 Apr 2021 20:42:05 +0000

Today I've had one of those irritating problems with Rails that I know one day I will have again. So, future me when you google it, here is the fix, you're welcome :)

The problem shows like this during rails new new_app

node_modules/node-sass: Command failed.

and

1 error generated.
make: *** [Release/obj.target/binding/src/binding.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/Users/andrzej/apps/stimulus_infinite_scroll/node_modules/node-gyp/lib/build.js:262:23)
gyp ERR! stack     at ChildProcess.emit (node:events:365:28)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
gyp ERR! System Darwin 19.6.0
gyp ERR! command "/usr/local/Cellar/node/16.0.0/bin/node" "/Users/andrzej/apps/stimulus_infinite_scroll/node_modules/node-gyp/bin/node-gyp.js" "rebuild" "--verbose" "--libsass_ext=" "--libsass_cflags=" "--libsass_ldflags=" "--libsass_library="
gyp ERR! cwd /Users/andrzej/apps/stimulus_infinite_scroll/node_modules/node-sass
gyp ERR! node -v v16.0.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok

As you can see, the node version used is 16.0.0. At the moment of writing this post, it's a relatively new version and not everything works fine with this.

The solution? Downgrade node.

In my case I did:

npm install -g n
sudo n 14

and that's it.

A possible alternative:

brew remove node
brew install node@14

If this helped you, let me know in the comments, so that I believe in value of such posts even more ;)

From Rails scaffold listing to Hotwire infinite scroll

Andrzej Krzywda — Thu, 22 Apr 2021 19:06:01 +0000

Rails scaffold is a technique for quickly generating a typical CRUD UI.

It's a server-side rendered html which allows listing/editing/creating/deleting records.

One of the promises of the modern approach to building UI (like hotwire or stimulus reflex) is how easy it is to just tweak the backend logic, without using JavaScript at all.

Let's look at the example of infinite scroll - full tutorial how to do it is here and here.

I just want to focus on the "diff" how to make it work.

This is a typical Rails scaffold preparing all the posts to be displayed.

  def index
    @posts = Post.all
  end

This is the equivalent Rails view (rendering html).

  <% @posts.each do |post| %>
    <div>
      <h1><%= post.title %></h1>
      <p><%= post.body %></p>
    </div>
  <% end %>

Now, in order to use Hotwire (actually Turbo Frame) we change the controller to this:

  def index
    @page = params[:page] ? params[:page].to_i : 1
    offset = (@page - 1) * PER_PAGE
    @posts = Post.offset(offset).limit(PER_PAGE)
    @next_page = @page + 1 if @posts.size == PER_PAGE
  end

In short, we pass 3 parameters to the view: page, posts, next_page.

And the view changes to this:

<%= turbo_frame_tag "posts_#{@page}" do %>
  <% @posts.each do |post| %>
    <div>
      <h1><%= post.title %></h1>
      <p><%= post.body %></p>
    </div>
  <% end %>

  <% if @next_page %>
    <%= turbo_frame_tag "posts_#{@next_page}", loading: :lazy, src: posts_path(page: @next_page) %>
  <% end %>
<% end %>

We wrap the whole thing with a turbo_frame_tag and we append more such frames for next pages. That's it.

As you can see the middle of the view stayed the same.

The UI now lists posts and keeps appending them when we scroll down.

I'm not claiming that it's amazing or something. But the practicality of getting our app more interactive, while not jumping to JavaScript is simple yet powerful.

I like it as it allows me to gradually extend my app with new features.

If you like audio to educate yourself then I recommend this podcast to learn more how it works:

https://www.codewithjason.com/rails-with-jason-podcast/episodes/092-vladimir-dementyev-5fK__ZQf/

E-commerce modules worth extracting in the code

Andrzej Krzywda — Wed, 21 Apr 2021 11:58:36 +0000

When you work on a non-trivial e-commerce application the complexity is usually, well non-trivial too.

E-commerce is one of this kind of apps, where just following a CRUD approach may not be enough.

Here is a list of modules (definitely not a full list) that might be worth having as separate, in order to avoid coupling.

Ordering

This is a classic one. If you have no modules in your e-commerce code then you can call it Ordering anyway, because that's the most important.
Some possible operations (commands) here:

initiate an order (usually with some items from the cart)
confirm the order (so that other modules are aware)

Ecommerce

I recently like to call this Ecommerce - a module which contains the logic around the cart concept.

Catalog

This module allows adding products to the offer visible to the clients.

Inventory

Keep track of how many items are left. Sometimes booking too.

Pricing

Pricing is usually not as trivial as just assigning a price to one product. USually we have a whole range of pricing strategies that change over time. It's nice to encapsulate them and keep it separate from other modules.

Payments

Keep track of what was paid for. Sometimes your order will be paid in multiple payments, sometimes it's not fully paid. Sometimes you pay for multiple orders with one payment - hadle this logic here.

Sample application with ecommerce modules

I'm working on sample application which shows some of those concepts. It started as a ordering system, but now it's in a process of becoming an e-commerce app too.

https://github.com/RailsEventStore/cqrs-es-sample-with-res

Are there any other modules that you like to extract in ecommerce apps? Share them in the comments :)