DEV Community: Antonio

How to create big aggregates in DDD

Antonio — Tue, 10 May 2022 13:29:50 +0000

Hi! My name is Antonio and I have been reading about DDD for quite some time. I think Domain Driven Design is the right tool for some enterprise applications, so recently I have been trying to use it on my company.

Before continuing reading, I'm assuming you have a good knowledge about DDD and related concepts (sorry for not including an introduction, but I think there are already too many introductory articles about DDD, so I don't feel like writing another one).

Problem

So, what problem am I facing with DDD? Big aggregates implementation (emphasis on implementation and not design). When I say big, I do not mean they contain a lot of different entities or a lot of dependencies, but many instances of the same entity. For example, a bank account aggregate has one child entity: a transaction. Now, that bank aggregate can have hundreds or thousands instances of that entity.

Let's suppose that my company domain is about Roads and Stops (this is just an example). Both things are entities because they have an identity. In this case, Road would be the root aggregate and Stop would be a child entity of that aggregate. Let's say they have two or three fields each, it does not really matter. Here is a quick implementation of that model in Python (I have not used dataclasses and a lot of the logic is missing because it's not important for this discussion):

class Road:
    id: int
    name: str
    stops: [Stop]
    ...

class Stop:
    id: int
    latitude: int
    longitude: int
    ...

So now, you need to create a repository to retrieve those entities from storage. That's easy enough, just a couple of SQL queries or reading a file or whatever you want to choose. Let's suppose this is our repository (let's avoid interfaces, dependency injection and so on because it's not relevant in this case):


class RoadRepository:
     def get(id: int) -> Road:
         ...
     def save(road: Road) -> None:
         ...

Easy enough, right? Okay, let's continue implementing our model. The get method is really easy, but the save method has a lot of hidden complexity. Let's suppose we are using a relational database like postgres to store our entities. Let's say we have two tables: roads and stops and they have a relationship and so on.

In order to implement the save method, we would need to update all of our child entities. And that's the problem. What happens if our Road instance has 345 different stops? How do we update them? I have not a final answer for that, but I have some proposals!

Solution 1

This would be the equivalent of solving the problem by brute force: delete everything and recreate it again.

Props

Easy to implement

Cons

Not sure about the efficiency of this one. but I estimate is not that good.
If you set the unique identifiers on the database level, you are going to have a problem keeping the same identifiers.

Solution 2

Keep track of all the changes at the aggregate level. Something like this:

class Road:
    id: int
    name: str
    stops: [Stop]

    def update_stop(self, stop: Stop):
        ... some logic to update the list ...
        self._changes.append({
           'type': 'UPDATE',
           'stop': stop,
        })

Then we would read that list of changes on the repository and apply them individually (or in bulk, depending on the change type, for instance, we can group together the deletions, creations, etc.).

Props

It's more efficient than the first solution because in average requires less db operations.

Cons

Our domain has been contaminated with logic not related to the business.
A lot of code is necessary to keep track of the changes.

Time to discuss!

What do you think about this problem? Have you faced it before? Do you have any additional solutions? Please comment it and we can discuss it :)

Work report. Week 8

Antonio — Wed, 24 Jul 2019 08:28:51 +0000

Welcome to a new work report! Sorry for being two days late, I've had a lot on my plate these days. Let's see what happened this week :D.

P6DOC

I am not sure if you know this tool: p6doc. It's being improved by one of my partners in #gsoc2019, Joel. It's a command line interface to consult the Official Perl6 Documentation. Something similar to man but for Perl6!

I tested it out this week and I thought it would be nice to integrate my project with that tool to improve the consulting time. So I write down my ideas in a issue and Joel and I are currently discussing this new feature!

In a few days some new changes will come (see next branch), so be alert!

GitHub Templates

I did not realize I did not have any template to use by issues or pull requests, so I have added them to the repository :D. I also have realized that perl6/doc was using the old template system so I made an issue to update it! One of the things I am enjoying the most about #gsoc19 is the huge amount of things that I am learning about git and github. There are so many features to help you develop your ideas like projects, milestones, etc.

Big TODO list

A few days ago, my mentor revised my code and suggested a lot of improvements on this TODO list. It's kind of long, but I am trying to complete it item by item.

A lot of them have already been made, but others, like generate documents in parallel, has proved impossible to me to resolve. It turns out that pod2html (used to convert all pods to HTML) cannot be executed in parallel. I tried it by all means without any results. I have reported this problem, but as you can see the error is a segfault. So this looks like a bug in Rakudo rather than the module itself. Debug this problem is the definition of hell, there are so many functions being called, several threads, etc. Hence, generate the documents in parallel will have to wait until this bug is resolved or a new HTML module is developed.

I have also refactored and deleted some modules, to improve the readability and understanding of the process. I have also made a new documentation under docs/reference explaining how the things are made. I did this because the previous docs were too much technical and it was hard for people to see the big picture. I hope that with this approach, the system will be easier to understand.

And that's has been my work this week. It does not look a lot, but the refactor and some items of the TODO list have caused several problems because some things were highly coupled (my fault). Now everything is more modular and less prone to errors.

Perl6 goodies!

Now is when I tell you something interesting about perl6! If you are a programmer, you are bound to work with Arrays, Lists, etc. Perl6 gives you everything you need to work with these data structures very easily.

One function I use a lot is grep (see the docs). This function is like a filter. You can filter by type, using regexes, functions, everything you can think of. Let's see some examples.

say ('hello', 1, 22/7, 42, 'world').grep: Int; 
# OUTPUT: «(1 42)␤» 

say ('hello', 1, 22/7, 42, 'world').grep: Rat; 
# OUTPUT: «(3.142857)␤» 

say ('hello', 1, 22/7, 42, 'world').grep: {.Str.chars > 3};
# OUTPUT: «(hello 3.142857 world)␤» 
~~~

![simply perl6](https://thepracticaldev.s3.amazonaws.com/i/x6smko21cv1tovbchlrc.jpg)

You will see how amazing is this once you start using it.

### Closing

And that's have been everything for today. I hope you give Perl6 a try and start using these amazing features.

See you next week with a new report and a new thing about Perl6! :D.

Work report, week 7.

Antonio — Mon, 15 Jul 2019 14:39:27 +0000

New feautres

Here I am another week! This week I have improving a lot of aspects of Perl6::Documentable:

Now it has a very useful update option to only regenerate the HTML documents coming from pod6 files that have been actually modified. This is a characteristic in the documentation's wishlist. Before this feature, if you wanted to see a change in the HTML you had to regenerate everything, what takes 2:30' approximately, plus processing the pod collection again. Now, with the cache and update option, it only takes around 10 seconds to see a change (and half of that time is spent is reprocessing the collection).
I have also made some little refactoring to distribute the responsibilities in a better way. In addition, you will find a new setup option. The aim of this feature is to initialize the directory where the documentation will be generated. What needs to be setup? Nothing, really, but if you want your freshly generated documentation to have an style sheet, search capabilities, etc., you need to download the necessary files or provide ones yourself.
Some minor bug fixes like: leading whitespaces files or the correct detection of subkinds.
CI testing! This is an important change because I made a terrible mistake! I did not test if the module actually could be installed, and it could not! Thanks to @ugexe for noticing this and tell me. I do not know if you remember Perl6::LinkHealth and the mini-doc repository, but they have also been integrated in this CI testing so now almost everything is covered :D.

And that's all the work I have made this week, if you want me to introduce whatever new feature you find interesting or useful, just tell me!

Next steps

Mmm, what should I do next? Perl6::Documentable is in a good state right now, so I will continue fixing bugs and make general improvements.

Nonetheless, I also want to start the work with Pod::To::HTML and Pod::To::BigPage. What needs to be done? Well, currently indexing is made in three different ways so it would be nice if we unify them into one. Moreover, the last two modules use different ways to convert a pod file into html, so it does not make a lot of sense to have duplicated logic. I may even create only one module joining both functionalities. I still do not know, I still have to discuss this with my mentors!

Perl6 curiosity

I have been thinking these weeks that write an article telling what I have done in a specific week is not that funny, so I will tell you about some characteristic of Perl6 that I have found useful and interesting.

Today: CLI! This week, I have had to add different options to the CLI version of Perl6::Documentable, and I thought this would be a little tedious, but I could not be more mistaken! As always, Perl6 makes your life easier and provides you with a very simple way of doing it.

Suppose you want to create your own CLI program, for instance, one that reproduces the zef command. First you should create a new file containing that functionality, let's call it Perl6::Zef. Now you need to create a new file where you will read the arguments the user has specified, Perl6::Zef::CLI.

This new file would be something like this:

Oh, what a weird code!. It really is not that weird, we only define "the skeleton" of the MAIN function (this function is called every time a file is executed) because it will have several different definitions.

After that, we simply specify two definitions of MAIN, but wait, you have written install and update as arguments, is that correct? YEP, in Perl6 this compiles and works. This is an amazing feature because it gives you the possibility to deal with the cases you need in an easy way.

I have also written #| and #=. These two are called declarator blocks. What does this kind of comment? The comment you write after that is attached to the source code below it. Why this is useful? Because, for instance, with those comments, you do not need to specify a --help option or something alike, because if you try to use this module with incorrect arguments, perl6 will tell you this:

$ > perl6 zef.pm6

Usage:
  zef.pm6 [--name=<install>] -- Install a new package
  zef.pm6 [--name=<update>] -- Updates a package

    --name=<install>    Name of the package to install
    --name=<update>     Name of the package to update

~~~

![simply perl6](https://i.pinimg.com/originals/7b/4f/6a/7b4f6a993a37feba36efd20910d03920.jpg)

I know, quite handy, isn't it? :D.

Summing up, I hope you try give a change to Perl6 because it has a lot of incredible characteristics!

See you next Monday :DDD.

GSoC report. Week 6

Antonio — Mon, 08 Jul 2019 10:20:01 +0000

First release is coming!

Here I am another week! This week I have made a lot of progress in the project! Perl6::Documentable now has a command line version with a lot of options to customize the build process or only generate the parts you want to.

I also want to release the 1.0.0 version of this module`. In order to that, I made a project in GitHub, to manage the related issues, where you can see a detailed report of my work this week.

But, why do I want to make a release? Because now it only contains the current logic, without any changes (well, there are some, but they do not change the final result), including the errors. So now it's the time to fix those errors, modify the current behavior and start to keep track of the changes. When that's done, we will have a stable system where we will be able to move towards the new documentation system.

Those errors/problems are described here and here. You can see there is a considerable amount of them. My intention is to fix them while I upgrade the initial version, releasing several patches (1.x.x).

In addition, I have started a wiki in the repo to explain more deeply some functionality, because in the README I have only added the API docs. At this moment it only contains one post about how Perl6::Documentable::Registry is initialized and what kinds of Perl6::Documentable objects you will find there. I will complete it with the following pages:

Working with the registry: some examples about how to use lookup or grouped-by to fully understand what's going on under the hoods.
Attributes: if you take a look at Perl6::Documentable you will see a lot of strange attributes like categories, kind, subkinds, etc. In the API docs is described how they are set but not what they mean or why they are chosen like that.

And that's all for this week! Among others things, the results of the Perl 6 annual survey has already been published here, check them out!

Thanks for reading

Don't be shy, let me know what you think of this project in the comments ☺️.

Work report! Week 5

Antonio — Mon, 01 Jul 2019 19:20:47 +0000

Here I am again! This week I have been a little busy with personal matters so I have not been able to work as hard as expected. Anyway, quite a few progress has been made.

Spinf Off Pod::Convenience issue #2696

This module is used at a lot of different files in perl6/doc. Its main aim is to provide functions to work with Pod objects: create them, modify them, etc. I have spun it off in Pod::Utilities, where code have been split between functions that modify pod structure and functions which create Pod objects.

Pod::To::Cached

As you can see in some issues ( #1952, #717, ... ), we really need a cache system to decrease testing and build time. Parse all pods to get Perl6 code takes a while and currently this is being made around three times in the process.

What we want with a cache system is something like this:

That means, compiles the pod once and use them everywhere. Luckily, @finanalyst made a module to compile and cache pod files => Pod::Cached.

Last Friday, my mentor, @JJ, and I were taking a look to how it was made and solving some issues (you can see my handsome face here).

Mini-doc set

Perl6::Documentable was published last week with all processing related logic so I already know almost all things being considered to generate the doc site.

Because of that, I have updated the doc set in the mini-doc repo, adding a Feature check list to the README.md (suggested by @jj), containing all things that should be present in that set.

Closing

Ant that has been all! I know this week has not been so productive but well, some work has been done. This week I will be working on:

Add index generation logic to Perl6::Documentable.
Start and hopefully finish the new module responsible of taking a Perl6::Documentable::Registry and generate the HTML files needed for the site.

First work report of GSoC!

Antonio — Mon, 24 Jun 2019 09:52:13 +0000

I'm back, as promised! Time to talk about what I have been doing all this time. Three weeks have passed since the coding period started and some work has been done since then:

Perl6::LinkHealth repo

This was my first module! I tried to do a basic tool to check if some files stopped being generated. It needs improvement and it's not finished yet, but my mentor recommended me to leave this project aside for a while.

Mini doc repository issue #2529

As you can see in the doc directory of Perl6 documentation repo, there are 383 pod files approximately. It takes quite a long time (~20 minutes) to process them all and build the doc site. This leads to one problem: if you want to make any change to the build process, first you need to follow these easy steps:

Code the change.
Build the site waiting 20 minutes.
Yep in this step you are still waiting.
See that you forgot a semicolon, come back to step 1.

Quite frustrating, isn't it? So, in order to change this, as discussed in issue #2529, a mini doc repository would be quite helpful. So I create it here, but, once the environment was set up and I started to search for the correct self-contained subset of pod6 files, I realized I did not know what needed to be in it! So to discover what was necessary I did the following:

Spin off Perl6::Documentable Issue #1937

In order to know what we need, first we need to know what it's being used and how. So, how are pod files being processed and where?

First they are read and parsed, then we get a data structure of Pod::* objects in an array. These objects contain the text written in the pod files, but to generate indexes and have a search system, an additional layer is necessary. That's where Perl6::Documentable and Perl6::Registry (source code) take part. A Documentable object is whatever piece of a pod file that is documented, but, what's being documented? Several things:

Pod files: every pod file in the doc directory is represented by a Documentable object with the following attributes:
- $name: By default is set to the filename. If the pod is representing a type, then it's set to the last word of the filename. Otherwise, it's set to the content of =TITLE.
- $url: This is set to /$kind/$link. $kind is the name of the folder where the pod file is allocated and $link can be set to the value you want adding link<your-link> just after of =begin pod. If you don't, then it's set to the filename.
- $pod: array containing Pod::* objects with the data.
- $summary: value of =SUBTITLE.
- $kind: is the name of the folder where the pod file is allocated (Language, Type or Programs).
- $subkinds and $categories: If it's a type, set to the value obtained from Perl6::Typegraph. If not, then $subkind is set to class and $categories is not set.
- pod-is-complete: Set to true because they represent an entire pod file.

When all of this is set, the new Documentable object is added to Perl6::Registry, which is that exactly, a registry where all Documentable objects are stored to be used later.

Next thing that needs to be done is search for definitions to be indexed. This a key part of the process because this definitions will be used to create TOCs, a search file for the site and intra-links. Let's start explaining what is a definition: a definition is a header, more precisely, a Pod::Heading object. They are something like this: =head2 Some text (the heading level does not matter). But as you can suppose, not all headings are valid definitions. There are two different types of valid ones: ambiguous and unambiguous. You can check more info about them here. Every new found definition is stored in a Documentable object and added to the registry.

But there's more: references. A reference is a X<> element. All and each one of X<> elements are processed as references, stored in a Documentable object and added to the registry as well.

Well, all this work is made in htmlify.p6 by three different funtions:

If you take a look at that functions, you will see that global variables are involved. Moreover, that code could be parallel (currently is parallel but the number of threads is set to 1). And, as you may know, global variables and parallelism, need monitors or some alike exclusion mechanism. It looks like monitors are not working as expected (by that reason the numbers of threads is set to 1). To fix that, I have moved all the logic described here to the Perl6::Documentable class, where there is not any global variable so no monitors are needed. You can check the new module here.

Perl6::Typegraph Issue #2573

I spinned this module to test it and document it. In addition, we want to get rid of the type-graph.txt file and get all the information necessary through introspection. This file should be added to that module too.

Pod::Load repo

This module is going to be used to read every pod file in the documentation, so I have been using it and finding some bugs.

esLibre19

Three days ago was #esLibre19! A congress about open source where I hosted a devroom about Perl and Perl6!

What's next?

This week I want to focus in several things:

Publish and polished Perl6::Documentable.
Spin off Pod::Convenience, which will be renamed to Pod::Utilities. In addition, all scattered functions related to this topic found in the repo, will be moved there to keep consistency.
Discuss where to move logic that reads and initializes the pod and the registry (it's not trivial because in the next weeks a cache system will be made and that logic will use it).
Create Perl6::RegistryToDoc module, where the rest of the logic of htmilify.p6 will be moved.

I have not explained several things, but if you go to the repository of each module, you will find a documentation explaining all the necessary!

And that's all for this week! See you next week, hopefully :D.

GSoC 2019!

Antonio — Wed, 19 Jun 2019 20:21:33 +0000

About me

My name is Antonio Gámiz, I'm a student at the University of Granada, where I'm studying a double degree in Computer Science and Mathematics. I have been programming in quite a few languages like JS, Python, C/C++ and Java, but I had never used Perl6 before!

I have been selected to work at the Perl6 organization! My mentors for this project are JJ Merelo (my main mentor), Richard Hainsworth, Tom Browder and Patrick Spek.

Brief introduction

First of all I need to tell you what is Perl6: is a language entirely made by the community and completely open source! So all the work has been made by people willing to improve it and support it. The community is very friendly and is always welcoming to new members willing to learn (you can join us in the IRC channel).

But why it's so cool? Because it unifies many great ideas that aren't usually found in other programming languages (quote from here). In addition, Perl6 is multi-paradigm! That means you can learn OOP at the same time you play with FP and a lot of more like declarative or imperative paradigms!

In addition, from my point of view, it's one of the best scripting languages at the moment, because it has a wide set of tools to make your life easier. Nonetheless, at the beginning, you can feel a bit scared of the syntax, but if you use it for a while, you will discover how amazing and powerful it is!

So what is your project?

Well, as I have said earlier, Perl6 is great, but in order to learn this incredible language you need an even better documentation. In Perl6, the documentation is also code and there is a special syntax to write i t, called Pod6 format. This is an example:

=begin pod

=TITLE Perl6

=head1 First article

Some comments

=end pod

Official documentation is entirely written using this format (you can see the source code here) and then is processed by a huge and scaring file: htmlify.p6 (this one) and a few modules. All of this code have been developed and maintained by the community along the past years, patching the problems whenever they appear and hoping anything stopped working.

So here is where I take part! My project aims to document the generation process, test it and spin off all the modules involved in the process. This is necessary due to there is a high level of coupling between the parts involved and it's quite hard to make changes by two reasons:

You cannot know if the changes you make will break some parts of the system because there is not enough test coverage.
If you want to change a specific behavior, you do not know where to modify it because the same task is made along several different files.

So, when all the process is documented, tested and we know why things are being made the way they are, will be time to create a doc specification. As my main mentor, JJ, said in this post, we need to settle on a specification of the document page format first. If we do that, generate the docs will be a lot easier because all documents will have the same format and there will not be strange situations prone to errors.

And that's the main part of my project, feel free to ask me anything about it!

Coming soon

I will be keeping a weekly blog explaining my work during that week and why do what I do. I will publish the first one next Monday, where I will talk about the work made these past weeks.

Closing

I know this article is a bit late but, in Spain, university tests are around this time and I have been terribly busy studying for them. Luckily, I have already finished and I'm totally free the rest of the summer! (huge thanks to my mentors, who have been very understanding with this problem).

Also, in two days is esLibre, a Spanish meetup where amazing people will be talking about open source! I will be hosting a devroom about Perl6 and Perl. Feel free to stop by and learn something new :).

See you again next Monday!