DEV Community: Yota Hamada

Mastering DDD: Repository Design Patterns in Go

Yota Hamada — Tue, 02 Jan 2024 05:45:00 +0000

This article aims to deepen the understanding of "Repository", a core concept in Domain-Driven Design (DDD). We will explore the fundamental roles and importance of a repository and present an example of its implementation in the Go language.

Prerequisites

Assumes an application that retrieves (and updates) data from a relational database.
Sample code is written in Go.

What is a Repository?

Let's start by looking at the definition of a repository.

Repository Pattern:

A repository maps the data retrieved from the database into a structure and provides an interface to access domain objects.

This aligns with the general understanding of a repository. Next, let's look at a more detailed definition in the context of DDD.

A More Detailed Definition of Repository

Check out the DDD Reference. This reference is based on Eric Evans's book on DDD, summarizing various DDD terms. On page 17, it states:

Repository Definition (DDD Reference):

Query access to aggregates expressed in the ubiquitous language.

Proliferation of traversable associations used only for finding things muddles the model. In mature models, queries often express domain concepts. Yet queries can cause problems.
The sheer technical complexity of applying most database access infrastructure quickly swamps the client code, which leads developers to dumb-down the domain layer, which makes the model irrelevant.

A query framework may encapsulate most of that technical complexity, enabling developers to pull the exact data they need from the database in a more automated or declarative way, but that only solves part of the problem.

Unconstrained queries may pull specific fields from objects, breaching encapsulation, or instantiate a few specific objects from the interior of an aggregate, blindsiding the aggregate root and making it impossible for these objects to enforce the rules of the domain model. Domain logic moves into queries and application layer code, and the entities and value objects become mere data containers.

The Role of Repository

The role of a repository is to provide access to aggregates. The aspect of "executing SQL to fill a structure with data and return it" is just a superficial facet of a repository.

A repository always exists in tandem with an aggregate. Therefore, a correct understanding of aggregates is essential for practicing DDD.

What is an Aggregate?

Let's look into the term "aggregate". What is an aggregate? On page 16 of the DDD Reference, just above the repository section, there's an explanation of aggregates.

Definition of Aggregate (DDD Reference):

Cluster the entities and value objects into aggregates and define boundaries around each. Choose one entity to be the root of each aggregate, and allow external objects to hold references to the root only (references to internal members passed out for use within a single operation only). Define properties and invariants for the aggregate as a whole and give enforcement responsibility to the root or some designated framework mechanism.

Use the same aggregate boundaries to govern transactions and distribution.

Within an aggregate boundary, apply consistency rules synchronously. Across boundaries, handle updates asynchronously.

Keep an aggregate together on one server. Allow different aggregates to be distributed among nodes.

What is an Entity?

An entity is also defined in the DDD Reference. An entity is a domain object uniquely identifiable by an ID. In a domain model, there are entities and value objects (Value Object), the latter not having an ID.

Aggregate ≠ "Model"

The term "model" has been used in application development for a long time, but its meaning varies depending on who uses it and the context. In the past, in web application development, "model" meant the model in the MVC (Model-View-Controller) pattern.

Have you heard of Active Record? It's a design pattern adopted by Ruby on Rails and ORM libraries. In Active Record, a "model" holds data for one record of a table.

In Ruby on Rails, classes implementing Active Record are called "models" and are placed in the models/ path. Influenced by Rails, in application development, "model" usually refers to an object holding data for one row of a table.

Aggregate in DDD is different from Ruby on Rails's "model"

As mentioned, an aggregate in DDD can hold data not only for the root entity but also for other entities. Therefore, an aggregate is not a 1:1 relationship with a table but a 1:n relationship (n ≥ 1). An aggregate in DDD is not just for manipulating one record of a table.

An aggregate is a domain model. Even if composed of a single entity, please consider it a type of aggregate. A repository provides an interface to access aggregates.

Entities Composing an Aggregate

We know an aggregate comprises one or more entities. So, how should we decide which entities compose an aggregate?

The DDD Reference states:

An aggregate defines a unit that sets the attributes (properties) and invariants, responsible for maintaining these.

In other words, an aggregate is a unit that defines the attributes of a domain model and is responsible for maintaining the invariants demanded by that domain model.

Invariants are essentially relationships between data that must always be maintained. That's data consistency. To maintain data consistency (invariants), updates must occur within the same transaction. This forms the basis of an aggregate. In summary:

Entities Composing an Aggregate:
These are clusters of entities and value objects that should be treated as a unit in maintaining database data consistency (invariants). One of these entities becomes the root of the aggregate. Other aggregates can only hold references to the root entity, not the encapsulated entities within an aggregate.

Boundaries of an Aggregate

There's no silver bullet for correctly determining the boundaries of an aggregate. Properly setting aggregate boundaries requires consideration of database design, scalability, and domain knowledge. It's more challenging in complex domains, like inventory management systems.

Rule of Thumb:
Keep the boundaries of an aggregate as small as possible. This is to reduce the size of database transactions and, in turn, reduce the technical debt of the domain model.

Minimizing Aggregate Boundaries

How can we keep aggregate boundaries small? To do this, we need to consider it from the stage of table design.

The boundaries of an aggregate will become the boundaries for asynchronous processing when the service or product scales in the future. Therefore, you must decide boundaries such that data inconsistency doesn't occur even when transactions are split. Sometimes, tables need to be separated in unexpected ways.

As aggregates grow larger, the number of tables updated in synchronous transactions increases, leading to performance degradation and maintenance issues. Even if it seems that entities should be updated in one transaction, they might be separable upon closer examination. It's crucial not to be constrained by preconceptions.

Note:
When splitting transactions at aggregate boundaries, consider the implications of partial transaction failures. Ideally, a single operation in the user interface should not span multiple aggregate boundaries. Since aggregate boundaries are deeply intertwined with the user interface, it's important to align understanding with product managers and designers early in the process.

Example of a Shopping Cart

Let's consider the interface of a shopping cart for an e-commerce site. Assume there are tables for the shopping cart and cart items. Application layer use case X utilizes the data of the shopping cart and cart items (e.g., add a cart item into a shopping cart).

ER Diagram

Non-DDD Repository

The interface for a non-DDD approach would look like this, with a one-to-one correspondence between tables and repositories.

Repository Interface:
Repositories are defined for the ShoppingCart and CartItem tables.

package repository

type ShoppingCart interface {
  GetByUserID(uuid.UUID) (*model.ShoppingCart, error)
  Insert(*model.ShoppingCart) error
  Update(*model.ShoppingCart) error
}

type CartItem interface {
  GetByShoppingCartID(id uuid.UUID) ([]*CartItem, error)
  Insert(*model.CartItem) error
  Update(*model.CartItem) error
}

Model Definition:

Structures representing one record of each table are defined as models. Without interfaces, domain logic cannot be encapsulated

package model

type ShoppingCart struct {
  ID uuid.UUID
  UserID uuid.UUID
  Status model.ShoppingCartStatus
}

type CartItem struct {
  ID uuid.UUID
  ProductID string
  Quantity int
}

Potential Issues:

This approach has the following potential issues:

Models become mere containers for table records.
Domain logic is implemented in use case X.
Aggregate invariants are ensured at the application layer.
Use case X needs to consider new and updated database operations.

DDD-Compliant Repository Example

Now, let's design a shopping cart repository following DDD principles.

The root of the aggregate is the ShoppingCart entity. CartItem entity is encapsulated within the ShoppingCart aggregate.

First, define a repository corresponding to the ShoppingCart aggregate. Unlike the previous example, there is no CartItem repository. This is because repositories are defined per aggregate, not per table. The ShoppingCart's AddItem method is an example of domain logic that is used by use case X.

Class Diagram:

Repository Interface:

The repository encapsulates the process of reading and updating aggregates, so there's usually no need to separate Insert and Update. The repository takes responsibility for this in the Save method.

package domain

type ShoppingCartRepository interface {
  GetByUserID(uuid.UUID) (ShoppingCart, error)
  Save(ShoppingCart) error
}

Aggregate Model Definition:

The root of the aggregate is the ShoppingCart entity. Aggregates encapsulate domain logic, so they are defined as interfaces, not structures, and are defined in the domain package. Here's the aggregate interface definition:

package domain

type ShoppingCart interface {
  ID() uuid.UUID
  AddItem(Product Product, Quantity int) error
}

Improved Points:

The repository can fully encapsulate database complexity (insertion/update decisions).
No need to create a repository for each table, reducing redundant SQL coding.
By interfacing models, domain logic can be encapsulated.
Domain logic and database complexity are eliminated from the application layer.

Adopting DDD clarifies the responsibilities of domain logic, database access, and the application layer. This improvement reduces application complexity and cognitive load, enhancing development speed over time.

Repository and Aggregate Implementation

Should Aggregate and Repository Implementations be Separated?

Where should the implementations of the model and repository interfaces be placed? There's no definitive answer, but I personally believe it's preferable to place the implementations of repositories and models in the same package.

Reasons for Defining Repository and Aggregate Model Implementations in the Same Package:

The role of the repository was to encapsulate the complexity of database access, creation, and saving of aggregates. The repository needs to initialize attributes for instantiating the aggregate. Since it references and updates attributes not exposed in the interface, it needs to be defined in the same package.

For example, consider the Save method implementation of the repository. In the following implementation, the decision to register new or update in the database is encapsulated.

package shoppingcart

// Struct implementing domain.ShoppingCartRepository
type repositoryImpl struct {}

// Struct implementing domain.ShoppingCart
type shoppingCartImpl struct {
  ID uuid.UUID
  // other properties...
}

func (r *repositoryImpl) Save(model domain.ShoppingCart) error {
  instance := model.(*shoppingCartImpl)
  if instance.ID == uuid.Nil {
    // If ID is nil, it's a new registration
    return r.insert(instance)
  }
  return r.update(instance)
}

Next, consider the GetByUserID method implementation of the repository. The repository can return an empty shopping cart if no data exists. Regardless of whether the aggregate exists, the right instance of the aggregate is returned, simplifying the repository interface.

package shoppingcart

func (r *repositoryImpl) GetByUserID(userID uuid.UUID) (domain.ShoppingCart, error) {
  data, err := r.findByUserID(userID)
  if errors.Is(err, sql.ErrNotFound) {
    // Return an empty shopping cart aggregate if no data exists
    return newEmptyShoppingCart(), nil
  }
  if err != nil {
    return nil, err
  }
  // Create an aggregate using the data of the already saved shopping cart
  return newShoppngCart(data), nil
}

Deep Module: The Importance of a Simple Interface

Deep Module is a concept explained in the book A Philosophy of Software Design by a Stanford University professor. Deep Module refers to a module that has a simple and narrow interface on the surface but contains rich functionality and complexity internally. Conversely, modules with complex interfaces but little internal functionality are called Shallow Modules. Deep Modules excel in low cognitive load, high reusability, and ease of understanding. For example, Go language's net/http package is a simple interface but encapsulates many features and complexities for implementing an HTTP server, making it easy to use. The file system of an OS is another example of a Deep Module. The concept of Deep Module has gained wide support in the IT industry in recent years.

Source: Depth of module

Conclusion

Implementing repositories and aggregates according to DDD principles allows encapsulation of domain logic and enhances application maintainability. Determining aggregate boundaries requires deep consideration of domain and table design, but this is arguably one of the more enjoyable aspects of software development. In future developments, I aim to actively adopt DDD to explore more enjoyable application development.

PS: Wishing everyone in Ishikawa Prefecture, Noto, and Wajima in Japan, who were affected by the earthquake on New Year's Day, a safe recovery.

I have built my first SaaS

Yota Hamada — Sat, 18 Nov 2023 05:00:00 +0000

Hello guys,
I am a Golang developer working in a small company and also making an another product as a side hassle.

I have just created a service that provide a chat GUI for a group of people who don't want their conversations to be learned by OpenAI for training their models.

The service is available at the URL below.
https://chat.corpgpt.jp/

Currently it's just a simple app that allows user to manage their OpenAI API token and their organization members.

It may be seemed boring to you. Indeed, it has nothing really interesting. However I think I try to use this simple service as an entry point to start a discussion with SMBs and enterprise companies that are eager to find an area where they can leverage the state-of-art generative AI technologies.

Here is the technology stack.
Backend: Go
Frontend: TypeScript / React
Infrastructure: Cloud Run on GCP

Maybe it seems a bit weird that I use Go to create this kind of service, but that has been a great experience so far!

If you are interested, please try it out and let me know your thoughts, ideas, advice, or anything.

The news that OpenAI fired Sam Altman and CTO makes me a bit sad today, I hope the advance of the technology would continue and Sam and Greg will start an another great company or service tomorrow or soon.

Thanks!

Automate ChatGPT with Workflow Engine built with Go (example of sending goal seek results by email)

Yota Hamada — Mon, 20 Mar 2023 09:52:47 +0000

Introduction.

This article shows how ChatGPT can be automated using Dagu, a free workflow engine. Specifically, we will show you how to use YAML definitions to run ChatGPT's API and automatically email the results.

What is a DAG workflow engine?

A workflow engine is, simply put, a tool for automatically executing the tasks you do on your computer. It does not simply execute tasks in simple sequence, but can also specify dependencies between tasks and execute tasks in parallel at the same time. The following graph is a DAG (Directed acyclic graph).

What is Dagu?

Dagu is a free, open source DAG workflow engine.

Features

DAGs can be created with no code (no programming required).
All operations, including creating and running DAGs, can be done from a browser.
Just install the command on your Mac and use it.

Example of calling ChatGPT API and sending an answer by email

The following is an example of calling ChatGPT's API and sending the results by email.

What each step does is simple.

call the ChatGPT API with the specified question
extract the answers from the API response
convert the response into an HTML email ready to be sent
send the email.

As a result, you will receive an email that looks like this

Steps to use ChatGPT to automate goal seek questions

In this section, the Goal Seek prompt is used to automate Goal Seek. Note that any prompt can be automated in the same way by rewriting the contents of the DAG.

The prompt used:

[Goal] Goal to achieve.
a[0,0]=[Goal]
a[i,1-5]=5 ideas to achieve a[i,0].
a[i+1,0]=a[i,1-5], which is the best idea to achieve a[i+1,0]=a[i,1-5]

Use this to calculate up to i=0-5 and put the results in an HTML table with 5 rows and 5 columns, 25 cells in total.

Installing Dagu

Dagu can be installed using Homebrew.
```
brew install yohamta/tap/dagu
```
You can check if the installation is successful with the following command:
```
dagu version
```

Launching Dagu Server

Dagu server can be launched by the following command.
```
dagu server
```
Access http://localhost:8080 in your browser. You should see a screen similar to the following.

Creating a ChatGPT workflow.

click on the three-line menu on the left. The following is what you will see in my environment.
press the 'NEW' button in the top right-hand corner and enter the name of the new DAG chatgpt_goalseek_example.
press the 'EDIT' button in the bottom right-hand corner.

paste the following into the edit screen and save it ('SAVE' button).

params: GOAL="Unique and interesting bullet-hell game."
env:
  - OPENAI_API_KEY="ChatGPT API Key"
  - MY_EMAIL="Recipient email address"
  - FORMAT: >-
      a[0,0]=[Goal]\n
      Five ideas to achieve a[i,1-5]=a[i,0]. \n
      The best idea to achieve a[i+1,0]=a[i,1-5]. \n
      \n
      Use this to calculate up to i=0-5 and put the results into a HTML table with 5 rows and 5 columns, 25 squares in total. \n

smtp:
  host: "smtp.mailgun.org"
  port: "587"
  username: "Mailgun username"
  password: "Mailgun password"

steps:

  - name: ask chatgpt
    executor:
      type: http
      config:
        timeout: 1200
        headers:
          Authorization: "Bearer $OPENAI_API_KEY"
          Content-Type: "application/json"
        silent: true
        body: |
          { "model": "gpt-3.5-turbo", "messages": [
              {"role": "system", "content": "Act as a state-of-art chat AI. Please output the result in a table format with 5 rows and 5 columns. Format your reply in HTML code styled with beautiful CSS."},
              {"role": "user", "content": "[Goal]${GOAL}\n${FORMAT}"}
            ]
          }
    command: POST https://api.openai.com/v1/chat/completions
    output: API_RESPONSE

  - name: get result
    executor:
      type: jq
      config:
        raw: true
    command: ".choices[0].message.content"
    script: "$API_RESPONSE"
    output: MESSAGE_CONTENT
    depends:
      - ask chatgpt

  - name: convert escaped unicode to plain HTML
    command: "sh"
    script: |
      input="${MESSAGE_CONTENT}"
      unescaped=$(echo -e "$input" | sed 's/\\u003c/</g' | sed 's/\\u003e/>/g')
      echo "$unescaped"
    output: MESSAGE_CONTENT
    depends:
      - get result

  - name: send mail
    executor:
      type: mail
      config:
        to: "$MY_EMAIL"
        from: "$MY_EMAIL"
        subject: "goal seek result"
        message: "$MESSAGE_CONTENT"
    depends:
      - convert escaped unicode to plain HTML

Setting up the ChatGPT API key and email sending server

set up the ChatGPT API key
Rewrite the ChatGPT API Key below with your ChatGPT API key.
```
env:
  - OPENAI_API_KEY="ChatGPT API Key"
```
set up a mail sending server
Register with Mailgun and you can send emails for free. Rewrite the following Mailgun username and Mailgun password with your Mailgun username and password. You can also use other email services.
```
smtp:
  host: "smtp.mailgun.org"
  port: "587"
  username: "Mailgun Username"
  password: "Mailgun Password"
```
set the recipient email address
Replace the following Recipient email address with the email address to which you want to send the ChatGPT response.
```
env:
  - MY_EMAIL="Recipient email address".
```

Try it out and run it.

Press the 'START' button in the top right-hand corner to run the DAG. In the text box, put in what you want to ask for in the Goal Seek prompt. For example, put in The best and most interesting roguelike game idea.

The execution status is displayed in the browser.

If an error occurs, press the 'RETRY' button in the top right-hand corner; the error may occur if ChatGPT's API is heavy.

If successful, you will see the following screen.

You will then receive the following email. Oh no.

Summary

As you can see, the workflow engine allows you to automate ChatGPT API calls with no-code.

If you always run similar prompts, it may be useful to define a DAG.

You can do more interesting things by combining multiple inputs and outputs of ChatGPT's API. For example, ChatGPT responses could be passed to further APIs to improve or transform the results, and if defined as a DAG, it could be useful to combine multiple roles of ChatGPT, as the APIs can be called in parallel.

I can only dream of the things we could do with it!

Dagu is open source software that I am developing using the Go language.
I would be happy if you could participate in its development.

https://github.com/yohamta/dagu