Building “The Better Store” an agile cloud-native ecommerce system on AWS — Part 1: Introduction to Microservice Architecture

A resilient cloud native app built using Domain Driven Design and Microservice Architecture on AWS

Overview

A lot has changed in IT over the last 10–15 years, with the advent of Cloud Native Computing. Its rise has enabled organisations from the smallest of start-ups to large organisations to design and implement systems that can:

Automatically scale on demand (and paying only for what you use).
Provide system resilience; i.e. such that a service can recover on failure.
Support change agility; i.e. to allow the implementation of new features within a short timeframe with minimal risk of outage.

As a Senior Consultant with several years experience specialising in Cloud-Native design, full-stack development and DevSecOps implementation on AWS within both start-up and large organisations, there has been much to keep up with and learn as new methodologies have emerged predominantly around Microservice Architectures (MSA) and Cloud Native patterns. Many of these topics are still largely debated; e.g. how big should a microservice be? Should microservices be allowed to communicate with other microservices, and are shared databases across microservices okay?

As an endeavour to expand and share my own knowledge of the topics with examples to demonstrate and build upon, “The Better Store” has been developed as a sample open-source eCommerce system with web (SPA) frontend to illustrate Domain Driven Design (DDD), MSA, Cloud Native development and implementation on AWS, with the intention of evolving this over time. This article is Part One of a series of seven planned articles, which will cover the following topics with reference to The Better Store:

Part One: An introduction to Microservices Architecture (MSA) to realize benefits of Cloud Native Computing, and Domain Driven Design (DDD) as a popular to assist in the formulation of an optimum MSA solution design for our given business domain.

Part Two: A focus on DDD Strategic Patterns, to assist in defining our business domain and appropriate Bounded Contexts, which help identify candidate microservices.

e.g. The Better Store context map

Figure 1. Sample Context Map for “The Better Store”, providing insights to potential decoupled microservices and their granularity/scope.

Part Three: Using DDD Tactical Patterns for further elaboration of The Better Store’s domain model and MSA design, with a focus on its Order Context.

e.g.

Figure 2: Illustrating a potential static view as a class diagram for designing an Order microservice, showing Domain (core logic), Infrastructure and Application Tiers

Part Four: Selecting Cloud Native Design patterns and AWS Services for implementation of The Better Store as an agile, highly scalable and resilient global solution; including at a high-level:

1. Decoupled services and communication styles
2. Database per Service
3. CQRS, and related data consistency patterns (i.e. Sagas, Aggregates and Event Driven Architecture)
4. API Gateway and composition
5. Global and auto-scaling architecture
6. Serverless implementation using AWS API Gateway and Lambda with NodeJS, Typescript, Inversify and the Onion Architecture

Figure 3: An envisaged implementation view of The Better Store

Part Five: Use of DevOps; specifically AWS Cloudformation combined with GitHub and AWS Pipelines for defining both applications and infrastructure as code, for fully-automated deployments.

Part Six: Development of the web frontend as a Single Page Application, using a javascript framework

Figure 4: The Better Store home page, whereby users may browse products, and after signup or sign-in, may add these to cart and purchase (using test payment services only)

We envisage that our final MSA landscape may look like the following:

Figure 5: An envisaged conceptual architecture for The Better Store, illustrating decoupled microservices

Part Seven: Monitoring and Site Reliability Engineering; how to get the most out of AWS Cloudwatch for early identification of potential issues and remediation, scaling and obtaining business insights.

For Part One though, we’ll now look further at a microservices introduction, its challenges, and then what we would like to initially see for The Better Store.

An introduction to Microservices

One of the most well-known methodologies adopted in recent years to help realize Agility, Scalability and Resilience capabilities in The Cloud, particularly for medium to enterprise-sized applications is Microservice Architectures (MSA). MSA takes the approach of designing applications as a composite of small, decoupled, stateless and independent services that are cohesive to a specific task, to provide the following advantages:

1. Services are kept small and cohesive to a specific function, to promote Change Agility, and Scalability.
   If a change is required to a specific application feature, e.g. cancelling an eCommerce order; only one independently-deployable service; e.g. Order Service that is specific to maintaining orders should require changing at the backend. This helps keep logic clear in one place, and the risk of a severe or unforeseen release defect is reduced when only 1 component needs to be released. This reduced change size and risk can (and should) allow changes to be deployed faster with reduced change management, allowing consumers to benefit from changes more quickly (while also further reducing risk as new changes are deployed in small increments). Smaller-sized services can also generally be started quicker by the Cloud platform when configured for horizontal scaling (i.e. starting multiple instances of a service to handle increased demand).
   Note that adoption of MSA to compose an application as many microservices does add a level of complexity to the solution, especially concerning their interaction and when multiple services are being developed concurrently. Releasing small-sized changes frequently with the support of mature DevOps processes for automated testing and deployment is often considered a prerequisite, which requires initial investment for organisations that wish to adopt MSA.

2. Services are decoupled; to promote Change Agility, Scalability and Resilience, and be Technology Agnostic
   We mentioned above that keeping services cohesive to a specific task helps reduce complexity and increase agility by keeping services small and independently-deployable, with reduced risk of impacting other services.
   The potential risk of a change to a service impacting other services and the overall application is also highly-dependent on the degree of coupling that exists between services and resources. For example, the following illustrates a hypothetical implementation of an order service, which is responsible for first completing payment of an order, before sending instructions for its shipping fulfillment:

Figure 6: Illustration showing coupling between services and the database

Order Service here is dependent (tightly coupled) to Fulfillment Service; meaning that any changes that are made to Fulfillment Service have the potential to impact Order Service. Regression testing of both services should be considered for any such changes. Furthermore, a runtime failure of Fulfillment Service would also likely impact Order Service and possibly the user’s experience. Additional coupling is also illustrated by the fact that both services are sharing the same database. Any changes that are required to be made to the database for one service may also have consequences for the other service.
In both these cases, clear communication and collaboration is needed between teams that manage both services whenever changes are required.

On the other hand, an MSA-based alternative may look like the following:

Figure 7: Illustration showing decoupled services, as favoured for MSA

Here, Order Service has been decoupled from Fullfilment Service, such that its order confirmation messages are sent en-route to Fulfillment Service via a publish-subscribe mechanism. The order service only needs to know how to send its messages to the message-based technology; for its order payment/confirmation use case it does not need to wait for a response from Fulfillment Service. The fulfillment service does not even need to be immediately available at the time messages are sent, as long as it is configured as a durable subscriber (for discussion in a future article).

The refactored solution has also decoupled the data store, such that each service has its own database. This provides benefits not only in terms of allowing the service’s development team having full control of changes to its data store with greatly reduced risk of impacting other services, they can also select the database technology or vendor that is most suitable to them, providing polyglot persistent to the overall application. Similarly, teams have a choice of implementing their services in a programming language that is most suited to their skill sets and uses cases, provided that it is supported by their Cloud provider.

3. Services are stateless; to promote scalability and resilience.
This means that services do not need to remember information of previous messages received from a client; each client request is required to be handled as though it were received from a client for the first time. This feature enables automated horizontal scaling and failover of services, whereby new service instances can be created and destroyed as needed based on demand or failure, and any new instance should be able to handle requests from clients that were served by another instance before.

Challenges of Microservices

Promoting change agility, scalability and resilience of applications are major benefits provided by MSA and Cloud Native Computing. However, many of its design principles introduce larger challenges that are not present with traditional monolithic solutions, and debates on many of these topics continue to rage, including:

1. How big is a microservice, or rather how should the granularity of a microservice be decided?
2. What are best practices for ensuring that microservices are effectively decoupled within a business domain, for optimal change agility and minimal maintenance in the future?
3. How can shared databases be eliminated if direct communications or ‘chattiness’ between microservices should be avoided?
4. How can work be roll-back if there is a partial failure; e.g. if a fulfillment operation cannot be completed, how can a user’s payment best be refunded?
5. What are the best communication technologies and patterns for different use cases?
6. How can global solutions best be implemented, to provide either regional disaster recovery/failover, or maximum performance for serving data consistent data to users no matter where they are?
7. What prerequisites are required for embarking on an MSA implementation?

Indeed, catering for much of the above is a complex task and can require much initial investment, particularly for organisations that do not yet have mature DevOps, Agile teams and practices, or Site Reliability Engineering (SRE) capabilities. Change agility and scalability advantages of MSA are valuable for medium to large applications, however these may not be so important and only add a restrictive overhead to start-ups. Each of these however will be discussed further in the design of The Better Store.

Coming next in Part Two: Using DDD Strategic Patterns to design The Better Store

References

1. “Domain Driven Design, Tackling Complexity in the Heart of Software”, Evans, E., Addison-Wesley (2003)
2. “Patterns, Principles, and Practices of Domain-Driven Design”, Millett & Tune, Wiley & Sons (2015)
3. “Design Patterns for Cloud Native Applications”, Indrasiri & Suhothayan, O’Reilly (2021)
4. DDD, Hexagonal, Onion, Clean, CQRS, … How I put it all together, Graca, H, _web _(2017)
5. “Intro to Amazon EventBridge”, Beswick, J. (AWS), web video

Disclaimer: The views and opinions expressed in this article are those of the author only.