Transitioning from Monolith to Modular: A Comparison of 3 Different

#career #monolith #microservices #architecturalpatterns

The Inevitability of Transitioning from Monolith to Modular

My years of experience in enterprise projects have shown that monolithic architectures, which initially offer rapid development and simple management, become a real headache once they reach a certain size. As the codebase grows, deployments can take weeks, adding new features becomes almost impossible, and even the smallest change can put the entire system at risk. At this point, transitioning to a modular architecture becomes inevitable. However, this transition is not a simple, overnight task; it requires careful planning and the right strategy.

One of the fundamental challenges encountered during the transition is the complexity of the existing monolithic structure. Database schemas, inter-service dependencies, and unexpected side effects can emerge at every step. Therefore, when defining a strategy, it's crucial to consider not only technical but also organizational and operational factors. In this post, I will deeply examine three different strategies that can make this transition more manageable, drawing on concrete examples from my own field experience.

Strategy 1: Strangler Fig Pattern

As its name suggests, the Strangler Fig Pattern is based on the principle of gradually building a new system around an existing one, eventually "strangling" the old system. This approach is particularly effective for minimizing risk in large and critical systems. Essentially, it begins by directing user traffic or specific functionalities to the newly developed modular structure. The old system remains operational, but the new system gradually replaces it over time.

The most critical step when implementing this pattern is deciding which functionalities to extract first. Typically, modules that cause the most problems, are updated most frequently, or require the highest performance become the initial targets. For instance, in an e-commerce platform, critical modules like payment processing or inventory management can be separated from the monolithic structure and moved to a new service. During this migration, it's important to create a "facade" or "proxy" layer between the old and new systems. This layer analyzes incoming requests and directs them to either the old system or the new system as appropriate.

ℹ️ A Concrete Example of the Strangler Fig Pattern

While working on a production ERP, the reporting module constantly experienced performance issues. To decouple it from the monolithic structure, we developed a new reporting service. When users requested a report, an Nginx reverse proxy analyzed the request, directing it to the old reporting module if relevant, or to the newly developed service if it concerned the new service. This reduced the load on the old reporting module, and as the new service's development was completed, traffic was entirely shifted to the new system. This transition took approximately 6 months, and the total system downtime was only 15 minutes.

The biggest advantage of this strategy is that it offers a seamless transition. Users typically don't notice any changes during most of the process. Furthermore, the ability to develop and test each new module independently allows for earlier detection of errors. However, this strategy requires a complex routing infrastructure, and ensuring synchronization between the old and new systems can be challenging.

Strategy 2: Branch by Abstraction

Branch by Abstraction is another effective method, particularly useful when the codebase needs to be deeply modified. In this approach, an "abstraction layer" is first created around the existing monolithic code. This layer preserves the interface that the existing code presents to the outside world, while allowing its internal structure to be changed. Subsequently, a new, modular structure is developed through this abstraction layer.

The first step of this strategy is to create an "abstraction" for the module or area to be modified. For example, if we want to modularize a user management module, we define an interface (or abstract class) that abstracts operations like retrieving, updating, or deleting user information, without accessing the entire existing code. This interface is initially implemented to call the existing monolithic user management code. This way, the rest of the code becomes dependent on this interface.

💡 The Importance of the Abstraction Layer

The primary purpose of creating an abstraction layer is to isolate other parts of the existing system. This minimizes the risk of unexpected side effects occurring in the rest of the system when modifying or rewriting the code behind the abstraction layer. This is vital for maintaining codebase consistency, especially in long-term transition projects.

Once the abstraction layer is ready, a new, modular implementation of this layer is developed. This new implementation can be an independent service or library. When development is complete, the abstraction layer's implementation is switched from the existing monolithic code to the new modular structure, ensuring that the rest of the system connects to this new structure. This strategy is ideal for incrementally refactoring the codebase and significantly reduces the risk of breaking existing functionality.

When implementing this strategy, correctly designing the abstraction layer is critically important. If the abstraction layer becomes unnecessarily complex or insufficient, the transition process can become even more difficult. Additionally, changes made through the abstraction layer might introduce an extra performance overhead.

Strategy 3: Decompose by Use Case

This strategy is particularly effective in systems where business logic has distinct and separable use cases. The core idea is to transform the monolithic application into a set of independent services, each fulfilling a specific functionality. This is often also referred to as "decomposition by business capability."

In this approach, the system's core functionalities and use cases are first identified. For example, in a cargo tracking system, use cases like "Package Creation," "Route Planning," "Delivery Update," and "Invoice Generation" can be defined as separate modules or services. Each use case is treated as an independent unit with its own data structure and business logic. This ensures that each module becomes consistent and self-contained.

⚠️ Considerations for Decomposing by Use Case

One of the biggest challenges when implementing this strategy is accurately identifying the subtle dependencies between functionalities. Business logic intertwined within a monolithic structure can lead to unexpected issues during decomposition. Therefore, detailed analysis and domain knowledge are critically important for the success of this strategy. For example, when separating one use case, you might inadvertently affect a data structure closely related to another use case.

Once independent services are developed for each use case, these services can be aggregated behind an API Gateway or communicate directly with each other. The biggest advantage of this strategy is that each service can be developed and scaled independently. This increases team efficiency and accelerates development cycles. Furthermore, each service can choose its own technology, leading to a more optimized architecture.

However, this strategy introduces the complexities of inter-service communication management and distributed systems. Careful planning is required to understand how changes in one service will affect others. Additionally, managing distributed transactions (e.g., the need for simultaneous operations across two different services) can add further complexity.

Trade-offs and Choosing the Right Strategy

Each of these three strategies has its unique advantages and disadvantages. Determining which strategy is best suited for you depends on your project's specific requirements, your team's capabilities, and the complexity of your existing system.

Strangler Fig Pattern: Ideal for those seeking a seamless transition and low risk. However, the complexity of the routing infrastructure and synchronization between old and new systems can be challenging. When choosing this strategy, utilizing powerful proxy solutions like Nginx or Envoy offers significant benefits.
Branch by Abstraction: Preferable when fundamental changes are needed in the codebase and when you want to isolate other parts of the existing system. However, it has disadvantages such as the need for correct abstraction layer design and potential additional performance overhead. Creating this layer with abstract classes in Java and abstract base classes in Python is a common practice.
Decompose by Use Case: Offers the fastest and most scalable solution for systems where business logic is clear and suitable for developing independent services. However, it comes with challenges such as distributed system complexity and inter-service communication management. In this strategy, tools like gRPC or Kafka can facilitate inter-service communication.