Book notes: Monolith to Microservices: Evolutionary Patterns to Transform Your Monolith

These are my notes on Monolith to Microservices by Sam Newman.

Practical and useful book on the subject.

Key Insights

• Microservices are independently deployable services modeled around a business domain.
• Don't focus on size but:
  1. How many microservices can you handle.
  2. How to define boundaries to get the most out of the microservices, without everything becoming a horrible coupled mess.
• Modular monolith still have the challenge of a monolith DB.
• DDD:
  • Aggregate: self-contained unit that have a life cycle around them.
  • Bounded context (BC) + Aggregate == unit of cohesion.
  • The aggregate is a self-contained state machine that focuses on a single domain concept in our system, with the BC representing a collection of associated aggregates, with an explicit interface to the wider world.
• Microservices are not the goal.
  • What are you going to achieve that you cannot with your current architecture?
  • Have you considered alternatives to using microservices?
  • How will you know if the transition is working?
• Reuse it not an outcome.
• When not to use microservices:
  1. Unclear domain.
  2. Startups.
  3. Customer installed and managed software.
• Dr John Kotter's eight-step process for implementing org change (from Leading Change):
  1. Establish a sense of urgency.
  2. Creating the Guiding Coalition.
  3. Developing a Vision and Strategy.
  4. Communicating the change vision.
  5. Empowering employees for broad-based action.
  6. Generating short-team wins.
  7. Consolidating gains and producing more change.
  8. Anchoring new approaches in the culture.
• Small changes, little steps.
  • The biggest the bet and bigger the accompanying fanfare, the harder is to pull out when it is going wrong.
• Pattern: Change Data Capture (CDC):
  • Use when there is no other option.
• Pattern: Database-as-a-Service interface.
• When you are relying on network analysis to determine who is using your database, you are in trouble.
• Split the DB first or the code?
  • Code first.
• Static reference data:
  • Duplicate, dedicated schema, static library.
  • Data service: only when creating microservices is cheap.
• BPM tools: issue is that they are non-dev tools that end up being used by devs.
• The more coupling, the earlier the pains will manifest:
  • 2-10 services:
    • Breaking changes.
    • Reporting.
  • 10-50:
    • Ownership at scale.
    • Developer experience.
    • Running too many things.
  • 50+:
    • Loads of teams.
    • Global vs local optimization.
    • Orphaned services.
  • In all:
    • Robustness and resilience.
    • Monitoring and troubleshooting.
    • End-to-end testing.
      • Solutions:
        • No cross team tests.
        • Consumer-driven contracts.
        • Use automated release remediation and progressive delivery in addition to end-to-end tests.
• Without strong code ownership (one and only one team can change service, other teams can do pull requests to propose changes) a microservices' architecture will grow into a distributed monolith.

TOC

• Chapter 1 - Just Enough Microservices
• Chapter 2 - Planning a Migration
• Chapter 3 - Splitting the Monolith
• Chapter 4 - Decomposing the database
• Chapter 5 - Growing Pains

Chapter 1 - Just Enough Microservices

• Microservices are independently deployable services modeled around a business domain.
  • Type of SOA.
• Start with the technology that you know.
• Don't focus on size but:
  1. How many microservices can you handle.
  2. How to define boundaries to get the most out of the microservices, without everything becoming a horrible coupled mess.
• Modular monolith still have the challenge of a monolith DB.
• Other monoliths:
  • Distributed monolith.
  • Third-party black-box systems, both on premise and SaaS.
• Coupling types:
  • Types.
  • Implementation coupling.
  • Temporal coupling (in the sense of sync calls).
  • Deployment coupling (release trains).
  • Domain coupling.
• DDD:
  • Aggregate: self-contained unit that have a life cycle around them.
  • Bounded context (BC):
    • Hide implementation and internal details.
  • BC + Aggregate == unit of cohesion.
  • The Aggregate is a self-contained state machine that focuses on a single domain concept in our system, with the BC representing a collection of associated aggregates, again with an explicit interface to the wider world.
• Start by targeting services that encompass entire BC.
  • You can split them further latter, hiding this decision.

Chapter 2 - Planning a Migration

• Microservices are not the goal.
• Key questions:
  • What are you going to achieve that you cannot with your current architecture?
  • Have you considered alternatives to using microservices?
  • How will you know if the transition is working?
• Reuse it not an outcome.
• Limit the scope of expected outcomes.
• Possible whys:
  • Improve team autonomy, alternatives:
    • Modular monoliths.
    • Assign responsibilities based on functional grounds.
    • Self-servicing.
  • Reduce time to market, alternatives:
    • Value stream mapping to identify bottlenecks.
  • Scale cost-effectively for load, alternatives:
    • Vertical or horizontal scaling.
  • Improve robustness:
    • Being able to react to expected variations.
    • Alternatives:
      • Running multiple copies of your monolith.
      • Use more reliable SW/HW.
      • Automate manual processes.
  • Scale number of developers, alternatives:
    • Modular monolith (but less).
  • Embrace new technology, alternatives:
    • None.
• When not to use microservices:
  1. Unclear domain:
     • Getting services wrong can be expensive due to large number of cross-service changes and overly coupled components.
  2. Startups.
  3. Customer installed and managed software:
     • Due to increased operational complexity.
• Dr John Kotter's eight-step process for implementing org change (from Leading Change):
  1. Establish a sense of urgency.
  2. Creating the Guiding Coalition:
     • Try to bring somebody from "the business".
  3. Developing a Vision and Strategy:
     • Vision: realistic yet aspirational.
     • Commitment to vision is important, but overly commitment to strategy can be dangerous
  4. Communicating the change vision:
     • Face to face + broadcast.
  5. Empowering employees for broad-based action:
     • Bandwidth change: increase capacity or reduce load.
  6. Generating short-team wins.
  7. Consolidating gains and producing more change:
     • Keep pushing on.
  8. Anchoring new approaches in the culture:
     • Communicate successes and failures.
• Small changes, little steps.
• Whiteboard is where the cost of change and the cost of mistake is the lowest.
• Where to start?
  • Identify BC and their relationships:
    • Just enough DDD to get started.
    • Maybe use Event Storming
    • Relationship show how easy/difficult should be to extract that BC (warn: code may disagree).
  • Plot BC in:
• Is the transition working?
  • Regular checkpoints. Agenda:
    1. Restate what you are trying to achieve. Does it still make sense?
    2. Review quantitative metrics.
    3. Ask for qualitative feedback: Happier in BVSSH.
    4. Decide if any change is needed.
• Avoid the skunk cost fallacy:
  • The biggest the bet and bigger the accompanying fanfare, the harder is to pull out when it is going wrong.

Chapter 3 - Splitting the Monolith

• You have more options if you can change the monolith.
  • Best if you can copy (not move) existing code.
  • Consider refactoring into modular monolith first.
• Pattern: Stranger Fig Application:
  • Steps:
    1. Identify what to migrate.
    2. Copy to microservice.
    3. Reroute calls to new microservice.
  • Safe to rollback routing.
  • There must be a clear way to redirect the calls to the new service.
  • In case of HTTP, a reverse HTTP proxy in front of the monolith.
    • Consider Ngnix + Lua if need something custom.
    • If custom logic is very complex (like changing protocol from SOAP to gRPC), consider:
      1. Avoid putting the logic in the proxy, as it is a shared service between teams.
      2. Implement it in the microservice.
      3. Use service mesh.
  • In message systems, the proxy consumes the monolith queue and does a content-based routing to two queues: new monolith queue and microservices one.
  • While migrating, avoid any functionality change.
• Pattern: UI Composition:
  • UI to call new microservice.
  • Widget or page level.
  • Mobile apps are monoliths, unless you can make changes without resubmitting them.
  • Micro-frontend.
• Pattern: Branch by Abstraction:
  • Steps:
    1. Create abstraction for the functionality to be replaced.
    2. Change clients to use new abstraction.
    3. Create new implementation that uses the new microservice.
    4. Switch over the new implementation.
    5. Cleanup.
  • Verify branch by abstraction:
    • If call to new implementation fails, call the old implementation.
• Pattern: Parallel Run:
  • In strangler and branch by abstraction, call both implementation and compare results.
  • Can check also performance.
  • N-version programming:
    • Implement functionality in several different ways, and do a parallel run, choosing the "correct" (quorum) one.
    • For fault tolerance and to avoid bugs.
  • Verification techniques:
    • Use spy in microservices to record what will do, but without doing it.
      • When duplicated side-effects are not ok.
    • Github scientist.
  • Not trivial, use just when there is a high risk.
• Pattern: Decorating Collaborator:
  • When cannot or don't want to change monolith.
  • Proxy works as a decorator that calls new microservice in addition to old monolith.
• Pattern: Change Data Capture (CDC):
  • React to changes happened in the data store.
  • Typical implementations:
    1. DB triggers:
       • Use very sparingly.
    2. Transaction log poll.
    3. Batch delta copier:
       • Process that on a regular schedule scans the DB to find what data has changed since last run.
  • Use when there is no other option.

Chapter 4 - Decomposing the database

• Shared DB:
  • It is ok with:
    • Read-only static reference data that is stable and has a clear owner.
    • Pattern: Database-as-a-Service interface.
  • Pattern: Database view:
    • Less coupling than shared DB.
    • When you are relying on network analysis to determine who is using your database, you are in trouble.
    • Useful only for read-only.
  • Pattern: Database wrapping service:
    • Move database dependencies to service dependencies.
    • When is too hard pulling the schema apart.
    • More flexible than DB view.
    • Can take writes.
    • Stepping stone, buys you time.
  • Pattern: Database-as-a-Service interface:
    • Create a specific and dedicated DB to be accessed externally.
    • Mapping engine options:
      1. CDC.
      2. Batch process.
      3. Build from an event log.
• Transferring Ownership:
  • Pattern: Aggregate exposing monolith:
    • Monolith expose a proper aggregate API.
    • When a newly extracted microservice still needs data owned by the monolith.
    • Maybe a future microservice.
  • Pattern: Change data ownership:
    • When monolith still depends on newly extracted microservice data.
    • Ideally monolith should call new service API:
      • Copying the data back to the monolith DB as an alternative.
• Data synchronization:
  • Pattern: Synchronize data in application:
    • Steps:
      1. Bulk synchronize data:
         • If monolith was kept online, implement CDC to copy data since snapshot created.
      2. Synchronize on write, read from old schema.
      3. Synchronize on write, read from new schema.
      4. Decommission old schema.
  • Pattern: Tracer write:
    • Same as synchronize data in app, but one table at a time (instead of the whole bundled context) using the new microservice API.
• Split the DB first or the code?
  • DB first:
    • Easy, little short-term benefit.
    • When concerned about performance or data consistency.
    • Pattern: repository per BC:
      • First step to understand dependencies.
    • Pattern: database per BC:
      • Bet for future split.
      • Recommend for greenfield.
  • Code first:
    • Most common.
    • Short-term improvements.
    • Pattern: Monolith as data access layer:
      • Same pattern as "aggregate exposing monolith".
    • Pattern: Multischema storage:
      • New microservice uses new schema for new functionality/data, old schema for existing data.
  • DB and code at the same time:
    • Avoid.
• Schema separation examples:
  • Pattern: Split table:
    • When table is owned by 2 or more BC.
    • Lose referential integrity.
    • Will need to chose the owner of the data.
  • Pattern: Move foreign-key relationship to code:
    • Increase latency: from 1 join query to 1 select + n service calls.
    • Data consistency, deletion options:
      1. Check with all services before deleting:
         • More coupling, reverse dependency.
         • Don't use.
      2. Handle 404/410 gracefully in dependant service.
      3. Don't allow deletion:
         • Soft delete/tombstone record.
• Static reference data:
  • Duplicate:
    • As it changes infrequently maybe ok.
    • For large volume of data.
    • Background process to update it.
  • Dedicated schema:
    • No duplication, always up to date.
    • Allows for cross-schema joins.
  • Static library:
    • For small datasets.
  • Data service:
    • When creating a new microservice is cheap.
    • Maybe can emit change events.
• Avoid distributed transactions.
• Sagas:
  • Model transactions as business processes.
  • Specially for long lived transactions.
  • Rollback with compensating transactions.
  • Two implementations:
    1. Orchestrated:
       • Central coordinator. Command and control.
       • Pro: easier to understand.
       • Cons: coupling.
       • Warn: Anemic microservices: coordinator having too much logic that should be in microservices.
       • Different services can play the coordinator role for different flows.
       • BPM tools: issue is that they are non-dev tools that end up being used by devs.
       • Camuda and Zeebe are targeted to microservices developers.
    2. Choreographed:
       • Distributed responsibility. Trust but verify.
       • Usually event based.
       • Pro: decoupled.
       • Cons: harder to understand whole flow.
       • Use correlation Id to build/know the state of the saga:
         • Service to read all events to show view.

Chapter 5 - Growing Pains

• More detailed in "Building microservices" book.
• Based on anecdotal experience.
• The more coupling, the earlier the pains will manifest:
  • 2-10 services:
    • Breaking changes.
    • Reporting.
  • 10-50:
    • Ownership at scale.
    • Developer experience.
    • Running too many things.
  • 50+:
    • Loads of teams.
    • Global vs local optimization.
    • Orphaned services.
  • In all:
    • Robustness and resilience.
    • Monitoring and troubleshooting.
    • End-to-end testing.
• Ownership at scale:
  • Without strong code ownership (one and only one team can change service, other teams can do pull requests to propose changes) a microservices' architecture will grow into a distributed monolith.
• Breaking changes:
  • Avoiding accidental breaking changes:
    1. Explicit schema to avoid structural breakages.
       • Protolock to prohibit incompatible changes.
    2. To avoid semantic breakages: testing.
       • Make it hard to change a service contract:
         • Make it obvious, no magic, or generate schemas from code.
  • Non-accidental:
    1. Do not break, but accrete.
    2. Give consumers time to migrate:
       1. Run two versions of the service.
       2. Support old and new endpoints in the service.
  • Be more relaxed if changes are within a team.
• Reporting:
  • Build a reporting specific schema.
• Monitoring and Troubleshooting:
  • Log aggregation:
    • First thing to do when implementing microservices.
  • Tracing:
    • API GW or service mesh to generate the correlation ID.
  • Test in production:
    • Synthetic transactions.
  • Towards observability:
    • Distributed Systems Observability.
• Local developer experience:
  • How many services to run locally?
  • Solutions:
    • Stubs.
    • Point to instance running elsewhere.
    • One remote env per developer:
      • Slow to deploy to test changes.
      • Cost.
    • Mix local/remote:
      • Telepresence for Kubernetes.
      • Azure's cloud functions.
• Running too many things:
  • Deployment, configuration and management of instances becomes more difficult.
  • Solution:
    • Kubernetes.
    • Function-as-a-Service (preferred).
• End-to-end testing:
  • Even slower and even more brittle.
  • Solutions:
    • No cross team tests.
    • Consumer-driven contracts:
      • Pact.
    • Use automated release remediation and progressive delivery in addition to end-to-end tests.
• Global vs local optimization:
  • Solving the same problem twice.
  • Divergent tech stack.
  • Solutions:
    • Cross-cutting group to raise awareness/make tech decisions.
• Robustness and resilience:
  • See Release it!.
• Orphaned Services:
  • In-house service registries that combines service discovery + code repository data.