Microservices: boundaries, sagas and failure

Part 2 of 8 · week 2 of the learning path

Microservices let many teams release and scale parts of a system independently, but they trade simple code for a distributed system, so use them when the organization needs the independence.

Learning objectives

• Decide between a modular monolith and microservices for a given team and product.
• Draw service boundaries, own data per service and choose how services communicate.
• Keep multi-service workflows consistent and resilient.

Principles

• Boundaries follow business capabilities, so a change usually touches one service.
• Each service owns its data; others reach it only through its API or events.
• Services deploy independently, and the team that builds a service runs it.
• Start with a well-structured modular monolith and split out services when a real need appears.

Communication

Style	Use it for	Watch out for
Synchronous (REST, gRPC)	Queries and steps where the caller needs an answer now	Latency adds up, failures cascade
Asynchronous (events, queues)	Notifying others, long steps, absorbing spikes	Harder to follow, needs idempotency
API gateway	One entry point for clients: auth, limits, routing	Keep business logic out of it
Service discovery	Finding healthy instances	Needs health checks and fast failover

Keeping workflows consistent: a single database transaction cannot span services, so use a saga: a sequence of local steps where each step has a compensating action if a later step fails. A saga can be orchestrated by one coordinator or choreographed through events.

Worked example, checkout. Cart, Order, Inventory, Payment and Shipping are separate services. The order saga reserves stock, charges the card, then books shipping. If the charge fails, the compensating action releases the reserved stock and marks the order failed. Each step is idempotent so a retry does no harm.

Resilience patterns: timeouts on every call; retries with exponential backoff and jitter; circuit breakers that stop calling a failing service for a while; bulkheads that isolate resources per dependency. Be careful with retries: if three layers each try a call three times in total, one user action can become 27 calls to the bottom service.

Delivery: one repository or several, but each service has its own pipeline, containers, canary releases and rollback. Use contract tests so teams can change services without breaking each other, and pass a trace ID through every call.

Real-world examples

• Amazon and Netflix: both moved from large applications to many small services owned by single teams, which let them release thousands of times a day, at the cost of heavy investment in tooling and operations.
• Segment (2018): the company published that it moved a set of tightly coupled microservices back into one service because the operational cost outweighed the benefits at its team size.
• Shopify: it runs a very large modular monolith with enforced internal boundaries, showing that good structure matters more than the number of deployables.

Design exercise: take the checkout example and write the saga as a table of steps, compensating actions and events. Then list two reasons you would keep it as a modular monolith for a team of eight.

Quiz 2 (5 questions, pass mark 4)

1. When are microservices a good choice?
   • A) A team of three building a simple app
   • B) You want fewer deployments
   • C) Many teams need independent releases and parts of the system scale differently
   • D) You want to share one database across services
2. What is a compensating action in a saga?
   • A) Releasing reserved stock when the payment step fails
   • B) Adding a cache
   • C) Rolling back one database transaction across services
   • D) Locking all tables
3. What does a circuit breaker do?
   • A) Encrypts calls
   • B) Caches responses
   • C) Balances load
   • D) Stops calling a failing dependency for a while so failure does not cascade
4. What do the Segment and Shopify examples show?
   • A) Monoliths are obsolete
   • B) Microservices bring operational cost, and a well-structured modular monolith can be the better choice
   • C) You should always use microservices
   • D) You should never split a system
5. Three layers each try a call three times in total. How many calls can reach the bottom service from one user action?
   • A) 3
   • B) 9
   • C) 27
   • D) 1

Quiz 2 answers

1. C
2. A
3. D
4. B
5. C three layers each making three attempts give 3 times 3 times 3, which is 27 calls.

Sources and further reading

• Microservices by James Lewis and Martin Fowler, the article that defined the style and its characteristics.
• Saga pattern on microservices.io, for orchestration versus choreography.
• Timeouts, retries, and backoff with jitter from the Amazon Builders' Library, the source for the retry pitfalls above.
• Why Segment returned to a monolith on InfoQ, summarizing Segment's own write-up.
• Deconstructing the monolith from Shopify Engineering, on the modular monolith.
• Prime Video: scaling up the monitoring service and reducing costs by 90%, with an InfoQ summary.

The Amazon and Netflix history in this module is from general knowledge and has no cited source here; add one if you want it referenced.

---

Part 2 of 8 in the System Design Learning Path series. Next: part 3.