My idea of a "developer" is not the ability of tackling complex algorithms but to be able to implement systems that are observable, loosely coupled, fault-tolerant, scalable... accepting changes easily. There are a lot of knowledge to learn to tend toward this absolute, in terms of technologies but moreover in terms of methodologies. Reactive Architecture can help us get closer to this absolute.
Reactive Architecture ?
Let's focus on a common known example. I wanna talk about the Inversion of Control pattern in software development and one of it's common implementation : the Dependency Injection.
In this example, Class A no longer need to implement the dependencies of Class B and Class C by itself since they're now injected. (yes it's even better if B and C are Interfaces)
Reactive architecture uses event driven to achieve this relationship applying IoC to the overall software architecture. When it comes to big softwares like SaaS, it's a key success architecture to keep scaling with multiple teams and multiple code bases.
In this example, the Inversion of Control is done via Publish/Subcribe principle with an event broker. The consumer is responsible of subscribing himself to a topic to recieve messages from the publisher. Hence, the publisher no longer care about calling the consumer, which protocol, or which OAS contract. It just sends the events to a topic and let the others decide if it's useful and how to react to it.
Why would I need this ?
We will take a simple example of ordering flow in an e-commerce application. Let's imagine those few steps :
- Create an order
- Check the stocks to see if everything is available
- Create the invoice
- Proceed payment
Implementation with RESTful 2 Phase Commit HTTP calls
First feeling is that it feels messy. And you're right at it. Each backend is responsible of executing its own code but also to call for the next one, waiting for the response. If something fails, the whole order process fails.
You may feel like it's obvious but from what I've seen in my career, this design is the most common : having backends calling each others via API Gateway and OAS contract is something very common.
Some of you would say "you got an obvious step function use case here". You're right but it would be an implementation of Orchestration SAGA patterns which leads to the same drawbacks.
So how do we make the things less coupled ?
Refactoring with Publish/Subscribe events
What did we changed :
- Each function is only responsible of itself and push events into a topic
- Each function has a queue to act as a buffer (not showing dead letter queues here)
- Each function (except first) is now triggered by an event
- The "Orders" topic uses filtering to redrive messages to the right SQS queue
The result is that backends are no longer concerned about what's going next. Interested ones just have to "react" on events.
By doing this, we're designed to accept changes and new features with few (to no) impacts.
Let's imagine that we would like to add a feature in a dedicated "statistics" backend that records the amount of dollars earned each day. You would just have to plug it to the right topics and react to events to process. New features are added following a "plugin" principle.
So what did we earned :
- Less coupled backends thanks to events
- Inversion of Control thanks to Pub/Sub
- Accept changes and new features with few to no impact
- Message buffering with queues to handle retry
Of course, if you tend to have multiple queues and topics, it would be better to use AWS Event Bridge but it would be exactly the same as previous :
Conclusion
Designing and implementing Reactive Architecture is a journey that can bring you a powerful platform able to accept changes and scaling but it's very demanding. Your initial development time will be higher and it will require competencies that junior or confirmed developers are not supposed to be comfortable with.
I would recommend to consider such an architecture only if there are real needs and developers wanting to grow, with the time to grow.
That said, once you're comfortable enough with such architectures, you will be facing new challenges like distributed transactions for example. That's why the next step would be Outbox, Event Sourcing and CQRS but... that's another story... soon on this blog :)
Top comments (2)
Nice article, however, I would argue that comparing pub/sub to DI is a little flawed: Class A is still aware of its dependencies. The main difference is that they are now injected, and not implemented by class A.
I took a too big shortcut here. The purpose was to explain that, as injected components, subscribers act like plug-ins. That's why I would argue that DI and Pub/Sub are two different implementations of IoC.
I'll rewrite this part since it's not cristal clear. Thank very much for your feedback !