In the last article we looked at projectors, the backbone of any CQRS/Event Driven system. This article was originally meant to be about implemen...
For further actions, you may consider blocking this person and/or reporting abuse
Awesome as always.
Initially, we have done immediate consistency across the board. It is easiest to reason about and create initially. And I would say if it is an internal business app, it probably never needs to scale, and therefore could stop there.
But even on a single node, full consistency some negative implications.
One is because of the coupling between events and view, deployments affect everything. When I deploy a new view, I risk also taking down the command API and vice versa.
Another is that other services interested in listening to events can become really awkward to implement. Take an emailer (that gets throttled by SES). To keep it consistent, we had to implement a fully-consistent projector to record the fact that emails need to be sent (based on events that got generated from the domain). Then we have a timed service that polls the query API every minute to see if there are emails to send. Then it calls the command side to actually send the emails and record they were sent. It's a really awkward workflow for the developer because of full consistency.
When we move over to eventual, the service can listen to events as they come in (instead of polling every minute), update it's own view of what it is doing, and save events indicating emails were sent. Far more straight-forward.
Another is scalability. Our load is more read-heavy. With full consistency, we would need to engage the view database's replica feature to add read capacity. This involves going deeper with (and being restricted by capabilities of) the projection DB -- settings, replication lag issues, etc. But with eventual, we don't even need to go that far. We can literally just spin up a new copy of the projector service, a new independent database for it, and add the service to a load balancer. We can also scale it down easily by doing the inverse. And basically every listener service works the same way (only with different integrations) so it is already something we know.
The really hard part of eventual consistency is accepting limitations around set validations and memberships. I've been meaning to write a post on that.
Thanks Kasey.
I've experienced the exact same issues you mention above, and once we made the switch to eventual we realised how easy it made things.
The funny thing is, you can architect your system as eventually consistent, then just force it to run as immediately consistent. Once you run into problems, switch over to eventual. It allows you to defer the problem, and it makes it easy to change when you need to. Building it as a immediate, then switching to eventual is a much more costly change, mainly because the immediate implementation is probably not event driven.
We used immediate for domain projections (mostly for set validations and memberships, as you phrased it) because we need them to guarantee domain constraints. There are other solutions though (think I discussed some in a previous article) but I'd love to see someone (maybe yourself :)) explore it further.
Immediate is nicer to reason about but quite hard these days. If you can get away with an architecture that is basically glorified LAMP with some transactional db, great, but a lot of projects are a bit more complicated than that. In architectures where you are dealing with search engines, asynchronous processing via queues, micro services, etc. you can't always wait for everything to catch up. A lot of this stuff is just fundamentally non transactional.
In my experience when building software like this with web based, and worse, mobile uis, things get quite hairy. These types of environments favor small payloads and fast requests. You don't want uis to block while a server waits for seconds to get some update through all its backend systems. At the same time you don't want to serve stale data to your users.
A good example is somebody is a user looking at a screen with search results. They 'edit' an item in some way and now they go back to the search results which are now out of date because it just takes a few seconds for updates to trickle through queues, be processed, and re-indexed in Elasticsearch. So there is a brief window where the results are inconsistent with what the user just did. That's a potential problem because it may cause the user to assume the update didn't happen.
A bad solution for this: grey out the screen and update a few seconds later and hope for the best. This creates a really bad experience because the system will be perceived as slow when in reality it is the frontend or some server side view doing a sleep to fake consistency. The better solution is to change the UX to not create this expectation of real time consistency when it is expensive to provide it and have network plumbing in place that ensures the UI is updated with fresh data when it comes available. This creates a more fluent experience and most users understand that their actions seconds ago may cause data to be updated in the UI. Use some nice animations, maybe some notification indicating success, etc. Blocking users and forcing them to wait is much worse than allowing them to continue to work but it requires UX to be aligned with reality.
This is the reason why most UI frameworks have switched from server side MVC to client side MVC with all the blocking stuff happening asynchronously. This is also why things like html 2, websockers, graphql, have become a thing.