re: Event Sourcing: What it is and why it's awesome VIEW POST

re: Thanks for your detailed reply, Barry. A few follow-up questions (my apologies for the lack of basic understanding they betray!) ↓ “When an aggre...

Hey Harrison,

Glad to answer, let's give this shot.

"When an aggregate is stored, it's events are pushed to a queue”

"Do you also store those events in a local database?"

Yes, this is a two phase operation. First the events are written to the database (or whatever storage you use for events), then they're pushed to the queue.

Now, you raise a valid point, what happens if the event is pushed to datastore, but not the queue, or vice versa (equally possible)? This is a long standing problem in any event driven system, and there are a couple of solutions.

In our implementation, writing to the datastore is transactional, once that completes the messages are sent to the queue. The messages are used to tell other systems that something has happened, they're just there to broadcast that a change has occurred, other system will read this message, then query the datastore to see what's actually happened. In other words, the "projectors" don't trust the events on the queue, they just use them as triggers for them to read events from the source of truth, the DB.

This still has the problem of "What if messages don't appear on the queue", but it becomes a problem that sorts itself out once another event appears on the queue, it'll trigger the projectors and they'll update normally.

BTW, we've never had this kind of failure. It's possible, but very unlikely. And even if it did happen, the system will handle it.

“we have a background PHP process that listens for events and then passes them to the appropriate projection whenever they are received”

"Could you speak a bit more about this?"

Yeah, you pretty got this in your exploration of an answer. It's receiving these events from a queue. In our current implementation, we have a single queue per service. Each service has a process that pulls events of the queue, in the same way as Laravel's queue workers. When an event is received, it queries the event log for the latest events. It then takes the new events, and play them into each projector.

If you'd like to discuss this further, drop me a DM on Twitter, we could arrange a skype call or something. Always glad to discuss.

code of conduct - report abuse