Building an Event Sourcing System in Rails, Part 1: What is Event Sourcing?

What is Event Sourcing?

Event Sourcing is a system design pattern that emphasizes recording changes to data via immutable events.

In other words: every time your data changes, you save an event to your database with the details.

Those events never change or go away. That way, you have a permanent, unchanging history of how your data reached its current state!

Martin Fowler’s important 2005 essay describes Event Sourcing this way (emphasis mine):

Event Sourcing ensures that all changes to application state are stored as a sequence of events. Not just can we query these events, we can also use the event log to reconstruct past states, and as a foundation to automatically adjust the state to cope with retroactive changes.
…
The fundamental idea of Event Sourcing is that of ensuring every change to the state of an application is captured in an event object, and that these event objects are themselves stored in the sequence they were applied for the same lifetime as the application state itself.

Cool! From this, we get a few key points about Event Sourcing systems that we’ll explore:

• Changes to data are stored as Event objects
• The Events have a fixed chronological order that can be “replayed”
• The Events cannot be changed or deleted (for as long as they might be needed)

Examples of Event Sourcing

The example app we will be building in these articles will focus on events related to creating and destroying a User model. Consider this situation:

• Client sends a request to create a User
• Server receives the request
• Server creates and saves a User
• Server creates and saves an Event, which records the details of the request and the created User

Obviously, the implementation (and order of operations) will be more complex--but at its core, that's the basic Event Sourcing system we'll be creating!

In a 2017 essay, Martin Fowler points out that version-control systems (like Git) are probably the most common example of Event Sourcing that programmers encounter (emphasis mine):

The core idea of event sourcing is that whenever we make a change to the state of a system, we record that state change as an event, and we can confidently rebuild the system state by reprocessing the events at any time in the future. The event store becomes the principal source of truth, and the system state is purely derived from it. For programmers, the best example of this is a version-control system. The log of all the commits is the event store and the working copy of the source tree is the system state.

For Rails-specific Event Sourcing, two resources/examples are often cited:

• Kickstarter’s minimal Event Sourcing system, described in this blog post
• Arkency’s RailsEventStore library, which has a number of versatile uses detailed in the GitHub repo

The focus of these articles will be on following Kickstarter’s example as I create a personal project with event sourcing architecture. Thanks Kickstarter, you folks are amazing!

Some implications of using an Event Sourcing system

Since working on an event sourcing system at my previous job, I’ve been building a personal project using one too. The key difference is that, at work, I was expanding on an already-existing system (and one that wasn’t initially designed for event sourcing).

Now that I’m building one from scratch, I’ve encountered some important considerations:

• Data is truly immutable—forget about using ActiveRecord.destroy! Instead, it’s important to design data for permanence, such as including a deleted field on User models.
• Recording payload dumps of event data can include potentially storing sensitive data in an unsafe format if the event is saved outside of auth features like has_secure_password—you must be extra cautious about when sensitive data might be exposed, and be ready to manually implement encryption where needed!
• Since events become “the principal source of truth” (in Martin Fowler’s words), tons of logic ends up being wrapped in Event objects—naming and organizing code (and making sure it’s future-proof) is more essential than ever! As Philippe Creux at Kickstarter described (emphasis not mine):

Naming is hard. And there are so many immutable events and attributes to name. The names you choose now will be the ones stored forever. So take a good dictionary and make sure that you nail down names that are explicit and future-proof.

I hope to explore and share these implications (and future ones that will surely arise!) in these articles, as we walk through how to implement an Event Sourcing system from scratch.

Next Up

The next article will go through the basics of creating a Rails app, centered around creating and destroying a User model, that uses basic Event Sourcing architecture. We will cover:

• Setting up the Rails app with a User model and controller
• Building the BaseEvent class architecture (per Kickstarter’s example)
• Implementing Events::User::Created and Events::User::Destroyed event classes

References

Special thanks to Philippe Creux and Kickstarter for sharing their Event Sourcing example.

Thanks to Martin Fowler for his important writings on Event Sourcing.

Thanks to Arkency for their great work with the RailsEventStore library.

And finally, thanks to fellow Dev.to user Alfredo Motta for sharing about this years ago (and keeping it up for me to catch up on!).

Comments

[Steve Geluso]

I was talking to someone today who is dipping in to programming running a bike share program in YVR Vancouver, B.C. They are learning Python and parsing a lot of transactional data. Event sourcing sounds like a great school of thought to come from to process data. The thing about that sort of data is you DEFINITELY don't want to ever delete or modify anything so you never lose the real record of what has happened.

Thanks for introducing me to this! I look forward to running in to it more in the future!

[Isa Levine]

Ooo that sounds like a rad project, Steve--and so good to hear from you! :)

And yes, event sourcing would be a great way to approach managing that data. In case you haven't seen, I wrote a tutorial for building an event sourcing system in Rails here!

Let me know if you end up using it, would love to hear you thoughts and catch up sometime!