RedEventStore

Recently, I wrote my first blog post on this blog about Sourcing, a Raku framework I'm developing for Event Sourcing. One crucial component of Event Sourcing is the Event Store, which is responsible for storing all domain events in an append-only manner. This allows for event replay, debugging, auditing, and system reconstruction.

The goal of Sourcing is to be Event Store agnostic. However, to demonstrate its functionality, it is useful to provide a small reference implementation. To achieve this, I decided to create a simple and lightweight Event Store using a database. The easiest way to do this in Raku (in my opinion) is by leveraging the Red ORM.

What is an ORM, and how is Red different?

An Object-Relational Mapper (ORM) is a tool that allows developers to interact with databases using objects and methods instead of raw SQL queries. ORMs typically map database tables to classes, making it easier to manage and query data programmatically.

Red is unique compared to most ORMs because instead of merely implementing SQL within Raku, it integrates SQL seamlessly with Raku's syntax and capabilities. This approach provides more expressive and powerful query-building capabilities while maintaining the idiomatic feel of Raku.

Query by ID and Type

An Event Store needs to store events, so let's start with a basic definition:

use Red:api<2>;
use Red::Type::Json;

unit model Event;

has UInt $.id   is serial;
has Str  $.type is column;
has Json $.data is column;

This defines a table with three columns: id (a primary key with auto-increment), type (a string indicating the event type), and data (a JSON column storing event details). To create a new event, you can run:

Event.^create: :type<EventType>, :data{bla => 1, ble => 2};

To retrieve all events after a specific ID (e.g., 42):

Event.^all.grep: *.id > 42;

To filter events by a specific type or a set of types:

Event.^all.grep: *.type ⊂ <Type1 Type2 Type3>;

In Raku, the .grep method returns a Seq, which lazily processes data. Red extends this behavior by returning a ResultSeq, which generates SQL queries without executing them immediately. This allows additional filtering before executing the query. Given this, we can define a method to retrieve events with optional filters for a minimum ID and a set of event types:

method get-events(UInt $seq = 0, :@types) {
    my $events = Event.^all;
    $events .= grep(*.id > $seq)     if $seq;
    $events .= grep(*.type ⊂ @types) if @types;
    $events;
}

Query by Parent Types

An Event Store should also support querying events based on their parent types. To achieve this, we create a related table to store type hierarchies:

use Red:api<2>;
use Red::Type::Json;

unit model Event;

has UInt $.id      is serial;
has Str  $.type    is column;
has Json $.data    is column;
has      @.parents is relationship( *.type, :model<EventType> );

use Red:api<2>;

unit model EventType;

has Str $.type   is id;
has Str $.parent is id;
has     @.events is relationship( *.type, :model<Event> );

To add an event and store its type hierarchy:

method add-event($event) {
    Event.^create:
        |$event.to-data,
        :parents[
            $event.^mro.map: {
                %( parent => .^name )
            }
        ]
}

This inserts the event into the Event table and its ^mro parents as (type, parent) pairs into the EventType table.

Now, we update the event retrieval method to support filtering by parent types:

method get-events(UInt $seq = 0, :@types) {
    my $events = Event.^all;
    $events .= grep(*.id > $seq)               if $seq;
    $events .= grep(*.parents.parent ⊂ @types) if @types;
    $events;
}

Filtering by Data

Filtering events based on specific data fields is another essential feature. Red supports filtering JSON column data directly. For example, to find events where field1 starts with "bla":

Event.^all.grep: *.data<field1>.starts-with: "bla"

However, this approach checks each row individually, which may be inefficient. To optimize this, we store indexed event data separately:

use Red:api<2>;
use Red::Type::Json;

unit model Event;

has UInt $.id      is serial;
has Str  $.type    is column;
has Json $.data    is column;
has      @.parents is relationship( *.type, :model<EventType> );
has      @.fields  is relationship( *.event-id, :model<EventData> );

use Red:api<2>;

unit model EventData;

has UInt $.event-id is column{ :id, :references{ .id }, :model-name<Event> };
has Str  $.field    is required is id;
has      $.value    is column{ :id, :nullable };
has      $.event    is relationship( *.event-id, :model<Event> );

To add events and store indexed data fields:

method add-event($event) {
    Event.^create:
        |$event.to-data,
        :parents[$event.^mro.map: { %( parent => .^name ) }],
        :fields[$event.get-data.kv.map: -> $k, $v {
            next if $v ~~ Positional | Associative;
            %( field => $k, value => $v )
        }]
}

Now, when querying events based on data fields:

my $events = do if %pars {
    my $fields = EventData.^all;
    for %pars.kv -> $key, $value {
        FIRST {
            $fields .= grep: { .field eq $key && .value eqv $value }
            next;
        }
        $fields .= join-model:
            :name("field_$key"),
            EventData, -> $prev, $field {
                $prev.event-id == $field.event-id
                && $field.field eq $key
                && $field.value eqv $value
            }
    }
    $fields.map: { .event };
} else {
    Event.^all;
};

Conclusion

By implementing an Event Store using Red, we gain a flexible, efficient, and Raku-idiomatic way to manage and query event data.
With its powerful ORM features and SQL integration, Red simplifies event storage, filtering, and retrieval.
This approach ensures that Event Sourcing in Raku remains both accessible and performant.
Future enhancements could include support for more advanced queries, indexing strategies, and additional database backends.