DEV Community

Cover image for Introducing Small Swoole Entity Manager Strates
sebk69
sebk69

Posted on

Introducing Small Swoole Entity Manager Strates

#php #swoole #architecture #opensource

I'm happy to announce the first release of Small Swoole Entity Manager Strates.

The project adds a snapshot-oriented persistence model on top of Small Swoole Entity Manager Core 3.

The main idea is simple:

Instead of progressively modifying the data currently visible to users, build a complete new version beside it and expose it only when it is ready.

This is useful when an application needs to update a complete relational graph while guaranteeing that readers never observe a partially updated state.

Typical examples include:

  • product catalogs;
  • shop configuration;
  • pricing structures;
  • availability rules;
  • booking-related data;
  • any dataset composed of several related tables that must become visible as one consistent version.

The package supports MySQL and PostgreSQL.

The problem

Imagine a catalog composed of several tables:

catalog
  |
  +-- catalog_settings
  |
  +-- products
  |     |
  |     +-- product_tags
  |
  +-- tags
Enter fullscreen mode Exit fullscreen mode

Updating this graph directly can be problematic.

Suppose we need to replace the current catalog.

A traditional process may look like this:

update catalog
update settings
delete old products
insert new products
update tags
insert relations
Enter fullscreen mode Exit fullscreen mode

During that operation, another request may read the catalog.

Depending on when that request happens, it could observe:

new catalog
old settings
half of the new products
old tags
Enter fullscreen mode Exit fullscreen mode

Transactions can solve some of these situations, but keeping very large graph updates inside a long database transaction is not always desirable.

Strates uses another approach.

Build first, release later

Every version of a dataset receives a build strate.

A strate is represented by a UUID v7.

For example:

019c85a7-bec1-7284-93c7-b1a7fc52faca
Enter fullscreen mode Exit fullscreen mode

Every row belonging to that version stores the same buildStrate.

Conceptually:

Catalog A
buildStrate = 019c85...

Products
buildStrate = 019c85...

Tags
buildStrate = 019c85...

ProductTag relations
buildStrate = 019c85...
Enter fullscreen mode Exit fullscreen mode

The currently visible version is not determined by which rows were inserted most recently.

Instead, Strates maintains a small central release table:

released_strates
Enter fullscreen mode Exit fullscreen mode

Its role is essentially:

(scope, scope_id) -> currently released buildStrate
Enter fullscreen mode Exit fullscreen mode

For example:

catalog / shop-42
        |
        v
019c85a7-bec1-7284-93c7-b1a7fc52faca
Enter fullscreen mode Exit fullscreen mode

Readers only load rows belonging to that released strate.

The architecture

The architecture revolves around three concepts:

Scope
Scope ID
Build Strate
Enter fullscreen mode Exit fullscreen mode

Scope

A scope defines the business domain being versioned.

Examples:

catalog
configuration
booking
pricing
Enter fullscreen mode Exit fullscreen mode

Scope ID

The scope ID identifies one independent instance of that domain.

For example:

scope    = catalog
scopeId  = shop-42
Enter fullscreen mode Exit fullscreen mode

Another shop can have its own independently released catalog:

scope    = catalog
scopeId  = shop-99
Enter fullscreen mode Exit fullscreen mode

Build strate

The build strate identifies one complete candidate version.

Together they give us:

catalog / shop-42 / strate-A
catalog / shop-42 / strate-B
catalog / shop-99 / strate-C
Enter fullscreen mode Exit fullscreen mode

Only one strate is released for each (scope, scopeId) pair.

The lifecycle

A build can have four states:

building
released
failed
garbage_collected
Enter fullscreen mode Exit fullscreen mode

The normal workflow is:

        create
          |
          v
      building
          |
          | complete graph persisted
          v
       release
          |
          v
      released
Enter fullscreen mode Exit fullscreen mode

If something goes wrong:

building
   |
   v
 failed
Enter fullscreen mode Exit fullscreen mode

Older versions can later be removed:

released
   |
   | superseded
   v
garbage_collected
Enter fullscreen mode Exit fullscreen mode

Creating a new strate

A new build is created with:

$strate = $stratifiedPersist->createNewStrate(
    [
        CatalogManager::class,
        CatalogSettingsManager::class,
        ProductManager::class,
        TagManager::class,
        ProductTagManager::class,
    ],
    'catalog',
    $shopId,
);
Enter fullscreen mode Exit fullscreen mode

At this point the new version exists, but it is not visible to readers.

The application can safely construct the complete graph.

Persisting the graph

Entities are persisted with the new strate:

$stratifiedPersist->persist($catalog, $strate);

$stratifiedPersist->persist($settings, $strate);

$stratifiedPersist->persistMany(
    $products,
    $strate,
);

$stratifiedPersist->persistMany(
    $tags,
    $strate,
);

$stratifiedPersist->persistMany(
    $productTags,
    $strate,
);
Enter fullscreen mode Exit fullscreen mode

The old released graph is still active during the entire operation.

So while the new version is being constructed:

Readers
   |
   v
Released strate A
Enter fullscreen mode Exit fullscreen mode

while writers are preparing:

Building strate B
Enter fullscreen mode Exit fullscreen mode

The two versions coexist.

DATABASE

strate A
  catalog
  settings
  products
  tags
  relations

strate B
  catalog
  settings
  products
  tags
  relations
Enter fullscreen mode Exit fullscreen mode

Only strate A is currently visible.

Releasing the new version

Once the graph is complete:

$stratifiedPersist->release(
    'catalog',
    $shopId,
    $strate,
);
Enter fullscreen mode Exit fullscreen mode

The release pointer changes:

Before

catalog / shop-42
       |
       v
    strate A
Enter fullscreen mode Exit fullscreen mode

becomes:

After

catalog / shop-42
       |
       v
    strate B
Enter fullscreen mode Exit fullscreen mode

Readers now see the complete second graph.

There is no period where they see half of A and half of B.

This is the core architectural principle of Strates.

Constant-size publication

An important property of this approach is that publishing a large graph does not require rewriting the graph.

Suppose the candidate version contains:

1 catalog
1 settings row
50,000 products
10,000 tags
150,000 relation rows
Enter fullscreen mode Exit fullscreen mode

The expensive work happens while the version is still invisible.

Publishing it only requires switching the release metadata.

Conceptually:

UPDATE released_strates
SET validated_strate = :newStrate
WHERE scope = :scope
AND scope_id = :scopeId
Enter fullscreen mode Exit fullscreen mode

The amount of data being published does not determine the size of the release operation.

That makes the architecture particularly interesting for large datasets assembled asynchronously.

Relational graphs remain stratified

Versioning the root entity is not enough.

Relations also need to remain inside the same snapshot.

Consider:

Catalog
   |
   +-- Product
          |
          +-- ProductTag
                   |
                   +-- Tag
Enter fullscreen mode Exit fullscreen mode

Every stratified relation includes buildStrate in its mapping.

For example:

#[ToMany(
    ProductManager::class,
    [
        'id' => 'catalogId',
        'buildStrate' => 'buildStrate',
    ],
)]
private ?EntityCollection $products = null;
Enter fullscreen mode Exit fullscreen mode

This is important.

Without buildStrate, a relation loader could accidentally connect:

Catalog from strate B
Enter fullscreen mode Exit fullscreen mode

with:

Product from strate A
Enter fullscreen mode Exit fullscreen mode

Strates treats the build identifier as part of the relational boundary.

The same principle applies to one-to-one, one-to-many and many-to-many relationships.

Many-to-many relations

Small Swoole Entity Manager represents many-to-many relationships using a join entity.

For example:

Product
   |
ProductTag
   |
  Tag
Enter fullscreen mode Exit fullscreen mode

The join entity is also stratified.

Its foreign keys therefore include the build identifier.

Conceptually:

product_id
tag_id
build_strate
Enter fullscreen mode Exit fullscreen mode

This prevents links from crossing version boundaries.

A relation from:

Product / strate B
Enter fullscreen mode Exit fullscreen mode

cannot accidentally reference:

Tag / strate A
Enter fullscreen mode Exit fullscreen mode

Reading released data

The service can directly retrieve the released root:

$catalogs = $stratifiedPersist->findReleasedByScopeId(
    'catalog',
    $shopId,
);
Enter fullscreen mode Exit fullscreen mode

Internally the query is constrained using both:

scopeId
buildStrate
Enter fullscreen mode Exit fullscreen mode

Applications can also obtain a query builder:

$query = $stratifiedPersist->createReleasedQueryBuilder(
    'catalog',
    'catalog',
    $shopId,
);
Enter fullscreen mode Exit fullscreen mode

and continue adding normal query conditions.

The release pointer remains the source of truth.

What happens when a build fails?

Building a graph may involve many operations:

API calls
database reads
transformations
validation
persistence
Enter fullscreen mode Exit fullscreen mode

If something fails before publication, the current released graph remains untouched.

The candidate can be marked as failed:

$stratifiedPersist->markBuildAsFailed(
    $strate,
);
Enter fullscreen mode Exit fullscreen mode

Readers continue using the previous released strate.

This gives us a useful property:

failed build != broken production state
Enter fullscreen mode Exit fullscreen mode

The failed snapshot simply never becomes visible.

Garbage collection

Keeping every historical version forever would obviously be expensive.

Strates therefore includes garbage collection.

After a new version has replaced an older released version:

strate A -> old
strate B -> released
Enter fullscreen mode Exit fullscreen mode

the rows associated with A can be removed.

The default strategy deletes rows using buildStrate:

DELETE FROM product
WHERE build_strate = :obsoleteStrate;
Enter fullscreen mode Exit fullscreen mode

The process is executed across all configured managers belonging to the graph.

Strates also supports a partition-oriented garbage collector for systems where each strate maps to a database partition.

Why UUID v7?

Build identifiers use UUID v7.

UUID v7 combines globally unique identifiers with a timestamp-oriented layout.

This makes them convenient for identifying independently generated snapshots while retaining useful chronological characteristics.

For example:

019c85a7-bec1-7284-93c7-b1a7fc52faca
Enter fullscreen mode Exit fullscreen mode

The UUID itself identifies the build, while the metadata table maintains explicit timestamps and lifecycle state.

Metadata remains small

The business data can be large, but the coordination model is intentionally small.

Strates primarily manages two metadata concepts:

stratified_build
released_strates
Enter fullscreen mode Exit fullscreen mode

stratified_build tracks builds:

strate
scope
scope_id
status
created_at
updated_at
Enter fullscreen mode Exit fullscreen mode

released_strates tracks publication:

scope
scope_id
validated_strate
Enter fullscreen mode Exit fullscreen mode

The large business graph stays in the application's own tables.

Strates coordinates its versions.

Why not just use active = true?

An active flag on every row seems simple at first.

But publishing a graph containing hundreds of thousands of rows would mean changing hundreds of thousands of flags.

It also makes synchronization across related tables more complicated.

With a release pointer:

many business rows
        |
        v
one buildStrate
        |
        v
one release pointer
Enter fullscreen mode Exit fullscreen mode

Publication becomes an indirection problem instead of a mass-update problem.

That indirection is the main architectural idea behind the project.

Architecture overview

The whole system can be summarized like this:

                  +----------------------+
                  |   released_strates   |
                  |                      |
                  | catalog / shop-42    |
                  |        |             |
                  +--------|-------------+
                           |
                           v
                    buildStrate B
                           |
       +-------------------+-------------------+
       |                   |                   |
       v                   v                   v
    Catalog             Products              Tags
       |                   |
       v                   v
   Settings            ProductTags
Enter fullscreen mode Exit fullscreen mode

While another version can simultaneously exist:

buildStrate C
      |
      +-- catalog
      +-- settings
      +-- products
      +-- tags
      +-- relations

status: building
visible: no
Enter fullscreen mode Exit fullscreen mode

When C is ready:

release pointer

B -> C
Enter fullscreen mode Exit fullscreen mode

and the complete graph becomes visible.

Installation

The package is installed through Composer:

composer require small/swoole-entity-manager-strates
Enter fullscreen mode Exit fullscreen mode

It currently targets:

PHP 8.3+
Small Swoole Entity Manager Core 3
MySQL 8
PostgreSQL 16
Enter fullscreen mode Exit fullscreen mode

Quality and database testing

Because the package coordinates persistence and release semantics, database behavior is a central part of the test suite.

The first release is tested against both:

MySQL
PostgreSQL
Enter fullscreen mode Exit fullscreen mode

The project also enforces:

PHPStan level 9
PHP syntax validation
100% line coverage
Enter fullscreen mode Exit fullscreen mode

The integration suite covers complete relational graphs including:

  • one-to-one relationships;
  • one-to-many relationships;
  • many-to-many relationships;
  • snapshot isolation;
  • release switching;
  • scope isolation;
  • failed builds;
  • garbage collection;
  • invalid cross-build relationships.

Where this architecture fits

Strates is not intended to replace normal CRUD persistence.

For a simple entity updated independently:

UPDATE user SET name = ...
Enter fullscreen mode Exit fullscreen mode

normal persistence is simpler.

Strates becomes useful when a group of related entities represents one logical version.

A good mental model is:

If your users should either see version A or version B, but never A-and-a-half, the snapshot model may be a good fit.

Examples include:

catalog publication
pricing releases
configuration deployments
large imports
external synchronization jobs
generated datasets
Enter fullscreen mode Exit fullscreen mode

What's next?

This first release establishes the main architecture:

build
persist
release
read
garbage collect
Enter fullscreen mode Exit fullscreen mode

The goal is to keep that model small and predictable while making it usable for increasingly complex relational graphs.

The package is part of the Small Swoole ecosystem and is built directly on Small Swoole Entity Manager Core 3.

I'm interested in feedback from developers working with large relational datasets, asynchronous imports, configuration publication systems, or similar snapshot-based architectures.

If that sounds like a problem you've encountered, I'd be very interested to hear how you currently solve it.

Links

Repository : https://git.small-project.dev/lib/small-swoole-entity-manager-strates
Packagist : https://packagist.org/packages/small/swoole-entity-manager-strates

Top comments (0)