Small Swoole DB 2.0: Bringing Safer, Faster In-Memory Data Access to Swoole and OpenSwoole

When you build applications with Swoole or OpenSwoole, one of the biggest advantages is that your PHP process stays alive.

You can keep state in memory, share data between workers, avoid unnecessary network round trips, and build services that behave much more like long-running application servers than traditional request-per-process PHP applications.

Swoole\Table is an important part of that model.

It gives us a fast shared-memory structure that can be accessed by multiple workers. But as applications grow, working directly with tables can become increasingly low-level.

You quickly start needing things such as:

• structured records;
• indexes;
• filtering;
• range queries;
• sorting;
• pagination;
• joins;
• safer updates;
• predictable behavior under concurrent workers.

This is the problem small/swoole-db is designed to solve.

It provides a database-like abstraction on top of Swoole/OpenSwoole shared-memory tables while keeping the performance characteristics that make them interesting in the first place.

---

Why use it in a Swoole or OpenSwoole application?

The main objective is not to replace PostgreSQL, MySQL, Redis, or another persistent database.

The goal is different.

small/swoole-db is useful when your application already has data that naturally belongs inside the lifetime of your Swoole server.

For example:

• shared application state;
• cached domain objects;
• routing information;
• service discovery data;
• temporary datasets;
• counters and runtime statistics;
• precomputed data;
• worker-shared lookup tables;
• fast intermediate results.

You still get the shared-memory benefits of Swoole\Table, but with a higher-level API.

Instead of progressively rebuilding a small query engine inside every application, you can work with tables, records, selectors and indexes directly.

---

A more database-like API over shared memory

A Swoole table is intentionally simple.

That simplicity is excellent for performance, but it means application code normally has to take care of a lot of additional logic.

small/swoole-db adds concepts developers already understand from database systems.

You can define columns, store records and query those records using selectors.

That makes application code easier to read because data-access logic becomes explicit rather than being spread across loops and conditional statements.

For example, instead of manually iterating through an entire shared-memory table to find matching records, the selector layer can express filtering, ordering and pagination directly.

This becomes increasingly useful when the same shared-memory dataset is accessed from multiple parts of an application.

---

Indexes for shared-memory data

Scanning a few rows is cheap.

Scanning thousands of rows for every request is not.

This is where indexes become important.

small/swoole-db supports indexes over table fields so queries can reduce the number of records they need to inspect.

Indexes support operations such as:

=
<
<=
>
>=

and can also be used with composite values.

For applications maintaining larger runtime datasets, this changes how Swoole\Table can be used.

Instead of treating it only as a key/value structure, it becomes practical to query shared data using secondary values as well.

---

Composite indexes

Real applications often need more than one field to identify useful subsets of data.

For example:

tenant_id + status
customer_id + date
service + environment

Composite indexes make these cases possible without creating application-specific lookup structures for every combination.

Prefix searches can also be useful when only the first part of a composite index is known.

This brings the API closer to the way developers already reason about indexes in relational databases.

---

Fast range queries

A shared-memory database abstraction only makes sense if it stays fast.

For ordered indexed values, the index structure allows the engine to avoid scanning unrelated records.

This is particularly useful for values such as:

• timestamps;
• prices;
• sequence numbers;
• priorities;
• dates;
• numeric metrics.

Range queries can therefore navigate the index rather than repeatedly filtering the entire table.

---

Swoole and OpenSwoole support

A project using this kind of low-level shared-memory functionality should not force developers into one ecosystem unnecessarily.

The library is tested against both:

• Swoole
• OpenSwoole

The intention is to keep the public API independent from the choice between the two runtimes wherever possible.

That makes it easier to use the library in existing projects and also gives teams more flexibility when choosing or migrating their runtime.

---

Shared-memory concurrency is not the same as normal PHP

This is probably the most important technical lesson when building abstractions over Swoole\Table.

The table itself is shared memory.

Your PHP objects are not.

And a sequence such as:

$value = $table->get($key);
$value++;
$table->set($key, $value);

is not automatically an atomic transaction just because the underlying table is shared.

Indexes make this even more complicated.

An index update may involve:

1. locating a tree node;
2. updating the node;
3. updating its children;
4. adding or removing a table key;
5. changing allocator metadata;
6. potentially rotating tree nodes.

If two workers mutate those structures simultaneously without coordination, perfectly valid individual operations can combine into an invalid final state.

A major focus of the latest work on small/swoole-db has therefore been index consistency under concurrent Swoole workers.

Index mutations are synchronized so a complete structural update is treated as one operation.

This is especially important for long-running applications, where a rare race condition can otherwise leave corrupted state alive for the rest of the process lifetime.

---

Balanced indexes matter

An earlier index implementation used a normal binary search tree.

That works correctly when data arrives in a favorable order.

Unfortunately, production data is often anything but random.

Consider values such as:

1
2
3
4
5
...

or timestamps that naturally increase over time.

A basic binary search tree can degenerate into something very close to a linked list.

The difference is dramatic.

In our tests with 2,000 sorted unique values, the old behavior could take almost a minute to construct the index, while randomly distributed inserts were dramatically faster.

The solution was to move to a balanced treap-based index structure.

The tree still follows normal search ordering, while deterministic priorities keep its shape balanced.

That avoids the catastrophic behavior produced by naturally sorted application data.

For Swoole applications storing timestamps, identifiers or sequential measurements, this is a particularly important improvement.

---

Better storage for duplicate indexed values

Another common situation is having many records with the same indexed value.

Think about:

status = pending
country = FR
active = true
category = 12

Thousands of records may legitimately belong to the same index entry.

Originally, keeping those keys inside the index node itself created two problems:

• the serialized value had a fixed size limit;
• inserting a new key became increasingly expensive as the list grew.

The current implementation separates index nodes from their associated table keys.

This removes the old JSON-size limitation and makes large duplicate groups much more practical.

---

O(1) duplicate membership lookup

Even after separating the keys from index nodes, there was another optimization opportunity.

Imagine an index containing 2,000 records with the same value.

Before adding another record, the index needs to know whether that table key is already registered.

Scanning the existing 2,000 keys every time makes building the group increasingly expensive.

The new implementation maintains a reverse membership map:

table key -> index node + slot

Normal membership lookup is therefore effectively constant-time.

In one duplicate-heavy benchmark using 2,000 records on OpenSwoole, insertion time dropped from roughly:

2,515 ms

to around:

307 ms

That's roughly an 8× improvement in that workload.

The implementation also keeps a compatibility path for indexes created before the reverse membership map existed.

---

Query execution now stops when it has enough results

Another deceptively expensive pattern is:

scan 20,000 rows
build 20,000 result objects
then return LIMIT 10

It produces the right answer, but it defeats the point of asking for ten records.

Simple selectors can now stream records directly from the table.

When there are no joins and no ORDER BY, the selector can:

1. iterate records;
2. evaluate the WHERE condition;
3. skip the requested offset;
4. collect the requested number of rows;
5. stop.

So:

LIMIT 10

no longer implies materializing the entire dataset first.

This is particularly useful for APIs, dashboards and internal services where pagination is extremely common.

---

Faster ORDER BY

Sorting has also been optimized.

A comparison function can run many thousands of times during a sort.

Small inefficiencies inside that comparator therefore multiply very quickly.

The ordering path now avoids repeated alias resolution and repeated value evaluation during the same comparison.

In a benchmark involving:

• 20,000 rows;
• five aliases;
• two ordering keys;

execution went from approximately:

2.3 seconds

to:

0.63 seconds

That's roughly a 73% reduction in execution time for that workload.

---

Fewer native table reads

Performance work is not always about a new algorithm.

Sometimes the fastest call is simply the one you no longer make.

Record hydration previously performed redundant native table accesses when reconstructing values and metadata.

The read path now reuses the data already obtained from the runtime wherever possible.

Depending on the workload and runtime, this reduced parts of the record/index read path by roughly 30–60% in our benchmarks.

For operations executed thousands of times per request or worker cycle, those small reductions add up.

---

Transactional index updates

Indexes also need to stay synchronized when existing records change.

Consider:

record X initially has status = pending
record X changes to status = accepted

It is not enough to insert X into the accepted index entry.

The previous pending membership also needs to disappear.

The library now handles indexed replacements explicitly and includes rollback behavior when an intermediate operation fails.

The same principle applies when table or index capacity is exhausted.

A failed storage operation should be visible to the application.

It should not silently leave half of an index mutation behind.

---

Deleted index nodes are reusable

Long-running Swoole servers make resource lifecycle especially important.

If an indexed value disappears completely, its tree node should not stay allocated forever.

Otherwise, an application that continuously creates and deletes values could slowly exhaust the configured index capacity even if only a small number of values are active at any given time.

Unused index nodes are now returned to a free list and can be reused by future values.

This makes index capacity reflect the active dataset much more closely.

---

Stability is a performance feature too

When talking about performance libraries, it's tempting to focus only on benchmark numbers.

For long-running Swoole applications, stability is just as important.

A fast operation that occasionally corrupts a shared index is not fast in any useful sense.

Recent development has therefore focused on both sides:

performance

and

predictable behavior under failure and concurrency.

The test suite now covers scenarios including:

• concurrent writers;
• indexed updates;
• deletion after updates;
• empty indexes;
• composite index prefixes;
• duplicate-heavy indexes;
• capacity exhaustion;
• node reuse;
• corrupted internal metadata;
• rollback behavior;
• range filters;
• constant-left comparisons;
• pagination;
• joins and ordering.

The current test matrix is run on both Swoole and OpenSwoole.

---

Announcing small/swoole-db 2.0.0

All of this work is coming together in small/swoole-db 2.0.0.

The 2.0 release is focused on two things that matter especially for Swoole and OpenSwoole applications:

stability under long-running, concurrent workloads and substantially better performance as datasets grow.

The major improvements include:

• concurrency-safe index mutations;
• balanced treap-based indexes;
• safe index replacement and rollback;
• reusable index nodes;
• scalable duplicate-key storage;
• O(1) duplicate membership lookup;
• faster record hydration;
• optimized ORDER BY;
• streaming LIMIT and pagination;
• improved composite-index behavior;
• explicit capacity failures instead of silent corruption;
• extensive testing on both Swoole and OpenSwoole.

The objective of 2.0.0 is not simply to make a few benchmarks faster.

It is to make Swoole\Table practical as the foundation of a richer shared-memory data layer that can stay alive alongside your application for days or weeks while remaining predictable.

If you are building APIs, workers, realtime services or other long-running PHP applications with Swoole or OpenSwoole, I'd be very interested to hear what kinds of shared-memory workloads you're using.

---

Repository: https://git.small-project.dev/lib/small-swoole-db

Packagist: https://packagist.org/packages/small/swoole-db