DEV Community: OceanBase User Group

OceanBase Vectorized Execution Engine — Accelerating Large-Scale Data Analytics

Charles Wu — Sun, 05 Jul 2026 13:33:46 +0000

What vectorized execution means for billion-row query performance

When people think about database query performance, the first things that come to mind are usually obvious optimizations such as indexes, partitioning, and parallelism. But in large-scale analytical scenarios, the factor that truly limits performance is often something more fundamental and far less visible — the way the query execution engine itself processes data. OceanBase introduced a vectorized execution framework starting in V3.2, and after several rounds of iteration, it completed a systematic rewrite of the data format, operators, expressions, and storage layer of the execution stack in V4.3, fundamentally improving the data-processing efficiency of the query execution engine. This article starts with the difficulties traditional execution models run into under analytical workloads, then gradually introduces the design philosophy and key technical implementation of OceanBase’s vectorized engine, along with the measured gains this technical evolution delivers.

1. Why a Vectorized Execution Engine Is Needed

To understand the value of a vectorized execution engine, we first need to understand what it replaces — the traditional Volcano model (also known as the Iterator model) — and the fundamental limitations this classic model, in use for three decades, exposes when facing modern analytical workloads. We will begin with how the Volcano model works, trace its execution through a real SQL query, and then analyze, one by one, the four layers of performance cost hidden behind the row-at-a-time processing pattern.

How the Volcano Model Works

Almost all relational databases use the Volcano model to execute queries. The core idea of this model is: each operator implements a next() interface that returns one row of data per call, and an upper-level operator pulls data by repeatedly calling the next() of the operator below it.

In the Volcano model, each operator only needs to care about its own logic; operators are composed through a unified interface, and adding a new operator does not affect the implementation of existing ones. This design works very well in OLTP scenarios (point lookups, small-range scans, processing tens to thousands of rows) — the cost of function calls is negligible compared to disk I/O and network latency.

The problem appears in analytical queries. Take a simple SQL statement as an example:

SELECT SUM(amount) FROM orders WHERE status = 'paid';

Under the Volcano model, the execution proceeds as follows:

The aggregation operator calls the next() of the filter operator
The filter operator calls the next() of the scan operator
The scan operator reads one row from the storage layer and returns it to the filter operator
The filter operator checks status = 'paid'; if not satisfied, it repeats steps 2-3 until it finds a row that meets the condition
The filter operator returns that row to the aggregation operator
The aggregation operator adds amount to the running SUM
Go back to step 1, until the scan is complete

If the orders table has 1 billion rows, of which 30% satisfy status = 'paid', then the scan operator's next() is called 1 billion times and the filter operator's next() is called 300 million times. That is roughly 1.3 billion virtual function calls just to complete a single summation.

Note that the business logic of this SQL statement is essentially just “iterate over an array and sum up the elements that meet a condition.” If you hand-wrote this logic in C, a simple for loop would do the job. But under the Volcano model, the same logic is scattered across three operators and coordinated through 1.3 billion function calls. The cost of this abstraction is acceptable in OLTP scenarios — when processing a few dozen rows, the overhead of a dozen-odd function calls is negligible. But when the data volume scales up to the billion-row level, this cost becomes impossible to ignore.

The Hidden Costs, Exposed Layer by Layer

1.3 billion function calls is already a startling number, but it is only the tip of the iceberg. When we look closely at the actual execution behavior of the Volcano model at the CPU microarchitecture level, we find that the row-at-a-time processing pattern systematically wastes hardware resources across four dimensions.

Function Call Overhead

Each virtual function call involves a series of operations: vtable pointer lookup, indirect jump, stack frame allocation, and register save/restore. On a modern x86 CPU, the cost of one indirect function call is about 5–10 nanoseconds. For 1 billion rows, the function calls alone consume about 5 seconds — 5 seconds during which no useful computation happens at all, with the system merely jumping back and forth between calls and returns.

CPU Cache Invalidation

The cache line of a modern CPU is typically 64 bytes. Under a row-store layout, even if the filter condition depends only on the status column, the system still has to load the entire row into the cache hierarchy, causing a large number of irrelevant fields to consume memory bandwidth and CPU cache. With main-memory access latency around 60-100ns, the cumulative wait time from cache misses over 1 billion rows can reach tens of seconds. If, instead, the same column is stored contiguously, a single cache line can hold 16 INT32 values, improving cache utilization by an order of magnitude.

CPU Pipeline Flushes

Modern CPUs use 15–20 stage pipelines to process instructions in parallel. Row-at-a-time processing introduces a large number of unpredictable branches through per-row NULL checks, filter-condition evaluations, and so on. For a filter condition with a 30% pass rate, the branch misprediction rate is about 20–30%, and each branch misprediction causes a pipeline flush of about 15 clock cycles. The cumulative loss from a single filter condition over 1 billion rows is about 1 second, and the compounding effect of multiple conditions and NULL checks in a complex query significantly slows down execution.

Idle SIMD Capability

Modern CPUs provide SIMD instruction sets (for example, AVX2’s 256-bit registers can process 8 INT32 values at once, and AVX-512 can process 16 at once). But in the row-at-a-time processing mode, a single comparison operation uses only 32 bits of data, leaving 94% of the hardware capability in a 512-bit register completely idle. This is like having a 16-lane highway but allowing only one car through at a time.

AP Workload Characteristics Amplify Every Problem

OLTP queries usually process a few rows to a few thousand rows, so the absolute amounts of function call overhead, cache misses, and pipeline flushes are all small and are masked by I/O latency. But the characteristics of AP queries are entirely different:

Full table scans: involving millions to billions of rows
Aggregation statistics: must traverse all the data
Multi-table JOINs: intermediate results may be larger than the original tables

When the number of rows processed jumps from the thousands to the billions, the per-row 5–10ns function call overhead swells from microseconds to seconds, and the per-row cache miss swells from milliseconds to minutes. There is a fundamental mismatch between the row-at-a-time processing of the Volcano model and the demands of analytical workloads.

The conclusion is now clear: the row-at-a-time processing pattern lags behind the capabilities of modern hardware across all four dimensions — function calls, cache utilization, pipeline efficiency, and SIMD utilization. We need a new execution model that changes, from the ground up, the way data flows between operators — so that every clock cycle of the CPU is used, as much as possible, on useful computation rather than wasted on waiting and jumping.

This is exactly the design starting point of the vectorized execution engine.

2. The Core Ideas of OceanBase’s Vectorized Engine

Once the fundamental problems of the traditional Volcano model are clear, the direction for a solution emerges as well: since the core ills of row-at-a-time processing are “one function call per row, one random memory access per row, one branch evaluation per row,” the most direct response is — to stop processing row by row. OceanBase’s vectorized engine is built around four core principles: batch processing to amortize overhead, a columnar memory layout to improve cache hit rates, SIMD instructions to unleash hardware parallelism, and branch elimination to protect the CPU pipeline. We will expand on each below.

Batch Processing

The core insight of vectorized execution is: amortize the overhead of a function call across N rows of data.

Returning to the earlier example SELECT SUM(amount) FROM orders WHERE status = 'paid', under vectorized execution (assuming a batch size of 1024):

The scan operator reads the status column and the amount column for 1024 rows at once
The filter operator compares 1024 status values in a tight loop and produces a bitmap marking which rows satisfy the condition
Based on the bitmap, the aggregation operator accumulates the amount values of the qualifying rows in a tight loop
The above process repeats until all data is processed

Number of function calls: 1 billion rows / 1024 = about 976,000 (roughly 2.93 million across the three operators combined). Compared with the 1.3 billion calls of the Volcano model, this is a reduction of nearly three orders of magnitude.

More importantly, the tight loops in steps 2 and 3 — simple for loops iterating over arrays — are exactly the pattern that CPUs are best at optimizing. The compiler can apply auto-vectorization, the CPU prefetcher can accurately predict the access pattern, and the branch predictor can maintain 100% accuracy when facing a branch-free loop body.

Columnar Memory Layout (Data Frame)

Inside OceanBase’s execution layer, data is organized by column, in a structure called the Data Frame. A Data Frame is a contiguous block of memory (up to 2MB) that holds the data of all columns in the current batch.

The reason the Data Frame size cap is set to 2MB: the L2 cache of a modern CPU is typically 256KB-1MB (per core). Keeping the hot working set near the L2 capacity minimizes L2 cache misses. The 2MB cap leaves headroom for multi-column scenarios; the effective amount of data actually used is usually smaller than the L2 capacity.

The tuning logic for vector size: suppose a query involves 10 INT64 columns (8 bytes per column). With a vector size of 1024, the data volume of one batch = 10 × 8 × 1024 = 80KB, which fits comfortably in the L2 cache. If the query involves 100 columns, the same vector size would produce an 800KB working set, which may exceed the L2 capacity. In that case the engine automatically reduces the vector size, ensuring the working set always stays resident in L2.

SIMD Instruction Utilization

Columnar batch data is naturally suited to SIMD processing. Take the filter condition WHERE amount > 100 as an example, assuming amount is of type INT32 and the batch size is 1024:

Scalar execution:

for (int i = 0; i < 1024; i++) {
    result[i] = (amount[i] > 100) ? 1 : 0;  // 1024 comparisons, 1024 branches
}

AVX2 SIMD execution:

__m256i threshold = _mm256_set1_epi32(100);
for (int i = 0; i < 1024; i += 8) {
    __m256i data = _mm256_loadu_si256((__m256i*)(amount + i));
    __m256i cmp = _mm256_cmpgt_epi32(data, threshold);  // 8 comparisons, 1 instruction
    // Write the result into the bitmap
}

128 loop iterations replace 1024 iterations, with zero branch instructions, processing 8 values per iteration. The theoretical speedup is 8×; in practice, due to memory bandwidth limits, it is usually 4–6×.

The OceanBase vectorized engine implements SIMD-accelerated paths on x86 (SSE/AVX2), ARM (NEON), and PPC architectures.

Branch Prediction Optimization

Vectorized operators internally adopt a “default-write-then-override” coding pattern. Take processing a column that contains NULL values as an example:

Traditional approach (one branch per row):

for (int i = 0; i < 1024; i++) {
    if (is_null[i]) {
        result[i] = NULL_VALUE;
    } else {
        result[i] = compute(val[i]);
    }
}

Every iteration of this loop has a conditional branch. If the NULL rate is 5%, the branch predictor has to handle a 95/5 distribution and will still produce about 5% mispredictions.

Vectorized approach (split loops, eliminating branches):

// First loop: branch-free, the CPU pipeline runs at full load
for (int i = 0; i < 1024; i++) {
    result[i] = compute(val[i]);
}
// Second loop: only processes NULL rows (usually very few)
for (int i = 0; i < null_count; i++) {
    result[null_positions[i]] = NULL_VALUE;
}

The first loop has no branches at all, so the compiler can fully unroll and vectorize it. The second loop only iterates over the positions of NULL rows (usually a tiny minority), and even though it contains a branch, its impact is negligible because the iteration count is small.

3. Key Technical Improvements

V4.3 is a systematic rewrite of the OceanBase vectorized engine. After the previous two versions validated the effectiveness of the core ideas — batch processing, columnar layout, SIMD utilization, and branch elimination — V4.3 shifted its focus to the “last mile” of performance engineering: deep, end-to-end optimization spanning the execution layer’s data format, the internal implementation of core operators, and the interaction between the storage layer and the execution layer. We expand on each technical module below.

Data Format Optimization

The execution-layer data format prior to V4.3 was inherited from an early design and carried indirect-addressing overhead: variable-length types referenced the actual data through an array of pointers, so accessing a single value required two memory hops (first read the pointer, then read the data). When processing billions of rows, this indirect addressing produced a large number of random memory accesses.

V4.3 redesigned the columnar data description format of the execution layer:

Fixed-length types (INT8/16/32/64, FLOAT, DOUBLE, DATE, TIMESTAMP, etc.): use a packed array to store values directly. This eliminates all pointer indirection, and data is laid out strictly contiguously in memory. The 1024 values of an INT32 column occupy only 4KB of contiguous memory, and the CPU prefetcher can perfectly predict the access pattern.
Variable-length types (VARCHAR, TEXT, BLOB, etc.): use a structure of an offset array plus a contiguous data region. The offset array records the starting position of each value within the data region, and the data region stores the actual content of all values. Accessing the i-th value simply requires: data[offset[i] ... offset[i+1]]. Compared with allocating memory row by row, this eliminates memory fragmentation and pointer hops.

The core design goal of these formats is: when the CPU processes a batch of data, all memory accesses are sequential and predictable. Sequential access patterns allow the CPU’s hardware prefetcher to load subsequent data into the cache in advance, reducing the actual cache miss rate to nearly zero.

Exploiting Batch-Level Metadata

In real business data, most batches contain no NULL values, and in most batches all rows pass the upstream filter condition. The V4.3 engine maintains metadata flags at the batch level:

The has_null flag: whether the current batch contains NULL values
The all_active flag: whether all rows in the current batch are unfiltered

When has_null = false, the operator can completely skip the NULL-checking logic internally — not skip it row by row, but skip the entire code path. This means that in the code the compiler generates for the "no NULL" scenario, the NULL-related instructions do not exist at all, occupy no instruction cache, and consume no execution resources.

When all_active = true, the operator does not need to check the skip bitmap and can perform an uninterrupted, continuous operation over the entire array. This allows both compiler auto-vectorization and manual SIMD optimization to reach theoretical peak performance.

These two seemingly simple flag bits have a cumulative effect over 1 billion rows: they skip billions of conditional evaluations and eliminate the corresponding branch-prediction overhead.

Sort Operator Rewrite

A traditional sort implementation moves entire rows of data during comparisons and swaps. Suppose we want to sort 1 million rows by 2 INT64 columns, where each row also has 10 non-sort columns (80 bytes of payload in total):

Total size per row: 2 × 8B (sort key) + 80B (payload) = 96 bytes
Each swap during sorting moves 96 bytes
Average number of comparisons in quicksort: 1 million × log₂(1 million) ≈ 20 million
Volume of data movement: 20 million × 96B = about 1.9GB

V4.3’s approach is to separate the sort key from the payload:

The sort phase materializes only the sort key columns and the row number (row_id): 2 × 8B + 4B = 20 bytes/row
After sorting completes, the payload columns are backfilled according to the sorted row numbers

The volume of data movement in the sort phase: 20 million × 20B = about 400MB, a reduction of nearly 5×. More importantly, 20 bytes/row means the L2 cache (256KB) can hold about 12,800 rows of sort data, whereas 96 bytes/row can hold only about 2,700 rows. The cache hit rate during sorting improves dramatically.

Hash Join Operator Rewrite

Multi-column join conditions are a common pattern in AP queries, for example:

SELECT ... FROM a JOIN b ON a.id = b.id AND a.dt = b.dt

The traditional implementation computes the hash value of the two join key columns separately and then combines them. V4.3 encodes a multi-column fixed-length join key into a single fixed-length column: two INT64 columns are encoded into a 16-byte contiguous value.

The benefits this brings:

Hash computation: a single hash operation on 16 bytes of contiguous memory, rather than two independent hashes plus a merge. This eliminates the overhead of one hash call and the merge operation.
Equality comparison: a single 16-byte memcmp (which can be done with one 128-bit SIMD comparison instruction), rather than two 8-byte comparisons plus a logical AND.
Memory access: probing the hash table only requires reading one contiguous block of memory, rather than jumping to the locations of two different columns.

When the hash table contains millions of records and the probe side has tens of millions of rows, the savings from these single operations accumulate into a substantial overall performance improvement.

Aggregation Operator Optimization

For high-frequency aggregation operations such as COUNT, SUM, and MIN/MAX, V4.3 provides specialized implementations that bypass the abstraction layer of the general aggregation framework. For example, SUM(int64_column) in a scenario with no NULLs and no filtered rows is compiled into a pure array-accumulation loop, and the compiler can directly generate SIMD accumulation instructions.

Storage-Layer Vectorization (Push-Down)

The execution flow before V4.3 was: the storage layer decodes the data → converts it to the execution-layer format → the execution layer applies the filter condition. V4.3 pushes a variety of computations down to the storage layer:

Filter Push-Down: filter conditions are executed directly on the encoded data. For dictionary-encoded columns, WHERE city = 'Shanghai' is converted into WHERE dict_code = 7 (assuming the dictionary code for 'Shanghai' is 7). Integer comparisons are an order of magnitude faster than string comparisons, and dictionary-encoded values are usually INT16/INT32 — more compact than the original strings and offering better cache utilization.

Aggregate Push-Down: leverages pre-aggregated metadata at the micro block level. If the storage layer has already recorded statistics such as the row count, SUM, and MIN/MAX of each micro block at write time, SELECT COUNT(*) FROM t can simply accumulate the row-count metadata of the micro blocks, with no need to decode or scan the actual data rows at all.

GroupBy Push-Down: when the grouping column is dictionary-encoded and the number of distinct values (NDV) is low, aggregation can be completed directly in the storage layer using the dictionary-encoded values as the grouping key, avoiding the overhead of decoding the original values and re-hashing.

SIMD-Accelerated Projection: projection templates are customized by column type and length, using SIMD instructions to decode and copy column data in bulk. For example, extracting 1024 INT32 values from the columnar format requires only 128 SIMD load + store operations using AVX2 instructions.

Adaptive Batch Size

OceanBase serves both TP and AP workloads simultaneously, and the vectorized engine automatically selects the batch size based on the query type:

TP queries (point lookups, short-range scans): use a smaller batch size. A point lookup usually returns only 1 row, and an overly large batch would bring unnecessary initialization overhead.
AP queries (full table scans, large-scale aggregations): use a larger batch size (such as 1024 or larger) to fully exploit the advantages of SIMD and caching.

This adaptive mechanism allows the same engine codebase to serve two completely different workload patterns without any manual tuning.

4. Measured Performance Gains

The following are OceanBase’s internal benchmark results, comparing the overall execution time with the vectorized engine turned on versus off under the same hardware environment:

The differences in speedup across the three benchmarks are worth analyzing. The improvement for TPC-H (40×) is significantly higher than for ClickBench (14×), because TPC-H contains a large number of multi-table JOINs and complex sort operations — exactly the scenarios where the Sort/Hash Join operator rewrites yield the greatest benefit. ClickBench is dominated by single-table, wide-table aggregations, where the bottleneck lies more in the I/O and decoding stages, and the function-call overhead of the Volcano model accounts for a relatively small share; therefore the speedup from vectorization is also relatively low. TPC-DS (26.5×) falls between the two, reflecting its query characteristics that mix simple aggregations with complex JOINs.

The incremental gains of V4.3 relative to the earlier vectorized implementations (V4.0/V4.2):

TPC-H: 68% of queries achieve a 10%-40% performance improvement
TPC-DS: 56% of queries achieve a 10%-40% performance improvement
ClickBench: an overall improvement of about 30%

The significance of this set of data is that V4.0 was already a complete vectorized engine, capable of achieving most of the acceleration shown in the table above. On top of that, V4.3, through its data-format rewrite and coordinated operator design, still achieved an additional 10%-40% improvement. This shows that “last mile” performance engineering — eliminating indirect addressing, optimizing memory layout, and exploiting batch-level metadata — can still unlock considerable performance headroom on top of an existing vectorized framework.

Summary

Through the four core techniques of batch processing, columnar memory layout, SIMD instruction utilization, and branch elimination, the vectorized execution engine frees the CPU from the inefficient state it endures under the Volcano model — where it spends most of its time waiting for data and jumping between functions — enabling it to concentrate its compute power on useful data computation. OceanBase introduced the vectorized framework starting in V3.2 to validate the technical direction, made vectorization the default execution path and implemented TP/AP adaptivity in V4.0, and then completed a systematic, coordinated rewrite of the data format and operator implementation in V4.3, achieving 14× to 40× query acceleration on industry-standard benchmarks such as ClickBench, TPC-H, and TPC-DS. Beyond this, the OceanBase vectorized engine continues to evolve, with broader operator coverage to come; operators that are not yet fully vectorized, such as Window Functions and Recursive CTEs, will be rewritten over time.

For users, all of these improvements take effect by default and require no additional configuration. Whether for everyday report queries, real-time data dashboards, or large-scale data exploration and multi-table correlation analysis, the vectorized engine quietly does its work at the underlying layer, allowing OceanBase to deliver significantly faster analytical query responses under the same hardware resources.

OceanBase Real-Time AP: From Data Scanning to Plan Stability

Charles Wu — Sun, 05 Jul 2026 13:18:10 +0000

How to keep real-time analytical queries cost-stable under sustained writes

Two Pain Points of Real-Time AP

Real-time AP is about whether a system can keep delivering stable analytical capabilities while writes, changes, and queries all happen concurrently over the same dataset. For the business, the key question is not “can the SQL execute,” but rather: as data keeps changing, can the system still maintain low-latency responses, high throughput, and controllable resource costs.

From an engineering perspective, the challenges of real-time AP mainly show up at two levels.

First, the analytical target is constantly changing. Queries must work directly against high-frequency, dynamically updated data rather than a stable, static snapshot. Under dynamic data, the cost of reading data grows, which in turn increases the compute burden.

Second, when there is a large amount of dynamic data, suboptimal execution plans become more likely. An unreasonable execution plan multiplies cross-node transfer and compute pressure.

Therefore, the key to real-time AP is not just making a single SQL statement run fast, but continuously controlling the overall cost of analytical queries under sustained writes.

OceanBase Handles a Dataset Where “Baseline + Incremental” Coexist

In real-time AP scenarios, OceanBase does not process a single static dataset, but rather a dataset where “baseline data + incremental data” coexist. Baseline data that can be reused stably corresponds to the Major SSTable; newly written and newly changed data first goes into the MemTable, then converts into the Minor SSTable, forming the incremental path.

The significance of this organization in OceanBase is that the system does not have to sacrifice analytical capability for the sake of the write path, nor give up analytical efficiency for the sake of writing data.
If all data were immediately reorganized into a form better suited for analysis, the higher data-reorganization cost would be pushed onto the write path, directly affecting sustained write efficiency. Conversely, if the system relied solely on the incremental path over the long term, analytical queries would bear higher read amplification and merge costs, making query latency and resource usage hard to keep stable.

Therefore, the difficulty of real-time AP is not simply optimizing one side — writes or analysis — to the extreme, but maintaining a long-term balance between these two kinds of constraints. OceanBase’s basic philosophy is: the incremental path prioritizes write efficiency, while the baseline prioritizes analytical efficiency.

The incremental path first ensures that new data can be continuously written and become visible to queries in a timely manner. As dumping (minor compaction) and merging (major compaction) proceed, this data is gradually folded into the baseline, and large-scale analytical scans are mainly handled by the columnar baseline.

The primary goal of the incremental side is still to serve write efficiency and transactional consistency, so overall it is closer to a row-based access path; but this does not mean it gives up analytical read efficiency. At the physical storage level, OceanBase balances compression and read efficiency for hot data, reducing the extra cost incurred when incremental data participates in analytical queries — without adding too much write overhead.

The six layers of capability discussed below are all built on this premise: what OceanBase aims to solve is not a one-time analytical speedup over static data, but long-term, stable support for analytical queries under dynamic data conditions.

Read Less: How Column Store, Encoding, Skip Index, and Pushdown First Drive Down Read Cost

In real-time AP, the first layer of cost usually comes from unnecessary reads. If large amounts of irrelevant data are already being moved during the read phase, the benefits of subsequent computation and parallel optimization will be noticeably weakened. So the first thing this layer must solve is: how to read as little as possible, and complete filtering as early as possible.

OceanBase’s approach is to push column pruning, filtering, and partial aggregation as close to the data as possible. For baseline data, it uses a columnar organization better suited for analytical reads; for incremental data, it keeps a row-based path better suited for sustained writes. During query execution, the system reads only the required columns where possible, reducing the I/O overhead from irrelevant columns.

Columnar encoding and Skip Index mainly operate on baseline column-store data. Columnar encoding helps reduce read and decode overhead, and supports partial filtering directly on compressed data; Skip Index maintains block-level statistics for block-level skipping — when a data block, statistically, cannot possibly satisfy the filter conditions, the system skips that block directly, avoiding unnecessary reads and decoding.

Incremental data is still primarily organized in a row-based form, and is merged with baseline results according to transactional visibility at query time. Compared with the baseline column store, the incremental path prioritizes write efficiency; however, at the physical storage level, it also tries to balance compression and filtering efficiency over recent data — though such capabilities should not be simply equated with the Skip Index on the baseline column store.

At the same time, filtering and common aggregations (including some Group By operations) can also be pushed down to the storage layer for execution, further reducing the volume of intermediate data sent up to the execution layer.

This layer addresses the read-cost problem: minimize ineffective scans, ineffective transfers, and ineffective intermediate results, providing a smaller and cleaner input for the next layer of “fast computation.”

Compute Fast: How Vectorized Execution, Storage Vectorization, and SIMD Improve Batch Computation Efficiency

Once the scan scope converges, the performance bottleneck usually shifts from I/O to compute efficiency. Real-time AP queries often involve aggregation, sorting, expressions, and join computation; if execution is still primarily row-by-row, CPU cache utilization and instruction execution efficiency will be constrained.

OceanBase’s approach is to replace row-by-row execution with vectorized execution. The vectorized execution engine processes data in batches, reducing frequent function calls and branch decisions, and lets operators and expressions run over a more contiguous data layout. The point is not simply to increase the number of threads, but first to improve the efficiency with which a single operator processes a batch of data.

Meanwhile, storage vectorization aligns projection, predicate, and aggregation-related computations more closely with the vectorized data format, reducing unnecessary data conversion and movement. Combined with SIMD, the system can process more data elements within the same compute cycle, further lowering the compute cost per unit of data.

This layer addresses: after “reading less,” can the data also be “computed fast.” Going one level deeper, the problem is no longer just local compute efficiency — it rises to the overall cost of distributed execution.

Globally Better: One-Phase Distributed Plans, Join Order/Algorithm, Data Distribution, Parallelism, and Auto DOP

Complex analytical queries typically involve multi-table joins, cross-partition access, and data exchange. At this layer, local optimality does not equal global optimality: a join path that looks reasonable in isolation may incur higher cross-node transfer costs; an access path that is faster locally may make overall execution slower.

During optimization, OceanBase adopts a one-phase distributed plan generation approach, evaluating access paths, join order, join algorithms, data distribution properties, and parallel properties within a unified cost framework. In other words, the system considers distributed execution cost during the plan generation phase, rather than first selecting a plan that looks reasonable on a single machine and then bolting on distributed processing afterward. The reason is that these factors are inherently coupled: join order affects the size of intermediate results, data distribution affects whether redistribution is needed, and the parallel strategy directly affects resource usage and response time.

On the parallelism side, OceanBase improves the response speed of complex queries through parallel execution, but a higher degree of parallelism is not always better. Auto DOP (Automatic Degree of Parallelism) leaves the decision of how much parallelism to use to the optimizer; it combines query cost and system state to decide whether to parallelize and what the degree of parallelism should be, balancing query latency against resource consumption.

This layer addresses: how to make distributed queries more reasonable from a holistic perspective. But “better now” does not mean “always effective afterward,” so the next layer must address plan stability under changing data.

Continuously Effective: Statistics, Cardinality Estimation, and Plan Stability

In real-time AP, performance fluctuations in the same class of SQL across different time periods are common. Often, the root cause is not that the SQL text changed, but that after data size, data distribution, and update cadence change, cardinality estimation and cost judgments become skewed, leading to plan regression or frequent plan switching.

OceanBase’s approach is to consider statistics maintenance and plan stability together. Statistics collection is not a one-time action; it must be continuously maintained in light of table size, partition characteristics, update patterns, and maintenance windows. For very large tables or high-frequency change scenarios, there are also more targeted collection strategies.

The quality of cardinality estimation directly affects access path selection, join strategy, and parallel decisions. Once statistics lag or become distorted, the optimizer may make wrong judgments in subsequent steps. For this very reason, in real-time AP, statistics relate not only to “whether this plan is chosen correctly,” but also to “whether the same class of queries can remain stable going forward.”

Scale Read Throughput: Column-Store Replicas, Routing, and Read-Traffic Isolation

When the concurrency of analytical queries keeps rising, path optimization of a single SQL statement alone can hardly guarantee overall throughput. As analytical read requests increase, a dedicated analytical read path needs to be scaled out, and routing strategies are used to direct different types of read requests to the corresponding replicas, reducing resource contention.

OceanBase’s approach is to introduce column-store replicas (C replicas) as an analytical read-only path. Column-store replicas support independent deployment and routing strategies, allowing analytical queries to execute on a more suitable read path and avoiding having all read requests concentrated on the same path.

At the routing layer, the system can combine policy conditions to decide whether to prioritize the column-store replica path, and work with weak/strong consistency reads and fallback mechanisms to balance consistency, availability, and throughput. Its core value mainly manifests in two aspects: first, scaling analytical read throughput; second, achieving TP/AP read-traffic isolation, reducing mutual interference between different workloads.

This layer completes the read-path expansion. The next step is to handle the repeated computation that recurs frequently in high-frequency analysis.

Reduce Repeated Computation: Materialized Views, Refresh, Mlog, and Query Rewrite

Real-time AP often involves a large number of periodic, repeated queries, such as fixed reports, repeated aggregations, and stable dimensional statistics. If everything is recomputed from the base tables each time, the system will repeatedly scan and compute, hurting both throughput and resource efficiency.

OceanBase’s approach is to move part of the repeated computation forward into maintainable precomputed results. Materialized views materialize and persist query results, relying on refresh to catch up with base-table changes: full refresh suits scenarios with few updates that can tolerate a longer maintenance window; incremental refresh suits scenarios with frequent changes that require controlling refresh overhead. Real-time materialized views typically no longer bind query freshness to refresh frequency; instead, they compute the already-materialized results together with incremental changes at query time, so that even under frequent changes, queries can still yield results closer to the current data state, supporting efficient real-time analysis.

Mlog (materialized view log) provides the change basis for incremental maintenance, enabling the system to update materialized results by the scope of changes rather than fully recomputing every time. The query rewrite capability lets the optimizer automatically hit materialized results when conditions are met, reducing the cost of manually rewriting SQL on the business side.

This layer addresses: for analytical tasks with high-frequency, stable patterns, how to avoid recomputing from raw detail every time. At this point, the six layers of capability form a complete optimization path for analytical cost.

Typical Scenarios: How the Six Layers Combine to Take Effect

These six layers are not used in isolation; they combine in different ways depending on workload characteristics. For single-table or wide-table aggregations, the main benefit usually comes from “read less + compute fast”; columnar reads, block-level skipping, and vectorized execution often determine the bulk of the performance gain.

It is worth adding that real-time AP does not correspond only to “ultra-large-scale data” scenarios. In gaming businesses, the more common need is online analysis with “moderate data size but high timeliness requirements,” such as campaign monitoring, retention-and-conversion dashboards, player-behavior segmentation, and real-time operational-metric reviews. Such scenarios likewise require second-level responses and stable throughput, so the six layers of capability still hold — only the optimization focus shifts more toward query latency stability and controllable resources under read-write concurrency.

For detail filtering and multi-dimensional statistics, the capability combination usually extends further to “globally better + continuously effective.” Such scenarios require not only more stable cardinality estimation and more reasonable distributed plans, but also rely more on an appropriate parallel strategy to avoid hard-to-explain performance fluctuations in the same class of queries at different times.

For periodic reports and fixed analytical tasks, “reducing repeated computation” is often key; materialized views, refresh, and query rewrite can significantly lower the cost of repeated aggregation. For systems with continuously growing analytical concurrency, “scaling read throughput” is a necessary complement; column-store replicas and routing strategies determine how far the analytical read side can scale, and how well it can be isolated from other read traffic.

Conclusion

The core difficulty of real-time AP is not getting a single analytical SQL statement to run, but keeping query latency, throughput, and resource cost within an acceptable range over the long term, under conditions of sustained writes and continuous change.
OceanBase’s answer does not rely on any single feature. Instead, on the premise of “baseline + incremental coexistence,” it progressively reduces the overall cost of online analysis across several layers: reading, computation, distributed execution, plan stability, read-path expansion, and precomputation.
For this very reason, when OceanBase real-time AP becomes a long-term capability rather than a one-off performance test, this technical framework is easier to understand, validate, and iterate on — and is better suited to continuously supporting business evolution.

Your OpenClaw Bill Is Bleeding Tokens. Here’s What We Measured — and How to Fix It

Charles Wu — Sun, 28 Jun 2026 13:57:43 +0000

Memory bloat, compaction loss, and a retrieval-first path: ~32% less token spend on the AppWorld dev split — without dumbing the agent down.

Photo by micheile henderson on Unsplash

Developers who actually ship with LLMs know one truth by heart: the context window is not free. Every extra thousand tokens nudges the invoice up and the latency out.

If you run OpenClaw (an agent stack that leans hard on long-horizon sessions), that anxiety gets concrete fast. Picture this: last week you spent two hours with your agent debugging production — logs, configs, experiments — and burned through 30k tokens of back-and-forth. This week you pick up where you left off, and the agent answers: Hi! Which refactor are we talking about?

So you spend a few thousand tokens re-explaining context. The model spends a few thousand more re-understanding. And you still might not land the same mental model you had last Tuesday.

Those 30k tokens? Mostly gone.

That is not a one-off glitch. OpenClaw’s default memory story quietly feeds two token black holes.

Two black holes that blow up your token budget

1) The more you remember, the more you pay

OpenClaw’s agent writes important state into MEMORY.md, and that file gets fully injected into the system prompt on every request. The longer you use the setup, the larger MEMORY.md grows—and every API call pays for the whole thing as input tokens.

Bootstrap caps exist (for example, a 20k-character default per file, 150k total), but long before you hit the ceiling, a bloated prompt starts crowding the model’s working space. OpenClaw’s agent knows information can get lost — so it writes even more aggressively into MEMORY.md, which accelerates the bloat.

2) The more you forget, the more you burn tokens fixing mistakes

When sessions get long, OpenClaw leans on two mechanisms:

Compaction: OpenClaw asks an LLM to summarize older conversation chunks to free context.
Memory flush: before compaction, OpenClaw spins up an embedded agent to decide what to persist into memory/YYYY-MM-DD.md.

But compaction is lossy compression by design, and OpenClaw’s retrieval-side slicing hard-cuts along line and character budgets (by default, 400 tokens per chunk) without respecting semantic boundaries. Important context can get cut mid-thought, recall quality drops, your agent makes mistakes, you rework, rework creates more chat, and you trigger compaction again sooner.

Tool calls are an accelerant

Tool outputs — web_fetch pages, exec dumps—can be huge per message—up to 400k characters per tool result in the worst case. That fills sessions fast. Those intermediates usually should not land in MEMORY.md, but they can still contain value you do not want to discard. Either way, tool-heavy runs tighten the doom loop.

The uncomfortable tradeoff: remembering everything gets expensive; forgetting costs correctness. You need a third path.

A third path: cloud memory that steers tokens instead of hoarding them

seekdb M0 is a cloud memory plugin for OpenClaw. The idea in one sentence:

Do not dump all memory into the system prompt. Before each turn, retrieve only the memory slices that match the current topic — and inject just those.

Unlike loading the full MEMORY.md on every request, M0 stores memory as discrete facts in a cloud database, with vector embeddings and full-text indexes. At conversation start, M0 runs hybrid retrieval (BM25 keyword scoring + vector similarity) and injects the top relevant facts. After each chat, M0 extracts new facts from the dialogue, compares them to what already exists, and decides whether to add, update, or skip.

What that buys you:

MEMORY.md stops ballooning—durable memory lives outside the always-on system prompt, so input tokens drop.
Session resets stop being catastrophic — memory persists and rehydrates without you paying again to restate context you already gave.
Cross-device continuity — your memory is not trapped on one laptop.

For most users, this is meant to feel invisible: you talk; M0 manages memory in the background.

OpenClaw’s native persistence tends to route through compaction over the full session (including tool outputs) and a flush agent that decides what to write — both are comparatively heavy and lossy. M0 splits what to store from how to store it into two phases.

Phase 1: fact extraction

After a conversation, M0 extracts facts from user ↔ assistant text only — not from tool-call intermediates — and uses an LLM to produce atomic facts.

Example: The user is Alex, a database engineer based in Austin. becomes three independent facts.

Hard rules we enforce during extraction:

Preserve time information (do not collapse went to Hawaii last year into a timeless went to Hawaii).
Keep the original language (no automatic translation during extraction).
Do not extract sensitive information.

Phase 2: memory decisions

M0 does not blindly insert facts. M0 retrieves similar existing memories, then asks an LLM whether the new fact should be:

ADD
UPDATE
DELETE (contradictions)
NONE (already covered)

In practice, M0 treats DELETE conservatively as NONE for auto-capture — M0 only adds and updates existing memories and does not proactively delete them, to reduce accidental erasure.

Example decisions:

New fact: "Went to Hawaii last May."
Existing memory: "Has been to Hawaii."
→ UPDATE (time detail added)

New fact: "Doesn't like pizza anymore."
Existing memory: "Likes pizza."
→ UPDATE (preference changed)

New fact: "Is a database engineer."
Existing memories: "Name is Alex" + "Is a database engineer."
→ NONE (already covered)

Implementation detail worth noting: in the memory-decision LLM call, each existing memory’s original ID is replaced with a short temporary index (0, 1, 2, …) so the decision model is less likely to hallucinate or garble long integer IDs. If the decision model returns an index that cannot be mapped back, M0 gracefully falls back to treating the fact as new.

Why this matters for tokens: M0’s fact-extraction stage ignores tool transcripts, so you avoid paying an LLM to read 400k-character blobs just to mint memories.

Tool-result compression: deterministic, zero LLM spend

M0 also attacks session inflation at persistence time. When OpenClaw persists tool results to session history, M0’s tool_result_persist hook replaces raw output with a structured summary—rule-based, no LLM tokens.

Illustrative shape:

Raw: curl returned a 3,000-line JSON payload

Compressed:
  tool: web_fetch
  status: success
  output: 3,000 lines / 48K characters
  preview: {"users":[{"id":1,"name":"Alice"... (300 chars)

M0’s summaries are not about perfect fidelity. They aim for high compression while preserving what happened, whether the tool succeeded, and a short preview.

Compared with OpenClaw’s native compaction, which feeds the entire session (including tool dumps) into a summarizer, M0’s hook-based compression is closer to upstream budgeting: you control what enters the LLM pipeline, instead of waiting until you overflow and then compressing reactively.

Experience + Skill: spend tokens on the right kind of reuse

M0’s memory layer answers who this user is and what they care about. Another common waste pattern in agent stacks is different:

Your OpenClaw agent may have skills, but not durable, reusable know-how distilled from real runs — so every similar task becomes another expensive exploration loop.

M0 splits playbooks into two layers:

Experience (strategy layer): a tight summary of approach + key cautions.
Skill (operations layer): structured steps, prerequisites, and pitfalls.

The two layers link by reference: your OpenClaw agent can pull strategy first, then expand operational detail only when needed — which helps keep the active prompt compact.

Under the hood, M0 stores these in OceanBase (a distributed SQL database) with separate tables for Experience and Skill, indexing title and description with both vector and full-text indexes. Retrieval runs four parallel signals — title vector, description vector, title full-text, description full-text — then merges with RRF (Reciprocal Rank Fusion).

Why four channels? In M0’s retrieval stack, title matching helps lock onto the right name, description matching helps lock onto the right content, vectors help with semantic equivalence (for example, build a playlist vs create a playlist), and full-text tends to win on exact strings like API names and error codes. That complementary mix is meant to make retrieval both accurate and broad: your OpenClaw agent should not need ten mid-confidence hits (think ~0.6 relevance) just to be safe, when three high-confidence items (~0.9) are enough to execute — and that gap maps straight to fewer tokens in the prompt.

M0 also stages knowledge ingestion: M0’s pipeline detects a procedure in traces → structures a Skill (steps / pitfalls / prerequisites) → dedupes (for example, vector similarity > 0.75 merges) → runs moderation → stores. When M0 extracts Experience records, M0’s extractor can see stored skills and reference skill IDs, which keeps links generated rather than hand-maintained.

AppWorld numbers: how much did we actually save?

Early on, we used LoCoMo to probe memory behavior, but found it skews toward chit-chat agents rather than work agents like OpenClaw — where evaluation is harder (skills, multi-step reasoning, structured API payloads).

For a fairer workload, we switched to the AppWorld benchmark — a suite of 750 autonomous agent tasks framed as realistic, stateful challenges. In short, AppWorld’s evaluation is built around state-based unit tests: an agent can complete tasks in different ways, and AppWorld’s harness still checks for unintended harm during the run.

The AppWorld benchmark paper (ACL 2024 resource paper, arXiv:2407.18901) states in the abstract:

The state-of-the-art LLM, GPT4O, solves only ~49% of our ‘normal’ tasks and ~30% of ‘challenge’ tasks, while other models solve at least 16% fewer.

The AppWorld blog (appworld.dev) puts it plainly:

Even the best LLM, GPT-4o, performs quite poorly. E.g., it completes only ~30% of the tasks in the challenge test set correctly.

In our controlled setup on AppWorld dev (54 tasks, 15-step cap, no pre-loaded distilled skills), GPT-4o’s baseline was ~24% (13/54 solved) — below the headline pass rates quoted in AppWorld’s public materials, which reflect a different task mix and evaluation harness than this stripped-down run.

Controlled comparison on AppWorld dev (54 tasks, 15-step cap)

Our setup: we ran traces with Hermes + Qwen 3.6-plus (34/54 solved, 63%), kept all 54 trajectories, then distilled into:

M0 path: 85 experiences (with skill_refs)
Hermes path: 44 SKILL.md files

Then we evaluated GPT-4o on each distilled knowledge base. Only two knobs differ: distillation + storage/retrieval.

Results:

Note: pp = percentage points (absolute change in pass rate, not relative % change).

Headline takeaways:

M0 net +8 tasks (examples mentioned: Spotify-style flows, cross-app tasks, Venmo-style flows), with some wins traded for losses.
Hermes net -1 on GPT-4o in this setup — no positive gain versus baseline.

Why M0 beat file-skill matching in our analysis

Retrieval precision: M0’s vector search can match the task description semantically; Hermes’ filename/tag matching does not understand semantics the same way, so Hermes misses paraphrases. Example (localized for a global audience): Create a Beyoncé playlist vs Bundle twenty Taylor Swift tracks together should route to the same underlying skill — M0’s vectors tolerate wording drift better than brittle naming.
Context hygiene: M0’s Experience records stay light (title-line scale); Hermes’ SKILL.md files can read like full manuals and crowd the model.
On-demand expansion + dedupe: M0 uses skill_refs to load operational detail only when needed, and M0 performs semantic deduplication by pairing vector-similarity checks with an LLM merge so near-duplicate skills fold together instead of piling up. Hermes may inject all matching skills at once, and collisions among Hermes’ SKILL.md filenames can overwrite useful variants.

Efficiency (same GPT-4o runs as the table): average steps 9.5 → 6.2 (-35%), tokens 2.56M → 1.74M (-32%). Even failures become cheaper failures — less thrash, less exploration tax.

Teach once with a strong model, run forever with a cheaper one

Rough cost sketch (our pricing assumptions — not a live vendor quote):

GPT-5.4 one full pass: ~$57.6 at $22.5 / 1M tokens
GPT-4o baseline: 2.56M tokens → ~$25.6 at $10 / 1M
GPT-4o + M0 distilled experience: 1.74M tokens → ~$17.4 at $10 / 1M

Note (GPT-5.4 line, illustrative): Blended $/M on ~2.56M tokens in our draft; not a literal line item on OpenAI’s price list. Recompute from your own traces, then confirm current rates on the OpenAI API pricing page before you budget.

Our playbook: let GPT-5.4 or Claude Sonnet 4.6 solve the hard version once; M0 distills traces into Experience + Skill; then route repeat work to GPT-4o (or cheaper) with higher pass rates, fewer steps, and a smaller bill than the old naive rerun.

The production takeaway is obvious: in a typical agent product, most requests are repetitive patterns. You do not need the most expensive model on every call — either let a strong model teach the task once, or have a human guide a weaker model through one clean run — and then later runs can finish on their own, grounded in distilled experience.

Beyond one user’s workspace: once an Experience picks up enough positive feedback, M0 can publish it to a shared space where any other M0-connected agent can retrieve it — your solved mistakes stop being only yours. M0’s vector dedupe folds overlapping discoveries together, contributor metadata accrues, and that crowd knowledge is meant to grow out of distillation itself — not through a separate manual editorial pipeline.

One-sentence install

OpenClaw is built around the idea that the assistant should do the heavy lifting, not a human babysitting every step — and seekdb M0’s install path is written the same way: you send your OpenClaw assistant a single line, for example:

Read https://m0.seekdb.ai/SKILL.md and install and configure M0 per the instructions.

After that, the agent is expected to check the installed OpenClaw version, obtain an Access Key, install the m0 plugin, apply the openclaw.json / gateway settings in one shot, and restart the gateway—without you clicking through a setup wizard.

Humans can still sanity-check the service:

# health check
curl -s https://m0.seekdb.ai/health

# create a memory instance
curl -s -X POST https://m0.seekdb.ai/api/instances/ \
  -H "Content-Type: application/json" \
  -d '{"name": "my-memory"}'

The returned ak field is your Access Key for authenticated memory operations.

Try it: wire up M0, then tell your OpenClaw agent a handful of real details about you — seekdb M0 will usually auto-extract about five or six facts, run them through the memory-decision step, and persist them in the cloud. On later chats it should pull your technical preferences back in instead of cold-starting the interview from zero.

At that point it already knows who you are — so you should not have to spend tokens re-introducing yourself.

Wrapping up

So why does OpenClaw token usage spike? Because the default memory path leans on MEMORY.md full-load plus reactive compaction and file-scattered recall. The prompt gets crowded; history gets summarized away; OpenClaw’s agent may not even know what to search for. You pay for remembering, you pay again for forgetting, and you pay a third time for re-discovery.

M0’s bet is simpler to state than it is to build:

Free memory from the always-on context — store independently, retrieve on relevance, persist across sessions.

More crucially: distill execution into reusable Experience + Skill, then retrieve sharply — M0-style high-precision recall beats padding the prompt with maybe relevant bulk.

Our AppWorld comparison is the punchline: same model, same tasks, swap the knowledge system, and you move from 2.56M → 1.74M tokens while pass rate climbs ~15 pp in our reported setup.

Spend tokens on thinking — not on re-learning what you already solved.

Sources

OpenClaw: https://openclaw.ai/
seekdb M0: https://m0.seekdb.ai/
PowerMem (open source): https://github.com/oceanbase/powermem
AppWorld: https://appworld.dev/
seekdb D0: https://d0.seekdb.ai/

Existing M0 users: this upgrade applies automatically — Experience and Skill records accumulate in M0 during normal agent use, with no extra configuration.

New users: send the one-liner install prompt to your OpenClaw agent and let it walk the setup.

The first time you pay tuition on a mistake, you should not have to pay full tuition again.

Why Your Vector Database Benchmark Is Wrong for AI Agents

Charles Wu — Sun, 28 Jun 2026 13:42:29 +0000

Key Takeaways

Most vector database benchmarks (ann-benchmarks, vendor pages) test bulk-load + read-only — but AI agents actually run streaming workloads: concurrent writes and reads at millisecond latency.
Under streaming load, P99 latency under concurrency — not QPS or serial latency — determines whether your agent’s SLA holds. Across 6 vector databases, P99 jitter ranged from 1.1× to 10.3× when concurrency was added.
The root cause is architectural: most engines accumulate index segments under streaming writes, so concurrent queries fanout and contend on CPU. We rebuilt seekdb v1.3.0 around two fixed indexes (delta + snapshot HNSW) to avoid this — and saw 22× QPS and 19× P99 improvement over our own v1.2.0.

If you’re picking a vector database for an AI agent, you’re probably looking at ann-benchmarks or vendor performance pages. Those benchmarks all run the same shape of test: bulk-load the dataset, build the index, then run read-only queries.

That is not what an agent does.

An agent’s real workload looks like this:

for step in agent.run():
    memory.write(step.observation)        # continuous writes
    relevant = memory.search(step.query)  # millisecond-later reads

Writes and reads happen together. They happen concurrently. The interval between them is milliseconds, not minutes. This pattern has a name — streaming workload — and VectorDBBench has a test case designed specifically for it: StreamingPerformanceCase. Sustained writes at a fixed rate plus concurrent queries. The same shape your agent runs in production.

VectorDBBench is maintained by Zilliz (the company behind Milvus), so it’s a third-party open-source benchmark.

The Metric Everyone Skips: How Much Does Your P99 Move Under Concurrency?

Test setup: Cohere 10M dataset (768-dim), 16 vCPU / 64 GiB, identical HNSW index parameters across all systems (M=16, ef_construction=256, ef_search=200), sustained write rate of 500 rows/sec.

A note on the two seekdb rows. You’ll notice seekdb appears twice in the chart — v1.2.0 and v1.3.0. v1.3.0 is the latest release, and it’s a deliberate architectural rewrite specifically targeting streaming workloads (we’ll explain what changed in the next section). We kept v1.2.0 in the comparison on purpose: it lets you see, on the same hardware and the same dataset, how a conventional vector-database design behaves under streaming load versus the redesigned one.

Most people look at benchmark charts and read off two numbers: QPS and serial latency. But your agent doesn’t run single-threaded in production. What actually determines whether your SLA holds is concurrent P99 — and how much it inflates when you add concurrency.

Look at the “P99 Jitter” group in the chart:

Elasticsearch: 10.3× — Serial P99 of 5.2ms (faster than seekdb on the cold path), but the moment you add concurrency it climbs to 53.6ms.
Milvus: 9.7× — Serial 15.9ms, concurrent 153.6ms.
seekdb: 1.1× — From 19.7ms to 21.7ms. Barely moves.

This is not a tuning problem. It’s an architecture problem. The next section explains why.

Full benchmark scripts and configs: github.com/oceanbase/vdb-streambench. PRs welcome to add more systems.

Why Streaming Workloads Blow Up P99

Milvus, Elasticsearch, and Qdrant all perform well in the workloads they were designed for: bulk ingestion followed by read-only queries. They were built around that shape, and they are good at it.

But streaming writes expose a structural assumption baked into all of them: every batch of new data produces a new index segment. At query time, the engine has to fan the request out to N segments, run a k-NN search against each one, and merge the results. With a single query thread that’s manageable. But once you run M concurrent query threads against N segments, you have N×M units of work contending for CPU, and P99 explodes.

Most vector databases let their segment count grow with streaming writes, so concurrent queries fight harder and harder for CPU. seekdb keeps the segment count fixed at exactly two — so it doesn’t.

Concretely, seekdb v1.3.0 introduced two mechanisms specifically for streaming:

The write path never touches the index. When a transaction commits, all that happens synchronously is a write to the redo log. A separate Change Stream pipeline asynchronously consumes the redo log in the background and applies vectors to an in-memory delta HNSW index. Writes and index construction are physically decoupled — writes never block on indexing, and indexing never blocks on writes.
The query path always hits exactly two indexes. seekdb maintains a delta HNSW (the incremental layer that absorbs new writes) and a snapshot HNSW (the steady-state main index), modeled after the LSM-tree pattern from KV stores. A query runs k-NN against both indexes and merges the result. The number of indexes is fixed regardless of how much data you’ve written, so concurrent queries don’t contend on a growing fanout.

We learned this the hard way. The seekdb v1.2.0 row in the chart — 69 QPS, concurrent P99 of 410ms — is what we shipped before the rewrite. The old write path built indexes synchronously, so we hit exactly the architectural problem described above. The 22× QPS and 19× latency improvement in v1.3.0 came entirely from these two changes. Same product, same dataset, same hardware.

Agents Need More Than Speed — They Need an Undo Button

Performance is one half of the agent problem. The other half is something most benchmarks don’t even try to measure: agents need to make speculative changes to their data.

An agent might:

Update memory with a hypothesis it isn’t sure about yet
Run an A/B experiment on its own state
Try a tool call that could write garbage rows

You don’t want any of that touching production state directly. You need a sandbox, and you need a clean way to roll back.

Most vector databases don’t have a primitive for this. seekdb implements Copy-on-Write directly in the storage engine:

-- Snapshot in seconds, no data copy
FORK DATABASE agent_state TO sandbox_42;

-- Agent does whatever it wants in the sandbox
USE sandbox_42;
INSERT INTO memory (embedding, content)
VALUES ('[0.1,...]', 'new observation');

-- Speculation succeeded → merge back
MERGE TABLE sandbox_42.memory INTO agent_state.memory
       STRATEGY THEIRS;

-- Speculation failed → drop it, mainline is unaffected
DROP DATABASE sandbox_42;

This is kernel-level COW, not application-layer snapshot/restore. The fork is instant, no data is copied, and each sandbox is a fully writable database — schemas, vector indexes, auto-increment columns all behave normally. Three conflict resolution strategies (FAIL, THEIRS, OURS) let you decide exactly how much of an agent's writes you trust. Both FORK DATABASE and FORK TABLE are supported, so you can branch at whichever granularity matches your use case.

Hybrid Retrieval in a Single SQL Query

Agent retrieval is rarely pure vector similarity. You usually want to combine vector distance with structured filters and sometimes full-text matching — show me the top 10 documents authored by user 42 since January, that match “quarterly report”, ranked by similarity to this embedding.

In seekdb, that’s one SQL statement:

SELECT id, title, l2_distance(emb, '[0.12,0.34,...]') AS dist
FROM docs
WHERE MATCH(content) AGAINST ('quarterly report')
  AND author_id = 42
  AND created_at > '2026-01-01'
ORDER BY dist APPROXIMATE LIMIT 10;

Vector, full-text, and scalar filters are pushed down into a single execution plan — no client-side merging of multiple round-trip results. Full MySQL wire protocol compatibility means LangChain, LlamaIndex, Dify, and any MySQL client connect with no adapter.

30 Seconds to Try It

pip install -U pyseekdb

import pyseekdb

client = pyseekdb.Client(path="./agent_state.db")
memory = client.get_or_create_collection(name="episodic")
# Round 1: write agent observations
memory.upsert(
    ids=["1", "2", "3"],
    documents=[
        "user prefers dark mode",
        "user speaks English and Chinese",
        "user timezone is UTC+8",
    ],
)
memory.refresh_index()
results = memory.query(query_texts="ui preferences?", n_results=1)
print(results["documents"])
# -> [['user prefers dark mode']]
# Round 2: write a new observation, refresh, query immediately
memory.upsert(ids=["4"],
              documents=["user saw pricing page 3 times today"])
memory.refresh_index()
results = memory.query(query_texts="purchase intent signals", n_results=1)
print(results["documents"])
# -> [['user saw pricing page 3 times today']]

No server, no schema migration, embedded mode runs in-process. Writes go through the same async indexing pipeline as the server build, so when you need a write to be queryable immediately, call refresh_index() once. Switching to server or distributed mode is a one-line connection-string change. There's also a hosted Cloud trial — no signup, free for 7 days, single curl command.

About seekdb

seekdb is fully open source under Apache 2.0, built by the OceanBase team. You’re probably already running on OceanBase indirectly — it’s in production at Alipay, Taobao, DiDi, and Xiaomi, among others. seekdb inherits the same storage engine and SQL executor, focused specifically on the hybrid vector + relational workloads that agents need. Six months in, the project has 2,500+ GitHub stars and is integrated with LangChain, LlamaIndex, Dify, and Coze.

If you’re picking a database for an agent, take 30 seconds and run the demo above. If your current vector database has a StreamingPerformanceCase number you're proud of, we'd love to add it to the comparison.

⭐ github.com/oceanbase/seekdb — a star helps more people find the project, and gives us reason to keep investing in it.

Questions or want to discuss your agent workload: GitHub Issues · GitHub Discussions

Why an All-in-One Data Foundation Matters: Harness, Tape, and a Database-Native Path

Charles Wu — Sun, 28 Jun 2026 13:31:22 +0000

From model + Harness to Tape and one data foundation — why agent runtime data should live in the database from the first line of code.

Photo by Brett Jordan on Unsplash

The agent race is quietly shifting from the model layer to the data layer. When agents run, they produce vast volumes of semi-structured trace data — high-frequency writes, long lifecycles — and traditional database architectures struggle to keep up. Shuttling that data back and forth across observability platforms, vector stores, and caches further erodes the efficiency of the record → distill → feed back loop.

This reveals a critical divide:

Building a database downward from an agent framework is not the same as extending a mature database upward to connect with agent frameworks. The starting points differ, and so do the cost structures.

On the latter path, data is a first-class citizen from the first line of code. Run, record, distill, evaluate, and feed back — all within one foundation, without the overhead of cross-system data movement.

That is where an all-in-one data foundation matters: it turns the agent data loop into an internal cycle, not a fragmented patchwork of engineering pieces.

Starting from the definition of Harness, this article draws on the open-source Bub project, discusses layered agent architecture, and lands on a database-native Harness approach — including OceanBase’s exploration and value in this space.

I. Understanding Agents, Harnesses, and How They Relate

A complete agent can be expressed as model + Harness.

Harness covers every engineering component outside the model. Like tack on a horse, Harness is the full toolkit for steering a model toward its destination — reins, saddle, route — which at the engineering level maps to feedback mechanisms, logging systems, and training methods.

A harness has a clear layered structure. Layer 1 is provided by the coding-agent builder or SDK vendor, including base tools and external interfaces. Layer 2 is extended on the user side with the components they need — business logic such as RAG systems, memory systems, and BI pipelines.

In agent scenarios, the model itself is not a continuously stateful system — it returns responses to requests without awareness of concrete business state. What lets agents work reliably in products and teams are the context management, tool invocation, state recording, run-trace tracking, effectiveness evaluation, and data flow responsibilities that Harness takes on.

In this process, we gradually identify and abstract key elements, defined as primitives (Primitive). System prompts, Skills, task-completion methodologies, and multi-agent communication mechanisms are all important primitives that emerge from practice. Standardizing these primitives and incorporating them into Harness improves business performance and expands capability on one hand, and gradually productizes Harness itself on the other.

Data collected from Harness is equally vital. It evaluates workflow effectiveness and, after de-identification, can form standard datasets for training the next generation of models. As models improve, they feed back into primitive discovery and refinement within Harness — even correcting past behavior — forming a continuously improving flywheel. The diagram below (from the LangChain blog) illustrates this loop clearly.

II. Building Extensible Agents: The Bub Project

Bub is an open-source Python agent project on GitHub. Its design reflects a key approach to controlling agent complexity: balancing stability and flexibility through a lean kernel and plugin-based extension.

Mainstream agent products such as ChatGPT, Qwen (Alibaba’s conversational AI), ModelScope services, and low-code platforms like Dify and Flowise already ship a built-in agent loop. A core problem remains: agent capability must match the business scenario precisely. Skills and tools can extend capability, but to complete tasks efficiently you still need to assemble a toolset for each specific scenario.

Products such as OpenClaw, Nanobot, and Hermes Agent bundle too many features together. That creates two problems: feature interference and cognitive load for users; for developers, high system complexity and difficult maintenance (for example, OpenClaw upgrades often break many features across the product). Such tightly coupled designs are hard to use as-is in production. Many vendors therefore repackage a specific version or build entirely in-house.

Bub takes a different architecture strategy: build a lightweight kernel and extend capabilities through plugins. Extra functionality is separated into plugins. Only a carefully designed lean kernel is maintained to implement a stable agent loop; required business capabilities are introduced step by step through feature plugins. Users need only verify that plugins are working correctly. If a plugin fails, they remove it and restore service — greatly improving maintainability.

Bub’s core design philosophy is not about how powerful a single agent is, but about how stages are divided within a single interaction. Whether it is Bub’s built-in agent or externally integrated Codex or LangChain, either can get the work done. Bub breaks each interaction into explicit stages — conversation state construction, prompt assembly, channel input/output definitions, and more. This staged breakdown makes flow control possible: hooks expose integration points at each stage, rather than piling all logic into a single agent.

A key design is removing mandatory binding on output. Traditional systems bind message replies strictly to the input channel. Bub allows an agent to stay silent in certain scenarios — returning no message. That looks like a flaw in a personal-assistant setting, but in multi-user or multi-agent collaboration, silence that avoids noise is a friendly trait.

The community is now seeing a wave of approaches that standardize and modularize agent design, for example:

Agents.md — inject system- and task-related prompts.
Skills — package general SOPs (documentation, code review) as distributable assets without hard-coding them into the agent loop.
MCP (Model Context Protocol) — through plugins, provide IM channel adapters, scheduled tasks, AG-UI visualization, and more.

This is the direction mainstream agent frameworks are moving in 2026. Bub is a practice of this idea: with only a few hundred lines of core interface code, it builds flexible infrastructure.

III. From Context to Data Loop: Tape and Database-Native Harness

1. Building the data loop around Tape

Tape (a core concept in Bub and in AgentSeek, which we are building) is not simple chat history. In some ways it resembles a trace, recording key facts from a single agent run.

Unlike traces in OpenTelemetry and similar observability systems, Tape’s view is simpler — related, but not overly focused on detail. Its distinctive value lies in:

Both observability data and a context model — Tape carries observability for critical tasks and serves as the agent’s runtime context model. That means humans and AI can collaborate on the same data view. The agent can read its own Tape to review past behavior.
Enabling agent self-reflection and diagnosis — Traditionally, when an agent fails, engineers troubleshoot through an observability platform. With Tape, users can talk directly to the agent and ask, “Why did that fail just now?” Engineering investigation also becomes a natural conversation with the agent, because root-cause information is already built into its context.
Supporting automated evaluation and analysis — From Tape records, an agent can compare different models, or the same model across different tasks, enabling automated comparative evaluation without relying on human-facing dashboards.
Serving model training — Through de-identified, formatted export, Tape can readily become task-specific datasets for model training and fine-tuning — truly connecting context and observability to model training in a closed data loop.

2. Why a database-native Harness is needed

Agent systems such as OpenClaw rely heavily on the filesystem (such as various .md files). That is friendly for humans and agents to read, but poor for data processing, analysis, and handling. Modern context engineering needs a Memory layer above raw task trajectories—both a summary of the trajectory and an index. Plugins such as lossless-claw in the OpenClaw community later began using databases like SQLite to connect call chains and memory, which shows databases are necessary in this layer.

Using a database as the foundation of Harness means all agent runtime data is natively a first-class citizen in the database. Observability, data extraction, and archival analysis can use native database capabilities without maintaining a complex heterogeneous data stack (such as MySQL + Elasticsearch + Redis). That provides a unified data foundation, simplifies architecture, and lowers operational cost.

OceanBase is a strong fit for this path. Why? Its core strengths include:

AI workload readiness — OceanBase and its derived tools provide vector search and hybrid retrieval optimized for AI agent workloads. SQL together with vector and full-text search are built-in capabilities, without maintaining multiple technology stacks.
HTAP capability — As a hybrid transactional/analytical processing database, it directly supports real-time queries and complex analysis on agent runtime data, supporting the data loop.
Unified storage with seamless scale-out — All kinds of data can be stored uniformly, supporting trace analysis, retrieval, and related workloads. From single-node deployment on the edge (such as OceanBase seekdb) it scales seamlessly to a distributed OceanBase cluster, offering a smooth upgrade path as the business grows.

IV. AgentSeek: Exploring Database-Native Harness

Through ongoing exploration of agent architecture, the OceanBase team is building AgentSeek — a Harness built entirely on database-native capabilities.

AgentSeek’s core idea: make agent runtime data a first-class database citizen from day one, helping users build data-loop scenarios. The project integrates OceanBase product capabilities with AgentSeek-related wrappers and is actively progressing.

Repository: github.com/ob-labs/agentseek

Closing Thoughts

From the layered definition of Harness, to Bub’s plugin-based extensible architecture, to Tape’s integration of observability and context, to the database-native Harness technical path — agent infrastructure is evolving from feature stacking toward data-driven design. OceanBase’s work in this space is both a natural extension of its technical architecture and a response to data-foundation needs in the AI era.

If you are building data-intensive agents today: where does runtime data land in your stack, and what still breaks when you try to close the loop? Share your setup in the comments — or dig into Bub and AgentSeek, and join Data4AI on LinkedIn to compare notes with other Data + AI practitioners.

References

Bub: https://github.com/bubbuild/bub
AgentSeek: https://github.com/ob-labs/agentseek
LangChain — The Anatomy of an Agent Harness: https://www.langchain.com/blog/the-anatomy-of-an-agent-harness

How Agent Memory Forgets: An Engineering Walkthrough

Charles Wu — Sun, 28 Jun 2026 13:14:31 +0000

Time coats memory in dust. Access is the only cloth that wipes it clean. When the dust grows thick and no one asks, that is forgetting.

Photo by Markus Winkler on Unsplash

In the previous post, we looked at forgetting from a cognitive-science angle — synaptic plasticity, the Ebbinghaus forgetting curve [1], spaced repetition, and desirable difficulty. Fascinating theory.

This piece takes a different angle. We follow a single message through an agent memory system — using PowerMem 2 as our running example — from write to eviction, and watch how forgetting is engineered at every step. In short: decay starts the moment a message is written, but what actually decides a memory’s fate is whether it gets accessed again.

1. Importance Scoring: Is This Worth Remembering?

Picture this message landing in the system:

Next Friday at 3:00 PM — Q2 requirements review with the product team in Conference Room B.

Once the message enters memory, the first question is not how to store it, but whether it deserves storage. If every piece of information is written and retrieved at equal weight, two problems compound as volume grows: retrieval signal-to-noise keeps dropping, and storage cost becomes unbounded.

Shannon’s information theory frames the same idea: high-probability events carry almost no information and are poor candidates for long-term retention; low-probability but critical events carry enormous information and should be persisted.

Importance scoring is the filter. Each item gets a score that drives decay speed, review cadence, and eviction priority downstream.

So how is importance assessed?

1.1 The six-dimension model

In PowerMem, scoring a single message is more involved than it looks. The system uses a six-dimension model — six axes, weighted and summed:

For our meeting message: highly relevant to “Q2 work” (relevance ≈ 0.8), concrete time and place (factual ≈ 0.8), attendance is non-optional (actionable ≈ 0.9), emotionally neutral (emotional_impact ≈ 0.2). Weighted:

0.3×0.8 + 0.2×0.5 + 0.15×0.2 + 0.15×0.9 + 0.1×0.8 + 0.1×0.6 ≈ 0.72

Importance score: 0.72.

The six-dimension model is the theoretical backbone of importance scoring.

1.2 Dual-path scoring: LLM vs. rule engine

PowerMem runs two execution paths so the system always returns a score. Path one uses the six-dimension model; path two is a rule engine that kicks in when path one is unavailable — graceful degradation.

Path one: LLM deep scoring (preferred)

When an LLM is available, the system asks it to analyze all six dimensions and return structured JSON, for example:

{
    "importance_score": 0.72,
    "reasoning": "Meeting schedule imposes a hard time constraint and clear action requirement",
    "criteria_scores": {
        "relevance": 0.8,
        "novelty": 0.5,
        "emotional_impact": 0.2,
        "actionable": 0.9,
        "factual": 0.8,
        "personal": 0.6
    }
}

PowerMem reads importance_score from the response.

Because LLM output is not perfectly predictable, parsing uses a three-level fallback: try JSON first; if that fails, regex-extract a numeric score; if that fails too, default to 0.5.

Engineering note: The per-dimension scores in the JSON are not fed back into the weighted formula. They exist to structure the model’s reasoning (chain-of-thought style) so the final score is more stable.

Path two: Rule engine (fallback)

When the LLM is down, the rule engine takes over. Precision drops, but the system keeps running.

Rules accumulate score from quantifiable signals:

Content length > 100 characters: +0.1; > 50 characters: +0.05
Keyword hit: +0.1 each
Contains ? or !: +0.05 each
Metadata priority high / medium: +0.2 / +0.1
Score capped at 1.0

This is graceful degradation in production: one external dependency failing should not stall the entire memory layer.

2. Classification and Parameter Initialization

2.1 Three-layer memory model

Once importance is set, the next step is classification: which memory layer does this message belong to?

If you read the previous post, you’ll recall how biological memory layers work: information enters the hippocampus (short-term buffer) with limited capacity, consolidates into neocortex (long-term storage), and only items that are repeatedly activated, richly linked to existing knowledge, or emotionally salient earn priority transfer.

PowerMem maps that to three layers:

Classification thresholds:

score ≥ 0.8 → long_term
score ≥ 0.6 → short_term
otherwise → working

if score >= self._algo.long_term_threshold:   # 0.8
    return "long_term"
if score >= self._algo.short_term_threshold:  # 0.6
    return "short_term"
return "working"

Our meeting message scores 0.72 — above 0.6, below 0.8 — so it lands in short_term.

Higher layers decay more slowly; memories live longer.

2.2 Forgetting parameter initialization

Classification answers where a memory lives. The sharper questions are: how fast should it fade? When should it be reviewed for consolidation?

After classification, PowerMem builds a full lifecycle metadata profile — a card tracking strength, decay parameters, review schedule, and management flags.

The profile splits into two blocks:

Below we walk through each parameter for the meeting example (importance = 0.72, short_term).

2.2.1 Initial retention

Initial retention captures how “solid” a memory is at birth. More important content should start with higher retention; low-importance noise should be fragile from day one and yield bandwidth to stronger competitors.

In PowerMem:

initial_retention = self.initial_retention * importance_score
# Meeting example: 1.0 × 0.72 = 0.72

Two fields are written:

initial_retention — snapshot at creation (“how firmly it was encoded”)
current_retention — live effective retention

They match at creation; only current_retention changes as decay and review proceed.

2.2.2 Decay rate by layer

working / short_term / long_term mirror working memory, hippocampus, and neocortex: closer to long-term storage, slower per-unit-time forgetting. Each layer gets its own decay coefficient:

{
    "working":    0.5,  # smallest S — fades fastest
    "short_term": 1.5,
    "long_term":  2.0,  # largest S — fades slowest
}

For our short_term meeting message:

0.1 (global base decay) × 1.5 (short_term coefficient) = 0.15

By comparison:

working: 0.1 × 0.5 = 0.05
long_term: 0.1 × 2.0 = 0.20

Larger coefficient → slower forgetting (within this parameterization).

2.2.3 Review scheduling

Spaced repetition’s core rule: review often at first, then stretch intervals. Hit the window where the memory is almost gone but still recoverable — not so early that review is wasted, not so late that it’s already lost.

PowerMem schedules review timestamps at creation, not on demand.

Step 1: Baseline intervals

Five global baseline intervals (hours): [1, 6, 24, 72, 168] — roughly 1 h, 6 h, 1 day, 3 days, 7 days. Shared by all memories.

Step 2: Compress by importance

Baseline intervals treat all memories equally, but a credential reminder should be nudged more often than small talk. Each interval is compressed:

adjusted_interval = interval × (1 - importance_score × adjustment_factor)

interval — baseline (e.g. 1 h, 6 h, 24 h)
importance_score — 0.72 for our meeting
adjustment_factor — default 0.3 (compression strength)
Floor: 0.5 hours

For the meeting (importance = 0.72):

1 - 0.72 × 0.3 = 0.784

Each baseline interval becomes 78.4% of its original length:

For a low-importance message (importance = 0.3):

1 - 0.3 × 0.3 = 0.91

Intervals shrink only slightly — reviews drift later:

Higher importance → earlier review windows → more chances to re-consolidate.

After computing five timestamps, PowerMem stores the full schedule and initializes companion fields:

Together:

next_review — when to review
review_count + last_reviewed — how much review has happened
reinforcement_factor — how much each review restores

When next_review arrives and a review completes: review_count increments, last_reviewed updates, current_retention rises by reinforcement_factor, next_review advances.

That closes the engineering loop of retrieval → reconsolidation.

2.2.4 Lifecycle state machine

Beyond continuous numbers, memories need discrete lifecycle flags:

Should this memory be promoted to a higher layer?
Should it be evicted?
Should it leave the active retrieval pool?

At creation, PowerMem initializes:

{
    "should_promote": false,
    "should_forget": false,
    "should_archive": false,
    "is_active": true
}

should_promote — e.g. working → short_term when a “scratch” memory is accessed repeatedly; slower decay upstairs.
should_forget — decay factor drops below threshold (0.3), or zero accesses in 7 days (silent forgetting).
should_archive — move out of active search. Archive ≠ delete; data remains, but skips routine retrieval.
is_active — participates in normal read/search paths.

An access_count counter is also initialized; promotion logic can require ≥ 3 accesses, among other rules.

2.2.5 Persisting the metadata profile

When all parameters are computed, the system packs them into a structured dict, merges into metadata, and writes alongside the message body.

The clock starts ticking here.

3. Decay Calculation

Time passes. Retention drifts downward.

Recall the Ebbinghaus form: R(t) = e^(-λt) — exponential decay.

3.1 PowerMem’s decay formula

In code:

rate = self.decay_rate if decay_rate is None else decay_rate
decay_factor = math.exp(-hours_elapsed / (24 * rate))

The denominator 24 × rate (hours) is the memory’s characteristic decay timescale. Call it S (Strength): S = 24 × rate. Larger S → slower decay → longer life.

Cleaner form:

decay_factor = e^(-t / S)

Elegant property: after elapsed time t = S, retention falls to e^(-1) ≈ 37% of its prior value — regardless of S. Know S in hours, and you know the forgetting rhythm.

For our short_term meeting memory, rate = 0.15:

S = 24 × 0.15 = 3.6 hours

Roughly every 3.6 hours, retention drops to ~37% of what it was.

One last engineering detail: PowerMem includes a fallback path. The caller first computes and passes a memory-type-specific decay rate. If none is provided, the system falls back to the global default decay rate. This ensures decay calculation still works even when legacy metadata lacks type information.

3.2 A telling engineering trade-off

PowerMem’s formula is equivalent to the classic Ebbinghaus write-up — different notation. Classic: R(t) = e^(-λt) where λ is the decay constant and λ = 1/S.

Back-solving classic Ebbinghaus lab data gives λ ≈ 0.821. PowerMem’s default config implies λ ≈ 0.417 — about half:

This is intentional, not a bug. Ebbinghaus used meaningless syllables — the fastest decay humans show. PowerMem stores semantically linked agent memory; gentler decay matches that reality.

Tune aggressiveness via INTELLIGENT_MEMORY_DECAY_RATE in .env:

Lower decay_rate → more aggressive forgetting.

3.3 Decay timeline for the meeting memory

A short_term memory crosses the 0.3 threshold in ~4.3 hours — but that does not mean immediate deletion. Eviction runs when the memory is accessed.

4. Access-Triggered Lifecycle

4.1 Defer forget decisions until access

Decay runs continuously in the background; forget / promote / archive decisions execute on access.

Cognitive background: retrieving a consolidated memory temporarily reopens it to plasticity; reconsolidation strengthens the trace. Engineering translation: memory fate should not be time-only — re-evaluate on every touch.

Access is the strongest feedback signal. Frequent access → retain or promote. Long silence → forget, even if it once looked important. Lazy evaluation avoids batch scans; no cron job walking every row.

4.2 Four checkpoints

On Memory.get() or Memory.search(), four stages run in order.

Stage 1: Forget

rate = self._resolve_decay_rate(memory)
decay_factor = self.calculate_decay(created_at, decay_rate=rate)
if decay_factor < self.working_threshold:  # 0.3
    return True

if access_count == 0 and time_elapsed > timedelta(days=7):
    return True

Either condition triggers forget. The second is blunt: never accessed in seven days is itself a strong forget signal. The caller performs deletion.

Stage 2: Promote

Any one condition promotes:

if access_count >= 3:
    return True

if time_elapsed > timedelta(hours=24):
    return True

if importance >= self.short_term_threshold:  # 0.6
    return True

Effect: working → short_term or short_term → long_term — lower decay multiplier, longer life.

Our meeting memory (importance = 0.72 ≥ 0.6) qualifies for promotion on first access. If still short_term, it upgrades to long_term — sticky note → durable knowledge. That mirrors biological consolidation.

Stage 3: Archive

if time_elapsed > timedelta(days=30):
    return True

if importance < self.working_threshold:  # 0.3
    return True

Archived memories are not deleted; they leave the active pool but remain reachable via archive APIs.

Stage 4: Periodic reprocessing

Whenever the access count hits a multiple of 5 (the 5th, 10th, 15th access, and so on), or when the memory type changes, the system recomputes all Ebbinghaus metadata. Parameters track evolving access patterns — spaced repetition stabilized in code.

5. Search Weighting

5.1 Interference theory in retrieval

The hard part of memory is not storage — it’s retrieval. As volume grows, cross-interference explodes.

Search for “Q2 review.” Pure semantic ranking might surface a three-month-old meeting note at the top and bury yesterday’s schedule change. Best semantic match ≠ what the user needs right now.

PowerMem injects time into ranking.

5.2 Ranking formula

final_score = relevance_score × decay_factor

relevance_score — keyword / semantic match
decay_factor — time decay from Section 3

This is a cross-rank: stale-but-perfect matches lose to fresher moderate matches:

Recency has veto power. No matter how well it matches, a nearly decayed memory sinks.

Search is also access: each hit triggers Memory.get() — batch lifecycle management for free.

6. Global Optimization

6.1 Why global passes matter

Everything above is per-memory, online. At scale you still need periodic housekeeping — duplicates, redundancy, fragmentation. Analog: sleep-dependent consolidation — replay, transfer, dedupe, merge, strengthen important links.

PowerMem offers three complementary strategies.

6.2 Three optimization strategies

(1) Exact deduplication

Content-hash exact match. Maintain hash → [memories]; keep earliest per group, delete rest. Batch cap: 10,000 records. Identical duplicates → one row.

(2) Semantic deduplication

Embedding cosine similarity finds near-duplicates with different wording. O(N×M) pairwise compare; default threshold 0.95. Delete newer duplicate; keep earliest.

Example:

“Q2 review moved to next Wednesday”
“Q2 review pushed to Wednesday next week”

Same meaning, different surface form — semantic dedup catches it.

(3) Memory compression

For similar-but-not-identical clusters, an LLM summarizes into one synthetic memory:

Greedy clustering — group pairs above threshold (default 0.85)
LLM summary — templated prompt → one replacement memory per cluster

Together with per-item decay, this spans micro (row-level fade) → macro (batch compress) — a full memory quality system.

Summary

Quite a bit of ground. Our meeting message’s path through PowerMem:

"Q2 requirements review, Friday 3 PM, Conference Room B" enters the system
  ↓
Importance scoring → "How important?" → 0.72
  ↓
Classification → "Which layer?" → short_term
  ↓
Parameter init → "Set the decay clock" → decay rate + review schedule locked in
  ↓
Time passes → retention falls
  ↓
Accessed → "Still alive? Promote?" → passes; promoted to long_term
  ↓
Searched → "Where in results?" → recency × relevance
  ↓
Global optimization → "Duplicates? Compress?" → dedup and merge

The through-line: use finite storage for the highest-value signal, and keep retrieval SNR high.

Six mechanisms, six jobs:

Importance scoring — what to remember
Classification — how long to remember
Decay — when to evict
Access triggers — dynamic adjustment
Search weighting — how to find it
Global optimization — how to stay lean

What we find most interesting about PowerMem: forgetting is not a binary “delete or not” afterthought. Decay begins at write time — but decay is a continuous weight, not deletion. What actually decides fate is whether the memory is touched again. That’s uncomfortably close to how human memory behaves.

Forgetting is not a patch on top of memory systems. It is a core design dimension across the entire lifecycle.

Based on PowerMem v1.1.1 source analysis; code references reflect the actual project.

From Neurons to Code: The Forgetting Design Behind PowerMem

Charles Wu — Sun, 28 Jun 2026 04:09:03 +0000

Brains forget on purpose. PowerMem does too — three memory tiers, exponential decay, and ranking by relevance × freshness, not just semantic match.

Photo by Fredy Jacob on Unsplash

Key Takeaways

PowerMem treats forgetting as a first-class capability — not a bug — using a three-tier memory model (working, short_term, long_term) backed by Ebbinghaus-style exponential decay.
Decay-rate multipliers differ by tier (×2.0 / ×1.5 / ×1.0), so unimportant memories fade quickly while frequently accessed ones are promoted and stabilized — directly mirroring synaptic plasticity and memory consolidation.
Retrieval ranking combines semantic similarity with a decay factor (final_score = relevance × decay), turning forgetting into a quality regulator rather than a delete switch.

If every memory carries equal weight at retrieval time, two problems compound:

Retrieval quality decays. New and old memories interfere with each other in the embedding space. As the corpus grows, the signal-to-noise ratio of any query drops.
Storage costs spiral. Most low-value content is never retrieved, yet it keeps consuming space, index time, and embedding budget forever.

PowerMem’s forgetting mechanism decides two things: when a memory dies, and how much weight it carries during retrieval. Before walking through the code in a follow-up post, it is worth tracing the cognitive-science principles the system is modeled on.

How Nature Forgets

Synaptic Plasticity

The biological substrate of memory is the synaptic connection between neurons. Those connections are anything but static — two opposing mechanisms continuously modulate them:

Long-Term Potentiation (LTP) — frequently used pathways are strengthened. This is the basis of remembering.
Long-Term Depression (LTD) — rarely used pathways are weakened. This is the basis of forgetting.

LTP and LTD are partners, not adversaries. If every synapse were strengthened equally, the brain would lose its ability to distinguish signal from noise. LTD selectively weakens inactive connections so that limited synaptic resources concentrate on the active pathways. Forgetting is the price memory pays for discrimination.

From Hippocampus to Neocortex

A newer memory is first held in the hippocampus — high-throughput, low-capacity, much like RAM. During sleep, the brain replays these traces and gradually transfers selected ones to the neocortex for long-term storage.

The transfer is selective. Only memories that are repeatedly activated, richly associated with prior knowledge, or marked by strong emotion are prioritized. Isolated, single-occurrence, emotionally neutral information falls off during the move. Nature performs filtering automatically during consolidation, and this is the direct biological blueprint for PowerMem’s three-tier model: working → short_term → long_term.

Forgetting Is a Retrieval Problem

Cognitive psychology adds another lens: interference theory. Forgetting is often not about information being erased, but about it being un-retrievable. Proactive interference — old memories disrupt the recall of new ones (you keep typing your old phone number). Retroactive interference — new memories disrupt the recall of old ones (learning Spanish makes Italian vocabulary slip).

The hard problem is not writing — it is reading under interference. As the store grows, cross-memory interference rises super-linearly. Decaying low-value entries reduces interference density and restores retrieval precision.

A Shannon-Information View

Claude Shannon’s 1948 definition of information quantifies surprise:

I(x) = -log₂(p(x))

The information content of an event is inversely related to its probability — common events carry little information; rare events carry a lot.

Mapped onto a memory system this gives a natural rule. “What I had for breakfast yesterday” (happens daily, p ≈ 1, I ≈ 0) is not worth long-term storage. “The master password for our production database” (almost never asked, tiny p, huge I) must be persisted.

A well-designed forgetting mechanism is therefore an information filter: high-information content (rare but critical) is retained, low-information content (frequent but trivial) is decayed and evicted, and everything in between is interpolated smoothly. PowerMem’s tiered architecture implements this filter; the forgetting curve gives it a time-varying weight, so classification keeps evolving instead of being decided once at write time.

The Ebbinghaus Forgetting Curve

Memory Becomes Measurable

In 1885, Hermann Ebbinghaus turned memory research from philosophy into laboratory science. Using roughly 2,300 nonsense syllables to avoid prior-knowledge bias, he ran a strict protocol on himself:

Learn a 13-syllable list until two consecutive error-free recitations.
Wait 20 minutes, 1 hour, 9 hours, 1 day, 2 days, 6 days, 31 days.
Re-learn using the savings method — measure how much faster than the first time.

The retention data:

Two conclusions, still standing more than a century later:

Forgetting is exponential, not linear — about 40% lost in the first 20 minutes, more than half within an hour, then a long slow tail.
Spaced review rewrites the curve — repeated reviews at the right interval slow subsequent decay.

From the Original Fit to Modern Exponential Decay

Ebbinghaus’s original fit was logarithmic:

b = 100k / ((log t)^c + k)

with b the savings percentage, t the time in minutes, and constants k ≈ 1.84, c ≈ 1.25.

Later work showed that a simpler exponential model approximates the data just as well, and it is now the standard form:

R(t) = e^(-λt)

R(t) — retention at time t, the fraction of the original information still recallable, in [0, 1].
e — the natural constant (≈ 2.71828), the mathematical base for any continuous, smooth exponential process.
λ (lambda) — the decay rate. Larger λ → faster forgetting (steeper curve). Smaller λ → more durable memory (flatter curve).
t — elapsed time since the memory was formed, typically in hours.

The graph is a fast-then-slow curve. Most of the loss happens early; whatever survives the early window is far more stable, simply because there’s not much left to forget. These equations are the mathematical foundation of PowerMem’s forgetting mechanism.

Why Exponential Is the Right Functional Form

The defining feature of forgetting is that the rate of forgetting is proportional to what remains. The differential statement is dR/dt = -λR — change rate proportional to current state — and its unique solution is exactly R(t) = e^(-λt).

Newton’s law of cooling, radioactive decay, capacitor discharge — apparently unrelated phenomena that share the same equation because they share the same self-consistent relationship between rate and state. Memory decay is no exception. Modern spaced-repetition systems (SuperMemo, Anki, PowerMem) converge on exponential decay because it offers the best balance between simplicity, computability, and empirical fit.

Spaced Repetition and Desirable Difficulty

Ebbinghaus also discovered that spaced repetition resets the curve, and each reset slows the next decay. Neuroscience explains why through memory reconsolidation: when a consolidated memory is actively retrieved, it briefly returns to a plastic state, and the brain re-stabilizes it through a fresh round of protein synthesis and synaptic reinforcement.

Reconsolidation needs time. Cramming ten repetitions into five minutes does not allow protein synthesis and synaptic remodeling to complete — the biological reason rote cramming is inefficient. Wait too long, however, and the trace has already decayed below retrieval threshold, leaving nothing to reconsolidate. Robert Bjork (UCLA, 1994) crystallized this into the concept of desirable difficulty: the most efficient learning happens when retrieval is just hard enough to trigger adaptation. This principle drives PowerMem’s review-scheduling logic.

PowerMem’s Three-Tier Memory Architecture

This is where the biology, the information theory, and the math all land in code. PowerMem is not the first system to talk about “memory tiers” — but the way it makes forgetting a tunable parameter at every layer is what makes the design worth examining in detail.

From Biology to Code

The cognitive-science principles above translate into three engineering tiers:

Classification is driven by an importance score:

importance ≥ 0.8  →  long_term
importance ≥ 0.6  →  short_term
importance < 0.6  →  working

The decay-rate multiplier is the key differentiating parameter. Over the same 24-hour window, a working memory decays at twice the rate of a long_term one. Importance directly controls expected lifespan: unimportant content disappears quickly, freeing retrieval space for the things that actually matter.

Global Architecture of the Forgetting Subsystem

PowerMem’s forgetting subsystem has four cooperating components, arranged along the lifecycle of a memory entry:

New input → ImportanceEvaluator
         → EbbinghausAlgorithm
         → EbbinghausIntelligencePlugin
            ├─ on_add():    inject decay parameters at creation
            ├─ on_get():    check decay / promotion / archival on access
            └─ on_search(): batch-process lifecycle during search
         → MemoryOptimizer
            ├─ exact dedup (MD5 hash)
            ├─ semantic dedup (cosine similarity)
            └─ memory compression (LLM summarization)

ImportanceEvaluator — judges how important a piece of information is and outputs a 0.0–1.0 score.
EbbinghausAlgorithm — pure-math layer providing decay computation, review scheduling, and the forget / promote / archive decisions.
EbbinghausIntelligencePlugin — injects management logic at the key lifecycle hooks: creation, access, and search.
MemoryOptimizer — periodic global pass that performs deduplication and compression.

Forgetting Is More Than Deletion

In retrieval, the forgetting mechanism plays an equally critical role as a ranking signal. Search results are ordered by:

final_score = relevance_score × decay_factor

relevance_score — semantic match (vector similarity).
decay_factor — temporal freshness (the exponential decay value).

These two parameters jointly determine the final ranking, which makes non-trivial cross-rankings possible. The numbers below are illustrative; the actual decay factor depends on the configured decay_rate:

Forgetting is not a simple delete switch — it is a quality regulator for retrieval. It guarantees that the result respects both the content match dimension and the time freshness dimension simultaneously.

Why Forgetting Matters

Pulling the threads together:

Forgetting is the foundation of ranking. Decay manufactures a second axis beyond semantic similarity, so otherwise-equivalent matches can be separated cleanly.
Forgetting lets memory evolve. Frequently accessed entries are promoted and assigned lower decay rates; repeated use stabilizes what is genuinely useful, exactly as reconsolidation does in the brain.
Forgetting is continuous, not binary. A smooth 1.0 → 0.0 spectrum mimics how human memory actually fades, and leaves room for future features — soft deletes, memory revival, tiered archival — without breaking the model.

Nature designed it this way. PowerMem translates that design into code you can configure, tune, and reason about.

The next post follows a single piece of information through the full PowerMem pipeline — importance evaluation → tier assignment → decay → access trigger → promotion or forgetting → global optimization — to see exactly how the theory becomes runtime behavior.

PowerMem on GitHub: https://github.com/oceanbase/powermem

If you find PowerMem helpful, please give it a ⭐ on GitHub. It would be a great help to the project!

Based on PowerMem v1.1.1. All code references come from the actual project files.

Technical Deep Dive: How OceanBase’s Native Column Store Powers HTAP

Charles Wu — Sun, 28 Jun 2026 03:34:43 +0000

This series covers how OceanBase Analytic Processing (AP) delivers strong transactional guarantees and high-concurrency support for real-time analytics. This article focuses on the native column-store engine’s architecture — tracing the full technical path from LSM-Tree’s baseline-delta separation, through adaptive compaction, columnar encoding, and Skip Index optimizations, to the vectorized execution engine 2.0, cost-model-driven row/column path selection, and system-wide adaptations across DDL, backup/restore, and transaction consistency. Together, these mechanisms show how OceanBase serves both TP and AP workloads within a single architecture.

1. From TP to HTAP: Why a Native Column Store

When building real-time analytics, enterprises face a classic architectural trade-off: deploy a separate OLAP database, or run analytical queries directly on the OLTP system. The first approach introduces data synchronization latency and operational complexity. The second hits a performance wall — row-store engines are not designed for analytical workloads.

Starting with V4.3.0, OceanBase offers a third path: a native column-store engine that delivers high-concurrency transaction processing and complex analytical queries within the same database instance — no additional data sync pipeline required. The key breakthrough is a deep integration of row store and column store within a single codebase and a single OBServer process, built on the LSM-Tree architecture.

Before V4.3.0, OceanBase’s AP capability relied on a lightweight row-store-plus-index approach. It handled simple analytical queries but bottlenecked on typical AP workloads involving multi-table joins, wide-range scans, and complex aggregations. The native column store closes this gap, enabling use cases like real-time reporting, real-time data warehousing, and user profiling.

This article walks through the technical details, starting with the column-store engine’s architecture.

2. Architecture of the Native Column-Store Engine

OceanBase’s native column-store engine is not a bolt-on layer over a row-store architecture. It redesigns data organization from the ground up. The core challenge: support both high-concurrency writes and efficient analytical queries in a unified architecture — balancing storage format, data flow, and query execution. This section explains the baseline-delta separation mechanism built on LSM-Tree.

Traditional column-store engines offer weak transactional support and struggle with complex transaction scenarios. OceanBase’s LSM-Tree architecture handles this by separating baseline and delta data:

Baseline data: Stored in columnar format, optimized for analytical query performance. When delta data accumulates to a threshold, it merges with the baseline to produce a new columnar baseline.
Delta data: Stored in row format for high-concurrency transactional updates. DML operations first write to an in-memory MemTable, then flush to disk as SSTables.
Merge mechanism: A background process periodically merges delta data into the baseline, avoiding performance degradation from heavy random updates.

This architecture delivers row-store and column-store unification in one codebase, one architecture, and one OBServer process — serving both TP and AP query performance.

3. Key Technical Mechanisms

With the baseline-delta separation established at the architecture level, the next challenge is engineering efficiency: how to compact columnar data efficiently, reduce storage overhead, and minimize I/O and compute costs at query time. This section covers four core mechanisms in OceanBase’s column-store engine.

Adaptive Compaction

Column-store compaction is more complex than row-store compaction. Columnar data is organized by column, so merges involve more files and data reorganization, consuming significantly more resources. OceanBase addresses this with an adaptive compaction mechanism.

The system intelligently selects which partitions to compact based on the volume of delta data written and query performance metrics per partition — avoiding resource overload from full-scale compaction scheduling. It borrows parallelization techniques from row-store compaction, splitting columnar merge tasks horizontally into sub-tasks for parallel execution. A more innovative feature is vertical splitting: column-level merge task scheduling that prioritizes hot or critical columns, optimizing resource allocation.

V4.3.0 also introduces tablet-level compaction, supporting partition-level merges triggered from the system tenant. When users observe query performance degradation, they can manually trigger a partition-level merge for quick resolution — providing greater operational flexibility.

Columnar Encoding

OceanBase V4.3.0 introduces a new columnar encoding algorithm, enabled via row_format=compressed. This encoding is deeply optimized for column-store access patterns, with full-stack optimization from low-level encoding to upper-level execution.

The new algorithm leverages CPU SIMD instructions to dramatically improve parallel processing of numerical computations. It applies efficient compression algorithms to numeric columns — delta encoding, run-length encoding, and others — significantly reducing storage footprint. Queries can execute filter operations directly on compressed data, eliminating decompression overhead and further boosting query performance.

Skip Index

Skip Index is one of the column-store engine’s core optimization features. In analytical queries, most time is spent on I/O and full data scans. Skip Index adds pre-aggregated data at the storage layer, intelligently skipping irrelevant data blocks to drastically reduce unnecessary disk access.

At the implementation level, Skip Index computes statistics at the smallest storage unit (micro-block) — including min, max, and null count — then aggregates upward layer by layer: micro-block to macro-block to SSTable, building a multi-level index structure. At query time, the system uses pre-aggregated min/max values to quickly determine whether a data block contains data within the query’s filter range, skipping large volumes of irrelevant blocks.

For DDL, OceanBase provides flexible Skip Index management. Users can create different types of Skip Index on specified columns at table creation, or modify them later via ALTER TABLE. For column-store tables, the system automatically creates MIN_MAX and SUM Skip Indexes on all columns — delivering performance gains with zero additional configuration.

Enhanced Pushdown

OceanBase further enhances query pushdown, moving more computation into the storage layer. All filter conditions can now be pushed down to storage, where they combine with Skip Index pre-aggregation to perform rapid data filtering at the storage level.

Aggregate function pushdown is also strengthened. count, max, min, sum, and avg can execute directly at the storage layer. For aggregation queries without GROUP BY clauses, the final result is computed entirely in the storage layer, eliminating the overhead of pulling data up to the execution layer.

The most innovative enhancement is GROUP BY pushdown. For low-cardinality columns, the system uses dictionary information within micro-blocks to perform localized GROUP BY computation, significantly reducing data transfer volume. This optimization is especially effective for typical analytical scenarios like user profiling and behavioral analysis.

4. System-Wide Adaptation

The column-store engine is not an isolated feature — its value depends on tight integration with every database module. From SQL parsing to execution plan generation, from DDL operations to backup/restore, OceanBase has systematically adapted multiple core modules since V4.3.0 to ensure column-store capabilities integrate seamlessly into existing workflows.

DDL Support and Table Types

OceanBase V4.3.0 provides flexible column-store DDL support. Users can create different table types based on workload requirements:

Column-store table: Creates a pure column-store table where all data is stored in columnar format, suited for analytics-heavy workloads.

create table t1 (c1 int, c2 int) with column group (each column);

Row-column redundant table: Maintains both row-store and column-store copies of the data. Supports both high-concurrency transactions and efficient analytical queries, at the cost of additional storage.

create table t2 (c1 int, c2 int) with column group(all columns, each column);

Column-store index: Creates a column-store index on a row-store table to accelerate specific query patterns.

-- Create a pure column-store index on columns c1, c2 of table t1
create index idx1 on t1(c1, c2) with column group(each column);

-- Create a row-column redundant index on column c1 of table t1
create index idx2 on t1(c1) with column group(all columns, each column);

Column-store tables support the full range of DDL operations: adding columns, dropping columns, modifying column types, and more. Skip Index DDL syntax was further refined in V4.3.5, supporting online DDL for maintaining pre-aggregated data.

Cost-Model Enhancement in the Optimizer

Cost-based row/column path selection is a key optimization in V4.3.0. OceanBase implements a unified optimizer codebase that estimates costs differently for row-store and column-store paths, enabling automatic path selection for user queries.

Cost Estimation

Storage-layer cost evaluation: The optimizer estimates I/O cost, CPU cost, and memory cost for scanning both row-store and column-store data.
Data characteristics: It considers the number of columns accessed, data distribution, and filter selectivity to dynamically select the optimal storage path.
Hybrid path support: For complex queries, the optimizer may use both row-store and column-store paths simultaneously, achieving best performance through intelligent data reorganization.

Path Selection Strategy

When the optimizer identifies an analytical operation (full table scan, multi-column aggregation, complex filtering), it prefers the column-store path. For point lookups and high-concurrency updates (TP operations), it selects the row-store path. This intelligent routing allows OceanBase to efficiently handle both TP and AP workloads within a single database.

Vectorized Engine 2.0

V4.3.0 introduces a new vectorized execution engine based on the Column data format. Compared to the earlier Uniform format, the new engine offers:

Native columnar data support: Optimized for column-store access patterns, eliminating data format conversion overhead.
Full SIMD utilization: More efficient data layout enables better use of modern CPU SIMD instructions for numerical computation.
Memory access optimization: Improved in-memory data arrangement increases cache hit rates and memory access efficiency.

Backup/Restore and Transaction Adaptation

OceanBase has adapted multiple modules around the column-store engine — from optimizer to executor, from DDL to backup/restore and transaction processing:

Backup/restore support: Column-store backup and restore is fully compatible with row-store mechanisms, supporting both full and incremental backups.
Transaction consistency: The column-store engine natively supports distributed strong-consistency transactions via MVCC, guaranteeing consistent data views.
High-concurrency processing: The LSM-Tree-based architecture supports high-concurrency transactional and query operations.
Mixed-workload capability: High-concurrency transaction processing and complex analytical queries coexist, providing a unified data processing platform.

Through these adaptations, OceanBase delivers a new technical option for modern enterprise applications requiring real-time analytics, strong transactional guarantees, and high concurrency — particularly in finance, e-commerce, and IoT where data freshness and consistency requirements are stringent.

5. Core Value

OceanBase’s native column-store engine leverages LSM-Tree architectural innovation to solve the traditional column-store bottleneck around strong transactions and high concurrency — a significant advance in HTAP database technology.

Key technical breakthroughs:

Architectural unification: Seamless row-column fusion under one architecture eliminates the data synchronization complexity and latency inherent in traditional HTAP systems.
Native transactional support: The column-store engine natively supports distributed strong-consistency transactions — rare in the industry.
Concurrency scalability: MVCC combined with LSM-Tree enables large-scale concurrent read/write operations.
Real-time analytics: Achieves second-level data freshness for analytical queries — delta data is immediately available for analysis.

From an industry perspective, the column-store engine redefines the direction of HTAP databases. It demonstrates that a unified architecture outperforms separated architectures in both performance and cost, that strong transactions and real-time analytics are achievable simultaneously, and that high-concurrency OLTP and complex OLAP can coexist.

6. Use Cases

OceanBase’s column-store engine applies broadly across multiple scenarios.

OLAP workloads: In data warehouse applications, the columnar format excels at large-scale data import and transformation, significantly improving ETL throughput. Complex report generation benefits from vectorized execution and pre-aggregation optimizations.

Real-time analytics: For user behavior analysis, business monitoring dashboards, and similar use cases, OceanBase delivers sub-second query latency. Anomaly detection systems perform rapid identification on real-time data for timely alerting.

HTAP mixed workloads: E-commerce platforms process high-concurrency transactions and complex sales analytics on the same platform, eliminating synchronization delays. Financial institutions achieve unified trading and risk control — real-time transactions and risk monitoring on one platform.

IoT and monitoring: High-volume device data collection and analysis, efficient time-series storage and querying, and predictive maintenance based on real-time device data — all demand strong real-time analytics capabilities.

7. Summary and Roadmap

OceanBase’s native column-store engine provides a new technical option for real-time analytics. From architectural unification to native transactions, from storage-layer pushdown to vectorized execution, the column-store engine forms a comprehensive technical system. Looking ahead, OceanBase will continue evolving across three dimensions: functionality, performance, and deployment flexibility.

Richer Functionality

Flexible column groups: Currently supports pure column storage; future releases will enable custom column group partitioning for diverse analytical needs.
Enhanced direct load: Further improvements to incremental direct load capabilities will shorten data preparation time for analytics.

Stronger Performance

Skip Index enhancement: Expand the statistical dimensions supported by Skip Index, covering more query patterns.
Unified storage format: Current storage formats are diverse; future releases will deeply integrate storage formats with the SQL vectorized engine, automatically recognizing different formats during SQL execution to reduce conversion overhead.

Flexible Deployment

Heterogeneous replicas: Support OLAP-specific heterogeneous replica types for specialized deployment requirements.
Storage-compute separation: Future support for storage-compute separation, enabling independent scaling of storage and compute for AP workloads at lower cost.

Continued evolution of OceanBase’s column-store engine will further strengthen its position as an enterprise-grade unified HTAP data platform.

How Does an Enterprise-Grade Query Optimizer Keep HTAP Workloads Accurate, Fast, and Stable Over Time?

Charles Wu — Sun, 28 Jun 2026 03:11:57 +0000

Photo by on Jungwoo Hong on Unsplash

Today, many enterprises treat “running real-time analytics directly on transactional data” as a baseline capability: the same dataset must support high-concurrency point lookups and short transactions, while also delivering reports, risk-control rule evaluation, and operational dashboards within minutes or even seconds.

This is not merely a question of whether something is “technically possible.” Enterprises want to minimize the servers, storage, synchronization, and operational overhead that come with building and maintaining a separate lakehouse pipeline just for analytics. At the same time, they want to shorten the path from when transactional data is generated to when analytical results become available, so that reporting, risk control, and business decisions can stay as close to real time as possible.

In many enterprise deployments, transactional and analytical systems are built separately. But as analytics increasingly emphasizes the timeliness of results while overall system cost must be kept in check, this model introduces new problems: data has to be moved and synchronized across multiple systems, pipelines grow longer, consistency governance becomes more complex, and analytical results are more likely to lag behind the true state of the business.

For exactly this reason, users rarely think of the word “optimizer” first — until a familiar yet unsettling pattern starts showing up in production: the same kind of SQL, on the same hardware, sometimes runs fast and sometimes runs so slowly it feels like a completely different system.

When troubleshooting, people tend to first suspect network jitter, disk bottlenecks, or a sudden traffic spike during a particular window. But in HTAP scenarios, a significant share of performance fluctuations can ultimately be traced back to the same root cause: the execution plan no longer fits the current data distribution, statistics, and workload state.

What an enterprise-grade query optimizer must solve is precisely this: continuously generating more reasonable execution plans in such a mixed environment — letting analytical queries complete at an acceptable resource cost, while avoiding situations where a wrong access path, a poor distributed execution strategy, or an unreasonable degree of parallelism drags every other workload on the same cluster into jitter.

Why HTAP Scenarios Are More Prone to Execution Plan Problems

Why SQL in HTAP Scenarios Runs “Sometimes Fast, Sometimes Slow”

Most slow SQL in production is not caused by “having no optimizer,” but rather by “the optimizer choosing the wrong plan under current conditions.”

The most typical cases include:

After data volume grows, the system still uses a join order or access path that was only suitable for a smaller data scale;
After the data distribution becomes skewed, the cost model still estimates as if the distribution were uniform;
After statistics are refreshed, the plan switches to a path that is theoretically better but performs worse in practice.

In HTAP scenarios, these deviations get amplified: response-time fluctuations of short queries turn into long tails, while analytical queries may — due to a poor distributed strategy or degree of parallelism — consume extra shared resources, ultimately stripping “real-time analytics” of its real-time nature.

Therefore, the first step in performance optimization is not “making one particular SQL run a bit faster,” but making plan selection as explainable and reproducible as possible: why columnar storage was chosen this time, why a redistribution was needed this time, and why the degree of parallelism is N rather than M.

Why HTAP Makes “Plan Selection” Harder Than Pure TP or AP

In transaction-oriented (TP) workloads, each query touches little data, has tight response-time requirements, and follows highly repetitive execution patterns; the goal is typically low latency and path stability. In analytics-oriented (AP) workloads, queries more often involve large-range scans, multi-table joins, aggregation, sorting, and window functions, and rely more heavily on columnar storage, distributed execution, and parallelism; the goal is typically throughput and execution efficiency for complex queries.

HTAP stacks both on top of the same engine, the same database objects (indexes, partitions, and so on), and the same set of statistics. This means the optimizer cannot simply answer “is this operator fast or not”; within a unified cost framework, it must simultaneously handle two completely different plan-selection problems:

For short queries: row store or index path;
For large queries: columnar scan or another access path;
Under distributed execution: stay local or perform data redistribution;
How to choose the join algorithm;
Serial execution or parallel execution.

What makes it even trickier is that the runtime environment is always changing: data growth, distribution skew, statistics refreshes, and version upgrades all change the answer to “which plan is cheaper.” Once row-count estimation or cost evaluation goes wrong, every subsequent choice compounds in the wrong direction, eventually manifesting as query latency jitter, wasteful amplification of CPU and I/O, and even mutual interference with other workloads during peak hours.

One boundary needs to be made clear here: the fundamental competition between transactions and analytics over physical resources is governed by system-level capabilities such as resource isolation, quotas, and scheduling. The optimizer’s value is to reduce the extra amplification caused by plan errors — not to replace system-level governance.

How OceanBase Selects, Runs, and Stabilizes Plans Well

Unified CBO: How OceanBase Handles TP and AP Within a Single Framework

OceanBase does not build two separate, unrelated optimizers for TP and AP. On top of the same CBO (Cost-Based Optimizer) framework and the same set of database objects — statistics, indexes, partitions, and so on — OceanBase selects different but cost-comparable execution plans for different SQL shapes.

For TP-leaning queries, the optimizer focuses more on path stability, index and table-lookback costs, and short-query response time. For AP-leaning queries, the optimizer brings columnar scans, pruning effectiveness, distributed data movement, join algorithms, and parallel costs all into consideration.

From an implementation standpoint, the OceanBase query optimizer follows the cost-based System-R approach: it enumerates and evaluates the cost of base-table access paths, join orders, join algorithms, and other operator combinations to generate the final execution plan. As a distributed HTAP engine, OceanBase’s main difference lies here: distributed properties and parallel properties must be considered alongside local operator costs from the plan-generation stage onward. Distributed and parallel properties are not add-on information tacked on at execution time — they are part of the plan’s cost.

While fully inheriting the unified CBO framework, OceanBase has made targeted enhancements for the parts of analytical queries that are more common and more prone to amplifying cost deviations. Viewed against the main line of optimizer work, these capabilities essentially solve three things: first select the plan accurately, then make the plan run well, and finally keep the plan stable.

Selecting the Plan Accurately: Row Estimation, Statistics, and Path Selection

For the optimizer, the prerequisite for “selecting accurately” is to estimate data scale and selectivity as accurately as possible first. Whether it’s the access path, join order, or subsequent query rewriting and parallel decisions, everything is ultimately built on cost evaluation; and whether cost evaluation can be trusted depends first on the quality of row estimation and statistics.

In HTAP scenarios where data changes continuously, relying solely on static statistics is often not enough. To address this, OceanBase combines online statistics collection, dynamic sampling, and storage-layer row estimation into its cost model, so that judgments of “row count and selectivity” stay closer to the real data state.

The core role of dynamic sampling is to improve estimation quality when statistics are insufficient or inaccurate, providing more reliable input to the CBO;
Storage-layer row estimation stays closer to the real data distribution, helping improve the accuracy of access-path selection;
Online statistics collection is used to reduce how far statistics lag behind the real data.

In particular, OceanBase’s row-estimation method — based on logical rows and physical rows — can account for both incremental and baseline data, unlike traditional approaches that rely more heavily on static statistics. This brings the statistical profile closer to a real-time state, and it also handles predicate dependencies more naturally in composite-index scenarios. For example, when an index (a, b) encounters a condition like a = 1 and b = 1, it can obtain an estimate close to the true selectivity without over-relying on extra multi-column histograms or complex compensation logic.

Only once row estimation becomes more trustworthy can the optimizer truly do a good job of “selecting the path accurately.” This matters especially in HTAP scenarios, where the same semantics can often be served either by a row-store index path or by a columnar scan path, and the optimizer must make an explainable comparison within a unified cost framework. For columnar scans, if the benefit of a Skip Index cannot be correctly evaluated, it’s easy to fall into a “looks cheap, actually expensive” path mis-selection; for row-store and index paths, row-estimation errors can likewise inflate an access that should have been local into a far more costly execution method.

The same logic applies to complex SQL. The value of query rewriting is not “the more rules the better,” but in turning SQL into a structure that more easily yields a globally optimal plan — providing a better starting point for subsequent path selection, join ordering, and parallel decisions. In other words, accurate estimation is not the finish line; it is the prerequisite that lets the entire chain of downstream decisions start from the right place.

Running the Plan Well: Distributed Global Cost and Parallel Decisions

In a Shared-Nothing architecture, the plan that is “cheapest on a single node” is often not the one that is “cheapest for the cluster.” Once data is spread across multiple nodes, joins and aggregations may introduce broadcasts, redistributions, and multiple rounds of data movement. These costs are often insignificant in a single-node model, but at the cluster level they can become the dominant expense.

Therefore, the real problem a distributed database optimizer must solve is not “can it generate a distributed plan,” but can it bring distribution properties, data movement, and operator implementations into a global comparison at the plan-generation stage. This determines whether it falls into the trap of “locally optimal, globally inefficient.”

From an optimization-framework perspective, distributed plan optimization is harder than local optimization in three main ways:

The operator implementation space is larger
Take Hash Join as an example: in a distributed environment, different data distributions correspond to different distributed implementation algorithms, giving a far larger choice space than single-node algorithms.
Physical properties are more complex
Beyond sort properties, a distributed plan must also maintain properties such as partition information and data location, which directly determine whether a given operator can adopt a particular distributed execution method.
Parallelism and partitioning further expand the search space
Partition pruning, intra-partition parallelism, inter-partition parallelism, and degree-of-parallelism selection all add to optimization complexity.

To address these issues, one important direction for OceanBase is to bring distributed properties into plan selection as early as possible, rather than simply “generating a locally optimal plan first and then bolting on a distributed execution method.”

Closely related to distributed planning is parallel decision-making. Parallelism can indeed shorten the wall time of complex queries, but a higher degree of parallelism is not always better. Too much parallelism amplifies CPU, memory, and system jitter; too little leaves long queries unable to finish for a long time. So letting the optimizer balance “whether to parallelize and how much” between response time and resource cost is more aligned with engineering reality than relying on fixed hints across large numbers of SQL statements.

This is exactly the point of OceanBase’s optimizer Auto DOP (Automatic Degree of Parallelism) capability: the optimizer automatically decides “whether to parallelize and what degree to use” based on query cost and resource state. It does not simply chase a shorter single-query response time; it brings parallelism itself into the plan cost, aiming to reduce “bad parallelism” within the given resource constraints.

Columnar storage and Skip Index mainly solve “read less, read the right things.”
Distributed planning and parallel decisions mainly solve “how to move less data and how to expand operators.”

Together, the two form a more complete real-time analytical execution-plan capability for HTAP scenarios.

Keeping the Plan Stable: Caching and Evolution Mechanisms for High-Frequency SQL

If row estimation, distribution, and parallelism mainly solve “selecting and running the plan well,” then for an enterprise-grade system there is a third thing to consider: can the plan stay stable over the long term.

For complex analytical queries, the core problems usually remain row estimation, distribution, and parallel selection themselves; plan caching and SPM are not the default main path. But for highly reusable SQL — especially high-concurrency, low-latency business requests — plan stability becomes a very practical concern. Changes in statistics, growth in data scale, and version upgrades all lead the optimizer to generate new plans, and a “theoretically better” new plan may not actually perform better under real workloads.

For scenarios like these, OceanBase has built a fairly complete plan-caching and plan-evolution mechanism.

First is parameterized plan caching.
For high-concurrency workloads, generating and caching a separate plan for every specific parameter is both costly and impractical. The value of parameterized caching is to let a large number of SQL statements that share the same shape but differ in parameters reuse the same plan, keeping the execution overhead low when a cached plan is hit.

But parameterized caching does not mean “one shared plan is always optimal for all parameters.” In real business, the data scale corresponding to different parameters can vary enormously. For example, even for the same task — “compute a merchant’s sales over the past year” — a large merchant may be better served by a main-table scan, while a small merchant is better served by an index. Forcing the same plan to be reused in both cases leads to obvious parameter-sensitivity problems. In other words, plan caching solves “can plans be reused efficiently,” but it does not automatically solve “whether this plan will remain suitable for all parameters and the current data state over the long term.”

At this point, the question evolves from “whether to cache the plan” into “when should the plan change, and can that change happen safely.” Building on this, the plan-evolution mechanism (SPM) further addresses “can a new plan be switched in safely.”
For highly reusable SQL, the core value of SPM is not to boost analytical performance, but to turn plan stability from a matter of human experience into mechanized governance: a new plan is not immediately and fully adopted the moment it is generated; instead, it is validated against real business traffic. Only when validation shows it is genuinely better than the baseline plan will subsequent SQL continue to use the new plan; otherwise, execution falls back to the baseline plan.

So SPM is more accurately positioned as: a means of governing plan stability for highly reusable SQL. It is part of the optimization loop, but it is not the main path for improving complex analytical performance.

From “Selecting the Right Plan” to “Keeping the Plan Under Control”

A query optimizer generally faces two structural constraints. First, statistics, dynamic sampling, and cost parameters describe “the data from some earlier period,” while the query happens “right now.” Second, search and decision time is limited — even within a mature framework like System-R, it is impossible to enumerate all equivalent plans and find the global optimum within an acceptable time.

Given this, “sometimes choosing the wrong plan” is not a weakness unique to any one product, but the normal state of the CBO problem itself. OceanBase weighs row-store versus columnar paths, distributed data, and the degree of parallelism within the same “estimate — compare — select” logic: row-estimation errors, join order, shuffle, and parallel decisions can all drift off course in a chain reaction; HTAP stacking (TP and AP sharing statistics and database objects on the same engine) only makes it easier for “sensitive SQL” to expose plan jitter during peaks or statistics-refresh windows.

The maturity of an enterprise-grade optimizer should not be measured by a single benchmark. What matters more is whether, when a plan no longer matches the real workload, the execution plan can be turned into a process that is observable, intervenable, evolvable, and reversible. Beyond “selecting plans,” OceanBase also shares the responsibility of “governing plans” together with the execution side, database objects (indexes, partitions, and so on), and the operations toolchain: letting plans be continuously corrected as data, versions, and business evolve, rather than treating planning as a one-time result.

The common sources of “choosing the wrong plan” fall into three categories: row estimation deviating from the data state (changes in table size, data skew, or correlated predicates causing inaccurate estimation; the first response should be refreshing statistics and adjusting the collection strategy, not jumping to hints); parameter sensitivity of high-frequency SQL (parameterized caching does not guarantee all parameters are equally well served, requiring coordination with business data traffic and the timing of baseline establishment); and database objects (indexes, partitions, and so on) or version upgrades (SPM compares new and old plans under real traffic, rather than switching the entire cluster instantaneously after a single statistics task).

OceanBase handles execution-plan risk through a combination of runtime reuse, switch-time governance, and manual intervention during troubleshooting.

At runtime, parameterized plan caching is used first to reduce hard-parsing overhead, letting SQL with the same structure but different parameters reuse existing plans as much as possible. If parameter distributions vary widely and a single plan cannot cover all scenarios, further mechanisms such as SPM, adaptive plan matching, or targeted hints are needed to constrain plan behavior.

Among these, SPM’s role is “validate the new plan first, then promote it.” A candidate new plan does not immediately replace the existing one; instead, it enters the baseline-and-evolution process and is validated under real traffic before deciding whether to switch. Through sql_plan_management_mode, the system can also make policy trade-offs between "online evolution" and "baseline plans."

For problems that have already been clearly diagnosed, Outline / Hint are better suited as local correction tools, used to pin down access paths, join orders, or parallel strategies. During the diagnosis phase, tools such as EXPLAIN, SQL Trace, and DBMS_XPLAN can help locate execution-plan problems.

Therefore, the OceanBase optimizer cannot guarantee that every SQL statement uses the optimal plan on every execution. Instead, it places plans within a continuously governable process: at runtime it relies on plan caching to reduce repeated optimization cost; when plans change, it controls switching risk through SPM; and after problems surface, it uses hints, outlines, and diagnostic tools to locate and correct them.

From Capabilities to Scenarios: Where the Optimizer Lands in Typical HTAP / Real-Time Analytics Workloads

The “select accurately, run well, stay stable” described above is not an abstract optimizer slogan; it translates into different benefit priorities across different business workloads. For enterprise systems, the value of the optimizer is often demonstrated not through a single benchmark, but through whether it can reliably support several typical query scenarios.

Complex queries and financial core scenarios: the focus is whether global cost is trustworthy

In scenarios like financial cores and real-time risk control, queries are often not simple point lookups; they need to combine transaction data, account information, rule conditions, market data, or credit-limit information to make high-frequency judgments. Such queries are characterized by many joined tables, strong real-time requirements, and sensitivity to result latency, while also being easily affected by data distribution and cross-node access.

In these scenarios, the most critical capability of the OceanBase optimizer is not making a particular operator locally optimal, but whether it can form a more trustworthy global cost judgment at the plan-generation stage. Whether statistics and row estimation are accurate determines whether the access path and join order are reasonable; whether the distributed plan considers data location and network cost early enough determines whether cross-node latency keeps getting amplified in high-frequency queries. For these mixed TP-and-real-time-risk-control queries, the value of the OceanBase optimizer shows up first in keeping complex joined queries stably within an acceptable latency, rather than letting plan fluctuations become a source of tail latency.

HTAP / real-time analytics scenarios: the focus is whether row/columnar paths, distribution, and parallel decisions can be handled together within a unified framework

In more analytics-leaning workloads such as reporting, operational analytics, and real-time data warehouses, the problems the optimizer faces become even more complex, because the same dataset may serve both high-concurrency short queries and large-range scans, multi-table joins, and aggregation analytics.

The OceanBase optimizer solves these problems within the same CBO decision framework: row/columnar path selection, analytical cost rewriting, distributed plan generation, Auto DOP, and — when necessary — plan-stability governance can all happen in coordination within a unified optimization process.

High-concurrency write scenarios: the optimizer’s focus lies more in stability governance for highly reusable SQL

For more OLTP-leaning high-concurrency write scenarios, the priorities differ from analytical workloads. Here, plan caching, parameterized matching, execution-plan stability, and the commit efficiency of the write path itself usually deserve more attention. Capabilities like Plan Cache are more directly relevant to typical transactional write paths; the performance of complex analytical SQL, on the other hand, often still needs to be considered together with sharding design, index design, and specific query patterns.

This precisely illustrates that the value of the OceanBase optimizer does not lie in “solving every scenario with the same single point,” but in whether it can, for different workloads, combine capabilities like row estimation, path selection, distributed execution, parallelism, and stability governance into plan-selection logic suited to that scenario.

The Key to the OceanBase Optimizer Is a Closed Optimization Loop Within a Unified Framework

Let’s return to the question from the beginning of this article: why does the same kind of SQL run “sometimes fast, sometimes slow” within the same system?
The real answer often lies not in whether some single feature exists, but in whether the system has a mechanism that can continuously generate reasonable plans under continuously changing data and workload conditions.

The OceanBase optimizer, with distributed-native + HTAP-integrated as its through-line, completes a full closed loop within a unified CBO framework:

Through statistics, dynamic sampling, storage-layer row estimation, and other capabilities, it makes cost judgments as trustworthy as possible;
Through row/columnar path selection, complex SQL rewriting, distributed plan generation, and parallel decisions, it selects and runs plans well;
Through parameterized plan caching, adaptive plan matching, and SPM, it brings stability governance for highly reusable SQL into a systematic mechanism.

This is the true value of an enterprise-grade query optimizer in HTAP and real-time analytics scenarios: it does not just make a single SQL run faster, but keeps the system able to maintain performance, resource efficiency, and stability within an explainable, governable, and dependable range — even under complex queries, distributed execution, and continuously changing data conditions.

AI Writes Code Faster. Why Hasn’t Delivery?

Charles Wu — Wed, 27 May 2026 15:59:00 +0000

The bottleneck didn’t disappear — it moved downstream.

When coding gets 10× faster but review, CI, and release don’t, you don’t get 10× delivery. You get a longer queue behind the keyboard.

If your stack includes a database layer, that queue often shows up twice — once in application CI, again in migrations, backups, and failover. The closing section links how we think about that at OceanBase.

I recently watched an interview with Cat Wu on how Anthropic’s product team went from shipping a feature every few months to every few weeks, sometimes days, and for small slices of work — even within a single day. Our team has been having a parallel conversation: what concrete practices actually turn AI speed into delivery speed?

My main takeaway isn’t “AI writes code scary fast.”

That’s the shallow read.

Lots of teams now claim delivery is 10× or 50× faster with AI.

I’m skeptical.

Because one thing gets conflated all the time:

Faster code generation is not the same thing as faster software delivery.

An agent can draft a patch in ten minutes. Sure. But whether that patch can land on main, be validated, reach real users, and be debugged or rolled back when it breaks—that’s a different system entirely.

If only the “writing code” step speeds up while review, testing, release, monitoring, and rollback stay the same, “50× faster” is often a local illusion: one part of the pipe got hot; the org is still stuck.

Most teams, in my view, aren’t at the “quantity becomes quality” inflection yet.

The real shift isn’t “everyone runs more coding agents.” It’s that how you set goals, verify code, ship features, and contain risk has to change. Otherwise AI doesn’t multiply delivery — it multiplies PRs waiting for review, features waiting for validation, and branches waiting to merge.

As CircleCI puts it, success in the AI era is “no longer determined by how fast code can be written” — the decisive factor is whether you can “validate, integrate, and recover at scale.” For most teams, that’s a sharper framing than asking whether AI can generate code at all.

What’s more interesting: Cat has said outright that new internal models weren’t the main driver of faster iteration. The lever was process — how goals are set, how docs are written, how previews ship, how cross-functional work runs, and who has authority to put something in front of users.

Shipping a feature a day isn’t about whether the model can code. It’s about whether the org removed what used to slow releases down.

Here’s the thesis up front:

When code gets cheap, the expensive thing becomes judgment.

Judgment about what’s worth building, how good is good enough to ship, where a human must decide, and where an agent can run to completion.

So “AI-native” speed isn’t “everyone uses AI to write code.” Two things have to happen together:

Less idle motion in the process. More clarity in the rules.

Less idle motion means fewer documents, handoffs, approvals, and waits that exist only because “that’s how we’ve always worked.” More clarity means goals, evidence, verification, permissions, release, and rollback are specified — not reinvented in every hallway conversation.

Only when both are true does “one release per day” stop being a gamble.

1. Most teams misunderstand “AI speedup”

Seeing Anthropic ship often, many people jump to: strong models, engineers on Claude Code, therefore fast coding.

That helps. It isn’t the main story.

If coding is the only thing that speeds up, what do you get?

More PRs waiting for review
More features waiting for validation
More half-finished branches
More production risk
More arguments about whether this is shippable at all

That’s not faster delivery. It’s congestion moved downstream from typing to everything after typing.

A routine feature might have taken days from kickoff to first PR. Now a crisp small ask can get a first diff in minutes. But “code exists” is a short leg of the journey.

After that comes a whole chain: Should we build this at all? Is this the right shape? Who owns blast radius? Is test evidence enough? Can we spin a preview? Should this sit behind a flag? How do we roll back?

When coding was slow, that chain was masked by coding time. Code in thirty minutes, CI still twenty, review still half a day, QA still queued — and the backlog becomes visible.

Qovery’s piece on AI and DevOps in 2026 makes the same point from the platform side: when AI coding tools explode code throughput, CI/CD, environment provisioning, and deployment pipelines — not typing speed — become the constraint. As they put it, the bottleneck has flipped: less time coding, more time waiting on builds, previews, and deploys.

So I’ve stopped framing R&D efficiency as lines of code per hour.

A better line:

AI compresses the coding segment and forces you to face the real system bottleneck.

The lesson from Anthropic’s public interviews isn’t simply “they had a better model.” Cat Wu has said internal model use raised shipping speed only “a little bit,” with “the bulk of the increase” coming from process and team expectations. When code gets cheaper to write, both Wu and Mike Krieger describe bottlenecks shifting — to deciding what to build, merge queues, CI, and the other steps that used to hide behind slow coding.

2. Layer one: nail the goal, thin the doc

In classic product development, the PRD (product requirements document) often functions as comfort food.

Longer feels more professional. More edge cases upfront feels more in control. Before engineering starts, you get fifteen pages of background, scope, flows, exceptions, competitors, and timeline.

In AI-native teams, that pattern ages badly.

Not because docs don’t matter — because long docs often disguise decisions as description.

What matters isn’t pre-specifying every button and edge case. It’s being explicit about goal, principles, and how you’ll know it worked.

Cat’s interview: routine features don’t need a novel-length PRD; large infrastructure bets still might. Many features need one page — goal, principles, metrics — and then people with context make local calls.

That’s a real role change.

PM value used to be: “I specified it completely; engineering executes.”

Now it’s closer to: “I made the goal and tradeoffs legible so people with context can decide quickly.”

That matters more with agents in the loop. If every micro-decision waits on PM, you’re back to pre-AI cadence. If the goal is fuzzy, agents and humans will sprint in the wrong direction together.

A one-pager isn’t laziness. It should answer:

## Goal
What user problem are we improving?

## Non-goals
What are we explicitly not solving this round?

## Principles
When tradeoffs appear, what wins?

## Success signals
What observable outcomes mean "keep going"?

## Risks
What must never be touched without a human? What needs explicit sign-off?

That beats twenty pages of “looks complete” for agents and engineers alike. Agents don’t need literary prose — they need bounds and decision rules. Engineers don’t need a script — they need to know where they can decide alone.

3. Layer two: ship research previews, lower the promise

Another Anthropic habit: lots of capabilities land first as research previews.

That can sound like “ship half-baked work.” It isn’t lowering quality — it’s narrowing the promise.

Why do traditional teams ship slowly?

Every release carries a hidden contract: stable, complete, friendly to all users, documented, consistent, no obvious landmines.

That contract is heavy.

So teams enter wait mode: one more edge case, one more design pass, one more test round, one more stakeholder sync.

Six months later, users still haven’t touched it.

Research previews change expectations upfront:

This is early. We’re still finding the shape. You can try it — it isn’t the final product.

Three effects:

You don’t have to wait for perfect.
Real user signal arrives earlier.
Bad product directions fail faster.

The trap: research preview ≠ irresponsible preview.

You can’t throw junk over the wall and blame “early access” for UX debt.

A healthy preview still needs three guardrails: bounded blast radius, risk can be turned off or rolled back, and users know what stage they’re in.

The promise can be lighter. Safety work cannot.

That’s why “less process idle time” only works alongside clearer rules — fewer approvals, not fewer checks; thinner docs, not thinner goals; earlier ship, but still flags, monitoring, and rollback.

4. Layer three: looser roles, pricier taste

Cat noted something representative: many PMs on Claude Code have engineering backgrounds; designers ship frontend code. Product, engineering, and design are less siloed.

That trend will accelerate.

When code is cheap, work that existed only to hand off starts to feel wasteful.

PMs used to validate an idea via spec → design → eng queue → wait. Now a PM who codes can spike a prototype; an engineer with product sense can fix interaction gaps; a designer who ships UI can get to something runnable.

That doesn’t mean everyone becomes full-stack or specialties vanish.

Role is default responsibility — not the ceiling on what you can do.

You might be a PM — but can you prototype when the team needs it? You might be an engineer — but can you flag when the problem framing is wrong?

The scarce skill isn’t typing code. It’s product taste.

When everything is buildable, the expensive question becomes:

Of all the things we could build, which ones should exist?

That’s what Cat means when she says that as code gets cheaper, choosing what to write gets more valuable.

Without taste, AI teams do something worse than traditional teams:

They ship the wrong things faster.

Bad ideas used to die on engineering cost. Not anymore — agents will diligently implement them.

In AI-native teams, PM value isn’t scheduling or chasing status. It’s picking the highest-leverage slice, defining a small but real unit of ship, and separating noise from signal in early feedback.

That’s harder than “writing good prompts.”

5. Layer four: don’t build for imaginary AGI

One of the harder points in the interview: many AI product people build for a future super-model, not today’s model.

“Models will catch up — ship the scrappy version; AGI will fix it.”

Dangerous.

The hard job is shipping for the model you have now:

Know where it’s strong and weak, what you can delegate, what needs guardrails, what needs external memory, task lists, or human takeover.

Early Claude Code big refactors sometimes stalled mid-flight — so the team added todo lists so work was decomposed and tracked. As models improved, some of that scaffolding could fade.

Cat’s line: the model eats your adaptation layer.

A lot of product design that feels essential today will matter less tomorrow as models improve.

Two buckets:

1. Model-gap scaffolding — todos, forced continuation, verifiers when the model won’t self-check. Let these retire as capability catches up.

2. Task-structural constraints — permissions, release policy, rollback, audit trails, user promises, gradual rollout, UX commitments.

Those don’t vanish because the model got smarter — strong human engineers don’t eliminate permission systems either.

Complexity often comes from mixing the buckets: scaffolding that should expire gets cast in concrete, while structural safety gets waved away as “the model will figure it out.” Both hurt.

6. What actually holds up “one feature per day”

Mapped to practice, I’d break “daily ship” into something concrete:

Step 1 — Intake: No novella. Goal, non-goals, principles, risks, acceptance.

Step 2 — Build: Agent works in an isolated worktree or sandbox — read, edit, run tests, add tests.

Step 3 — Evidence, not “I’m done”: Files touched, entry points affected, tests run / not run, touches on auth/data/money/security, flag needed?, rollback path.

Step 4 — Review by risk tier: Low risk → lean on automation + evidence completeness. Medium/high → human eyes on architecture, UX, security.

Step 5 — Merge without blind exposure: Default behind a flag; internal environments first when possible.

Step 6 — Release train: A fixed daily window for changes that are merged, verified, and within risk appetite.

Step 7 — Observe: Error rates, latency, key conversion, user feedback, log anomalies — not “we shipped, done.”

Step 8 — Incidents: Flip the flag or roll back first; don’t spend thirty minutes in a blame meeting.

Agents matter. Clear rules at each gate matter more.

I call that bundle a Release Harness — not a single tool, but constraints on how work is sliced, how evidence is submitted, how risk is tiered, what must be automated, what must stay human, and when merge / ship / rollback is allowed.

A minimal release checklist is seven fields: goal, scope, risk tier, verification evidence, ship method, rollback method, watch metrics. The point isn’t a pretty template — it’s that every PR can answer them.

Without that, “one release a day” becomes a daily merge of opaque AI diffs with users as QA.

That isn’t leverage. That’s risk moved downstream again.

7. Traps I’ve stepped in

7.1 Treating “daily ship” as “one big feature per day”

Common mistake.

Daily ship means small changes can enter production safely each day — not a finished epic every sunset.

Big bets should decompose: schema → API → internal entry → flag → user-visible UI. Each step can ship; not each step must be user-visible.

If asks stay huge and agents only accelerate implementation, you get bigger PRs, harder review, nastier integration.

AI amplifies how well you slice work. Slice well → parallel progress. Slice badly → a fast, scary monolith.

7.2 One agent, one shot, whole feature

“Here’s the full ticket — do all of it” works in demos. In production, it’s risky.

Agents finish local tasks well; they don’t automatically know your release bar. They may refactor the wrong layer, add a “reasonable” compatibility shim, and bundle UI, API, tests, and config into one diff.

The PR looks productive. Reviewers suffer.

I prefer task types:

Explore — read-only, options, no edits
Implement — bounded scope, tests required
Verify — reviewer mindset, hunt risks
Fix — confirmed issues only, no scope creep

More stable than one agent from start to finish.

7.3 Green tests, broken product

Worse and common: unit tests pass, the PR claims “done,” but no one walked the real user path — flag missing in staging, menu hidden, role matrix gap, field mismatch on the live API.

AgentField’s writeup on ~200 autonomous agents writing production code describes the same gap: parallel agents can leave every issue green while the merged product still fails — tests passed with mocked dependencies, every acceptance criterion met while a module stayed invisible to consumers. The system optimizes for the criteria and checks you encode, not for coherence you never specified.

Green means covered paths didn’t explode. Not the product entry works.

Evidence needs an integration slice: where’s the entry, routes, config, permissions, analytics, end-to-end path.

Without that, you ship a half-plugged-in feature.

7.4 No feature flags, so long branches become “the plan”

“We want high frequency” without feature flags doesn’t add up.

No flags → unfinished work can’t live on main → long-lived branches → merge pain → release frequency drops.

With agents touching many surfaces at once, branch merges get uglier.

Flags aren’t just “gradual rollout.” They let incomplete capability exist safely on main.

7.5 Automation that’s 95% right

Cat’s advice for individuals applies to teams: if automation isn’t reliable, it isn’t automation — it’s a new chore.

95% correct release notes that omit risks → humans re-read every PR.

Auto-fix CI that patches wrong → humans re-audit every patch.

Auto-verify that false-passes → humans redo manual QA.

The last 5% has to be trustworthy enough to depend on.

Closing: speed isn’t the goal — feedback is

Not every team should ship daily.

Core infrastructure, regulated financial flows, heavy compliance — release cadence isn’t the only metric.

Anthropic’s practice is useful because it surfaces a sharper question:

After AI made code fast, did your feedback loop get fast?

If not, you mostly get more half-done work, more PRs, more verification load, more integration risk.

If yes, the shape of the team changes: clearer goals, thinner docs, looser handoffs, faster path to trunk, hidden-by-default features, evidence that accumulates automatically, rollback that isn’t improvised.

Then AI becomes organizational R&D capacity, not a typing sidecar.

When I gauge maturity, I don’t count agents, LOC, or open PRs.

I ask three questions at end of day:

What actually reached production today?
Why was that safe?
If we were wrong, how fast can we revert?

Teams that answer those three consistently are the ones that can talk about shipping every day without gambling.

What this means if you ship database changes

Application delivery is only half the picture when agents touch schema, tenants, replication, or ops runbooks. The same rules apply: thin goals, explicit evidence, flags or staged rollout, and rollback you have practiced — not “the agent said the cluster is fine.”
At OceanBase, we see the same bottleneck shift in the open-source community: AI speeds up how people generate deploy scripts and config, but production still depends on verification, integration, and recovery — especially for distributed databases where a green unit test does not prove a safe cutover.
If you are experimenting with agent-assisted database work, three places to start:

oceanbase-skills (including oceanbase-deploy) that wrap deployment, tenant management, and benchmarks in governed, repeatable flows — not one-off prompts.
OceanBase documentation — canonical steps for install, upgrade, and operations so agents and humans share the same source of truth.
Contributions and feedback — open an issue or PR on GitHub if you are hardening a Release Harness that includes the data plane; we are interested in what breaks when coding gets 10× faster but cluster validation does not.

Your move: Pick one change you shipped (or almost shipped) in the last month. Ask whether evidence covered app + data + rollback — not just “tests passed.” If the data path was hand-waved, that is your downstream bottleneck.

Stop Editing JSON by Hand

Charles Wu — Sun, 24 May 2026 16:36:32 +0000

How ClawMaster helps you set up OpenClaw, manage Skills, and use PowerMem — without living in config files

TL;DR: ClawMaster is an open-source OpenClaw companion with a setup wizard, channel and model management, cost observability, Skill hosting, and PowerMem-backed memory. Install with two commands, then open http://localhost:16223.

When Skills pile up

Once Skills under an agent multiply, overlaps and clutter show up fast. A common pattern today is filesystem scanning: Skills live as Markdown on disk, and the runtime may walk through every SKILL.md when it needs one.

As the library grows, a few frictions appear:

Finding the right Skill in a large tree takes longer.
Long SKILL.md files are easy to read incompletely; recall gets less stable.
Version, dependencies, and “when do I use this?” are hard to track with folders and filenames alone.
Context windows limit how much you can load in one turn.

That is the backdrop for tools that make OpenClaw easier to run day to day — not only to install once.

What ClawMaster is

ClawMaster is an open-source project from openmaster-ai. It is described as an “OpenClaw companion for real life” — a path from install to daily use, not just getting config files right.

You get an OpenClaw management console from install through configuration: a setup wizard so you can configure models and channels and see what each AI call costs — without memorizing commands or hand-editing JSON.

It also supports:

PaddleOCR document parsing
ERNIE (Wenxin) image generation
Cost observability
Scheduled tasks
In-browser Skill refresh and hosting

Who it’s for

Quick start

npm i -g clawmaster
clawmaster

Open http://localhost:16223. The setup wizard walks you through the rest.

After launch:

Pick or create an OpenClaw profile.
Connect at least one model provider and set a default.
Add channels, plugins, Skills, or MCP servers as needed.
Enable gateway or observability when you want runtime inspection.

PowerMem and LLM Wiki

ClawMaster integrates PowerMem — the memory engine open-sourced by the OceanBase team. Instead of a pile of Markdown files, memory becomes structured, queryable storage with a forgetting curve.

ClawMaster also aligns with Andrej Karpathy’s LLM Wiki idea: content goes in once, the knowledge base keeps compounding, and the agent can draft with views you have already captured — even when you do not paste a link again. ClawMaster tracks a related LLM Knowledge module on the v0.4.0 milestone.

Try it

npm i -g clawmaster && clawmaster
Finish the wizard at http://localhost:16223
Configure one channel or model you actually use
Install or refresh one Skill from the UI

If you want guided exercises, see clawmaster-workshop.

References

ClawMaster: https://github.com/openmaster-ai/clawmaster
PowerMem: https://github.com/oceanbase/powermem
LLM Wiki (Karpathy): https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f

Based on the OceanBase community write-up on ClawMaster and the public ClawMaster README (v0.3.1). Check the repo for the latest release and roadmap.

When Your Database Agent Speaks Plain English

Charles Wu — Sun, 24 May 2026 13:25:30 +0000

The OceanBase community just shipped agent Skills that turn natural language into OBD commands.

TL;DR: The OceanBase community just open-sourced oceanbase-skills —
agent Skills that turn natural language into OBD commands. First release:
oceanbase-deploy for cluster ops, benchmarks, and backup/restore.

Someone on the OceanBase community team recently shipped something that made me pause: a repository called oceanbase-skills.

Here is the pitch: instead of memorizing OBD flags and config files, you describe what you want in plain English and let a Skill handle the rest.

The first release — oceanbase-deploy — is already usable. I ran a TPC-H benchmark with a one-line prompt. All 22 queries finished in under 10 seconds.

Let me show you how it works.

Why this matters

If you have operated OceanBase, you know OBD is powerful. You also know the CLI has a broad command surface: config files, required flags, switchover vs failover, benchmark paths that need precise parameters.

Running a TPC-H benchmark the old way:

# 1. Open docs to confirm syntax
# 2. Remember --remote-tbl-dir is mandatory
# 3. Create the directory
mkdir -p /tmp/tpch
# 4. Assemble the full command
obd test tpch ob-test --tenant=mysql_test --remote-tbl-dir=/tmp/tpch --scale-factor=1
# 5. Run it and pray...

With the Skill installed, you just say:

Run TPC-H for tenant mysql_test on cluster ob-test.

The agent fills in --remote-tbl-dir, creates the path, and runs the job. In my test, all 22 TPC-H queries finished in under 10 seconds.

Same pattern works for deploying seekdb (OceanBase’s AI-native hybrid search database) and driving mysqltest — describe the goal, not the subcommand tree.

What oceanbase-deploy covers

Today’s scope:

Cluster deployment and lifecycle management
Tenant operations
Benchmarks (TPC-H and more)
Backup and restore workflows
Other routine OBD operations

No new APIs — just curated OBD workflows in Skill form, so agents follow documented flags instead of guessing.

The roadmap: a Skill catalog, not a one-off

The repository currently focuses on deployment and ops, but the README outlines what’s next:

More skills are on the way. Planned areas include OceanBase kernel tuning, SQL diagnostics, migration, and more.

Contributions are welcome at github.com/oceanbase/oceanbase-skills.

Think of this as the start of an OceanBase Skill layer for agents: small, composable packages instead of one monolithic “MCP server” that tries to do everything.

Closing

The pattern here is incremental by design: ship a focused Skill, prove the value, then expand. Not a grand “one platform to rule them all” narrative — just one Skill, one workflow, one problem solved at a time.

Try it: clone oceanbase-skills, install oceanbase-deploy in your agent, and replace one manual OBD command you run weekly with a one-line prompt.

If it saves you a doc tab, star the repo and tell the team what Skill should ship next.

References

oceanbase-skills: https://github.com/oceanbase/oceanbase-skills
oceanbase-deploy: https://github.com/oceanbase/oceanbase-skills/tree/master/skills/oceanbase-deploy
seekdb: https://github.com/oceanbase/seekdb

Building agent tooling for databases? What’s your biggest friction point with CLI-based ops? Drop a comment below.

👏 Clap if this helped · 🔔 Follow for more database engineering deep dives

Note: oceanbase-skills and seekdb are OceanBase community projects. The author contributes to the broader OceanBase ecosystem.