DEV Community: Fred Feng

Kookaburra: an Embedded Object Database for Java, with Type-Safe Queries

Fred Feng — Sun, 11 Oct 2026 12:38:44 +0000

1. Overview

Kookaburra: query Java objects like a database, right inside your application.

An embedded object database for Java 17+. Annotated classes in, type-safe chained queries or SQL-style OQL out, answered from indexes in milliseconds. Transactions at three isolation levels, unique and foreign-key constraints, exact decimals, online backup, and a desktop tool, Kookaburra Studio.

Video tutorial (2:57, English subtitles):

If the player does not show, watch it on the project website or open the MP4.

2. What Problem Does It Solve

Keeping your own objects usually means an ORM plus a relational database; the light alternative, stream().filter(), scans everything on every query. Kookaburra sits in between: write queries like streams, run them like a database, in your process.

Pain point	Kookaburra
Heavy ORM / SQL mapping	store annotated objects directly
`stream().filter()` scans everything	indexes: milliseconds on 10 M objects
Field names in strings	`where(Book::getYear)` or the generated `Book_`
Duplicates and dangling references	`@Unique` (one field or a combination), `@ForeignKey` with RESTRICT / CASCADE / SET_NULL
Several objects must change together	transactions: READ COMMITTED, SNAPSHOT, SERIALIZABLE
Money rounding errors	exact `BigDecimal`
Backups of a live store	`db.backup(dir)` while writing
A binary store you cannot see into	Studio: browse, edit, query, back up

3. Quick Start

<dependency>
    <groupId>com.github.paganini2008</groupId>
    <artifactId>kookaburra-core</artifactId>
    <version>1.0.0-SNAPSHOT</version>
</dependency>

try (ObjectDB db = ObjectDB.open(Path.of("data"))) {
    db.batch()
      .insert(new Book("Dune", "Frank Herbert", Genre.SCIFI, 1965, 9.99, "classic", "desert"))
      .insert(new Book("Neuromancer", "William Gibson", Genre.SCIFI, 1984, 12.50, "cyberpunk"))
      .insert(new Book("Hyperion", "Dan Simmons", Genre.SCIFI, 1989, 14.20))
      .commit();

    List<Book> books = db.query(Book.class)
            .where(Book::getGenre).eq(Genre.SCIFI)
            .where(Book::getYear).ge(1960)
            .orderBy(Book::getYear).desc()
            .limit(2)
            .list();
    System.out.println(books.stream().map(Book::getTitle).toList());
}

[Hyperion, Neuromancer]

Studio: mvn -f kookaburra-face/pom.xml -P dist package, then double-click bin/kookaburra-face.jar.

4. Requirements

Item	Requirement
Java	17+
Runtime dependencies	RoaringBitmap 1.6.22, ANTLR 4.13.2 runtime, SLF4J API 2.0
Build	Maven 3.8+
Studio	Java 17+; one jar for Windows x64, Linux x64 and macOS Apple Silicon (JavaFX bundled)

Supported field types: boolean, all integer types, float / double, BigDecimal, String, enums, LocalDate / LocalDateTime / Instant, Ref<T>, lists and sets of these, @Embeddable classes (and lists / maps of them), Map, byte[], any Serializable.

5. How It Works

You	Kookaburra
annotate plain classes	stores them in a data directory (or in memory), no server
query with method references or OQL	answers from indexes; `explain()` shows which one
write objects, batches or transactions	commits atomically and durably, checks constraints
open the directory in Studio, or `db.backup(dir)`	visual browsing and editing, consistent copies while running

6. Code Examples

Example 1: group and aggregate

Input: six books (4 sci-fi, 1 fantasy, 1 crime)
Execution:

db.query(Book.class).groupBy(where(Book::getGenre))
  .agg(Q.count().as("books"), Q.avg(where(Book::getPrice)).as("avgPrice"))
  .sortedBy(Q.count().as("books").desc())
  .toList();

Output:

SCIFI   books=4 avg=11.36
FANTASY books=1 avg=10.00
CRIME   books=1 avg=7.40

Example 2: OQL query and bulk update

db.oql("SELECT b.author, b.year FROM Book b WHERE b.year < :y ORDER BY b.year").param("y", 1950).rows();
db.oql("UPDATE Book b SET b.price = b.price * 0.9 WHERE b.genre = 'CRIME'").execute();

J.R.R. Tolkien 1937
Raymond Chandler 1939
updated=1

Example 3: constraints, decimals, callbacks

Input:

@Entity
public class Order {
    @Id long oid;
    @Unique String number;
    @ForeignKey(onDelete = OnDelete.CASCADE) Ref<Customer> customer;
    @Index BigDecimal total;
    Instant created;

    @PrePersist void stamp() { created = Instant.now(); }
}

Execution and output:

Code	Output
`db.insert(new Order("A-1", ann, "5"))` (A-1 exists)	`ConstraintViolationException: duplicate value 'A-1' for unique Order.number`
`db.query(Order.class).sum(where(Order::getTotal))` over 19.90 + 10.15	`30.05` (exact)
`db.referrers(Order.class, ann)`	`[A-1, A-2]` (backlinks)
`db.delete(Customer.class, ann)`	Ann and her orders, one atomic commit

Example 4: SERIALIZABLE stops write skew

Input: two doctors on call; each one goes off call if the other is still on call
Execution:

db.inTransaction(TxOptions.defaults().isolation(Isolation.SERIALIZABLE), tx -> {
    long onCall = tx.query(Doctor.class).where(Doctor::isOnCall).eq(true).count();
    if (onCall > 1) {
        tx.update(tx.get(Doctor.class, me).offCall());
    }
});

Output: under SNAPSHOT both commit (nobody on call); under SERIALIZABLE the second commit fails with ConflictException ("serialization failure"), is retried, and one doctor stays on call

Example 5: online backup

BackupInfo info = db.backup(Path.of("backups/2026-10-17"));   // writers keep going
try (ObjectDB copy = ObjectDB.open(info.path())) { … }        // restore = open

Example 6: nested objects and large values

db.oql("SELECT s.code FROM Shipment s WHERE s.route.to.city = 'Perth'").list();        // [S-1]
db.oql("SELECT s.code FROM Shipment s WHERE 'kiwi' MEMBER OF s.items.sku").list();     // [S-2]
db.insert(new Document("scan", text, Files.readAllBytes(Path.of("scan.png"))));        // MiB-sized values

In Studio: backlinks of the selected object, constraints of every field, nested values as a tree.

7. Configuration

One file: kookaburra.properties.

Property	Default	Description
`kookaburra.durability`	`SYNC`	`SYNC`: a write returns once it is on disk; `ASYNC`: faster, the last moments may be lost on a power failure
`kookaburra.transactions`	`false`	enables transactions; off = every write commits directly
`kookaburra.tx-isolation`	`SNAPSHOT`	default level: `READ_COMMITTED`, `SNAPSHOT`, `SERIALIZABLE`
`kookaburra.tx-timeout` / `tx-retries`	`60s` / `3`	transaction timeout; `inTransaction` retries
`kookaburra.data`	`data/` next to the jar	Studio default database

Tuning keys (memory, file sizes, background work) are commented in the file; the defaults suit most applications.

8. Performance / Comparison

Environment: Apple M2 Max (12 cores), 32 GB, internal SSD, JDK 17, -Xmx8g; 10 M objects; harness kookaburra-bench (warm-up, p50 / p99, consistency checks after every phase).

Operation (10 M objects)	p50
bulk insert, batches of 100 (`ASYNC`)	696 k objects/s
durable single inserts (`SYNC`), 16 / 128 threads	3.2 k/s / 21.2 k/s
`get(oid)`	0.6 to 1.1 µs
count with an indexed filter	0.24 ms
equality + range + limit 100	0.21 ms
top-10 by an indexed column of 2.3 M matches	0.39 ms
`GROUP BY city COUNT(*)`	4.0 ms
unique-key lookup (OQL)	0.12 ms
full scan with an unindexed predicate	0.81 s
transfers, 16 threads: SNAPSHOT / READ COMMITTED / SERIALIZABLE	136 / 142 / 140 k/s
atomic `increment` of one hot counter, 16 threads	265 k/s, no lost updates
insert with `@Unique` / without (in memory)	235 / 258 k/s
online backup of 10.5 M objects	25.4 s, writers not blocked
OQL `DELETE` of all objects, disk space afterwards	1.32 M objects/s, 0 B left
reopen / heap after reopen	2.0 ms / 6.5 MiB

9. Limitations & Trade-offs

Each durable single write waits for the disk; batch or use ASYNC for bulk loads.
Single-collection queries, no joins (references with fetch, or the semi-join in(...)).
Optimistic transactions: conflicts surface at commit and are retried, never blocked.
Not a fit for: several processes writing one directory, the SQL / JDBC / BI ecosystem, terabyte-scale data.

10. Summary

Stream-style code, database-style speed from indexes.
Method references or a generated metamodel for type-safe fields.
OQL covers the single-table SQL-92 core.
Three isolation levels; SERIALIZABLE closes write skew without locks.
@Unique and @ForeignKey with ON DELETE actions, checked atomically at commit.
Exact decimals, nested objects, large values.
Lifecycle callbacks and listeners, JPA style.
Online backup of a consistent snapshot; restore = open.
Sub-millisecond to a few milliseconds on 10 M objects; instant reopen, tiny heap.
Studio from one jar; every feature tested in memory and on disk, coverage ≥ 90 %.

SqlJam: Move Tables Across Databases with Schema, Data and All

Fred Feng — Mon, 05 Oct 2026 12:23:54 +0000

Tables, indexes, constraints, sequences, partitions and rows, in one go.

1. Overview

SqlJam: Move tables across databases. Schema, data and all.

A Java 17 + JavaFX desktop tool that copies tables between relational and OLAP databases (MySQL, MariaDB, PostgreSQL, Oracle, SQL Server, Db2, H2, SQLite, DuckDB, ClickHouse and more) in any direction:
import directly into a target database or into the same schema as copies, or save a self-describing export package of SQL, Parquet or CSV files and import it later. Imports into a database run in one transaction which an error rolls back, and rows are inserted, updated or skipped by key.

2. What Problem Does It Solve?

The hard part of a cross-database migration is not INSERT, it is the details: type names, identity columns, default value syntax, bigint unsigned overflow, timestamps shifted by time zones, BLOBs inflating scripts, and old servers rejecting new syntax. Hand-written scripts are slow and miss things.

SqlJam reads one metadata tree and generates SQL with a dialect chosen by target type and version. Export packages carry a manifest.json, so an import knows the target, checks completeness and verifies every file.

Pain	SqlJam
Dialect differences	Types, identifiers, defaults, identity and literals translated by the dialect of each target
Old versions	Version subclasses: `Oracle11gDialect`, `PostgreSQL9Dialect`, `MySQL56Dialect`, `SQLServer2008Dialect` …
Missing keys & indexes	Primary keys, unique/regular indexes, foreign keys, comments, sequences, partitions of every kind (range, list, hash, interval, composite, reference, nested), shown with their partitioning in the tree
Huge scripts	Data files split by size: SQL and CSV at 10 MB (`orders.sql`, `orders_2.sql` …), Parquet at 256 MB (`orders.parquet`, `orders_2.parquet` …), imported part by part in order
LOBs	Stored as files, restored by primary key
Scripts without context	`manifest.json`: source, target, options, SHA-256 per file, row counts
Analytics needs Parquet	Parquet packages through an embedded DuckDB, Parquet files of other tools loaded into any table
Spreadsheets need CSV	CSV packages with your delimiter, quote, NULL text, header, encoding (UTF-8 with BOM, GBK and more) and date format, exact types kept in `manifest.json`, CSV files of other tools loaded too
An import went wrong	One transaction by default, rolled back on any error, or commit every N rows. Skip rows with errors goes on and logs each skipped row. Confirm before commit reviews a successful import first
Rows exist already	Update or insert by key or skip existing by key: the primary key or the columns of a unique index match the rows, insert is the default
Warehouses apart from OLTP	DuckDB and ClickHouse data sources, grouped as OLAP databases
Only part of a table	Data viewer query (columns, WHERE, GROUP BY, ORDER BY) and export of the rows of the query
Rows of a join or a report	SQL Query editor with syntax highlighting, the first rows are shown and all rows of the query are exported as a table
A copy next to the original	Copies in the same schema with `{table}_copy`, keys and indexes renamed, source tables protected
Custom and modern types	PostgreSQL enums and domains, vectors, unions, maps, tuples and variants

3. Quick Start

git clone git@github.com:paganini2008/sqljam.git && cd sqljam
./mvnw -DskipTests package       # Windows: mvnw.cmd, output goes to bin/
bin/sqljam.sh                    # Windows: bin\sqljam.bat, or double click the jar of your platform

bin/
├── sqljam-1.0.0-SNAPSHOT-win.jar           # runnable jar of each platform,
├── sqljam-1.0.0-SNAPSHOT-mac-aarch64.jar   # JavaFX and all JDBC drivers inside
├── sqljam-1.0.0-SNAPSHOT-mac.jar
├── sqljam-1.0.0-SNAPSHOT-linux.jar
├── sqljam.sh / sqljam.bat                  # launchers, pick the jar of the platform
├── sqljam.properties                       # configuration, edit and restart
├── sqljam.vmoptions                        # JVM options, one per line
├── sqljam.png                              # Dock icon of macOS
└── sqljam-tutorial.mp4                     # tutorial video, export and import in under 3 minutes

sqljam.properties and sqljam.vmoptions sit next to the runnable jar. Edit them and restart, no rebuild needed.

A splash screen shows the loading of the configuration, the data sources and the JDBC drivers:

The first start brings an Example Shop (H2) data source with a few e-commerce tables (categories, customers, products, orders, order_items) to try everything right away. Example on the login page or Help → Restore Example Database brings it back at any time.

Right-click a data source to Connect, and once it is connected the same menu offers Disconnect, which also closes its table tabs. When an import or export needs a target data source, the + button next to it creates one on the spot. Exit sits at the right end of the toolbar. Help → About SqlJam shows the version, the home page, the repository and how to reach the author, and the GitHub icon at the bottom right opens the repository.

SQL Query on the toolbar opens an editor for the selected data source. Ctrl+Enter (⌘+Enter on macOS) runs the statement at the caret, Ctrl+Shift+Enter runs them all, and Export… sends all rows of the query to SQL, Parquet, CSV or another database:

A short tutorial video walks through the example database: query rows, export into files, import the package into PostgreSQL, and copy the tables straight into SQL Server and Oracle.

1. Log in to a data source	2. Export wizard	3. Progress

4. Data query	5. Package viewer	6. Parquet files

4. Requirements

Only Java 17+ is needed to run SqlJam. JavaFX and the JDBC drivers are inside the runnable jar, and building needs no Maven installation thanks to the Maven Wrapper (./mvnw).

Platform	Runnable jar	Launcher	Supported
Windows 10 / 11 (x64)	`sqljam-<version>-win.jar`	`sqljam.bat`	✅
macOS Apple Silicon	`sqljam-<version>-mac-aarch64.jar`	`sqljam.sh`	✅
macOS Intel	`sqljam-<version>-mac.jar`	`sqljam.sh`	✅
Linux x64 (GTK 3)	`sqljam-<version>-linux.jar`	`sqljam.sh`	✅

Database	Version
MySQL	5.5 to 9.x
PostgreSQL	9.x to 16
Oracle	11g to 23ai
SQL Server	2008 to 2022
MariaDB	10.3 to 11.x
H2 / SQLite	2.x / 3.x
DuckDB (OLAP)	1.x
ClickHouse (OLAP)	24.x to 26.x

All JDBC drivers are bundled in the fat jar.

5. How It Works

flowchart LR
    S[(Source DB)] --> T[Metadata tree]
    T --> D[Target dialect<br/>type + version subclass]
    D --> I[Import directly] --> TD[(Target DB)]
    D --> E[Export package<br/>schema.sql · data*.sql · lob/ · manifest.json]
    E --> SI[ScriptImporter] --> TD
    D --> P[Parquet or CSV package<br/>DuckDB workspace · data/*.parquet or *.csv]
    P --> PI[ParquetImporter · CsvImporter] --> TD

Read metadata once: XxxMetaDataOperations completes comments, identity, generated columns, partitions and sequences per database.
Generate for the target: DbType.createDialect(major, minor) picks the version subclass.
Stream rows: count rows (percentage progress) → page by primary key → normalize values → batch insert or SQL literals.
Parquet and CSV: rows pass an embedded DuckDB workspace, COPY ... TO writes them, read_parquet or read_csv with the column types of manifest.json loads them back.
Transactions: an import runs in one transaction, committed when it succeeds or after your confirmation. MySQL, MariaDB, Oracle and H2 commit around DDL, so keys and identity values are set after the commit.
Upserts and skips: ON CONFLICT, ON DUPLICATE KEY UPDATE or MERGE of each database, kept tables, indexes and keys are skipped.
Skipped errors: a failed page is written row by row under savepoints, each bad row is logged with its key and the others are kept.

6. Code Examples

Example 1: Export as a PostgreSQL package

Input: MySQL database shop

ScriptExporter exporter = new ScriptExporter(new File("export"), DataFileStrategy.FILE_PER_TABLE, 10 * 1024 * 1024);
Exporter.ExportConfiguration config = exporter.getConfiguration();
config.setDbType(DbType.MYSQL);
config.setUrl(DbType.MYSQL.getUrl("localhost", 3306, "shop"));
config.setUsername("root");
config.setPassword("secret");
config.setIdReused(true);
exporter.setTargetDbType(DbType.POSTGRESQL);
exporter.exportDdlAndData();

Output

export/
├── schema.sql   ├── data/orders.sql   ├── data/orders_2.sql
├── lob/         ├── lob-manifest.json ├── constraints.sql
└── manifest.json

The same tables as Parquet files:

ParquetExporter parquet = new ParquetExporter(new File("export-parquet"));
parquet.setTargetDbType(DbType.POSTGRESQL);
parquet.setCompression("ZSTD");
parquet.exportDdlAndData();   // schema.sql, data/orders.parquet, constraints.sql, manifest.json

CsvExporter csv = new CsvExporter(new File("export-csv"));
csv.setDelimiter(";");
csv.exportDdlAndData();       // data/orders.csv, data/orders_2.csv ...

Example 2: Oracle → SQL Server directly (schema created)

ImportExporter importer = new ImportExporter();
Exporter.ExportConfiguration source = importer.getExportConfiguration();
source.setDbType(DbType.ORACLE);
source.setUrl(DbType.ORACLE.getUrl("ora-host", 1521, "ORCL"));
source.setUsername("hr");
source.setPassword("secret");
source.setIncludedSchemaNames(new String[]{"HR"});
ImportExportHandler.ImportConfiguration target = importer.getImportConfiguration();
target.setDbType(DbType.SQLSERVER);
target.setUrl(DbType.SQLSERVER.getUrl("mssql-host", 1433, "demo"));
target.setUsername("sa");
target.setPassword("secret");
target.setTargetSchemaName("hr");
importer.exportDdlAndData();

Output: tables, keys, indexes, foreign keys, comments and sequences in demo.hr. Identities continue from the imported max value.

Rows of a query and copies in the same schema:

source.getTableQueries().put("EMPLOYEES", new TableQuery(List.of("EMPLOYEE_ID", "NAME", "SALARY"),
        "SALARY > 1000", null, "SALARY DESC"));   // target table created with these columns
target.setTableNamePattern("{table}_copy");       // EMPLOYEES_COPY next to EMPLOYEES

Source column	→ PostgreSQL	→ Oracle	→ SQL Server
MySQL `bigint unsigned`	`numeric(20, 0)`	`NUMBER(20,0)`	`decimal(20,0)`
PostgreSQL `jsonb`	same	`CLOB`	`nvarchar(max)`
SQL Server `datetime2(7)`	`timestamp`	`TIMESTAMP(7)`	same
PostgreSQL enum / domain	same (created in the target schema)	`VARCHAR2` / base type	`varchar` / base type
MySQL / Oracle `VECTOR`	`text`	same	`nvarchar(max)`
ClickHouse `Array(T)` / `Map(K, V)`	`text`	`CLOB`	`nvarchar(max)`

7. Configuration

All settings live in bin/sqljam.properties next to the runnable jar. Settings changed in the UI are saved to ~/.sqljam/sqljam.properties and win over the file.

Property	Default	Description
`sqljam.export.max-file-size`	`10485760`	Max bytes of a SQL or CSV data file
`sqljam.export.parquet-max-file-size`	`268435456`	Max bytes of a Parquet file, Parquet reads best in large files
`sqljam.export.csv-delimiter`	`,`	Delimiter of CSV files
`sqljam.import.confirm-commit.reminder-minutes`	`10`	Reminder for an import still waiting for commit
`sqljam.export.data-file-strategy`	`SINGLE_FILE`	Single file or one file per table
`sqljam.export.lob-separated`	`true`	Write LOBs as files
`sqljam.export.page-size`	`5000`	Rows per page, tuned by benchmark
`sqljam.export.lob-page-size`	`100`	Rows per page for tables with LOB columns
`sqljam.banner.mode`	`console`	Startup banner with the version: console, log or off
`sqljam.import.batch-size`	`1000`	INSERT statements per batch when importing a package
`sqljam.ui.theme`	`Primer Dark`	7 themes
`sqljam.pool.maximum-size`	`10`	Connection pool size

8. Performance

Scenario (200,000 rows × 6 columns)	Time	Rows/s
H2 → Oracle 23ai	2.1 s	95.3k
H2 → export package (32 MB, 4 files)	2.9 s	69.4k
H2 → PostgreSQL 16	3.2 s	62.4k
H2 → MySQL 9.6	3.3 s	60.7k
H2 → SQL Server 2022	5.0 s	40.1k
Export package → PostgreSQL	4.5 s	44.5k

Environment: Apple M2 Max, 32 GB, JDK 17. MySQL/PostgreSQL local, Oracle/SQL Server in Docker. Default settings.

Quality: a full cross-database import matrix with every column type of every database, SQL, Parquet and CSV package round trips, commit and roll back verified on every database, copies in the same schema, old-version SQL executed on real servers, UI and boundary tests.

9. Design Trade-offs

Table-level focus: tables, keys, indexes, constraints, sequences, partitions and comments: the objects data migration depends on.
Rows paged by primary key: stable, deterministic reads on every database.
Tables copied one by one: predictable load on production servers.
Plain SQL in packages: readable, editable and portable. Direct import uses batched inserts for speed.
Version-specific SQL: older servers get the syntax they understand instead of a lowest common denominator.
Parquet and CSV through DuckDB: one embedded engine writes and reads both, no Hadoop libraries.
One transaction by default: an import is all or nothing, confirm before commit when you want to look first.
Grouped rows are for viewing: exports carry real table rows, a GROUP BY result is a view of them.
Copies need their own names: a copy in the same schema takes a table name such as {table}_copy, so the originals are never replaced.

10. Summary

Any-to-any copies. Every cross-database pair passes automated tests.
Dialects are chosen by type + version. Old versions are subclasses.
Tables, indexes, constraints, sequences, partitions and comments move together.
Export package = SQL + LOB files + manifest.json. Export and import are paired.
Data files split by size (SQL and CSV 10 MB, Parquet 256 MB) and imported in order. LOBs stored separately.
Timestamps, unsigned numbers, bits, money, UUID and JSON survive the trip.
Pooled connections and batch inserts: 40k to 95k rows/s for direct imports.
A mainstream dark UI with 7 themes, percentage progress and cancel.
Parquet and CSV packages and files of other tools, DuckDB and ClickHouse as OLAP data sources, imports in one transaction and rows updated by key.
A SQL Query editor exports the rows of any query through the same exits, columns of tables keep their types.
Query the rows of a table, export just those rows, or copy tables inside the same schema.

Vortex TSDB: Live Metrics in One Command — HTTP Writes, Minute Aggregates, Sliding Windows, Replicated in Memory

Fred Feng — Mon, 05 Oct 2026 11:22:32 +0000

1. Overview

Lightweight. Replicated. Real-time.

Vortex TSDB is a distributed time series database for live metrics. Push numbers over HTTP;
read per-minute aggregates, instant values and tumbling or sliding windows. Every node holds the
whole dataset in memory.

Version	Images (amd64 · arm64)	Source	License
1.0.0	`fredfeng033/vortex-tsdb` · `ghcr.io/paganini2008/vortex-tsdb`	GitHub	Apache 2.0

2. What Problem Does It Solve?

Live metrics — devices online, API QPS, orders, temperatures — need exact counts from many
writers, aggregation by the minute and over any window, and a service that survives a dead node.
General-purpose TSDBs do this with a disk engine, a query language and several components; Vortex
does only this layer, in memory, behind plain HTTP.

Pain	Vortex
Many components to deploy and run	One service + optional Redis; `docker compose up -d`
Counts drift under concurrent writers	Writes serialised on the leader, then replicated: exact, no locks
Windows need tables or a query language	`range` · `step` · `window` · `z`, folded at read time
One node down, service down	Full replica on every node; new leader in ~5 s
Memory grows with every series	Per-node key limit; overflow to Redis

3. Quick Start

Free on Docker Hub and GHCR · linux/amd64 + linux/arm64 · no JDK, Node or build needed

curl -fsSLO https://raw.githubusercontent.com/paganini2008/vortex/main/deploy/docker-compose.yml
docker compose up -d                     # 3 nodes + web console + gateway

curl -X POST 'http://localhost:9080/tsd/push?t=long&c=car&d=speed&v=44'
curl 'http://localhost:9080/tsd/last?t=long&c=car&d=speed'

{"code":1,"data":{"value":44,"timestamp":1791182928819},"elapsed":1,"msg":"ok","requestPath":"/tsd/last"}

Open	URL
Web console	http://localhost:9080/
Swagger UI	http://localhost:9080/swagger-ui.html

Also	Command
Pull	`docker pull fredfeng033/vortex-tsdb:latest` · `docker pull fredfeng033/vortex-tsdb-web:latest`
GHCR	`docker pull ghcr.io/paganini2008/vortex-tsdb:latest` · `VORTEX_REGISTRY=ghcr.io/paganini2008 docker compose up -d`
Your Redis	`VORTEX_REDIS_HOST=host.docker.internal VORTEX_REDIS_PASSWORD=secret docker compose up -d`
From source	`git clone https://github.com/paganini2008/vortex.git && cd vortex && ./run-docker.sh`

Nodes take 20–30 s to start; wait for (healthy) in docker compose ps.

4. Requirements

To	You need
Run the images	Docker with compose v2 (Docker Desktop, Rancher Desktop, OrbStack, Engine 23+)
Overflow (optional)	Redis — tested with 7.4 and 8.6
Build from source	JDK 17+, Maven 3.9, Node.js 20+
Stack	Spring Boot 4.1 · openspreader · Next.js 16 · React 19 · Tailwind 4 · Traefik v3

5. How It Works

Data	Key	Structure
Minute bucket	`tsd:<type>:<category>:<dimension>:<minute>`	max · min · sum · count; retention as TTL
Latest value	`tsd:last:<category>`	Hash: a category in one read
Catalog	`tsd:catalog`	Sorted set by last sample
Health	`tsd:health`	Hash; each node every 5 s

6. Code Examples

Example 1 — push and read back


Input	`for v in 44 67 112; do curl -X POST "localhost:9080/tsd/push?t=long&c=car&d=speed&v=$v"; done` then `curl 'localhost:9080/tsd/retrieve?t=long&c=car&d=speed&z=Asia/Shanghai'`
Execution	Three samples land in one minute bucket; `retrieve` returns the latest 60 buckets in zone `z`
Output	`"09:29:00": {"count": 3, "highestValue": 112, "lowestValue": 44, "totalValue": 223, "averageValue": 74.3333}`

Example 2 — a sliding window


Input	`curl 'localhost:9080/tsd/query?t=long&c=car&d=speed&range=6h&step=5&window=15'`
Execution	72 points; each folds the 15 minute buckets before it. `window = step` gives tumbling windows
Output	`{"step": 5, "window": 15, "last": {"value": 61}, "summary": {"count": 21540, …}, "points": [{"from": …, "to": …, "count": 900, "averageValue": 69.8}, …]}`

Example 3 — the whole cluster


Input	`curl 'localhost:9080/tsd/categories'` · `curl 'localhost:9080/tsd/health'`
Execution	Any node answers from replicated memory
Output	Categories busiest first; per node QPS, error rate, cache, replication lag, JVM

Categories	Zoom	Light theme

7. Configuration

Variable	Default	Description
`VORTEX_RETENTION`	`24h`	How long data is kept
`VORTEX_SPAN_MINUTES`	`1`	Bucket width; must divide 60
`VORTEX_TIME_ZONE`	`UTC`	Zone when a request has no `z`
`VORTEX_CACHE_MAX_KEYS`	`10000`	Keys per node in memory
`VORTEX_REDIS_HOST`	blank	Overflow Redis; blank deletes keys beyond the limit
`VORTEX_SNAPSHOT_INTERVAL`	`5m`	Snapshot interval
`VORTEX_GATEWAY_PORT`	`9080`	Host port

All settings: docs/configuration.md.

8. Performance & Comparison

Environment: Docker Desktop, 4 CPUs / 8 GB, 3 nodes, k6 through the gateway.

Scenario	Result
826,781 writes at rising rates	0 failures, identical totals on all nodes
Sustained	~1,000 samples/s
Peak	1,400–1,500 samples/s, leader ~2.3 CPUs, heap ≤ 310 MB
`kill -9` leader at 600/s	new leader in 4.7 s, 0.28% of requests failed
Graceful leader stop	immediate hand-over, 0 failures

	Vortex	Prometheus	InfluxDB	TimescaleDB
Writes	HTTP push	pull	HTTP push	SQL
Queries	HTTP params	PromQL	InfluxQL / SQL	SQL
Storage	memory + snapshots	disk	disk	PostgreSQL
HA	built in	extra setup	by edition	PG tooling
Best for	live metrics, days	monitoring	time series	analytics

9. Limitations & Trade-offs

Limit	Consequence
One leader applies every write	~1,000–1,500 samples/s; does not grow with nodes
Data in memory	Every node killed at once → back to the last snapshot (≤ 5 min)
Async replication	Writes in flight when the leader crashes fail; some may still be stored
Redis overflow	Reads of spilled keys cost a network round trip
No query language, alerting, auth	Keep it on a private network

Not for: ledgers that must never lose a record · months or years of retention · tens of thousands of writes per second.

10. Summary

One command — docker compose up -d, images for amd64 and arm64.
Just HTTP — push with t/c/d/v, query with range/step/window/z.
Exact — leader-serialised writes, no drift, no locks.
Local reads — every node a full replica.
Windows for free — tumbling and sliding, folded from minute buckets.
Self-healing — new leader in ~5 s; graceful stops unnoticed.
Durable enough — snapshots at stop and every 5 minutes.
Bounded — key limit per node, optional Redis overflow.
Observable — /tsd/health and a web console out of the box.
A clear niche — live metrics with days of retention.

GitHub: https://github.com/paganini2008/vortex · Website: https://paganini2008.github.io/vortex/

Openspreader: The missing piece of Java concurrency. `java.util.concurrent`, scoped to the cluster.

Fred Feng — Thu, 01 Oct 2026 04:19:12 +0000

Thirteen coordination primitives as one Spring Boot starter. The APIs you already know,
spanning every instance of your application instead of one JVM. Zero lines of configuration to
start.

// A lock across every replica of this application
ProcessingMutex mutex = syncs.applicationMutex("daily-report");
if (mutex.tryAcquire()) {
    try { generateReport(); } finally { mutex.release(); }
}

// A scheduled task that runs on one instance per round. One annotation.
@Scheduled(cron = "0 0 2 * * *")
@MultiProcessingScheduled
public void syncOrdersDaily() { ... }

What Problem Does It Solve?

synchronized only covers one JVM. Semaphore only limits one process. @Scheduled on three
replicas runs three times. The usual fix is Redis for a distributed lock, ZooKeeper for
coordination, and hand-written leader election for the scheduler: three pieces of
infrastructure, so that three replicas can agree on who does what.

openspreader does it with one dependency. Locks acquire in about a millisecond, cache reads are
served from local memory at over four million per second and never touch the network, and a
task dispatched to a peer runs there without your code knowing it moved.

Quick Start

<dependency>
    <groupId>com.chaconne-ai</groupId>
    <artifactId>openspreader</artifactId>
    <version>1.0.0-SNAPSHOT</version>
</dependency>

<repositories>
    <repository>
        <id>central-snapshots</id>
        <url>https://central.sonatype.com/repository/maven-snapshots/</url>
        <snapshots><enabled>true</enabled></snapshots>
    </repository>
</repositories>

Not a line of configuration is required. Discovery defaults to 127.0.0.1, so two instances
on one machine with different server.port values form a cluster by themselves. Local
debugging works out of the box.

One line has to change for production:

spring.spreader.ip-addresses=192.168.0.111,192.168.0.63,192.168.0.77
spring.spreader.name=order-cluster
spring.spreader.multiprocessing.mutex.enabled=true

With three instances started:

Node started: cluster=order-cluster, name=order-service, clusterPort=22000
Mutex service started

[instance 1] acquired nightly-report, generating
[instance 2] tryAcquire returned false, skipping this round
[instance 3] tryAcquire returned false, skipping this round

Every component is off by default, so adding the starter to a running service changes
nothing until you ask for one.

Requirements


Java	17 or later
Spring Boot	4.1, built and tested against it
Optional	Micrometer for Prometheus, Kryo for faster serialisation, `spring-data-redis` for cache overflow, Netty/MINA/Grizzly for an alternative transport
Ports	one cluster port (22000 by default, identical on every node) plus one work port per node

How It Works

Everything rests on spreader: the cluster runs inside the same JVM as
your beans, so there is no broker and no registry.

  ┌─────────────────── your application ───────────────────┐
  │                                                        │
  │   @Scheduled   mutex.tryAcquire()   cache.get(k)       │
  │        │              │                  │             │
  │   ┌────┴──────────────┴──────────────────┴─────────┐   │
  │   │            openspreader components             │   │
  │   │  lock  semaphore  latch  cache  pool  rpc  dag │   │
  │   └────────────────────┬───────────────────────────┘   │
  │                        │                               │
  │   ┌────────────────────┴───────────────────────────┐   │
  │   │      spreader: gossip, members, leader         │   │
  │   └────────────────────┬───────────────────────────┘   │
  └────────────────────────┼───────────────────────────────┘
                           │ TCP/UDP
              ┌────────────┼────────────┐
         other replica           other replica

Writes go through the leader, reads stay local. The registers for locks, permits, latches
and barriers live in the leader's memory; the cache replicates operations rather than data.

cache.incr("views", 1) on a follower
    │
    ├─ forwarded to the leader
    │
    ├─ leader applies it, assigns version 1007
    │
    └─ broadcasts {v1007, INCR, "views", 1} to every replica
                      │
         each replica applies it to its OWN copy

That is why setbit on a 100MB bitmap ships one datagram rather than the bitmap, and why a
cluster-wide Bloom filter is practical at all.

Code Examples

Five representative components. Every component has a runnable example under
com.chaconneai.openspreader.example.

Lock

Input

ProcessingMutex mutex = syncs.applicationMutex("bulk-export");

Execution: three ways to take it, matching three attitudes.

mutex.tryAcquire();                    // skip if taken, do not wait
mutex.acquire(3, TimeUnit.SECONDS);    // wait three seconds, then skip
mutex.acquire();                       // wait indefinitely

Output

mutex.isHeld();              // is anyone holding it
mutex.currentOccupied();     // WHO holds it, which answers "why did my task not run"
mutex.release(5_000);        // release, but keep others out for five seconds

That cooldown solves a real problem: a task finishes, another replica immediately takes the
lock and runs it again. A cooldown window removes the duplicate.

Semaphore

Input: protect a third party API at five concurrent calls.

ProcessingSemaphore sem = syncs.applicationSemaphore("third-party-api", 5);

Execution

if (sem.tryAcquire(3, TimeUnit.SECONDS)) {
    try { callThirdParty(); } finally { sem.release(); }
}

Output: the five permits are shared cluster-wide, whatever the replica count. That is
exactly what a single process Semaphore cannot do: three replicas each holding a local
semaphore of 5 gives you an actual concurrency of 15.

Cache

Input: 50 commands over four data structures, plus bitmaps and Bloom filters.

cache.set("k", bytes, 10, TimeUnit.MINUTES);
cache.hset("user:1", "email", bytes);
cache.zadd("leaderboard", member, 99.5);
cache.setbit("seen", offset, true);
cache.zrangeByScore("series", from, to, 0, 500);     // a page, not a million members
cache.zremrangeByScore("series", 0, cutoff);         // a retention policy in one operation

Execution: reads come from local memory; writes are forwarded to the leader and broadcast
back incrementally.

Output

reads     over 4,000,000 per second, never touching the network
writes    roughly 2,000 per second (TCP), 8,300 (UDP)

So it suits shared state that is read far more than written: configuration, allow-lists,
rate-limit counters, sessions. A write becomes visible elsewhere after milliseconds. It is
eventually consistent, not a database.

When memory is not enough: register a CacheStore bean and eviction becomes a move
rather than a loss.

spring.spreader.multiprocessing.cache.external.enabled=true


A key lives in exactly one half	Memory, or outside. Reads ask memory first; writes go where the key already is
Eviction moves instead of deleting	Same sampling, same policy, but the key is written across before the memory copy goes
Only the leader writes outside	A follower replaying the replication stream touches memory only
Bitmaps stay in memory	A Bloom filter lookup does seven bit tests, which outside would be seven round trips
Nothing is enabled by default	Without the bean, `store` is the local store: same object, no wrapper, no extra call

MapReduce

Input: implement a job, register it as a bean.

@Component("wordCount")
public class WordCountJob implements MapReduceJob<String, String, Integer, Integer> {

    public List<String> split(String text, int suggestedShards) {
        return splitByLines(text, suggestedShards);
    }

    public void map(String shard, Emitter<String, Integer> emitter) {
        for (String word : shard.split("\\W+")) {
            if (!word.isBlank()) {
                emitter.emit(word.toLowerCase(), 1);
            }
        }
    }

    public Integer reduce(String word, List<Integer> counts) {
        return counts.stream().mapToInt(Integer::intValue).sum();
    }
}

Execution: where each phase runs.

split     the submitter only, once
map       every node, the submitter included, once per shard
reduce    wherever the key's hash sends it, once per distinct key

Output

MapReduceResult<String, Integer> result = mapReduce
        .<String, String, Integer, Integer>submit("wordCount", input)
        .get(5, TimeUnit.MINUTES);

Combining the partial results is not yours to write: one key is reduced on exactly one node.

Every node needs this bean, at the same version. Half the nodes on a new map and half
on the old one produce a wrong answer rather than an error. Do not submit jobs during a
rolling deployment.

RPC

Input: enable the scan, declare an interface pointing at another application's name.

@SpringBootApplication
@EnableRpcClients(basePackages = "com.example.client")
public class Application { }

@RpcClient(serviceId = "inventory-service",
           timeout = 3000,
           maxConcurrent = 50,                       // in-flight ceiling, fail fast when full
           fallback = InventoryFallback.class)
public interface InventoryApi {

    int stockOf(String sku);
}

Execution: inject and call it like any bean. The server side needs a bean with a matching
method carrying @MultiProcessingCall.

int stock = inventory.stockOf("widget-1");

Output: routing is by application name rather than URL, and the target is chosen afresh on
every call, so scaling on the other side takes effect immediately.

Setting	Why it matters
`maxConcurrent`	Without it, requests pile up locally when the downstream slows, and what falls over is your own process
`maxRetries`	A failure is retried on a different instance each time, so the called method should be idempotent. Retrying after a timeout is ambiguous: the request may have completed with only the reply lost

Configuration

Property	Default	Description
`spring.spreader.name`	`default`	Cluster name, the only means of isolation between dev, staging and prod
`spring.spreader.ip-addresses`	`127.0.0.1`	Where to look for peers
`spring.spreader.advertise-host`	auto	Set this in containers
`...cache.max-keys`	`1000000`	Per process, not per cluster: every node holds a full replica
`...cache.eviction-policy`	`LRU`	Samples 5 keys and evicts the least recently used, as Redis does. Exact LRU would need a global lock on every read
`...cache.external.enabled`	`false`	Build the Redis-backed overflow store
`...mutex.request-timeout-ms`	`3000`	Shorter than the cache's: a lock request not answered quickly is better retried
`...mutex.lease-ms`	`15000`	Renewed while held, so a task may run as long as it likes. Only a crashed holder loses it

Performance

Conditions: single machine over loopback, 3-node cluster in one JVM, 4-core container, JDK
serialization. Absolute values do not transfer; the ratios do.

Reads never leave the process

Operation	QPS
`exists`	6,562,196
`size`	4,566,915
`stats` (aggregate, four numbers)	4,130,498
`hget` (single field)	2,104,340
`getbit`	1,548,987
`zscore`	1,210,631
`lrange` (50 elements)	345,364
`zrange` (50 members)	287,519
`hgetAll` (all fields)	68,489

hget against hgetAll is 30x. Calling hgetAll in a loop is the easiest accidental
mistake in this API, and it costs more than the read/write gap does.

Writes go through the leader

Operation	QPS
`max` (aggregate)	7,314
`incr`	6,524
`min` / `sum` (aggregate)	6,356 to 6,539
`zadd`	5,504
`rpush`	4,500
`hset`	4,080
`setbit` (fixed capacity)	3,419
`setbit` (growing)	1,991

Aggregates are the fastest writes. max/min/sum keep four numbers, so there is no
allocation, no resizing and no hash lookup, and the value travels in the arg field rather
than as a payload. That is what makes them suitable for downsampling a time series.

The ceiling, and where it comes from

Writes are serialised globally: every write goes through the leader and takes one state lock,
because a monotonic version number is what lets all replicas replay in the same order. The
lock's granularity is the version number's granularity.


Read scaling	Linear with node count. Reads are local
Write scaling	Flat. Adding nodes does not raise write throughput; it raises the leader's broadcast fan-out
Practical write ceiling	~2,000/s on TCP, ~8,300/s on UDP

So the question to ask is how far your write rate is from those numbers. Far means no problem.
Close means that data may not belong here.

Limitations & Trade-offs

Every coordination capability rests on one thing: whoever holds the cluster port is the
leader, and the write path for locks, permits, latches, barriers and the cache is all there.

Premise	Consequence
Leader uniqueness rests on timing, not consensus	Under partition each side may have a leader, and two parties hold the same lock. Fine for work where a repeat is wasteful: scheduled tasks, cache warming, batch de-duplication. Not for a transfer or a debit, which belongs in a database transaction or behind an idempotence key
A change of leader leaves a gap	Takeover measured at 3.3 to 4.4 seconds, during which lock acquisitions and cache writes retry. Leave business timeouts headroom, or you will see "it failed once and was fine a few seconds later"

Limit	Detail
The cache is a cache	Every node holds a full replica in heap. Not a database, not durable
Eviction is local	Nodes can disagree about which keys are resident
Everything is in-process	No persistence by default: a full cluster restart starts from empty
Resume is at-least-once	A DAG node that ran and reported back, whose record was not yet written when the coordinator died, is a step that happened and is not in the record
Cross-machine performance is unmeasured	Every number above is one JVM over loopback

Summary

Thirteen coordination primitives, one starter, zero configuration to start
The APIs you already know, scoped to every instance of the application rather than one JVM
Reads never leave the process: over four million per second, because every node holds a full cache replica
Writes have a clear ceiling: ~2,000/s on TCP, from a monotonic version number that must be assigned serially
Aggregates are the fastest writes (7,314/s), four numbers in constant memory, suited to downsampling
hget is 30x faster than hgetAll, a trap easier to fall into than the read/write gap
Every component is off by default, so adding the starter changes no behaviour
Memory can overflow to an external store, turning eviction from a loss into a move. Off by default
Two premises to accept: leader uniqueness rests on timing not consensus, and a change of leader leaves a gap
The cluster layer underneath is spreader, where the same post builds a replicated H2 on it

Spreader: Your JVM processes are the cluster. Nothing else to install.

Fred Feng — Thu, 01 Oct 2026 04:17:21 +0000

Turn plain JVM instances into a cluster that elects a leader, tracks every member and passes
messages between them. One dependency, one line of configuration, and no ZooKeeper, etcd,
Consul or registry anywhere in the picture. The processes are the cluster.

GossipCluster cluster = GossipCluster.create(
        GossipConfig.builder().ipAddresses("10.0.0.1", "10.0.0.2").build());
cluster.start();

if (cluster.isLeader()) {
    runTheNightlyJob();          // exactly one instance does this
}

What Problem Does It Solve?

Three machines running the same application, and you need them to agree on which one runs the
nightly job, to notice within seconds when one dies, and to send each other a message. The
standard answer is a new piece of infrastructure: something to deploy, monitor, upgrade and get
paged about at 3am, for a problem that is not your product.

spreader keeps it inside the processes. Every node runs the same code and holds the same
complete member list, with no special node and no role to assign. Measured: 18,464 cross-node
round trips per second, and a dead leader replaced in 3 to 4 seconds.

Quick Start

<dependency>
    <groupId>com.chaconne-ai</groupId>
    <artifactId>spreader</artifactId>
    <version>1.0.0-SNAPSHOT</version>
</dependency>

Snapshots need the repository declared too:

<repositories>
    <repository>
        <id>central-snapshots</id>
        <url>https://central.sonatype.com/repository/maven-snapshots/</url>
        <snapshots><enabled>true</enabled></snapshots>
    </repository>
</repositories>

Open two or three terminals and run one in each:

java -cp spreader-1.0.0-SNAPSHOT.jar \
     com.chaconneai.spreader.example.BestPractice \
     --cluster=demo --port=22000 --ipAddresses=127.0.0.1 --portRetry=10

What you see:

Node started: cluster=demo, id=a1b2c3d4, clusterPort=22000, transport=TCP/NioTcpTransport
joined: the only node so far, so this one leads

===== cluster [demo] =====
this node: demo-app@127.0.0.1:30001  [leader]
leader:    demo-app@127.0.0.1:30001
members:   1

>>> [joined] demo-app@127.0.0.1:30002, members now 2
>>> [joined] demo-app@127.0.0.1:30003, members now 3

Kill the leader, and within seconds:

>>> [leader changed] 127.0.0.1:30001 -> 127.0.0.1:30002  (this node is now the leader)

Requirements


Java	17 or later
Optional	Netty, MINA or Grizzly, to replace the built-in NIO transport
Ports	one cluster port (22000 by default, identical on every node) plus one work port per node, chosen automatically

How It Works

Two mechanisms, both deliberately boring.

Membership spreads by SWIM gossip. Every second each node picks a few peers at random and
exchanges member lists directly, never through a coordinator.

    Node A ◄──────── gossip ────────► Node B
       ▲  holds :22000                   ▲
       │  = is the leader                │
       └──────────── gossip ─────────────┘
                      │
                   Node C

Leadership is pre-emptive. The cluster has one fixed port and whichever node holds it
leads. A new node scans the configured addresses, knocks on that port, and either finds a
leader or becomes one.

leader dies
    │
    ├─ failure detection notices (probe, indirect probe, suspect timeout)
    │
    ├─ every eligible node computes its rank in the takeover order
    │
    ├─ rank 0 waits 0ms, rank 1 waits takeoverDelayMs, rank 2 waits 2x ...
    │
    └─ first to bind :22000 is the new leader; the rest see it and stand down

Pre-emptive and consensus-based election are two routes, not two grades

Where a leader's legitimacy comes from is the root of every other difference. Pre-emptive: from
holding a physically exclusive resource. Consensus-based: from a majority's authorisation,
since two majorities must intersect and a node in that intersection will not vote twice in one
term.

	Pre-emptive (this library)	Consensus-based (Raft)
How a leader arises	whoever takes the port	whoever a majority votes for
Must the leader be up to date?	No. A node that just restarted can lead	Yes. A stale log cannot win a vote
Terms	none, so no built-in fencing	monotonic; an older term is refused
One node	works	works
Two nodes, one dies	the survivor takes over	stalls, no majority
Membership changes	soft state, gossiped, free	configuration state, changed through the log
Disk	none	fsync before voting
Under partition	both sides can lead and accept writes	the minority elects nobody and refuses writes
Takeover time	3 to 4 seconds, dominated by failure detection	typically a few hundred milliseconds

Two consequences shape how to use it:

A pre-emptive leader carries no data authority. It is not required to be up to date, so any state kept on it has to be rebuildable.
With no terms there is no built-in fencing. A leader already replaced can still write to an outside resource. Where that matters, have the leader issue a monotonic token, carry it into whatever you write, and have that resource refuse older ones.

The dividing line is the nature of the work, not its importance. Coordination where a
repeat is merely wasteful fits this route. Work where a repeat causes real harm needs
consensus-based election.

Code Examples

Example 1: members and the leader

Input

List<Node> all = cluster.members();                        // the whole cluster
List<Node> peers = cluster.membersOf("order-service");     // just my own application
Node leader = cluster.leader();                            // can be null

Execution: reach for membersOf more often than members. "The other replicas of my own
application" is usually the question actually being asked, especially once a cluster runs more
than one application.

Output: each Node carries what you need.

node.address();     // an address that can be dialled
node.name();        // application name
node.state();       // ALIVE / SUSPECT / LEFT
node.metadata();    // your own business metadata, gossiped cluster-wide

metadata is an extension channel: to let everyone know this node's version, zone or canary
flag, put it in the configuration with .metadata("zone", "az-1") and every node reads it off
the member list.

Example 2: a real trap

Input: three replicas, and a scheduled task that should run once.

// Wrong
if (cluster.isLeader()) {
    syncOrdersDaily();
}

Execution: the leader is a cluster-wide notion that does not distinguish applications.
If your cluster also runs an api-facade and the cluster port happens to be held by one of its
instances...

Output: every replica of order-service sees isLeader() == false and the task never runs
at all.

The right tool is openspreader's @MultiProcessingScheduled, which picks one executor per
application. Or elect within your own application against membersOf(self().name()).

Example 3: a replicated H2

Input: any node writes, and callers do not need to know whether they are the leader.

store.write("INSERT INTO product (name, price) VALUES ('widget-1', 10.99)");

Execution: the write takes one path only.

write on a follower          write on the leader
       │                            │
 sendToLeader(sql)                  │
       │                            │
       └────► the leader applies it locally
                      │
             multicast to every replica
                      │
             each applies it to its own H2

public void write(String sql) throws SQLException {
    byte[] message = encode(cluster.isLeader() ? REPLICATE : WRITE_REQUEST, sql);
    if (cluster.isLeader()) {
        // Apply first, then tell the others. The other order would have a replica
        // holding a row the leader itself does not
        apply(sql);
        cluster.multicastOn(CHANNEL, cluster.self().name(), message, false);
    } else if (!cluster.sendToLeaderOn(CHANNEL, message)) {
        // No leader at this instant. Surfacing it beats silently dropping the write
        throw new SQLException("No leader to take the write; retry in a moment");
    }
}

Output: every node reads its own copy, and no read leaves the process.

java -cp spreader.jar:h2.jar \
     com.chaconneai.spreader.example.ReplicatedH2Example --port=22000

[127.0.0.1:30001 leader] rows=8 applied=8 members=3
[127.0.0.1:30002]        rows=8 applied=8 members=3
[127.0.0.1:30003]        rows=8 applied=8 members=3

One step is easy to miss: a node joining an established cluster starts empty. Replicating
only the increments leaves it permanently short of history, answering reads with nothing, which
is worse than answering slowly. So ask for a full copy on join:

@Override
public void onClusterJoined(Node self, boolean alone) {
    if (!alone) {
        cluster.sendToLeaderOn(CHANNEL, encode(SNAPSHOT_REQUEST, ""));
    }
}

H2 has a convenient statement for this: SCRIPT dumps the whole database as SQL, one statement
a row. The leader dumps it and sends it over, the new node runs DROP ALL OBJECTS and replays
it, and it is caught up.

The complete code is in com.chaconneai.spreader.example.ReplicatedH2Example, about 350 lines.
It needs no H2 at compile time, using only java.sql from the JDK, so the spreader jar
does not carry it.

Configuration

Defaults are tuned for a small cluster on a reliable LAN. One line genuinely has to change.

Property	Default	Description
`ip-addresses`	`127.0.0.1`	Where to look for peers. A few seeds are enough; a new node learns the full member list from any one of them
`cluster-name`	`default`	The only means of isolation. Nodes with different names ignore each other even when the network can reach them
`node-name`	`default`	The application name. Instances sharing one are replicas, and the layers above use it to tell who belongs with whom
`cluster-port`	`22000`	The port that decides leadership. Identical on every node
`advertise-host`	auto-detected	Set this in containers. Auto-detection may pick an address no peer can reach
`leader-eligible`	`true`	`false` makes this application join and work but never contend for leadership
`transport-type`	`TCP`	TCP unless you have measured otherwise
`payload-ack`	`true`	Wait for an ACK and resend. Off is roughly 3x the throughput, since the ACK round trip dominates small-message cost
`probe-timeout-ms`	`800`	Raise this first on a congested network. Too low and healthy nodes get suspected during a GC pause

Performance

Numbers expose a property of the design. Absolute values will not transfer to your
hardware: every cross-node call here costs a loopback hop instead of a real network one. The
ratios will.

Conditions: single machine over loopback, 8 threads x 2000 synchronous round trips, ~300
byte payloads, JDK serialization, 4-core container.

Transport x protocol: the pairing is the unit of choice

Combination	QPS	avg	P50	P99	max
NIO / TCP	18,464	0.426 ms	0.367 ms	1.69 ms	5.4 ms
NIO / UDP	13,023	0.606 ms	0.423 ms	6.46 ms	15.0 ms
MINA / TCP	12,249	0.640 ms	0.438 ms	5.18 ms	15.4 ms
NETTY / UDP	10,682	0.746 ms	0.486 ms	7.19 ms	28.9 ms
MINA / UDP	10,494	0.751 ms	0.464 ms	7.24 ms	21.8 ms
GRIZZLY / TCP	9,171	0.864 ms	0.580 ms	7.68 ms	21.3 ms
GRIZZLY / UDP	8,751	0.910 ms	0.644 ms	6.96 ms	20.2 ms
NETTY / TCP	7,081	1.115 ms	0.495 ms	14.46 ms	56.8 ms

Pick the cell, not the framework. Netty is second-best on UDP and last on TCP, with a P99
twice anyone else's. MINA is the opposite. "Which transport is fastest" has no answer.

Treat anything under ~20% as noise: an earlier run on a 12-core host put Netty/UDP on top, and
NIO/TCP measured 16,351 and 14,637 on two consecutive runs.

The same four transports, through the components

Operation (TCP)	NIO	NETTY	MINA	GRIZZLY
lock acquire + release	875	686	616	648
cache write (`incr`)	2,110	1,733	1,440	1,748
task dispatch to a peer	11,581	10,028	11,473	9,148

Operation (UDP)	NIO	NETTY	MINA	GRIZZLY
lock acquire + release	2,878	2,162	3,009	720
cache write (`incr`)	8,326	8,237	2,645	1,618
task dispatch to a peer	5,809	10,787	10,407	8,856

Two findings:

Coordination runs 3 to 4 times faster over UDP (locks 875 to 2,878, cache writes 2,110 to 8,326). These are short one-way messages plus a reply, and TCP's acknowledgement and congestion control are pure overhead on that shape. A sustained request/response stream is the opposite, and TCP wins there.
Grizzly collapses on UDP: 720 against NIO's 2,878 is four-fold, far outside the noise band, and it appears on every coordination row while leaving task dispatch alone.

Reads are absent from both tables on purpose: they come from local memory and never touch the
transport, so comparing them would only mislead.

Dispatch threads are not a throughput dial

Mode	msg/s	per message
4 producers, 4 dispatch threads	5,334,329	187 ns
serial dispatch (`payload-dispatch-threads=1`)	5,100,959	196 ns
concurrent dispatch (`payload-dispatch-threads=4`)	3,453,596	290 ns

Rows two and three share a single producer, and the serial dispatcher wins by almost 50%.
Coordination between dispatch threads costs more than the parallelism returns. Raise this
setting when several threads publish concurrently, leave it at 1 otherwise.

Limitations & Trade-offs

Limit	Detail
Election is pre-emptive, not consensus-based	Under partition two sides can each elect a leader. It heals when the partition does, and every occurrence is counted. Work where a repeat causes real harm does not belong on this route
Membership is eventually consistent	Right after a change, different nodes briefly hold different views
UDP messages can be lost	`payload-ack` covers it with resend and dedup, at a throughput cost
Cross-machine performance is unmeasured	Every number above is one JVM over loopback

Where it does not fit: thousands of nodes, clusters spanning datacentres, or a leader that must
stay correct under partition. The first two make gossip convergence unpredictable; the third
calls for Raft.

Summary

One jar, one line of configuration. No registry, no coordinator. The processes are the cluster
Fully decentralized: every node runs the same code and holds the same complete member list
Election needs no consensus round: whoever holds the cluster port leads, and the OS already guarantees one port, one process
Pre-emptive and consensus-based are two routes, not two grades. The line falls on the nature of the work, not its importance
18,464 cross-node round trips per second measured, and a dead leader replaced in 3 to 4 seconds
Pick the cell, not the framework: Netty is last on TCP and second on UDP. There is no "fastest transport"
Coordination runs 3 to 4x faster over UDP, while a sustained request/response stream favours TCP
The two failures that log nothing each have a counter: dropped messages and a second leader
Limits are stated, not hidden: partition dual leaders, eventual consistency, UDP loss, cross-machine unmeasured
For ready-made locks, caches and scheduled-task exclusion, see openspreader

Cronflower: Turn a Spring Boot app into a distributed cron cluster

Fred Feng — Tue, 29 Sep 2026 11:32:26 +0000

cronflower is an open-source, distributed cron scheduler for the JVM with a web console. It forms
its own cluster and needs no external database, broker, or coordinator. This post is the usage tour of
its distributed task scheduling (its DAG side has its own post).

What problem does it solve?

@Scheduled runs in one JVM, so the moment you scale to two instances the job fires twice. It has no
retry, no timeout, no history, and no view of what ran; if the box reboots at 02:00 the nightly rollup
just quietly doesn't happen.

The usual fix is to bolt on Quartz, a database, a lock table, and a dashboard you wrote yourself.
cronflower is that whole stack behind two dependencies: you annotate a method, run the scheduler
alongside it, and get a clustered, persistent, retrying scheduler with a console.

Quick start

git clone https://github.com/paganini2008/cronflower
cd cronflower/deploy
./run-local.sh -e 1          # scheduler + console + 1 executor  (embedded H2)

Open http://localhost:7200, sign in admin / admin123, and the example tasks are already running.
Scale into a real cluster with ./run-local.sh -n 3 -e 2.

Requirements

Need	Version / note
JDK	17+ (builds via the bundled Maven Wrapper)
Node	20+ (builds the console)
Docker	optional (container path only)
Database	optional — none → embedded H2; MySQL / PostgreSQL for a shared store

How it works

Two kinds of process: a scheduler owns the schedule and durable state; an executor is your app,
which declares tasks and runs the code on callback. Run several schedulers and they elect a leader over
gossip; state lives in the store, so a restart or failover loses nothing.

flowchart LR
  App["your app = executor<br/>@Task beans"] -->|register on boot| L
  subgraph Cluster["scheduler cluster"]
    L["leader"] <-->|gossip| F1["follower"]
    F1 <-->|gossip| F2["follower"]
  end
  Cluster --> Store[("store<br/>H2 · MySQL · PostgreSQL")]
  L -->|"dispatch when due (HTTP callback)"| App

The leader loads only the tasks due in the next few minutes into a timing wheel and claims the rest
from the store as they come due, so a cluster with hundreds of thousands of tasks still starts
instantly.

Code examples

A task is a `@Task` bean method

Input — annotate a method on an executor (0 args, or a single String parameter):

@Component
public class DemoTasks {

    @Task(cron = "*/5 * * * * ?", group = "showcase", name = "sayHello", initialParameter = "world")
    public String sayHello(String who) { return "hello, " + who; }

    // Fixed interval, no cron. Or iso = "PT1H30M" for an ISO-8601 duration.
    @Task(interval = 10, intervalUnit = TimeUnit.SECONDS, group = "showcase", name = "every10s")
    public void every10s() { /* ... */ }

    // YCRON: noon on the 200th day of the year — no classic cron can say this.
    @Task(cron = "0 0 12 ? ? 200", parser = "ycron", group = "showcase", name = "dayOfYear200")
    public void dayOfYear200() { /* ... */ }
}

Execution — discovered on boot, registered with the scheduler, which owns the schedule from then on.

Output — each task shows up in the console with its schedule, run count, and next fire time.

Reliability is configured, not coded

Input — add attributes; the scheduler enforces them, so every executor behaves the same:

@Task(cron = "*/20 * * * * ?", group = "reliability", name = "flaky",
      maxRetryCount = 2, retryInterval = 1000,   // retry with back-off
      timeout = 2000,                            // fail a run that overruns 2s
      misfirePolicy = "SKIP",                    // FIRE_ONCE_NOW (default) / FIRE_ALL / SKIP
      repeatCount = 10)                          // finish after 10 fires
public String flaky(String parameter) { /* fails twice, then succeeds */ }

Output — in the execution history a single fire is three rows (attempt #0 fail, #1 fail, #2
success), each tagged with the scheduler that dispatched it and the executor that ran it:

Build a schedule in code, or skip the executor

// Fluent, self-validating schedule instead of a hand-typed string:
@Bean CronExpressionBuilder mondayMornings() {
    return () -> new CronBuilder().everyWeek().Mon().at(9, 0).toString();
}
@Task(builder = "mondayMornings", description = "weekly report")
public void weeklyReport() { /* ... */ }

For a job that is just "call this URL on a schedule", create an HTTP-API task from the console's
New task form or the REST API — the scheduler makes the call itself, no executor involved, and
operators add or edit tasks with no redeploy.

The @Task attributes, for reference:

Attribute	What it does
`cron` / `parser`	a cron schedule, or YCRON with `parser = "ycron"`
`interval` + `intervalUnit` / `iso`	a fixed interval, or an ISO-8601 duration
`builder`	a `CronExpressionBuilder` bean (wins over `cron`; the only way to set a deadline)
`initialParameter`	a constant, or a SpEL template evaluated on each fire
`maxRetryCount` / `retryInterval`	retry a failed run, with back-off
`timeout`	fail a run that overruns, in milliseconds
`misfirePolicy`	`FIRE_ONCE_NOW` (default) / `FIRE_ALL` / `SKIP`
`repeatCount`	total fires before the task finishes

Configuration

Property	Default	Description
`cronsmith.client.server-urls`	—	On the executor: scheduler seed URL(s)
`cronsmith.server.scheduler.zone`	`UTC`	Fire-time zone — must match cluster-wide
`cronsmith.server.scheduler.window-minutes`	`5`	Windowed-loading horizon
`cronsmith.server.scheduler.sharding`	`true`	Group sharding over a shared store (else leader-only)
`cronsmith.server.dispatch.routing`	`ROUND_ROBIN`	…`WEIGHTED` / `CONSISTENT_HASH` / `RANDOM` / `FIRST` / `LAST`
`spring.datasource.url`	H2 file	Point at MySQL/PostgreSQL for a shared, sharding-capable store

Limitations & trade-offs

It is a platform to run (its own gossip cluster + store), not a tiny in-process library. For one JVM with a couple of fixed jobs, plain @Scheduled is lighter.
Group sharding needs a shared database (MySQL/PostgreSQL); on node-local H2/SQLite it safely degrades to leader-only.
Fire-time zone must match across nodes; everything is computed and stored in UTC.
Bean tasks run on executors, so the scheduler reaches them over HTTP — they must be reachable from it.

Summary

A distributed, stateful cron scheduler for Spring Boot, two dependencies, no external infra.
Nodes gossip and elect a leader; a restart or failover loses nothing, and no cron fires twice.
@Task gives you cron / YCRON / interval / ISO-8601, plus retry, timeout, misfire, repeat — declarative.
HTTP-API tasks need no executor; operators add or edit tasks with no redeploy.
Every run is recorded with result, timing, attempt, and the node that ran it.
Group sharding + weighted dispatch scale it horizontally; a timing wheel + windowed loading keep startup instant.
A console shows tasks, cluster, executors, health, and live settings from one endpoint.

Run it: cronflower on GitHub.

Cronflower: Run a DAG workflow across your cluster, instead of chaining cron jobs

Fred Feng — Tue, 29 Sep 2026 11:30:51 +0000

cronflower is an open-source, distributed scheduler for the JVM with a web console. cronflow is
its optional DAG add-on: declare the steps and how they depend on each other as a graph, and the same
cluster runs the graph node by node, with data flowing between nodes over typed channels. (Its
distributed @Task scheduling has its own post.)

What problem does it solve?

Teams end up chaining cron jobs: one at 02:00 charges the orders, another at 02:15 ships them, set
fifteen minutes later because that is usually long enough for the first to finish. There is no real
dependency, no shared data, and no way to tell afterwards whether step two ran because step one
succeeded or just because the clock moved.

That is the wall plain cron hits: it schedules single jobs, it cannot orchestrate a flow of them. A
DAG makes the dependency real, lets data flow between steps, and records every run.

Quick start

git clone https://github.com/paganini2008/cronflower
cd cronflower/deploy
./run-local.sh -e 1          # scheduler + console + 1 executor

Open http://localhost:7200 (admin / admin123) and the example DAGs are registered under DAG >
Workflows, ready to trigger.

Requirements

Need	Version / note
JDK	17+ (builds via the bundled Maven Wrapper)
Node	20+ (builds the console)
cronflow add-on	`cronflow-spring-boot-starter` on the scheduler, `cronflow-executor-spring-boot-starter` on the executor
Database	optional — none → embedded H2; MySQL / PostgreSQL for a shared store

How it works

A DAG is a Spring bean: @Dag names it, each @DagNode method is a step, and the to list is the
edges. A node returns a Map of named channel writes; downstream nodes read them from a
DagState. Each @Channel says how concurrent writes to it merge, via a reducer. The engine
drives the graph across the whole cluster, dispatching each node to a live executor.

flowchart LR
  input((input)) --> intake
  intake --> credit & income & collateral
  credit --> decide
  income --> decide
  collateral --> decide
  decide -->|"score >= 70"| APPROVED
  decide -->|else| REJECTED

Code examples

Declare a workflow — `@Dag`

Input — a bean with @Dag + @DagNode methods and typed channels:

@Dag(name = "scoring-flow", inputs = {"input"}, channels = {
        @Channel(name = "score",   reducer = ChannelReducer.SUM_INT),
        @Channel(name = "factors", reducer = ChannelReducer.JOIN_CSV)})
@Component
public class ScoringFlow {

    @DagNode(entry = true, to = {"credit", "income", "collateral"})     // fan out to 3 parallel scorers
    public Map<String, Object> intake(DagState state) {
        return Map.of("applicant", state.getString("input"));
    }

    @DagNode(to = {"decide"})
    public Map<String, Object> credit(DagState state) {
        return Map.of("score", 40, "factors", "credit");
    }
    // income(), collateral() … same shape, writing to the same channels

    @DagNode(trigger = "ALL")                                           // join: wait for all three
    public Map<String, Object> decide(DagState state) {
        long total = state.getLong("score");                           // reducer already summed them
        return Map.of("decision", total >= 70 ? "APPROVED" : "REJECTED");
    }
}

Execution — three scorers run in parallel, each writing score; the reducer merges the concurrent
writes (summed to 90) so decide reads one combined value. No shared mutable state, no ordering
assumptions.

Output — the console renders the graph; a run lights up node by node, each node showing which
executor ran it, so a wide fan-out really runs in parallel on different machines:

Edges beyond straight lines

@DagNode(when = @When(expr = "#risk > 80", to = "humanReview"), otherwise = {"fulfilment"})
public Map<String, Object> riskScore(DagState state) { /* writes risk */ }

@DagNode(subgraph = "fulfilment-flow", to = {"notify"})                // nest a whole other DAG
public Map<String, Object> fulfilment(DagState state) { /* ... */ }

@DagNode(shard = @Shard(input = "ids", output = "squares"), to = {"report"})
public Map<String, Object> square(DagState state) { /* runs once per id, at run time */ }

when + otherwise: route by a SpEL expression over the channels.
trigger: ALL waits for every upstream edge; ANY fires on the first arrival.
subgraph: a node is a whole other DAG, so you compose instead of copy.
@Shard: fan out dynamically, once per element of a list known only at run time.

Three ways to start it

By hand from the console's Trigger button, with an optional JSON seed for the input channels.
From a finished @Task in the same bean: its return value becomes the DAG input.
On a schedule directly: pick Trigger DAG in the Create Task form.

Configuration — channels & reducers

Each @Channel merges concurrent writes with a built-in reducer, or your own via customReducer:

Reducer	Merge
`SUM_INT` / `SUM_LONG`	add the numbers
`MAX` / `MIN`	keep the largest / smallest
`AND` / `OR`	boolean fold
`CONCAT_LIST` / `UNION_SET`	append / union collections
`CONCAT_STRING` / `JOIN_CSV`	join text
`MERGE_MAP`	merge maps
`LAST_WINS` / `FIRST_WINS` / `WRITE_ONCE`	pick one writer

The executor points at the scheduler with cronflow.client.server-urls; the server prefix is
cronflow.server.api-prefix (default /cronflow).

Limitations & trade-offs

The DAG engine rides on the cronflow add-on over the scheduler cluster; it is not a standalone workflow server. You run cronflower, then add @Dag beans.
Nodes are dispatched to executors over HTTP, so a node's bean must be reachable and idempotent enough to be retried.
Channel values cross the wire between nodes, so keep them serializable and reasonably small.
It orchestrates your steps; it is not a data-pipeline engine for streaming large datasets.

Summary

A DAG is a Spring bean: @Dag + @DagNode, edges via to, data via typed channels.
Reducers merge concurrent writes, so joins need no locks or ordering assumptions.
Branching (when), join modes (ALL/ANY), subgraphs, and dynamic fan-out (@Shard) are annotation attributes.
The engine drives the graph across the cluster, dispatching each node to a live executor.
Trigger a flow by hand, from a finished task, or on a schedule.
Every run is recorded node by node, showing what ran where and what it produced.

Run it: cronflower on GitHub.

Greenfinger 2.0: One url in, a searchable archive out

Fred Feng — Sun, 27 Sep 2026 10:49:39 +0000

Crawl the whole site. Keep every page, every picture, every version.
Search it by words, by meaning, or by describing a picture you remember.
Add a node and it goes faster. Kill one and it keeps going.
Nothing to install. Nothing to sign up for.

A distributed web crawler for the JVM. One pass writes plain files, a full text index and a vector
collection. Every node runs the same jar, there is no coordinator to deploy, and the first crawl
needs no database, no search server and no API key.

A rebuild in progress. v1 keeps answering searches while v2 is being written, and each node
reports what it pulled.

What problem does it solve

Most crawler code starts the same way. Someone needs the content of a site, writes a hundred lines
around an http client and an html parser, and it works. Then reality arrives. One advert link and
the crawler is downloading the rest of the web. The process is killed at page 40,000 and the queue
was in memory. Half the site turns out to be pdfs. Navigation and cookie banners get indexed
alongside the article, so every result looks the same.

And then storing it is a second project: files here, an index there, embeddings somewhere else, and
three pieces of glue that each fail differently. Greenfinger answers all of that in the product
rather than in your code, and the first crawl needs nothing running beside it.

Quick start

git clone https://github.com/paganini2008/greenfinger.git
cd greenfinger/backend && mvn clean install
cd ../deploy

./greenfinger-cli.sh --cluster=demo crawl --url=https://books.toscrape.com

Result:

Crawling 'books.toscrape.com' from https://books.toscrape.com

  pages kept    1,000      urls seen   28,411      1 in every 28
  images        3,204      indexed      1,000      elapsed  0h 4m 12s

Finished: reached maxFetchSize
Catalog 01a0c3d5-7d3d-7000-81f6-76057a403db8, version 0, now searchable.

Prefer the console:

./run-local.sh                # nodes plus the page on http://localhost:9700
GF_NODES=3 ./run-local.sh     # three nodes sharing one crawl
./run-docker.sh               # the same, in containers
./run-local.sh stop

Sign in with admin and the password in deploy/config/api/users.xml. Seven example catalogs ship
with a fresh install, so there is something to crawl before you have picked anything.

Requirements

	Version	Needed for
JDK	17 or later	Running anything
Maven	3.9 or later	Building from source
Node	20 or later	Building the web interface
Docker	any current	`run-docker.sh` only

Everything else is optional and opt in one variable at a time: PostgreSQL, MySQL, SQL Server,
Oracle or SQLite instead of the H2 file, Elasticsearch instead of the embedded Lucene index, Qdrant
or Weaviate instead of the embedded vector store, MinIO or any S3 compatible store instead of local
disk, and Ollama or OpenAI instead of the local ONNX models.

A browser is needed only for the playwright and selenium extractors. The default adaptive
falls back to HtmlUnit, which is pure Java and needs nothing installed.

How it works

One pass, three outputs

                            ┌──────────────┐
                            │  file        │  local disk, MinIO, any S3 store
   one crawl  ───────────►  ├──────────────┤
                            │  index       │  Lucene embedded, Elasticsearch
                            ├──────────────┤
                            │  vector      │  Lucene embedded, Qdrant, Weaviate, ES
                            └──────────────┘
                                   │
                    replay ────────┘   rebuild index or vectors from the files,
                                       without fetching the site again

The file layer is always on, because the database keeps metadata only and the other two rebuild
from what it wrote. Search never reads the database, and that single constraint is where the
rest of the design comes from.

What one page goes through

  url from the frontier
        │
        ├─ UrlPathAcceptor chain    domain, start url prefix, assets, robots.txt,
        │                           depth, path patterns. First refusal wins
        ├─ ExistingUrlPathFilter    seen before? RocksDB
        ├─ Extractor                plain http, browser only for unrendered shells
        ├─ ContentExtractor         the article, not the navigation
        │     └─ DocumentContentParser   when it is not html
        ├─ ContentDedupFilter       SHA-256 or SimHash
        ├─ OutputChannels           file, index, vector
        └─ new links ─────────────► back to the frontier, to whichever node owns them

Every drop is counted and named. That is why the monitor can say where 22,000 urls went rather than
only that 140 pages were kept.

How the cluster shares one crawl

   node A ── finds /a/b ──► node C owns it ──► fetches, finds /a/b/c ──► node A owns it ──► ...

   no central queue        no leader in the fetch path        join or leave mid crawl

A crawl is a recursive function, and the only thing distribution changes is that the recursive call
crosses a process. There is no join, because a parent page does not care what its children found.

Completion is decided by everyone. Every node checks the shared counters against
maxFetchSize and fetchDuration, and the first to notice writes the reason. A leader that dies
mid crawl cannot leave a crawl that never ends.

Code examples

Crawl, update, rebuild, replay

Input. A catalog id, and a verb.

./greenfinger-cli.sh --cluster=nightly crawl   --id=<id>            # from the start url
./greenfinger-cli.sh --cluster=nightly update  --id=<id>            # urls that appeared since
./greenfinger-cli.sh --cluster=nightly merge   --id=<id>            # and revisit what is held
./greenfinger-cli.sh --cluster=nightly rebuild --id=<id>            # new version, old one served
./greenfinger-cli.sh --cluster=nightly resume  --id=<id>            # continue after a pause
./greenfinger-cli.sh --cluster=nightly replay  --id=<id> --layers=index+vector

Output.

Verb	Version	Fetches	Writes
`crawl`	current	from the start url	everything it saves
`update`	current	only urls not seen before	new pages only
`merge`	current	new urls and pages already held	only the pages that changed
`rebuild`	a new one	the whole site again	the new version, old one keeps serving
`resume`	current	what is left on the frontier	as the interrupted run would have
`replay`	a named one	nothing	rebuilds an output from what is stored

replay is the one to remember. Your Elasticsearch was down for an hour, you changed the analyzer,
or you decided six months in that you want embeddings after all. None of those need the site to be
polite to you a second time.

Search three ways

Input. One box, three modes, from the console or the prompt.

greenfinger:> search --query="bread"
greenfinger:> search --query="what happens when a star runs out of fuel" --mode=meaning
greenfinger:> search --query="a bright spiral galaxy against black sky" --mode=pictures

Output. Words goes to the index, so this is exact terms with the matches highlighted.

Meaning goes to the text vectors. The top answer below is a supernova remnants page that never
contains the sentence that was typed.

Pictures goes to the image vectors, matched against the picture itself rather than the filename or
the alt text.

The same thing at the prompt:

╭────────┬──────────────────────────────────────────┬────────────────────────────────────────────────╮
│ Score  │ Title                                    │ Url                                            │
├────────┼──────────────────────────────────────────┼────────────────────────────────────────────────┤
│ 0.9356 │ APOD: 2008 June 4 - Chasing the ISS      │ https://apod.nasa.gov/apod/ap080604.html       │
│ 0.9317 │ APOD Index - Nebulae: Supernova Remnants │ https://apod.nasa.gov/apod/supernova_remnants… │
│ 0.9289 │ APOD Index - Stars: Binary Stars         │ https://apod.nasa.gov/apod/binary_stars.html   │
╰────────┴──────────────────────────────────────────┴────────────────────────────────────────────────╯

The embedding models run locally and need no account: multilingual-e5-small for text and
SigLIP 2 for images, both ONNX, both preloaded at startup on a background thread.

Add a pdf parser

Input. A site that links pdfs.

Code. Every default component is @ConditionalOnMissingBean, so publishing a bean is the whole
registration. There is no plugin registry and no ordering property.

@Bean
DocumentContentParser pdfParser() {
    return new DocumentContentParser() {
        public Set<String> fileTypes() {
            return Set.of("pdf");
        }

        public String extractText(byte[] content, String url, Charset encoding) throws Exception {
            return new Tika().parseToString(new ByteArrayInputStream(content));
        }
    };
}

Output. With GF_DOCUMENTS=true and GF_DOCUMENT_TYPES=pdf, pdfs linked from a crawled page
now contribute text to the index and the vectors. Collecting the links was always happening and
costs nothing. Fetching and reading them is what you just switched on.

The same shape works for every decision the crawler makes:

Interface	Decides	What ships
`Extractor`	How a page is fetched	restclient, htmlunit, playwright, selenium, adaptive
`UrlPathAcceptor`	Whether a link is followed	Domain, start url, assets, robots.txt, depth, patterns
`ContentDedupFilter`	Whether two urls are the same page	sha256, simhash
`ContentExtractor`	The text inside a page	Link density boilerplate removal
`DocumentContentParser`	Text out of a non html file	text, markdown, csv
`CompletionChecker`	When the crawl is over	Saved count, elapsed time
`OutputChannel`	Where results are written	file, index, vector
`BlobStore`	Pages and images as bytes	local, minio
`Searcher` / `VectorStore`	Full text, and meaning	lucene, elasticsearch, qdrant, weaviate
`EmbeddingClient`	What turns text into a vector	local ONNX, ollama, openai

Embed it in your own application

Input. A Spring Boot application of your own.

@EnableGreenfingerServer
@SpringBootApplication
public class MyApplication { }

Output. The whole REST api, the login and the page, inside your process. It is explicit rather
than auto configured, because sitting on a classpath is not a reason to open RocksDB and take a
crawl permit. To drive a crawl without the server, CrawlerLauncher.crawl(catalogId, onReady)
returns a CrawlerEngine.Result with the counters and the reason it ended.

Configuration

Per catalog

One url is required. Everything else has a default chosen to give a useful crawl of a site you know
nothing about. Run options at the prompt for the live values.

Property	Default	Description
`url`	required	http:// or https://. The identity and the outer boundary
`name`	the domain	Unique text
`cat`	`other`	One of nine categories, used to filter and group
`start-url`	`= url`	Where fetching begins. Must sit under `url`
`sitemap-url`	empty	Empty discovers it from robots.txt
`include`	`.<domain>/`	Ant path pattern, comma for several
`exclude`	empty	Ant path pattern, comma for several
`encoding`	`UTF-8`	Page charset, when the server is wrong about it
`extractor`	`adaptive`	adaptive, restclient, htmlunit, playwright, selenium
`max-size`	`10000`	Saved pages before the crawl stops
`depth`	`-1`	Link depth. -1 for no limit
`duration`	`30`	Minutes before the crawl stops
`interval`	`1000`	Milliseconds between fetches, per node
`retry`	`1`	Retries per url
`images`	`true`	Whether pictures are fetched at all
`output-types`	`file`	`file+index+vector`. file is always on
`content`	`text+image`	What reaches the index and the vectors
`max-versions`	`10`	Versions kept before the oldest is pruned

Node wide, in `deploy/.env`

Property	Default	Description
`GF_DB_URL`	an H2 file	PostgreSQL, MySQL, SQL Server, Oracle, SQLite
`GF_INDEX_PROVIDER`	`lucene`	Or `elasticsearch`, with `GF_ES_URIS`
`GF_VECTOR_STORE`	`lucene`	Or `elasticsearch`, `qdrant`, `weaviate`
`GF_FILE_TARGET`	`local`	Or `minio`. Object keys match the local paths exactly
`GF_EMBEDDING_PROVIDER`	`local`	Or `ollama`, `openai`
`GF_LUCENE_ANALYZER`	`standard`	`standard`, `smartcn` or `cjk`
`GF_ES_ANALYZER`	`standard`	`ik_max_word` needs the analysis-ik plugin
`GF_DOCUMENTS`	`false`	Whether linked documents are fetched and read
`GF_CLUSTER_TRANSPORT`	`NETTY`	Or the built in `NIO`
`GF_NODES`	`1`	How many processes a launcher starts
`GF_MEMORY`	`2g`	Per container. Caps heap and off heap together

Analyzers matter more than they look. The standard analyzer cuts Chinese into single characters,
and a Chinese analyzer drops French characters outright, so chaîne comes back as cha î ne.

Performance

Our own numbers only. There is no comparison here against other crawlers, because we have not run
the controlled experiment that would make one honest.

Test environment. Apple M2 Max, 12 cores, 32 GB, macOS 26.3.1, JDK 17.0.12. Two nodes started
by run-local.sh, each -Xms256m -Xmx2g. H2 file, embedded Lucene index, embedded Lucene vector
store, pages and images on local disk, local ONNX embeddings. Home broadband. Polite crawling, so
fetchInterval is the floor on throughput rather than the hardware.

Site	Nodes	Kept	Images	Urls seen	Elapsed
apod.nasa.gov	2	46 pages	16	3,773	1m 41s
simplefood.blog	2	140 pages	2,489	22,857 already known	5m 01s
books.toscrape.com	1	1,000 pages	3,204	28,411	4m 12s

The gap between urls seen and pages kept is the point rather than an inefficiency. On simplefood
22,340 urls were filtered out by the boundary rules and 561 were duplicate content, which is work
the outputs never had to do.

How evenly the work spreads.

two nodes, apod.nasa.gov
  node 18bbe738   76 handled   51%   26 pages   9 images
  node 96c82b18   74 handled   49%   19 pages   7 images

three nodes, a 61 page site, one shared PostgreSQL
  dispatched 61      handled 66      saved 61
  node-1: 18 fetches   node-2: 18   node-3: 25     each page exactly once

Everything else we measured.

	Measured
Word search, embedded Lucene, 151 documents	38 matches in 16 ms
Cluster wire format, one `CrawlTask`	JSON 327 bytes, encode and decode 1,394 ns
Both ONNX models preloaded	4.47 s, on a background thread after the app is ready
Idle RSS with both models loaded	1.77 to 1.88 GB
Image vector duplication, before and after the fix	43× then 1.00×
Container memory, vectors or a browser	1g is OOMKilled, 2g passes, and they want separate runs
Tests	133 classes, 1,098 methods, 80% line coverage gate on four modules

Limitations and trade-offs

The embedded index and vector store cost the extracted text once per node. Fine for one or two nodes. For a real cluster, point it at Elasticsearch or Qdrant, which the startup report recommends out loud.
Replication is asynchronous. Immediately after a crawl, one node can answer a search before another has caught up. Seconds, not minutes.
A node killed mid crawl leaves runningState set. The registry says nothing is running, the row says otherwise, and interrupt has nothing to interrupt. On the list to fix.
Article extraction leaves some template text in the chunks. Share buttons and footer credits turn up in search snippets.
Politeness is the throughput ceiling. fetchInterval defaults to a second per node, and that is deliberate. This is not the tool for hammering one host as fast as it will answer.
One crawl at a time per cluster. Two crawls means two clusters, which is a name and a port.
Pdf, Word and Excel need a bean. Each is another dependency with its own licence and its own appetite for memory, so the choice belongs to the application.

Summary

One url in, and you get a whole site on disk, a full text index and vectors, from one pass.
Nothing to provision. H2, an embedded Lucene index and local ONNX models are the defaults. No database, no search server, no API key, no model download by hand.
Decentralised by default. Every node runs the same jar. No coordinator, no central queue, no leader in the fetch path.
Scaling is starting another process and pointing it at the same cluster name, including joining a crawl already running.
Replay rebuilds an output from the files, so a lost index or a changed analyzer never means crawling the site again.
Versions make a rebuild safe. The old version keeps answering searches, and an interrupted rebuild publishes nothing.
Three ways to search, including finding a picture by describing it, with the models running locally.
Every decision the crawler makes is an interface with a shipped default that a bean replaces.
The crawl cannot leave the site. Two boundary rules, neither of which can be switched off.
Apache 2.0, JDK 17, Spring Boot 4.1.

Source, issues and full documentation: github.com/paganini2008/greenfinger

EasyJPA: Criteria API Power in Three Lines, Not Thirty

Fred Feng — Sun, 20 Sep 2026 13:41:02 +0000

1. Overview

EasyJPA — Criteria API power. Three lines, not thirty.

EasyJPA is a Spring Boot starter that puts a fluent, lambda-driven API over the JPA Criteria API.
Every condition is a method reference, every join has a name, and not a single JPQL string is
written. Joins, subqueries, grouping, pagination, updates and native SQL all compose in one chain
you read top to bottom.

Here is the entire pitch:

// ── The Criteria API ──────────────────────────────────────────────────────────
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaQuery<User> cq = cb.createQuery(User.class);
Root<User> root = cq.from(User.class);
List<Predicate> predicates = new ArrayList<>();
predicates.add(cb.equal(root.get("username"), "Jack"));
predicates.add(cb.equal(root.get("password"), "123456"));
cq.select(root).where(cb.and(predicates.toArray(new Predicate[0])));
User user = em.createQuery(cq).getSingleResult();

// ── EasyJPA ───────────────────────────────────────────────────────────────────
User user = userDao.query()
        .filter(new FilterList().eq(User::getUsername, "Jack").eq(User::getPassword, "123456"))
        .selectThis().one();

Same query, same type safety, same generated SQL.

2. What Problem Does It Solve?

The Criteria API is the only type-safe way to build a dynamic JPA query, and almost nobody enjoys
using it. A three-condition filter costs ten lines of CriteriaBuilder plumbing; a multi-branch
join means tracking Join<?, ?> variables by hand; a correlated subquery is close to unreadable.
So teams fall back on JPQL strings or Spring Data method names — and lose type safety, or
composability, or both.

The second problem is quieter and costs more. A paginated query is really two statements, the
listing one and the counting one, and they are written separately. They drift. And when the query
has a group by, the obvious count(*) counts rows rather than groups, so the total is simply
wrong. EasyJPA derives both statements from one definition, and counts groups through a derived
table.


Keeps	Criteria's type safety and dynamic composition
Drops	`CriteriaBuilder` / `Root` / `Predicate[]` boilerplate
Fixes	pagination counts that drift, and group counts that count rows

3. Quick Start

Install — the current version is 2.0.0-SNAPSHOT, in the Central Portal snapshot repository,
which Maven reads from no project that has not named it:

<dependency>
    <groupId>com.github.paganini2008</groupId>
    <artifactId>easyjpa-spring-boot-starter</artifactId>
    <version>2.0.0-SNAPSHOT</version>
</dependency>

<repositories>
    <repository>
        <id>central-portal-snapshots</id>
        <url>https://central.sonatype.com/repository/maven-snapshots/</url>
        <releases><enabled>false</enabled></releases>
        <snapshots><enabled>true</enabled></snapshots>
    </repository>
</repositories>

Name the provider:

@EntityScan(basePackages = {"com.example.entity"})
@EnableJpaRepositories(repositoryFactoryBeanClass = HibernateEntityDaoFactoryBean.class,
        basePackages = {"com.example.dao"})
@Configuration(proxyBeanMethods = false)
public class JpaConfig {
}

Extend EntityDao:

public interface UserDao extends EntityDao<User, Long> {
}

EntityDao is a JpaRepositoryImplementation, so save, findById and deleteAll are
untouched. Query:

List<User> users = userDao.query()
        .filter(Restrictions.eq(User::getVip, true))
        .sort(JpaSort.asc(User::getUsername))
        .selectThis().list();

That is the whole setup. No properties to add, nothing to tune.

4. Requirements

	Minimum	Notes
Java	17
Spring Boot	4.0	Spring Boot 3.1 – 3.5 takes the `1.0.x` line
Jakarta Persistence	3.2
JPA provider	Hibernate 7.4	or EclipseLink 5.0.1, or any Criteria implementation
Build	Maven 3.9

The version line follows the Spring Boot line, and the two are maintained apart:

Your Spring Boot	Use
4.0 and later	`2.0.x`, this one
3.1 – 3.5	`1.0.x`
3.0 and earlier	not supported

5. How It Works

┌─────────────────────────────────────────────────────────────────────────────┐
│  YOUR CODE        userDao.query().filter(...).sort(...).selectThis().list()  │
└──────────────────────────────────┬──────────────────────────────────────────┘
                                   │  method references + aliases
┌──────────────────────────────────▼──────────────────────────────────────────┐
│  EASYJPA          Model ─────────── resolves an alias to a table            │
│                   Filter · Field · Column · JpaSort                         │
│                   JpaPageResultSet ─ one definition, two statements         │
└──────────────────────────────────┬──────────────────────────────────────────┘
                                   │  "may I use a derived table here?"
┌──────────────────────────────────▼──────────────────────────────────────────┐
│  JpaProvider        Hibernate   │   EclipseLink   │   Standard              │
│                     supportsDerivedTable() · supportsRightJoin() · …        │
└──────────────────────────────────┬──────────────────────────────────────────┘
                                   │
┌──────────────────────────────────▼──────────────────────────────────────────┐
│  JAKARTA PERSISTENCE   CriteriaBuilder · CriteriaQuery · Root · Predicate    │
└──────────────────────────────────┬──────────────────────────────────────────┘
                                   │
┌──────────────────────────────────▼──────────────────────────────────────────┐
│  DATABASE    H2 · PostgreSQL · MySQL · SQL Server · SQLite · Oracle         │
└─────────────────────────────────────────────────────────────────────────────┘

Mechanic	In one line
Alias resolution	A lambda names an entity, not a table; the alias is looked up inside the statement being built and never outside it — so an outer query and its subquery keep their tables apart
Two statements, one definition	`JpaPageResultSet` holds the listing query and derives the counting query from it, so they cannot drift
Providers are asked, not assumed	Every optional feature is a `boolean` on `JpaProvider`, readable at runtime

And the pagination split, which is the part worth seeing:

      customPage().join(...).filter(...).groupBy(...).select(...)
                              one definition
                                    │
                 ┌──────────────────┴──────────────────┐
                 ▼                                     ▼
            list(10, 0)                           rowCount()
                 │                                     │
   select … offset ? rows                   no group by → select count(1) from (…) d
          fetch first ? rows only              group by → select count(1) from
                                                          ( <the groups> ) d

6. Code Examples

Every SQL block below is what Hibernate actually emitted, captured from the test suite against H2.
The model:

User ──< Order ──< OrderProduct >── Product        User.vip, Order.status, Order.totalPrice
                                        │          Product.name/price/location
                                      Stock        Stock.amount

Example 1 — Nested conditions

Input — vip or (username in ('Scott','Lee') and email is not null), by username.

Execution

List<User> users = userDao.query()
        .filter(Restrictions.eq(User::getVip, true)
                .or(new FilterList().in(User::getUsername, List.of("Scott", "Lee"))
                        .and().notNull(User::getEmail)))
        .sort(JpaSort.asc(User::getUsername))
        .selectThis().list();

Output

select u1_0.id, u1_0.email, u1_0.password, u1_0.username, u1_0.vip
from example_user u1_0
where u1_0.vip = ? or u1_0.username in (?, ?) and u1_0.email is not null
order by u1_0.username

Example 2 — Four tables, two branches

Input — order lines with their order, customer and product; only discounted products on paid or
shipped orders; newest first.

Execution

orderProductDao.customPage()
        .join(OrderProduct::getOrder, "o", null)      // OrderProduct → Order
        .join(Order::getUser, "u", null)              // Order        → User
        .join(OrderProduct::getProduct, "p", null)    // OrderProduct → Product, second branch
        .filter(new FilterList().notNull(Product::getDiscount).and()
                .in(Property.forName("o", "status"), List.of(OrderStatus.PAID, OrderStatus.SHIPPED)))
        .sort(JpaSort.desc(Order::getOrderDate), JpaSort.asc(Product::getName))
        .select(new ColumnList().addColumns(User::getUsername)
                .addColumns(Order::getId, Order::getOrderDate, Order::getStatus)
                .addColumns(Product::getName, Product::getPrice)
                .addColumns(OrderProduct::getAmount)
                .addColumns(Fields.multiply(Property.forName(Product::getPrice),
                        Property.forName(OrderProduct::getAmount)).as("subtotal")));

Output

select u1_0.username, o1_0.id, o1_0.order_date, o1_0.status,
       p1_0.name, p1_0.price, op1_0.amount, (p1_0.price * op1_0.amount)
from example_order_product op1_0
join example_order   o1_0 on o1_0.id = op1_0.order_id
join example_user    u1_0 on u1_0.id = o1_0.user_id
join example_product p1_0 on p1_0.id = op1_0.product_id
where p1_0.discount is not null and o1_0.status in (?, ?)
order by o1_0.order_date desc, p1_0.name
offset ? rows fetch first ? rows only

The third join branches back off OrderProduct rather than continuing from User: the lambda
carries its own entity, so the tree comes out the way it reads.

Example 3 — A grouping pagination that counts groups

Input — best sellers: units sold, distinct orders and turnover per product, keeping products
sold more than once. Then the total, for the page navigation.

Execution

JpaPageResultSet<Tuple> resultSet = orderProductDao.customPage()
        .join(OrderProduct::getProduct, "p", null)
        .groupBy(new FieldList().addFields(Product::getName))
        .having(Restrictions.gt(Fields.count(OrderProduct::getId), 1L))
        .sort(JpaSort.desc(2))
        .select(new ColumnList().addColumns(Product::getName)
                .addColumns(Fields.sum(OrderProduct::getAmount).as("soldAmount"),
                        Fields.countDistinct(Property.forName("this", "order.id")).as("orderAmount"),
                        Fields.sum(Fields.multiply(Property.forName(Product::getPrice),
                                Property.forName(OrderProduct::getAmount))).as("turnover")));

long groups = resultSet.rowCount();
List<SalesVo> rows = resultSet.setTransformer(Transformers.asBean(SalesVo.class)).list();

Output — the listing query

select p1_0.name, sum(op1_0.amount), count(distinct op1_0.order_id),
       sum((p1_0.price * op1_0.amount))
from example_order_product op1_0
join example_product p1_0 on p1_0.id = op1_0.product_id
group by p1_0.name having count(op1_0.id) > ?
order by 2 desc
offset ? rows

Output — the counting query, derived from the same definition

select count(1) from (
    select 1 from example_order_product op1_0
    join example_product p1_0 on p1_0.id = op1_0.product_id
    group by p1_0.name having count(op1_0.id) > ?
) derived1_0(c)

One statement, having honoured, no groups fetched. This is the bug you do not have to find.

Example 4 — A correlated subquery

Input — the users who have ever ordered.

Execution

JpaQuery<User, User> query = userDao.query();
JpaSubQuery<Order, Long> subQuery = query.subQuery(Order.class, "o", Long.class)
        .filter(Restrictions.eq(Order::getUser, User::getId))
        .select(Order::getId);

List<User> customers = query.filter(Restrictions.exists(subQuery)).selectThis().list();

Output

select u1_0.id, u1_0.email, u1_0.password, u1_0.username, u1_0.vip
from example_user u1_0
where exists (select o1_0.id from example_order o1_0 where o1_0.user_id = u1_0.id)

The subquery is created from the outer query, which is what correlates the two. Negate it with
.not() for not exists; swap exists for in and it becomes an in subquery.

Example 5 — Joining an aggregate as a derived table

Input — every product with its sales, without a correlated subquery per row.

Execution

JpaQuery<Product, Tuple> query = productDao.customQuery();
JpaSubQuery<OrderProduct, Tuple> sales = query.subQuery(OrderProduct.class, "op", Tuple.class);
sales.groupBy(new FieldList().addFields(Property.forName("op", "product.id")))
     .select(new ColumnList()
             .addColumns(Property.forName("op", "product.id").as("productId"))
             .addColumns(Fields.sum("op", "amount", Integer.class).as("soldAmount"),
                         Fields.count("op", "id").as("orderAmount")));

query.joinSubQuery(sales, "s", Restrictions.eq(Property.forName("s", "productId"),
                                               Property.forName("this", "id")))
     .sort(JpaSort.desc(Property.forName("s", "soldAmount")))
     .select(new ColumnList().addColumns(Product::getName, Product::getPrice)
             .addColumns(Property.forName("s", "soldAmount").as("soldAmount")))
     .setTransformer(Transformers.asCaseInsensitiveMap()).list();

Output

select p1_0.name, p1_0.price, s1_0.soldAmount, s1_0.orderAmount
from example_product p1_0
join (select op1_0.product_id c0, sum(op1_0.amount) c1, count(op1_0.id) c2
      from example_order_product op1_0
      group by c0) s1_0(productId, soldAmount, orderAmount)
  on s1_0.productId = p1_0.id
order by 3 desc

Example 6 — Update by an expression, delete by a subquery

Input — shout a username; decrement a stock; drop the users who never ordered.

Execution

userDao.update().setField(User::getUsername, Fields.concat(Fields.upper(User::getUsername), "!"))
       .filter(Restrictions.eq(User::getUsername, "Lee")).execute();

stockDao.update().setField(Stock::getAmount, Fields.minusValue(Stock::getAmount, 1L))
        .filter(Restrictions.gt(Stock::getAmount, 0L)).execute();

JpaDelete<User> delete = userDao.delete();
JpaSubQuery<Order, Order> subQuery = delete.subQuery(Order.class)
        .filter(Restrictions.eq(Order::getUser, User::getId));
delete.filter(Restrictions.exists(subQuery).not()).execute();

Output

update example_user  u1_0 set username = (upper(u1_0.username)||?) where u1_0.username = ?
update example_stock s1_0 set amount = (s1_0.amount - cast(? as bigint)) where s1_0.amount > ?

delete from example_user u1_0
where not exists (select o1_0.id from example_order o1_0 where o1_0.user_id = u1_0.id)

The rest of the surface, in one table

You want	Call
The entities	`dao.query()`
Columns into a VO	`dao.query(Vo.class)`
Any columns	`dao.customQuery()` → `Tuple`
…with a total count	`dao.page()` · `dao.customPage()` → `JpaPageResultSet`
Fetch an association	`fetch(Order::getUser)` · `leftFetch(User::getOrders).distinct()`
`CASE WHEN`	`new IfExpression<>(attr).when(a, b).otherwise(c)`
Date parts	`Fields.year/month/day(...)`
Any database function	`Function.build("LOWER", String.class, Product::getName)`
A result shape	`Transformers.asBean/asMap/asCaseInsensitiveMap/asList/noop`
Native SQL	`dao.queryForMap(sql, args)` — keeps `rowCount()` and `paginate()`

7. Configuration

EasyJPA has no properties of its own — no spring.easyjpa.* namespace. What you configure is
the provider and, for three databases, the driver.

What	Where	Value
Provider	`@EnableJpaRepositories(repositoryFactoryBeanClass = …)`	`HibernateEntityDaoFactoryBean` (default) · `EclipseLinkEntityDaoFactoryBean` · `StandardEntityDaoFactoryBean`
EclipseLink	`spring.autoconfigure.exclude`	`…orm.jpa.HibernateJpaAutoConfiguration`
SQL Server	JDBC url	`calcBigDecimalPrecision=true` — otherwise every `BigDecimal` binds as `decimal(38,0)` and `0.90` reads back as `1`
SQLite	entity id	`GenerationType.IDENTITY` — `AUTO` needs a sequence table SQLite locks against
Oracle 23+ on Hibernate	`OracleDialect` subclass	`DatabaseVersion.make(21)`

8. Comparison

Feature-level. No timings are claimed — nothing here is a benchmark.

	EasyJPA	Criteria API	Spring Data `Specification`	QueryDSL
Type-safe attributes	method references	metamodel or strings	metamodel or strings	generated `Q` classes
Code generation step	none	none	none	annotation processor
Dynamic composition	✅	manual `Predicate[]`	✅	✅
Multi-branch joins	one call each	track `Join<?,?>` by hand	manual	✅
Correlated subqueries	built from the outer query	manual	awkward	✅
Pagination count over `group by`	derived table, one statement	write it yourself	write it yourself	write it yourself
Derived-table join	`joinSubQuery(...)`	manual	❌	limited
Native SQL, pagination kept	✅	❌	❌	separate API
Provider gaps discoverable at runtime	`JpaProvider` flags	❌	❌	❌

Provider and database coverage

The same 202 tests run on three providers against six databases.

	H2 2.3	PostgreSQL 16	MySQL 9.6	SQL Server 16	SQLite 3.49	Oracle 23
Hibernate	✅	✅	✅	⚠️ 1	⚠️ 3	⚠️ 4
Criteria API alone	✅	✅	✅	⚠️ 1	⚠️ 3	✅
EclipseLink	✅	✅	✅	✅	❌ no platform	⚠️ 2

Each ⚠️ belongs to the database or to the provider on that database, never to a query EasyJPA built.
The README names every one of them.

9. Limitations & Trade-offs

Aliases are strings. A typo in "op" is a runtime error, not a compile error.
Not auto-configured. You name the factory bean yourself; there is no @AutoConfiguration.
The Criteria API is the ceiling. Window functions, CTEs and UNION need native SQL.
EclipseLink reaches less than Hibernate. No derived table, no right join, no subquery as a selected column — all of it reported through JpaProvider rather than failed over silently.
A fetched collection paginates in memory. Keep paginated fetches to to-one associations.
No reactive support. Blocking JDBC only.
Not a good fit for a project that writes mostly static queries — Spring Data method names are shorter for those.

10. Summary

One chain instead of ten lines. query().filter().sort().select().list() replaces the whole CriteriaBuilder / Root / Predicate[] ritual.
Method references throughout, so a renamed field is a compile error rather than a runtime surprise. No code generation, no annotation processor.
Joins branch by themselves. A lambda carries its own entity, so a four-table, two-branch join is four calls that read in order.
A pagination is one definition, two statements. The counting query is derived from the listing query and cannot drift from it.
A grouping pagination counts groups, through a derived table, in one statement, having included — the count most hand-written pagination gets wrong.
Subqueries correlate themselves because they are created from the query that uses them. Filter, column, comparison or nested — all the same way.
Aggregates join once, as a derived table, instead of running a correlated subquery per row.
Provider gaps are a boolean, not a stack trace. JpaProviders.getProvider().supportsXxx() answers before you build.
Tested where it matters — 202 tests, three providers, six databases, and every documented failure attributed to the database or the provider rather than hidden.
Zero configuration. No properties; name one factory bean and extend EntityDao.

GitHub — paganini2008/easyjpa, MIT licensed.
Spring Boot 4 takes the 2.0.x line, Spring Boot 3 the 1.0.x line.

Cronsmith: Stop hand-writing cron: build and parse it in Java

Fred Feng — Sun, 20 Sep 2026 08:55:55 +0000

Cronsmith is a Java library that builds, parses and runs cron through a fluent, object-oriented API
instead of hand-typed strings. You describe when in plain method calls and get back a cron expression,
the exact date-times it fires at, or a running task — across Quartz, Spring, AWS EventBridge and crontab.

// "the last Friday of every month at 18:00" — readable, and it tells you the next fire
CronExpression cron = new CronBuilder().everyMonth().lastDayOfWeek(DayOfWeek.FRIDAY.getValue()).at(18, 0);
cron.toString();              // 0 0 18 ? * FRIL
cron.getNextFiredDateTime();  // the next last-Friday-of-the-month, 18:00

What problem does it solve?

Quick, what does 0 0 18 ? * FRIL fire at? Cron strings are write-only: easy to fumble, hard to review,
and impossible to unit-test by eye. Worse, "cron" isn't one language — Quartz, Spring, AWS EventBridge
and Unix crontab disagree on fields, on whether 1 is Sunday or Monday, and on which of L / # / W
they even support. Copy a Quartz line into crontab and it can silently fire at a different time.

Cronsmith makes the schedule code you can read, iterate and port: build it with method calls, print
it for the target you deploy to, and ask it for its next fire times before you trust it.

Quick start

<dependency>
    <groupId>com.github.paganini2008</groupId>
    <artifactId>cronsmith</artifactId>
    <version>1.0.0-SNAPSHOT</version>
</dependency>

CronExpression weekdays9 = new CronBuilder().everyWeek().Mon().toFri().at(9, 0);
weekdays9.toString();              // 0 0 9 ? * MON-FRI
weekdays9.getNextFiredDateTime();  // next weekday at 09:00

Requirements

JDK 17+
The core parser's only runtime dependency is ANTLR — no Spring, HTTP or database.

How it works

A CronBuilder builds a CronExpression (an iterable, serializable model of the schedule). From there
you render it for a dialect, or walk its fire times:

CronBuilder  ──build──▶  CronExpression  ──┬── toString / toQuartz / toAws / toUnix  (render)
   or CRON.parse(str) ───────────────▶     ├── consume(..) / getNextFiredDateTime()  (iterate)
                                            └── serialize() / deserialize(..)         (store)

Code examples

Build — describe it, don't assemble strings

Input → Output:

new CronBuilder().everySecond(5);                                       // */5 * * * * ?
new CronBuilder().everyMonth().day(10).andDay(15).andLastDay().everyHour(2).everyMinute(5);
// 0 */5 */2 10,15,L * ?
new CronBuilder().everyMonth().dayOfWeek(3, DayOfWeek.SATURDAY).everyHour(2);  // 0 0 */2 ? * SAT#3 (3rd Sat)
new CronBuilder().everyMonth().lastDay(3).at(23, 30);                   // 0 30 23 L-3 * ?
new CronBuilder().everyMonth().latestWeekday(15).at(9, 0);              // 0 0 9 15W * ? (nearest weekday to 15th)

Look at a schedule instead of trusting it

new CronBuilder().setStartTime(LocalDate.of(2027, 1, 1).atStartOfDay())
        .everyMonth().latestWeekday(15).at(9, 0)
        .consume(System.out::println, 5);
// 2027-01-15T09:00 … 2027-05-14T09:00  <- the 15th is a Saturday, so it moves to the nearest weekday

Parse a string back into a model

CRON.parse("*/5 * * * *");    // 0 */5 * * * ?      a crontab line, seconds filled in
CRON.parse("0 9 * * 1-5");    // 0 0 9 ? * MON-FRI  crontab: 1 is Monday
CRON.parse("0 0 12 ? * 1");   // 0 0 12 ? * SUN     Quartz:  1 is Sunday

The field count picks the reading: 5 = crontab, 6 = Quartz without a year, 7 = Quartz with a year.

YCRON — schedules month-based cron can't say

Seven fields: <sec> <min> <hour> <day-of-week> <week-of-year> <day-of-year> [<year>].

new CronBuilder().year(2026).day(100).at(12, 0, 0);        // 0 0 12 ? ? 100 2026  (100th day of the year)
new CronBuilder().year(2026).week(20).Mon().at(9, 0, 0);   // 0 0 9 MON 20 ? 2026  (Mon of ISO week 20)
YCRON.parse("0 0 12 ? ? 100 2026").getCronType();          // CronType.YCRON (its own grammar)

Turn a plain interval into cron, or run it in-process

CRON.setInterval("PT30S");                    // every 30s (PT1H30M / sub-second are rejected, not rounded)

ScheduledExecutorService exec = Executors.newScheduledThreadPool(4);
CronFuture f = new CronBuilder().everySecond(5).scheduler(exec).runTask(() -> doWork(), 10); // run 10x

Configuration — syntax reference

 sec min hour day-of-month month day-of-week [year]
  *   *   *       *           *        ?        *
 day-of-month: 1-31, L, LW, L-n, nW, ?     day-of-week: 1-7 (SUN=1) or SUN-SAT, nL, n#m, ?
 YCRON: <sec> <min> <hour> <day-of-week> <week-of-year> <day-of-year> [<year>]

Tag	Meaning
`L` / `L-n` / `LW`	last day / `n` days before last / last weekday of the month
`nW`	weekday nearest the `n`th, without leaving the month
`<dow>L` / `<dow>#n`	last `<dow>` (`FRIL`) / `n`th `<dow>` (`TUE#2`, skipped in months without an `n`th)
`a-b` / `a/n` / `a,b`	range / step / list (entries may themselves be ranges or steps)

How it compares across schedulers

One schedule, printed for whatever you deploy to. Unsupported features are reported, not silently
rewritten:

Schedule	Quartz	Spring	AWS EventBridge	Unix crontab
weekdays 09:00	`0 0 9 ? * MON-FRI`	`0 0 9 ? * MON-FRI`	`0 9 ? * MON-FRI *`	`0 9 * * MON-FRI`
every 15 seconds	`/15 * * * ?`	`/15 * * * ?`	no seconds field	no seconds field
last day of month	`0 59 23 L * ?`	`0 59 23 L * ?`	`59 23 L * ? *`	no `L`
2nd Tuesday	`0 0 10 ? * TUE#2`	`0 0 10 ? * TUE#2`	`0 10 ? * TUE#2 *`	no `#`
restricted to 2027-2029	`… 2027-2029`	no year field	`… 2027-2029`	no year field

CronExpression daily = new CronBuilder().everyDay().at(9, 30);
CRON.toUnixString(daily);   // 30 9 * * *   (throws if the schedule needs L/#/seconds crontab lacks)

Limitations & trade-offs

It is a cron builder / parser / in-process runner, not a distributed job system. For persistent, clustered scheduling with retries and a console, use cronsmith-spring-boot-starter in cronflower.
Rendering is honest, not lossy: a schedule a target can't express throws instead of quietly changing when it fires.
ISO-8601 intervals must be an exact multiple of one unit (PT1H30M, PT90M, sub-second are rejected).

Summary

Build cron with a fluent, readable API; toString() for the string.
Parse any dialect back into an inspectable, iterable CronExpression.
Port one schedule to Quartz / Spring / AWS / crontab; unsupported features are reported.
YCRON expresses year-, week- and day-of-year schedules classic cron can't.
Iterate the real fire times (consume / getNextFiredDateTime) before you trust a schedule.
Serializable, with an in-process timing-wheel scheduler for simple runs.

Get it: cronsmith · JDK 17+, Apache 2.0.