DEV Community: Cies Breijs

Postgres pipelines from the JVM with Bpdbi

Cies Breijs — Tue, 24 Mar 2026 17:00:31 +0000

Every time your JVM app talks to Postgres over JDBC, something wasteful happens. Your code sends a query, waits for the response, sends the next query, waits again, etc. Each wait is a full network round-trip with a typical latency of 0.5-2ms in a cloud environment. For a simple transaction with a few queries, that's 8-10ms of just... waiting.

This problem is exacerbated by using db transactions and the additional queries that are commonly added when using Postgres' Row-Level Security (RLS). For instance when using Supabase, a role and JWT claims need to be set before executing the actual query. Usually this is done more than once for each HTTP request, as some db queries need to be executed with a different role than others.

Bpdbi is a new JVM database library for the Postgres database. It exposes pipelining, a feature that's been part of the Postgres wire protocol since version 14.

JDBC cannot do pipelines. Bpdbi can.

Where this came from

Bpdbi started as a port of the Vert.x SQL Client — the database layer behind Quarkus and one of the fastest JVM database drivers. Vert.x already speaks the Postgres wire protocol directly (no JDBC), and exposes the pipelines feature.

Vert.x is fully reactive (async/non-blocking). Reactive code is not for everyone: it adds significant complexity ¹ ².

Reactive code looks like this:

// Reactive — same logic, now good luck
public Uni<Invoice> generateInvoice(long orderId) {
  return orderRepo.findById(orderId)
      .flatMap(order ->
          Uni.combine().all()
              .unis(
                  customerRepo.findById(order.getCustomerId()),
                  itemRepo.findByOrderId(orderId))
              .asTuple()
              .map(tuple -> invoiceService.build(
                  order, tuple.getItem1(), tuple.getItem2())));
}

While the blocking equivalent look like this:

public Invoice generateInvoice(long orderId) {
  Order order = orderRepo.findById(orderId);
  Customer customer = customerRepo.findById(order.getCustomerId());
  List<Item> items = itemRepo.findByOrderId(orderId);
  return invoiceService.build(order, customer, items);
}

Reactive allows for higher throughput in high-traffic scenarios, but comes at a cost: less readable code, useless stack traces, and a paradigm that infects your code base. With Java 21's virtual threads, blocking I/O became much cheaper — you get thousands of concurrent connections without platform threads.

Bpdbi started as a port of Vert.x SQL Client, stripped of all its async/reactive machinery. Where Vert.x SQL uses Netty to connect with the database, Bpdbi uses a good old java.net.Socket.

Bpdbi employs Postgres' binary protocol and pipelines for all db queries, even single queries without parameters. This results in a simpler library with a much smaller footprint, while being very performant (as shown in the benchmarks).

TL;DR: Bpdbi provides blocking, pipelined, small-footprint and performant Postgres access for the JVM.

Pipelining in practice

Here's the core idea. Say you need to start a transaction, set some config, and run a query. With JDBC, that's four separate round-trips:

conn.createStatement().execute("BEGIN");
conn.createStatement().execute("SET statement_timeout TO '5s'");
conn.createStatement().execute("SET LOCAL role TO 'authenticated'");
PreparedStatement ps =
  conn.prepareStatement("SELECT * FROM orders WHERE id = ?");
ps.setInt(1, 42);
ResultSet rs = ps.executeQuery();

With Bpdbi, it's one:

conn.enqueue("BEGIN");
conn.enqueue("SET statement_timeout TO '5s'");
conn.enqueue("SET LOCAL role TO 'authenticated'");
RowSet result =
  conn.query("SELECT * FROM orders WHERE id = $1", 42);

enqueue() buffers statements locally. query() flushes everything —all the enqueued statements plus itself— in a single TCP write. The Postgres instance processes them all and sends all responses back at once. This feature is called pipelining.

When you need all the results

Sometimes you want results from multiple pipelined queries. flush() returns them all:

conn.enqueue("BEGIN");
int aliceQx = conn
  .enqueue("INSERT INTO users (name) VALUES (:name) RETURNING id")
  .bind("name", "Alice");
int bobQx = conn
  .enqueue("INSERT INTO users (name) VALUES (:name) RETURNING id")
  .bind("name", "Bob");
conn.enqueue("COMMIT");
List<RowSet> results = conn.flush();

long aliceId = results.get(aliceQx).first().getLong("id");
long bobId   = results.get(bobQx).first().getLong("id");

Four statements, one round-trip. Each enqueue() returns an index so you know which result is which.

The benchmark numbers

We ran JMH benchmarks using Toxiproxy to simulate 1ms network latency per direction (2ms round-trip). This simulates what you'd see talking to a database in the same cloud region.

Scenario	Bpdbi	JDBC (`pgjdbc`)	Speedup
10 SELECTs (pipelined vs sequential)	310 ops/s	18 ops/s	17x
Transaction (BEGIN+SELECT+COMMIT)	360 ops/s	185 ops/s	~2x
10 INSERTs in a transaction	116 ops/s	18 ops/s	6.5x
Cursor fetch (1000 rows)	281 ops/s	30 ops/s	9.3x
Bulk insert (100 rows)	313 ops/s	171 ops/s	1.8x
Single row lookup	370 ops/s	370 ops/s	op par
Multi-row fetch (10 rows)	358 ops/s	358 ops/s	on par

The pattern is clear: anything that touches the network more than once gets a massive speedup. While single-query performance is on par with JDBC + pgjdbc.

Postgres-only, and that's the point

Bpdbi only supports Postgres, the only open source database that truly supports pipelining. This is intentional, and it buys us a lot.

Binary protocol everywhere

The Postgres extended query protocol lets you request results in binary format. An integer comes back as four raw bytes instead of the text string "12345". A UUID is 16 bytes instead of 36 characters. No string allocation and string parsing.

Most JDBC drivers use the text format for simple queries (the "simple query protocol") and only switch to binary for prepared statements. Bpdbi uses the extended query protocol with binary format for everything — even BEGIN, COMMIT, and SET. This is what makes uniform pipelining possible: every statement uses the same wire protocol, so they can all be batched in a pipeline.

Small footprint

Bpdbi's Postgres driver is about 1,400 lines of Java. The whole library is under 200KB and that includes a connection pool.

Compare that to a typical Postgres/Jdbi/HikariCP stack:

Component	Size
`pgjdbc` (JDBC driver)	~1.1 MB
Jdbi (developer experience)	~1 MB
HikariCP (connection pool)	~160 KB
Total	~2.3 MB
Bpdbi (everything)	< 200 KB

And that's the modest stack. Hibernate is ~15MB, jOOQ is ~15MB.
The Vert.x SQL Client with Netty clocks in at 5MB+.
Bpdbi has no transitive dependencies beyond SCRAM-client (a small crypt lib for connection auth).

No Netty, no event loops

Plain java.net.Socket with unsynchronized buffered I/O streams.
No channel pipelines, no allocator frameworks, no thread pools. This works because Bpdbi connections are single-threaded by design — just like JDBC connections. With virtual threads, blocking on a socket is cheap. The simplicity pays off in readability, debugging, and startup time.

GraalVM native-image ready

The core library uses zero reflection. native-image just works, no configuration needed.

Developer experience: Jdbi-level, not JDBC-level

Directly using the JDBC API in your application code is low-level and verbose. That's why libraries like Jdbi, Spring JDBC Template and Sql2o exist. They provide: named parameters, row mapping, pluggable data type binders/ row mappers/ JSON mappers.

Bpdbi has all these features built-in: you don't need an additional library. It comes with add-on modules: bpdbi-record-mapper for mapping rows to Java records, and bpdbi-bean-mapper for mapping rows to JavaBeans. The bpdbi-kotlin add-on contains a mapper for mapping rows to Kotlin data classes using kotlinx.serialization which, unlike the other row mappers, does not use reflection.

Under the hood

The performance doesn't just come from pipelining. Bpdbi borrows optimization ideas from pgjdbc, Vert.x, and Jdbi, and even adds some of its own:

Column-oriented storage. A 100K-row, 10-column result creates 10 byte arrays instead of 1,000,000. Each Row is a lightweight view (buffer reference + row index) with no per-row allocation.
Lazy decoding. Rows store raw wire bytes. Columns you never read are never decoded. Your SELECT * that only reads 3 columns out of 20? Only those 3 get decoded.
Binary parameter encoding. Sending an int4 as 4 raw bytes instead of the ASCII string "12345" saves wire bandwidth and server CPU.
Unsynchronized I/O. Java's BufferedOutputStream acquires a lock on every write. Since Bpdbi connections are single-threaded, the buffered streams skip all synchronization.
Prepared statement cache. An LRU cache avoids re-parsing the same SQL. Oversized queries that would flush the cache are rejected outright.
Deadlock prevention. Large pipelined batches can deadlock if both TCP buffers fill up. Bpdbi estimates response sizes and inserts mid-pipeline syncs to prevent this — transparently.

Who is Bpdbi for?

Requirements for using Bpdbi:

Use Postgres (it's the only database that has pipelines).
Use Java 21+ (virtual threads make blocking I/O practical at scale).

Bpdbi makes a lot of sense if you:

Want to use Postgres' pipelining.
Use RLS (most Supabase users) or otherwise make a lot of "prefix" queries.
Prefer simple, blocking code over reactive/async/non-blocking code.
Care about small dependencies and fast startup (GraalVM, serverless, CLI tools).
Want to write SQL by hand (not ORM).

If you're already happy with Hibernate or jOOQ and their compile-time SQL validation, Bpdbi is probably not what you need.

Web application libraries/frameworks that work well with Bpdbi

Bpdbi uses blocking I/O and is designed for virtual threads.
It pairs well with HTTP frameworks that are not mandatorily reactive/async and do not dictate JDBC:

http4k — Functional, zero-reflection, tiny. The philosophical twin of Bpdbi on the HTTP side.
Javalin — Minimal Jetty wrapper with built-in virtual thread support. Very popular in both Java and Kotlin.
Helidon SE 4+ — Oracle's lightweight framework. Versions 1–3 were reactive (Reactive Streams); 4.x was rewritten around virtual threads and blocking I/O.
Undertow — Embedded, low-level. Blocking handlers run on a worker thread pool (or virtual threads).
Micronaut — Compile-time DI, GraalVM-first. Supports both reactive and imperative, controller methods can simply return values.
Spark — Dead-simple Java micro-framework with the same "just enough" philosophy.
Jooby — Modular micro-framework, explicit about dependencies, virtual thread support.
com.sun.net.httpserver — The JDK's built-in HTTP server. Zero dependencies, pairs naturally with Bpdbi's minimalism.

Frameworks like Spring Boot are opinionated about their own data stacks (Spring Data, Hibernate) and assume a JDBC DataSource integration for transactions, health checks, and connection management.

Give it a spin!

Everything you need to get started van be found on Bpdbi's GitHub repository. If you miss something, raise an issue to let us know.

Languages in the Linux kernel

Cies Breijs — Wed, 19 Feb 2025 22:06:22 +0000

The Linux kernel it written in many languages, but at the time of writing (2025) the main language of the project is C, clocking in at 98% (from github.com/torvalds/linux on 2025-02-19):

I believe the only language —or more correct "language family"— in the Linux kernel that will always be there is Assembly. At some point you need to write code, without any abstraction, directly for the hardware. Assembly is just that.

Allowing C++ in the kernel

Linus Torvalds, the benevolent dictator for life of the Linux kernel project, has rejected proposals for allowing C++ in the kernel. This while C++ has formidable interoperability with C, and was successfully used in many other kernel projects (Google's Fuchsia, Nintendo's Horizon and Nokia's Symbian).

I believe Linus was right to reject C++ for the following reaons:

C++ is a gigantic language, allowing it would create an endless discussion on which of its features would be allowed/forbidden,
being a loose superset of C it has the same archaic syntax, thus little improvement to the developer experience, and
no matter how many features would be allowed C++ does not improve enough on C to offset the cost of porting it.

The end of C

In recent year one thing became clear: C is no longer the best tool for the job. It's still the best understood tool, especially by the current team. It's what 98% of the Linux kernel is written in today. But the "unchallenged best" status it used to have for so many years in pragmatic kernel development is shaky.

Better languages have come up, specifically Zig and Rust.

There has been a lot of discussion on allowing Rust in the kernel. But since the main honchos, Linus and Greg Kroah-Hartman, have put their weight behind it, it is clear that Rust will find it's way into the project. We still have to wait and see how much of the project will be written in Rust.

Unlike C++, Rust has lots of features that are very useful in kernel development, and very few features that would need to be forbidden. Especially features related to memory safety, which constitute a large part of the bugs in Linux. Compared to C, Rust provides much a improved developer experience, which is not weird considering Rust is 40 younger.

Rust's main downside is: slower compile times. Compile times matter very much, but so does safety. While this is a hard trade-off, the decision seems to be final: Rust is to stay in the Linux kernel.

I expect Zig will come from another angle. The Zig compiler compiles C as well as Zig code. It's just a matter of time before the Zig compiler will be able to compile the Linux kernel. Once this is achieved, C-files can be ported to Zig one-by-one. I expect LLMs will help a great deal with the initial port of the Linux kernel's C code to Zig. Once in Zig, the code can be optimized by humans.

Zig is very similar to C. This makes the initial port rather straight forward. LLMs will perform much better on C-to-Zig than on C-to-Rust.

Compared to C, Zig brings serious improvements in the developer experience at similar-to-C compile times. Features like comptime are really cool and may allow the kernel project to do away with lots of crufty old C preprocessor macros on the one side, while allowing for interesting optimizations on the other.

Conclusion

I expect we will have a Linux kernel in Zig, Rust and Assembly in 10 years. It may be a fork. It may be the Linux mainline: it all depends on the open-mindedness of the main devs.

So far they seem up for it!

The case against ORMs

Cies Breijs — Wed, 29 Jan 2025 16:41:51 +0000

Allow me to kick off with a bold statement: ORMs —like Hibernate (JVM), Entity Framework (.NET), ActiveRecord (Ruby), Django ORM (Python), Eloquent (PHP/Laravel), Doctrine (PHP), GORM (Go), etc.— are never a good idea. Period.

Even when using object oriented (OO) languages.

No matter how good or lacking your SQL skills are.

Never.

Why? The short (TLDR) version of this argument is:

They make the simple queries slightly simpler, but they do not help you for the harder queries. — me

And with AI coding assistants an ORM's added value is even further reduced.

For the long version continue reading the rest of this article...

Simple gets simpler with an ORM!

Instead of the potentially dangerous:

db.execute("select * from User u where u.id = $userId")

You can use an ORM and simply write:

Users.getById(userId)

With the following benefits:

Harder to write code vulnerable to SQL injections.
The result of the ORM call is usually mapped to some User object, where in case of the SQL you merely get the raw result (a list of tuples).
Less to type and easier to read! (and code should be optimized for readability)

But there's more that makes ORMs great! Following associations is really easy when using an ORM. Let's say we want to fetch the user's permissions from the database. Using SQL this would look like:

db.execute("select * from Permission p where p.user_id = $userId")

When using an ORM this could be as simple as:

val user = Users.getById(id)
user.permissions()

Finally, I believe ORMs greatly simplify saving and deleting data in the database. Let's add a permission for the user and then delete the user altogether:

db.execute("insert into Permission (user_id, value) values ($userId, $permissionEnum)")
db.execute("delete from User u where u.id = $userId")

When using an ORM this would be:

Permissions.create(userId, permissionEnum)
Users.getById(userId).delete()

With all these benefits, why do I argue against using an ORM?

Disadvantages of ORMs

Every database-backed application at some point needs more complex queries: you will need a query that cannot be expressed using the ORM (like complex aggregations, recursive queries or bulk inserts/updates). And then? You could just retrieve much more data and do the processing in the application layer, which is very slow. Usually in those cases the ORM is bypassed by writing SQL-in-strings, after which the code base contains two ways to query the db: (1) through the ORM and (2) using SQL-in-strings.

We can conclude ORMs are a very leaky abstraction (thanks Joey). In other words: you cannot merely learn how to use an ORM, you still need to learn SQL as well!

Thus, the whole reason to learn an ORM and the myriad concepts you need to master in order to use that ORM effectively, is just to make the simple queries slightly simpler.

What are these concepts common to most ORMs? Well...

Entities or Models – For mapping database table rows to objects. This may require quite a bit of coding: every table that you want to query from using the ORM needs some mapping code.
Associations – Like one-to-one, one-to-many, and many-to-many. These need to be specified on the models.
Lazy/eager loading of associations – To controlling when related data is fetched.
Transactions – Ensuring data integrity with commit and rollback operations. Most ORMs have a specific way of achieving this (by abstracting over the db's TRANSACTIONs).
Caching – Optimizing performance by avoiding redundant queries. Not all ORMs have caching layers, but many do. This also ties into the way your application is clustered.
Concurrency – Managing conflicts when multiple users update data. Also know as optimistic locking.
Identity mapping – To ensure the same object instance represents the same database row within a session.
Sessions or Persistence contexts – To manage object life-cycles and database commits efficiently.
Life-cycles – ORM objects usually life-cycles that tie in with translations, sessions and clean/dirty tracking.
Clean/dirty tracking – data that was not changed, does not need to be updated in the database.
Cascading of operations – Describes if the associated ORM object should be persisted. This ties in to the ORM,s notion of clean/dirty.

Not all these concepts apply to all ORMs. This list is merely to illustrate what concepts you may have to learn when using an ORM.

Code that uses an ORM —as the name implies— looks just like any object oriented code. For example, when we want to have retrieve the ISO code of the country that a user's company is registered in, the ORM-based solution would look like:

Users.getById(userId).organization().registeredAddress().country().isoCode

In this example it is not obvious when the query/queries to the database are made, how many queries are made, and what data is retrieved. With an ORM's lazy/eager settings it could be that fetching a "user" also fetches it's organization. The lazy/eager settings are usually set once: and thus cannot be tweaked per query.

While the following SQL-in-string alternative is longer, it is also very clear in what value is being retrieved (only the "iso code"):

db.execute("""
  select c.isoCode
  from User u
  join Organization o  on u.organizationId = o.id
  join Address a       on o.registrationAddressId = a.id
  join Country c       on a.countryId = c.id
  where u.id = $userId
""")

Since ORMs tend to hide queries, analyzing performance bottlenecks of ORM-based code is much harder compared to "just SQL".

When the performance of a relational database backed application is critical, the business logic is usually for some part described in the database queries. In these kind of applications the library used to interact with the database has many call-sites in the application code, and often becomes tightly coupled with the business logic. Changing the library that is used to interact with the database for a large code base quickly becomes prohibitively difficult. Therefor, it is very important to carefully consider your options before making the investment.

So, when dealing with your data base, would you rather learn both...

...and...

...or just the latter?

If not an ORM, then what?

There are other ways to make integrating SQL queries in your language of choice easier. Usually these approaches do not alleviate you from learning SQL, but may provide some of the other benefits ORMs are known for, namely:

Improved type safety.
Some checking of SQL queries (so you get compile errors or failing tests when you change the name of a table but forget to update the query accordingly).
Make it harder to write code vulnerable to SQL injections.
Reduce boilerplate.

These non-ORM options that improve embedding SQL in general purpose languages generally come in two flavors:

Improving on SQL-as-strings.
As an embedded domain specific language (eDSL).

Compared to ORMs both approached are much thinner abstractions. In both cases it is very easy to "see the SQL". The solutions in category two have the potential to provide more features. The solutions in category usually perform slightly better (the eDSL solution are basically query builders: the queries are built at runtime).

Examples of libraries in the first category are:

Jdbi (Java/JVM),
SQLDelight (Kotlin),
sqlc (C),
sqlx (Rust), and
another library by the same name sqlx (Go).

Some of these can tie into your build process with a task that generates code based on the database schema (for this to work a database needs to be available at build time). Depending on the chosen library the generated code may contain:

Tables names, column names, columns types and sometimes also index names; to add some type safety and IDE code completion.
A "record" DTO definition for each (main) table to represents a row in that table.
Methods to reduce the boilerplate code needed to follow an association.

Examples of solutions in this category are:

jOOQ (JVM),
LINQ's method syntax (.NET),
Kysley and Drizzle with SQL-like syntax (TypeScript),

Conclusion

Using an ORM you will:

Find it hard or impossible to write efficient queries:
- easy to write queries in loops,
- often way too much data is being fetched (not easy to reduce this), and
- only a limited set of the database's features are available.
Become locked-in to your ORM library: the API surface is big and it will be used in a large portion of your application's code which results in a large portion of your code needs to be rewritten if you ever want to migrate to another library for accessing the database.

It is said that ORMs are a good fit for very simple CRUD applications. Many programmers learned about ORMs because they "came with the framework": Hibernate in Java EE frameworks like Spring, Entity Framework in (ASP.NET), ActiveRecord (Ruby on Rails), Django ORM in (Django) and Doctrine (Symfony). When the application is small and gets barely any traffic, the ORM does not hurt yet. But as the application grows the ORM becomes a liability.

It seems to me we have optimized these web frameworks for super clean code "Getting Started" guides. Maybe that's the only thing ORMs are good at: super clean code "Getting Started" guides.