Why Build gRPC Directly on Reactor Netty

Wang Lee — Fri, 07 Aug 2026 14:23:22 +0000

I recently started building grpc-reactor: an experimental gRPC implementation built directly on Reactor Netty HTTP/2. The project uses Protobuf but has zero runtime dependency on grpc-java transport, ClientCall, ServerCall, or StreamObserver.

This is not about proving grpc-java is bad. grpc-java is mature, stable, and covers load balancing, NameResolver, retries, and rich observability. This project answers a different question: if your programming model is already Reactor, can Mono and Flux flow all the way from the generated API down to the HTTP/2 stream, without adapting between two async abstractions?

This post is based on JDK 25, Gradle 9.2.1, Reactor 3.8.6, Reactor Netty 1.3.6, and Netty 4.2.15.Final. The main branch has completed Stage 0 through Stage 10: beyond protocol, four RPC cardinalities, production transport, DNS, codegen, standard services, and Stage 9 hardening, Stage 10 adds an optional load-balancing extensions artifact for grpclb, RLS, load reporting, and ORCA.

The API Goal Is Not Wrapping StreamObserver

Protobuf methods have four cardinalities. The target API maps them directly:

Protobuf Method	Reactor Signature
unary	`Mono<Resp> method(Mono<Req>)`
server streaming	`Flux<Resp> method(Mono<Req>)`
client streaming	`Mono<Resp> method(Flux<Req>)`
bidirectional streaming	`Flux<Resp> method(Flux<Req>)`

Why does gRPC define four instead of just unary? It's essentially a 2x2 combination where request and response each independently choose "single value or stream":

	Single Response	Streaming Response
Single Request	unary	server streaming
Streaming Request	client streaming	bidirectional

They solve different scenarios: unary covers classic request-response; server streaming handles server push (event subscriptions, large paginated pulls); client streaming handles bulk uploads (file chunks, batch writes); bidirectional streaming handles real-time two-way communication (chat, collaborative editing). These aren't invented patterns — HTTP/2 streams are inherently full-duplex. A single connection can multiplex hundreds of concurrent streams, with both request and response sending multiple data frames independently. gRPC elevates this transport capability to first-class API semantics: not a variant of chunked transfer encoding, but compiler-level type checking that callers and implementers match.

The low-level transport retains a single unified model:

Flux<Req> -> HTTP/2 stream -> Flux<Resp>

Generated code applies cardinality checks like single() at the boundary. This avoids implementing four separate network logic paths for four RPC types, while explicitly rejecting empty requests, duplicate requests, or duplicate responses in unary calls.

Compatibility Targets the Wire Protocol

The project's compatibility target is the public gRPC over HTTP/2 protocol spec, not grpc-java internal APIs:

generated Reactor API
        |
call dispatcher / service registry
        |
marshaller + message framer/deframer
        |
metadata + status + deadline
        |
Reactor Netty HTTP/2 stream

Each RPC corresponds to one HTTP/2 stream. Requests start with a HEADERS frame containing at minimum:

:method      POST
:path        /<package.Service>/<Method>
content-type application/grpc+proto
te           trailers
grpc-timeout <relative timeout, e.g. 100m for 100ms>

Message bodies use Length-Prefixed-Message format encapsulated in DATA frames — each Protobuf message is preceded by a five-byte envelope (1 byte compression flag + 4 bytes big-endian length). A single DATA frame may contain multiple gRPC messages, and a large message may span multiple DATA frames.

Why does gRPC require HTTP trailers? This is one of the protocol's most counterintuitive designs. HTTP status codes are nearly useless for gRPC — the protocol requires the HTTP layer to always return 200, with the actual call result (grpc-status and grpc-message) placed in trailers. The reason: in streaming scenarios, the server may have already sent thousands of messages, and whether it ultimately succeeded or failed can only be determined after processing the last piece of data. HTTP headers are sent before the body and cannot carry this posterior result. Trailers are the only mechanism in HTTP/2 that can append metadata after the body.

The protocol also defines trailers-only mode: when the server can determine failure before reading the body (e.g., path not found, authentication failed), grpc-status is returned directly in response headers without sending a body, saving one round-trip. Clients must check both headers and trailers to correctly extract the final status.

Modules Split Along Protocol Boundaries

The project currently has nine modules:

grpc-reactor-protocol      → Transport-independent protocol primitives
grpc-reactor-transport     → Reactor Netty HTTP/2 mapping
grpc-reactor-codegen       → Protoc plugin, generates Reactor stubs
grpc-reactor-gradle-plugin → Gradle integration
grpc-reactor-maven-plugin  → Maven generate-sources integration
grpc-reactor-services      → Optional Health, Reflection & Channelz standard services
grpc-reactor-binlog        → Optional, bounded canonical binary logging
grpc-reactor-lb-extensions → Optional grpclb, RLS, load reporting & ORCA
grpc-reactor-interop-test  → grpc-java compatibility tests

protocol handles only transport-independent values and codecs: message framing, metadata, status, timeout, compression, and protobuf marshallers. It depends on Reactor Core and Netty Buffer but not Reactor Netty — meaning the protocol layer can be tested independently without real HTTP/2 connections.

transport maps the protocol onto Reactor Netty HTTP/2, handling client, server, service registry, and per-call context.

codegen consumes Protobuf CodeGeneratorRequest and generates type-safe Reactor client and service binders.

services uses the same codegen to generate canonical gRPC service bindings, implementing Health v1, Reflection v1, and Channelz v1 on top of transport's immutable descriptor/diagnostics snapshots. This module requires explicit registration — adding the dependency alone won't expose management endpoints.

binlog is also an explicitly-enabled standalone module. It captures canonical binary-log v1 events via a transport interceptor, controlling information exposure and memory limits through metadata/message truncation, sensitive key redaction, and fixed-capacity sinks.

interop-test introduces grpc-java in test scope. grpc-java serves as the compatibility oracle here, never entering the project runtime.

Dependency Selection and Version Pinning

The runtime dependency chain is intentionally kept short:

Dependency	Version	Purpose
Reactor Core	3.8.6	Mono/Flux programming model
Reactor Netty	1.3.6	HTTP/2 client/server
Netty	4.2.15	ByteBuf, HTTP/2 codec
Protobuf-java	4.35.1	Message serialization

The project does not introduce Spring, Micrometer, or any DI framework. Tests use JUnit 6.1.2 and Reactor Test's StepVerifier. grpc-java 1.82.2 appears only in interop-test's test classpath with zero runtime intrusion.

Versions are centrally managed via gradle/libs.versions.toml with Gradle dependency locking generating lock files, ensuring fully reproducible builds across machines.

Why the Protocol Layer Must Come First

It's tempting to start with "spin up an HTTP/2 Server" — you quickly get an echo demo, but it pushes the truly difficult problems to later:

DATA may split in the middle of the five-byte gRPC header;
A single DATA buffer may contain multiple messages;
Metadata allows duplicate keys, and binary values require Base64;
Timeout wire values are at most eight digits with a unit suffix;
Final status comes from trailers;
ByteBuf must be released on success, failure, and cancellation paths;
Reactive Streams demand counts messages, HTTP/2 flow control counts bytes.

Therefore the project progresses by Stage: first lock down the build and interop fixtures, then complete the protocol layer, then implement unary transport, streaming, production features, and codegen. Each Stage has executable exit criteria — "classes have been created" is not a completion standard.

Stage 0: Build Baseline and Interop Fixture

Before writing any protocol code, Stage 0 solves "how to prove code is correct":

JDK 25 compilation — The project uses -Xlint:all -parameters -encoding UTF-8 for strict compilation. All warnings are compilation errors; silent suppression is not allowed. JDK 25 was chosen to validate Netty and Protobuf compatibility on the latest JVM early.

Spotless formatting — Unified Eclipse formatter config plus ktlint. spotlessCheck is the first gate in CI. This eliminates all code review discussions about formatting.

interop.proto test fixture — Defines a test service covering all four RPC cardinalities:

service InteropTestService {
  rpc Unary (TestRequest) returns (TestResponse);
  rpc ServerStreaming (TestRequest) returns (stream TestResponse);
  rpc ClientStreaming (stream TestRequest) returns (TestResponse);
  rpc BidirectionalStreaming (stream TestRequest) returns (stream TestResponse);
}

GrpcJavaFixture — In the interop-test module, a test utility class starts both a grpc-java server and client, providing start() / close() lifecycle. Through it, bidirectional verification is possible: Reactor client calls grpc-java server, and grpc-java client calls Reactor server. Both use the same .proto generated code, ensuring wire compatibility.

Utilities:

FreePorts: Allocates independent ports for each test, avoiding parallel test conflicts;
TlsTestCertificates: Pre-generates self-signed certificates for subsequent TLS tests;
LeakDetection: Integrates Netty's ResourceLeakDetector, ensuring ByteBuf leaks are immediately exposed in tests.

Stage 0 exit criteria: ./gradlew clean test passes from fresh checkout, CI is green on Linux + Java 25, protoc generation is deterministically reproducible, grpc-java fixture communicates bidirectionally.

Staged Verification Strategy

The project progresses through 12 Stages, each with clear goals, executable exit criteria, and regression coverage:

Stage	Goal	Key Deliverable
0	Build baseline	CI, formatting, interop fixture
1	Protocol foundation	Frame codec, metadata, status, timeout, compression
2	Unary transport	End-to-end h2c unary call
3	Server streaming	Multi-message response stream
4	Full cardinality	Client streaming + bidirectional
5	Production transport	TLS, gzip, deadline, GOAWAY, connection pool, keepalive
6	Name resolution	DNS, subchannel, pick_first, round_robin
7	Codegen & build integration	Protoc plugin, descriptor registry, Gradle/Maven plugins
8	Standard services	Health v1, Reflection v1, Channelz v1
9	Operations & hardening	Interceptor, observer, binlog, canonical smoke, fuzz/churn
10	Load-balancing extensions	grpclb, RLS, load reporter, ORCA
11	Diagnostics (planned)	Channelz v2 over the bounded diagnostics registry

Each Stage completion requires: ./gradlew clean spotlessCheck test --no-daemon passes in full, and all prior Stage tests continue running as regression. This means when Stage 4 completes, Stage 2's unary tests are still green.

Where Reactor semantics differ from grpc-java

The wire contract is shared, but the application contracts are not. The tests therefore compare the two implementations at the protocol boundary and test each runtime's lifecycle rules separately:

Concern	Reactor contract	grpc-java contract	What the tests must prove
Demand	`Subscription.request(n)` counts decoded messages; transport demand must also respect HTTP/2 byte windows	inbound flow is controlled through `ClientCall.request(n)` / readiness callbacks	no response DATA is delivered before downstream demand, and coalesced frames do not bypass demand
Cancellation	disposing a `Mono`/`Flux` cancels both application publishers and resets an open stream	`ClientCall.cancel` or `Context` cancellation reaches `StreamObserver` callbacks	both sides stop, exactly one terminal signal wins, and late DATA/trailers are ignored
Trailers	success trailers are retained in the call result; non-OK trailers become `GrpcException.trailers()`	trailers are exposed through `ClientCall.Listener#onClose` or `StatusRuntimeException#getTrailers()`	`grpc-status`, `grpc-message`, and custom trailers survive in both directions, including trailers-only errors
Buffer ownership	Netty `ByteBuf` is reference-counted and must be released on success, error, and cancel; decoded protobuf values cross the API boundary	generated protobuf messages hide transport buffers from application code	every rejected, partial, compressed, and cancelled frame reaches `refCnt() == 0`

For example, the Reactor demand test starts with zero demand and asks for one response at a time:

StepVerifier.create(client.bidirectionalStreaming(BIDI, requests), 0)
        .thenRequest(1)
        .expectNext(first)
        .thenRequest(2)
        .expectNext(second, third)
        .expectComplete()
        .verify();

The corresponding grpc-java test uses StreamObserver callbacks and completion signals. The messages and trailers must be wire-compatible, but it would be incorrect to describe the two tests as asserting the same backpressure mechanism. This distinction is why the project keeps both cross-runtime interop and runtime-specific failure tests.

The repository now also has two server-streaming cancellation interop cases in ServerStreamingInteroperabilityTest: a Reactor client receives one response from a grpc-java server and cancels, while a grpc-java client cancels a Reactor server after its first response. Both sides wait for the peer's cancellation callback under paranoid leak detection. The lower-level GrpcFrameCodecTest keeps the direct ownership assertion by checking that cancelled and partial inputs finish with refCnt() == 0.

The essential cancellation paths are deliberately small:

// Reactor client: cancel the HTTP/2 call after the first response.
TestResponse first = reactorStub
        .serverStreaming(Mono.just(request(5)))
        .take(1)
        .single()
        .block(Duration.ofSeconds(5));

// grpc-java client: cancel from the response callback.
@Override
public void onNext(TestResponse response) {
    requestStream.cancel("cancel after first response", null);
}

On the grpc-java server side, the test installs setOnCancelHandler; on the Reactor server side, the response Flux uses doOnCancel. The complete test, including latches, peer status assertions, shutdown, and leak-detection scope, is in ServerStreamingInteroperabilityTest.java.

Current Verification Results

The project uses this unified gate that simultaneously checks formatting, compilation, protocol tests, transport tests, and grpc-java interop tests:

./gradlew clean spotlessCheck test --no-daemon

This command passes as of this writing. The gate now includes the optional Stage 10 load-balancing extension tests in addition to protocol, transport, codegen, services, binary-log, and grpc-java interop tests. The dependency baseline is Protobuf 4.35.1, grpc-java 1.82.2, and JUnit 6.1.2. Under JDK 25, you'll still see Protobuf Unsafe, Netty/Gradle native access, and Gradle deprecated feature warnings — they don't affect test results but need ongoing tracking through future JDK and Gradle upgrades.

Subsequent posts will continue discussing the five-byte gRPC message envelope, ByteBuf ownership, four RPC cardinalities, production transport, DNS, code generation, standard management services, and how to integrate interceptors, observation, and binary logging without breaking Reactive Streams semantics.

DEV Community: Wang Lee