From Thread Pools to Virtual Threads: How Spring Boot on Java 21 Scales in Production

Layoffs, hiring freezes, uncertainty—that’s the current vibe. So I decided to revisit basics. While I was noodling on multithreading, Spring Boot, and Java 21, it clicked: Spring’s request handling has evolved from one‑thread‑per‑request to tuned pools to reactive loops, and now to virtual threads on the servlet stack. That shift changes how we think about scaling in production.

Spring Boot doesn't actually handle incoming HTTP requests itself. It embeds a server underneath, Tomcat by default, though Jetty and Undertow are options too, and that embedded server is what accepts each connection and decides which thread runs your controller code.

How that embedded server assigns threads to requests has changed more than once over the years. Each shift fixed a real limitation in the one before it, and introduced a new trade-off. Here's how we got from one request per thread to virtual threads.

One request, one thread

In the original servlet model, every incoming request got its own dedicated thread for its whole lifecycle. At the raw Java level, that's:

new Thread(() -> handleRequest(request)).start();

Easy to reason about, but a thread isn't free. Each one costs roughly a megabyte of stack space, plus OS-level context-switching overhead. A few thousand concurrent requests, and the server spends more time managing threads than doing actual work.

Thread pools took over, and mostly still are the default

Instead of spinning up a new thread per request, the embedded server reuses a fixed pool:

 ExecutorService pool = Executors.newFixedThreadPool(200);
 pool.submit(() -> handleRequest(request));

This is what's running under your Spring Boot app right now unless you've changed it—Tomcat's embedded pool, 200 threads out of the box. You never write this code the container manages it but when your monitoring shows requests queuing up because all 200 threads are tied up in slow I/O, bumping that pool size to 400 or 500 becomes your first line of defense. That's why "increase the Tomcat thread pool size" is a real production tuning lever.
The catch: a thread blocked on a slow database call or downstream API still occupies a pool slot, doing nothing. Enough slow requests at once, and the whole pool backs up.

Reactive programming tried removing the wait entirely

Spring's reactive stack, WebFlux, usually running on an embedded Netty server instead of Tomcat, took a different approach: never let a thread sit idle waiting on I/O in the first place.

Mono<Response> handle(Request request) {
    return externalCall(request).map(this::process);
}

When externalCall() fires, the thread doesn't wait for the response—it registers a callback and immediately moves on to handle the next request. When the external service eventually responds, a thread (possibly a different one) picks up the result and continues the chain. One thread can juggle dozens or hundreds of in-flight requests this way, because it's never stuck waiting.
Throughput genuinely improves—a small number of threads can handle what would've required 10x as many in the blocking model.

But the code stops reading like normal Java. Straightforward, sequential logic turns into chained operators, and a stack trace no longer maps cleanly to your actual call flow. A lot of teams adopted this out of necessity, not love.

Caveat: The virtual thread switch below is for servlet containers (Tomcat/Jetty). If you’re on WebFlux with Netty, you’re already on an event-loop model; virtual threads don’t apply there unless you switch to the servlet stack.

Virtual threads changed the trade-off itself

Java 21's virtual threads earn the "game changer" label for what they let you keep, not just for being fast.

Thread.ofVirtual().start(() -> handleRequest(request));

A virtual thread is cheap, kilobytes instead of megabytes, and when it blocks on I/O, it steps aside and frees the real OS thread underneath instead of holding onto it. If you're on Spring Boot 3.2+, you get this without switching to Netty or rewriting a single controller: setting
spring.threads.virtual.enabled=true tells the embedded Tomcat to hand out virtual threads instead of platform threads for each request.

That's the real unlock: the scalability reactive programming was built for, without giving up plain, sequential, readable code.

The trade-off you used to face whether to write readable code or scalable code isn't forced anymore. Though scalability still depends on your database connection pool, downstream services, and avoiding synchronized blocks (which quietly "pin" virtual threads and bring back the exact bottleneck you were trying to escape).

If you’re evaluating virtual threads, here’s one test that matters: under load, run a thread dump and search for “pinned”. For example:

jcmd <pid> Thread.dump_to_file -filename /tmp/threads.txt

If you see many pinned virtual threads, you’ve got some refactoring to do before virtual threads help.

Quick start checklist:

• Enable: spring.threads.virtual.enabled=true (servlet apps on Spring Boot 3.2+).
• Bound resources: ensure DB and HTTP clients have explicit concurrency limits to prevent overload.
• Validate: load test with realistic latency; capture p95/p99 and thread dumps; search for “pinned”.
• Refine: remove synchronized hotspots and long native calls that cause pinning.

Times are rough, so I’m shoring up fundamentals. What topics are you revisiting these days—concurrency, SQL tuning, backpressure, JVM internals? Share a link or cheat sheet that helped.