In modern Java web application development, choosing the right embedded server and understanding its concurrency model can significantly impact your app's scalability and performance. This post explores what embedded servers are, compares key concurrency models, and offers an inside look at how Undertow — a high-performance embedded server — works under the hood.
What Are Embedded Servers?
Embedded servers are web servers integrated directly into an application instead of operating as standalone services. This allows developers to build self-contained applications that are easier to develop, deploy, and test.
Some popular embedded servers in the Java ecosystem include:
- Tomcat: The classic servlet container, known for robustness.
- Jetty: Flexible and modular, supports both synchronous and asynchronous servlets.
- Undertow: Lightweight, designed for reactive and asynchronous workloads.
- Netty: A low-level asynchronous event-driven network framework, powering custom protocols and reactive HTTP servers.
Concurrency Models: Thread-per-Request vs. Event Loop
How a server manages multiple simultaneous requests drastically affects performance and scalability. Two main concurrency models prevail:
Thread-per-Request Model
- Assigns one thread per incoming request
- Threads handle the full request lifecycle, including any blocking calls (like database queries)
- Concurrency directly limited by thread pool size
- Can lead to thread exhaustion if many requests block simultaneously
- Found in traditional servers like Tomcat and Jetty
- Easier programming model but less scalable under heavy load
Event Loop Model
- Uses a small, fixed number of event loop threads
- Each event loop handles many connections via non-blocking, asynchronous I/O
- Blocking or long-running tasks are offloaded to worker thread pools
- Enables handling thousands of concurrent connections efficiently
- Common in servers like Undertow and Netty
- Requires writing asynchronous or reactive code but yields superior scalability
How Undertow Works Internally
Undertow is designed for ultra-low latency and high throughput, making it well-suited for modern reactive applications.
Undertow’s threading model distinguishes two main types of threads that work together to deliver high performance and scalability:
IO Threads (Event Loop Threads)
- These are the event loop threads in Undertow.
- The number of IO threads is typically set equal to the number of CPU cores.
- Each IO thread runs an event loop responsible for handling all non-blocking network I/O operations on multiple socket connections.
- Every incoming request on a connection is initially handled by the IO thread bound to that connection.
- IO threads never perform blocking operations directly to avoid stalling the event loop.
- Their job is to efficiently multiplex many concurrent connections with minimal threads.
Worker Threads (Core Threads)
- Worker threads come from a dedicated worker thread pool.
- These threads handle blocking or long-running tasks, such as database queries, file operations, or heavy computations.
- When a blocking task arises, Undertow dispatches the work from the IO thread to a worker thread.
- Once completed, the worker thread posts a callback back to the same IO thread to resume request processing.
- The worker thread pool size is usually larger than the number of IO threads, for example, 8 times the IO thread count.
| Thread Type | Function | Thread Count (Typical) | Is it Event Loop? |
|---|---|---|---|
| IO Threads | Run non-blocking event loops, manage connections | Equal to CPU cores | Yes, these are the event loop threads |
| Worker Threads | Handle blocking/long-running operations | Multiple times IO thread count | No |
Here’s what happens under the hood:
- IO Threads Are Event Loops: Undertow runs a group of IO (input/output) threads, each executing an event loop handling multiple socket connections asynchronously.
- Connection Thread Affinity: Each network connection is permanently assigned to a single IO thread, ensuring all logic and IO for that connection stay on one thread, avoiding concurrency hazards.
- Worker Threads for Blocking Work: When the processing requires blocking operations (database calls, file IO), Undertow dispatches work to a worker thread pool to keep IO threads free for non-blocking tasks.
- Callback Mechanism: After completing blocking operations, the worker thread posts a callback to the original IO thread to continue request processing.
- Thread Count Balance: Typically, Undertow configures IO threads equal to CPU cores, with a larger pool of worker threads to handle blocking workloads efficiently.
This hybrid approach preserves responsiveness and throughput by separating lightweight asynchronous IO from heavyweight blocking operations.
Comparing the Models: At a Glance
| Feature | Thread-per-Request | Event Loop (Undertow) |
|---|---|---|
| Threads per request | One | Few event loop threads serve many |
| Handling blocking calls | Threads blocked | Blocking offloaded to worker threads |
| Scalability | Limited by thread pool size | High concurrency with fewer threads |
| Programming complexity | Simpler synchronous code | Requires async/reactive programming |
How Undertow Supports Both Blocking and Non-blocking Paradigms
Undertow is built to handle web requests in two different ways, giving you flexibility depending on your application's needs: non-blocking (asynchronous) and blocking (synchronous) processing.
Non-blocking Mode — The Efficient Event Loop
By default, Undertow uses IO threads known as event loops to manage incoming requests. These event loop threads work asynchronously using non-blocking IO, which means:
- One thread can manage many connections at once without waiting or getting stuck.
- When your request processing involves operations that take time (like database calls), Undertow offloads this work to a special pool of worker threads.
- Once that work finishes, a callback is sent back to the original event loop thread to pick up where it left off and send the response.
This method is very efficient for handling large numbers of concurrent connections because the event loop threads are never blocked waiting for slow operations.
Blocking Mode — Support for Traditional Synchronous Code
Some applications, especially traditional Spring Boot projects using Spring MVC, work in a blocking way:
- Your controller methods and other request handlers run synchronously, meaning they wait (block) until operations finish.
- Undertow supports this by letting you switch into blocking mode using a call to
HttpServerExchange.startBlocking(). - When blocking mode is used, the request handling is moved off the event loop threads and onto worker threads.
- This way, even though your code blocks while waiting for operations, it does not freeze the important event loop threads that keep the server responsive.
What Happens in a Non-Reactive Spring Boot App?
If you use Undertow as the embedded server in a typical non-reactive Spring Boot application:
Incoming requests land on Undertow’s non-blocking IO event loop threads.
Since Spring MVC uses blocking, synchronous request handlers, Undertow automatically dispatches these requests away from the event loop threads to its worker thread pool.
This dispatch ensures that blocking processing happens on worker threads, keeping the event loop threads free to handle other incoming connections efficiently.
The whole “switch to blocking mode” and running on worker threads is managed internally by Undertow as part of its integration with Spring Boot.
This lets you write traditional blocking code without worrying about blocking event loop threads yourself.
This approach strikes a balance between traditional blocking programming and efficient server resource usage.
Why Is This Important?
- The non-blocking event loop model is great for scalability and handling thousands of concurrent connections efficiently.
- The blocking mode keeps compatibility with existing synchronous code and frameworks.
- Undertow’s hybrid model allows you to build on top of proven blocking paradigms without sacrificing event loop performance.
- You get the best of both worlds: wide compatibility and high scalability.
If your application has mostly synchronous, blocking code (like typical Spring MVC controllers), Undertow will safely handle blocking calls on worker threads so the event loop threads remain free. If you want even better scalability with highly concurrent workloads, consider using reactive programming models where you can take full advantage of the event loop’s non-blocking nature.
Top comments (0)