In modern application architectures, designing a backend that can efficiently handle heavy loads while providing timely responses is crucial. In this blog post, we'll explore how to build a scalable asynchronous backend service using Spring Boot. We'll cover traditional asynchronous processing with DeferredResult and @Async, and then introduce virtual threads (from Project Loom) that allow you to write blocking code in a natural style while still scaling to massive concurrency.
Table of Contents
- Introduction
- Asynchronous Processing in Spring Boot
- Using DeferredResult to Manage Responses
- Customizing Thread Pools for Heavy Loads
- Virtual Threads: Simplifying Blocking Code
- Complete Example
- Conclusion
Introduction
When your backend service needs to handle up to a million concurrent requests, every millisecond counts. Traditional synchronous processing can easily lead to thread exhaustion and increased latency. Asynchronous processing decouples request handling from heavy business logic, ensuring each request is processed independently. In this post, we'll dive into how you can achieve this using Spring Boot's asynchronous support—and how virtual threads can further enhance scalability by letting you write blocking code without the overhead of traditional threads.
Asynchronous Processing in Spring Boot
Spring Boot makes it easy to run tasks asynchronously. By using the @Async annotation on service methods, you can offload heavy processing tasks to a separate thread pool. This approach ensures that your web container (like Tomcat or Netty) isn't blocked by long-running operations.
For example, consider a service method that simulates a heavy computation:
@Service
public class AsyncService {
@Async("taskExecutor")
public CompletableFuture<String> processTask(String input) {
try {
// Simulate a heavy process, e.g., I/O or computation.
Thread.sleep(3000); // 3-second delay
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
return CompletableFuture.completedFuture("Processing interrupted");
}
return CompletableFuture.completedFuture("Processed input: " + input);
}
}
In this example, the @Async annotation dispatches the method to a separate thread, freeing up the web request thread almost immediately.
Using DeferredResult to Manage Responses
While asynchronous processing decouples the heavy logic from the request thread, you still need a way to provide a response once the processing is complete. This is where Spring's DeferredResult comes in.
When a request is made, you immediately return a DeferredResult from your controller. This object holds the eventual response and is updated once the asynchronous task finishes. Here’s how you can use it:
@RestController
public class AsyncController {
@Autowired
private AsyncService asyncService;
@GetMapping("/process")
public DeferredResult<ResponseEntity<String>> process(@RequestParam(value = "input", defaultValue = "default") String input) {
// Set a timeout of 5000ms (5 seconds)
DeferredResult<ResponseEntity<String>> deferredResult = new DeferredResult<>(5000L);
CompletableFuture<String> futureResult = asyncService.processTask(input);
futureResult.thenAccept(result ->
deferredResult.setResult(ResponseEntity.ok(result))
).exceptionally(ex -> {
deferredResult.setErrorResult(
ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body("Error processing request")
);
return null;
});
return deferredResult;
}
}
With this pattern:
-
Individual Request Contexts: Each HTTP request creates a unique
DeferredResult. -
Asynchronous Invocation: The heavy processing runs on a separate thread (or virtual thread, as we'll see later) and updates its corresponding
DeferredResultwhen done. - Timeout Management: If processing exceeds the 5-second timeout, you can handle it gracefully.
Customizing Thread Pools for Heavy Loads
Under heavy loads, the choice of thread pool is critical. Configuring a custom Executor lets you fine-tune the number of threads, queue capacity, and more:
@Configuration
public class AsyncConfig {
@Bean(name = "taskExecutor")
public Executor taskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(10); // Minimum threads
executor.setMaxPoolSize(50); // Maximum threads
executor.setQueueCapacity(100); // Queue capacity before new threads are created
executor.setThreadNamePrefix("Async-");
executor.initialize();
return executor;
}
}
This custom executor ensures that your asynchronous tasks are efficiently scheduled and executed, even under the strain of a million requests. If you don’t configure an executor that leverages virtual threads (as we’ll cover next), your code will run on these heavier platform threads.
Virtual Threads: Simplifying Blocking Code
Traditional asynchronous design forces you to avoid blocking calls or rewrite your code in a non-blocking, reactive style. However, not every use case requires a fully reactive model. With virtual threads—a feature introduced with Project Loom—you can write blocking code in a natural, synchronous style while still achieving high concurrency.
What Are Virtual Threads?
- Lightweight Concurrency: Virtual threads are much lighter than platform threads, allowing you to create thousands or even millions without incurring significant overhead.
-
Simplified Coding: You can write blocking operations (like
Thread.sleep()) directly. Virtual threads manage the underlying blocking efficiently.
Configuring a Virtual Thread Executor
Here’s how to create an executor that uses virtual threads:
@Configuration
public class AsyncConfig {
@Bean(name = "virtualThreadExecutor")
public Executor virtualThreadExecutor() {
// Creates an executor that spawns a new virtual thread per task.
return Executors.newVirtualThreadPerTaskExecutor();
}
}
Using Virtual Threads in Your Service
You simply annotate your service method to use the virtual thread executor:
@Service
public class AsyncService {
@Async("virtualThreadExecutor")
public CompletableFuture<String> processTask(String input) {
try {
// Blocking code written in a natural style
Thread.sleep(3000); // 3-second delay
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
return CompletableFuture.completedFuture("Processing interrupted");
}
return CompletableFuture.completedFuture("Processed input: " + input);
}
}
This means that if you don't explicitly use virtual threads, your asynchronous tasks will run on traditional platform threads. Virtual threads let you write blocking code without worrying about the resource overhead typically associated with them.
Complete Example
Below is a complete Spring Boot project example that combines everything we've discussed—from asynchronous processing with DeferredResult to leveraging virtual threads for scalable blocking code.
Main Application Class
package com.example.asyncdemo;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.scheduling.annotation.EnableAsync;
@SpringBootApplication
@EnableAsync
public class AsyncDemoApplication {
public static void main(String[] args) {
SpringApplication.run(AsyncDemoApplication.class, args);
}
}
Virtual Thread Executor Configuration
package com.example.asyncdemo.config;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
@Configuration
public class AsyncConfig {
@Bean(name = "virtualThreadExecutor")
public Executor virtualThreadExecutor() {
return Executors.newVirtualThreadPerTaskExecutor();
}
}
Asynchronous Service with Blocking Code
package com.example.asyncdemo.service;
import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;
import java.util.concurrent.CompletableFuture;
@Service
public class AsyncService {
@Async("virtualThreadExecutor")
public CompletableFuture<String> processTask(String input) {
try {
Thread.sleep(3000); // Simulate blocking call
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
return CompletableFuture.completedFuture("Processing interrupted");
}
return CompletableFuture.completedFuture("Processed input: " + input);
}
}
REST Controller Using DeferredResult
package com.example.asyncdemo.controller;
import com.example.asyncdemo.service.AsyncService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.context.request.async.DeferredResult;
import java.util.concurrent.CompletableFuture;
@RestController
public class AsyncController {
@Autowired
private AsyncService asyncService;
@GetMapping("/process")
public DeferredResult<ResponseEntity<String>> process(@RequestParam(value = "input", defaultValue = "default") String input) {
// Configure DeferredResult with a 5-second timeout
DeferredResult<ResponseEntity<String>> deferredResult = new DeferredResult<>(5000L);
CompletableFuture<String> futureResult = asyncService.processTask(input);
futureResult.thenAccept(result ->
deferredResult.setResult(ResponseEntity.ok(result))
).exceptionally(ex -> {
deferredResult.setErrorResult(
ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body("Error processing request")
);
return null;
});
return deferredResult;
}
}
Conclusion
In this post, we've explored how to build a scalable backend service with Spring Boot that can handle massive loads asynchronously. By leveraging:
-
DeferredResultto maintain unique request contexts, -
@Asyncto decouple heavy processing from the request thread, - Custom executors to manage thread pools, and
- Virtual threads to write natural blocking code while scaling concurrency,
you can design a robust service that responds promptly even under high load. If you choose not to use virtual threads, your asynchronous tasks will run on traditional platform threads, which are heavier. Virtual threads offer a game-changing approach by simplifying your code without sacrificing performance.
Happy coding, and may your backends scale gracefully!
Top comments (0)