Bound Spring MVC's Async Executor or Pay for It Under Load

This article was originally published on Jo4 Blog.

Spring MVC's default async task executor spawns a brand new thread for every async request. No queue. No ceiling. No back-pressure. Spring itself logs a WARN on first use: "This executor is not suitable for production use under load." That's a load-bearing warning, and most teams scroll past it.

We learned the hard way and ended up wiring a bounded ThreadPoolTaskExecutor, a typed rejection path, and a Micrometer saturation gauge. Here's the whole arc.

What Spring MVC Defaults To

When a controller method returns a DeferredResult, Callable, WebAsyncTask, Flux, or Mono, Spring MVC needs an executor to run the async dispatch. If you don't configure one, you get SimpleAsyncTaskExecutor.

SimpleAsyncTaskExecutor does exactly what its name says: every submission spawns a fresh thread. There is no pool. There is no queue. There is no rejection policy, because there is no ceiling to reject against. Send it a thousand concurrent requests and it will cheerfully start a thousand threads, each with its own ~1 MB stack, and watch your container OOM.

Spring 6 logs this WARN on first use:

This executor is not suitable for production use under load. Consider configuring a ThreadPoolTaskExecutor via WebMvcConfigurer.configureAsyncSupport.

If you ignore the warning, the failure mode is exotic: latency stays fine until traffic crosses some invisible threshold, at which point the JVM either runs out of native threads (Linux pthread_create returns EAGAIN) or runs out of heap because every parked thread is holding stack frames and a Tomcat request scope.

Why That's Wrong For Real Apps

Three patterns push you onto this executor whether you realize it or not:

1. DeferredResult / Callable / WebAsyncTask returns. Anyone using the classic async-servlet pattern.
2. Flux / Mono returns from a Spring MVC controller. Spring MVC (not WebFlux) bridges reactive types onto the async-dispatch path. Every chunk emission round-trips through the executor.
3. SSE endpoints (SseEmitter). Long-lived streams that hold a dispatch thread for the entire lifetime of the connection.

We ship all three. Our analytics SSE stream in particular pins one async thread per subscriber for up to five minutes (our ASYNC_TIMEOUT). With SimpleAsyncTaskExecutor, 500 concurrent dashboard users would mean 500 unbounded threads with no warning sign before the container falls over.

Bounded Pool Setup

The fix is a real ThreadPoolTaskExecutor with explicit core, max, and queue settings. Here's the actual bean from WebMvcAsyncConfig.java:

@Bean(name = "mvcAsyncExecutor", destroyMethod = "shutdown")
public ThreadPoolTaskExecutor mvcAsyncExecutor() {
    ThreadPoolTaskExecutor exec = new ThreadPoolTaskExecutor();
    exec.setCorePoolSize(8);
    exec.setMaxPoolSize(200);
    // queueCapacity = 0 → Spring uses SynchronousQueue: direct hand-off, no buffering.
    exec.setQueueCapacity(0);
    exec.setThreadNamePrefix("jo4-mvc-async-");
    exec.setRejectedExecutionHandler(new ThreadPoolExecutor.AbortPolicy());
    exec.setWaitForTasksToCompleteOnShutdown(true);
    exec.setAwaitTerminationSeconds(30);
    exec.initialize();
    return exec;
}

@Override
public void configureAsyncSupport(AsyncSupportConfigurer configurer) {
    configurer.setDefaultTimeout(ASYNC_TIMEOUT.toMillis());
    configurer.setTaskExecutor(mvcAsyncExecutor());
}

A few decisions worth calling out, because the obvious choices are wrong for long-lived async:

queueCapacity = 0. Spring's ThreadPoolTaskExecutor interprets queueCapacity = 0 as a SynchronousQueue — a direct hand-off with no buffering. The instinct is to "set a big queue, it's free." For short-lived tasks that's right. For SSE streams (which hold the thread for up to 5 minutes), a non-zero queue would park new connections waiting for an existing core thread to free — effectively "hang the client until someone else disconnects." With a 0-capacity queue, the pool grows from corePoolSize=8 toward maxPoolSize=200 as new subscribers arrive, and only true saturation hits the rejection handler.

AbortPolicy, not CallerRunsPolicy. This one we learned by pain. Our first attempt used CallerRunsPolicy (the seemingly-graceful "if the pool is full, run inline on the caller's thread"). Under load that pins the Tomcat request thread executing the task synchronously — which turned multi-minute Playwright teardowns into multi-minute hangs because the test runner's "close all contexts" call was racing pool saturation. Fail-fast with AbortPolicy is preferable: back-pressure surfaces as a RejectedExecutionException that the caller can translate into an HTTP 503, and the Tomcat thread is freed immediately for the next request.

waitForTasksToCompleteOnShutdown = true + awaitTerminationSeconds = 30. SSE streams need time to flush their final event and let the client see the close cleanly. Hard-killing the pool on shutdown means subscribers get a TCP reset instead of a proper stream-end signal.

destroyMethod = "shutdown". Pairs the bean lifecycle with the executor lifecycle so Spring's context shutdown drains the pool. Without it you leak threads on every context refresh.

Rejection With Meaning

AbortPolicy throws RejectedExecutionException. By default that propagates out of the async-dispatch machinery, hits the generic exception handler, and becomes a 500 Internal Server Error — which is a lie. It's not an internal error; it's capacity. Calling clients, CDN edges, and retry-aware HTTP libraries all behave differently against 500 vs 503: 500 says "give up, this is broken"; 503 says "back off and try again."

We give it a typed home in our ErrorCode enum:

SERVICE_SATURATED("SERVICE_SATURATED", "Server is temporarily at capacity. Please retry."),

And we catch it explicitly in GlobalExceptionHandler:

@ExceptionHandler(RejectedExecutionException.class)
public ResponseEntity<ResponseBody<EmptyResponse>> handleRejectedExecution(RejectedExecutionException ex) {
    String requestUri = getRequestUri();
    log.warn("Async executor saturated on {}: {}", requestUri, ex.getMessage());

    Fault fault = Fault.builder()
            .code(ErrorCode.SERVICE_SATURATED.getCode())
            .description(ErrorCode.SERVICE_SATURATED.getMessage())
            .build();

    ResponseBody<EmptyResponse> body = ResponseBody.<EmptyResponse>builder()
            .requestId(UUID.randomUUID().toString())
            .faults(List.of(fault))
            .build();

    return ResponseEntity.status(HttpStatus.SERVICE_UNAVAILABLE)
            .header(HttpHeaders.RETRY_AFTER, "5")
            .body(body);
}

Three things matter here. First, the response is 503 SERVICE_UNAVAILABLE — semantically correct for "we're at capacity," not "we're broken." Second, the Retry-After: 5 header is honored by fetch retry middlewares, the AWS SDK, and most CDN edges, which means callers self-throttle without us writing client code. Third, the structured fault carries SERVICE_SATURATED so the SPA can render a friendly "we're at capacity, try again" toast instead of a generic 5xx splash.

The mapping back to HTTP status lives in mapErrorCodeToStatus:

case SERVICE_SATURATED -> HttpStatus.SERVICE_UNAVAILABLE;

Without that explicit handler, the rejection would fall through to the catch-all Exception handler and become a 500. Same root cause, completely different operational signal.

Make It Observable

Bounded pools are useless if you can't see how close you are to the wall. We expose five gauges via a MeterBinder, scraped from /actuator/prometheus:

@Bean
public MeterBinder mvcAsyncExecutorMetrics(ThreadPoolTaskExecutor mvcAsyncExecutor) {
    return registry -> {
        Gauge.builder("mvc.async.executor.active",
                mvcAsyncExecutor, ThreadPoolTaskExecutor::getActiveCount)
            .description("Threads currently executing tasks")
            .baseUnit("threads")
            .register(registry);

        Gauge.builder("mvc.async.executor.pool.size",
                mvcAsyncExecutor, ThreadPoolTaskExecutor::getPoolSize)
            .description("Current pool size (grows from core toward max under load)")
            .baseUnit("threads")
            .register(registry);

        Gauge.builder("mvc.async.executor.pool.max",
                mvcAsyncExecutor, e -> (double) e.getMaxPoolSize())
            .description("Configured maxPoolSize ceiling")
            .baseUnit("threads")
            .register(registry);

        Gauge.builder("mvc.async.executor.queue.size",
                mvcAsyncExecutor, e -> {
                    ThreadPoolExecutor tpe = e.getThreadPoolExecutor();
                    return tpe == null ? 0 : tpe.getQueue().size();
                })
            .description("Queued tasks waiting for a thread")
            .register(registry);

        // The headline metric: active / max. 0.8 = alert; 1.0 = saturated.
        Gauge.builder("mvc.async.executor.saturation",
                mvcAsyncExecutor, e -> {
                    int max = e.getMaxPoolSize();
                    return max == 0 ? 0.0 : (double) e.getActiveCount() / max;
                })
            .description("Active-to-max ratio (0..1). >0.8 = capacity warning, 1.0 = rejecting")
            .register(registry);
    };
}

The first three (active, pool.size, pool.max) are diagnostic — useful in a Grafana dashboard to see how the pool actually grows. queue.size is included for forward compatibility; with queueCapacity = 0 it's always zero, but if we ever tune the queue we don't want to be the team that forgot to publish the gauge.

mvc.async.executor.saturation is the headline. It's a single derived ratio (active / max) bounded between 0 and 1, which means one alert threshold works regardless of how we resize the pool over time. Cardinality of one, semantics of "how close are we to rejecting" — exactly what a pager wants to wake you up about.

Alerts That Actually Mean Something

Two thresholds, both honest:

• WARN when mvc_async_executor_saturation > 0.8 for 5 minutes. We're at 80% of the ceiling for long enough to mean it's not a spike. Likely time to bump maxPoolSize, investigate slow async handlers, or check if SSE clients are leaking connections.
• CRIT when mvc_async_executor_saturation == 1.0. We're rejecting. SERVICE_SATURATED 503s are going out the door. Page on-call.

The warning level matters because saturation is a leading indicator. Once you're rejecting, customers have already seen 503s. Once you're at 80%, you have minutes to act before that happens — assuming you alert on it. The corollary: a Grafana dashboard with saturation as the headline tile (not request rate, not latency) tells you at a glance whether you're about to have a bad day.

We pair saturation with the Retry-After header on the 503 so even when we are rejecting, well-behaved clients smooth it out and we recover without a thundering herd hammering us the moment the pool drains.

Lessons Learned

• SimpleAsyncTaskExecutor is a development convenience, not a production executor. If you return DeferredResult, Flux, Mono, or use SseEmitter from a Spring MVC controller, configure AsyncSupportConfigurer.setTaskExecutor with a real bounded pool. The default's WARN log message is correct.
• Queue capacity is a function of task duration, not of "more is better." For sub-second tasks, a generous queue smooths spikes. For long-lived tasks like SSE, a queue silently hangs new connections behind old ones. Use queueCapacity = 0 + grow the pool to maxPoolSize.
• AbortPolicy over CallerRunsPolicy for HTTP work. CallerRunsPolicy pins your Tomcat request threads under load and turns a capacity problem into a hung-process problem. Fail fast, return 503.
• Map rejection to a typed error code, not a generic 500. RejectedExecutionException → SERVICE_SATURATED → HTTP 503 + Retry-After tells callers something actionable. 500 tells them nothing.
• Saturation is the gauge worth alerting on. A derived active/max ratio is dimensionless, bounded 0..1, survives pool resizing, and gives you a single threshold to define "we're about to have a bad day."
• The WARN threshold is the one that prevents pages. Alert at 0.8 with 5-minute hysteresis to get warned before you reject. Alert at 1.0 to know you're already rejecting. Both, not either.

---

What's your async-executor war story? Tail-latency, thread leaks, OOM? Drop it in the comments.

Building jo4.io — a URL shortener with analytics for developers who ship.