Threads Are Not Connections: A Spring Boot Concurrency Incident

Gaddeti Rohit Babu — Fri, 16 Oct 2026 10:17:40 +0000

Description:

A Spring Boot service slowed down under load because API calls and database calls shared one thread pool. How the team found it, fixed it, and why virtual threads would not have helped.

Load went up, and the service slowed down with it. p99 latency climbed from 400 ms to 18 s. CPU sat around 20%. Connection timeout errors started showing up in the logs.

A slow service usually has something working too hard. This one did not. Requests were piling up, yet very little real work was getting done. Most of the time was going to waiting, and the engineers had to find out what for.

What the service was doing

Each unit of work was a short chain of blocking calls. Some went to external APIs. Some went to the database. A single unit could touch both, more than once, one after another.

The service ran many units at the same time with CompletableFuture. Every stage of every unit ran on one shared thread pool of 100 threads. Database access went through a Hikari connection pool with 20 connections.

Both kinds of calls block the thread that makes them. A thread that sends an HTTP request sits idle until the response comes back. A thread that needs the database first asks Hikari for a connection. If none is free, it waits. Then it waits again for the query. While a thread waits, it does no useful work. It is also unavailable for anything else.

So the 100 threads were one shared budget for two different kinds of waiting. When the APIs were fast and connections were free, the budget was enough. When either side slowed down, it spent the budget for both.

That design was the team's own choice, and it was easy to set up. The problem was that 100 did not describe any real limit in the system. It was just a number.

Finding the cause

The team took thread dumps during the slowdown and lined them up with Hikari's metrics. Nearly every pool thread was blocked. Some were waiting on HTTP responses. Many more were parked inside getConnection. Hikari showed all 20 connections in use and up to 60 threads waiting for one.

A thread could get stuck in two ways, and each made the other worse.

The first was slow API calls. Under load, external calls took longer and held their threads longer. That left fewer threads for every other stage, including database work, which sat in the executor's queue.

The second was the database. With 100 threads and only 20 connections, most of the pool could end up waiting inside Hikari. A thread waiting for a connection is just as unavailable as one waiting on HTTP. API stages then queued behind them.

Because both kinds of work drew from the same threads, slowness on one side became delay on the other. Every unit of work waited in the same queue, whatever resource it needed next.

The root cause was a mismatch between thread demand and available connections. The thread pool and the connection pool had been sized as two separate settings. They should have been one decision. This also explained the low CPU. Blocked threads use almost none.

Transaction scope made it worse. One code path opened a transaction, called an external API, and wrote the result before committing. The connection stayed checked out for the whole API call. In that path, every slow response took a connection out of the pool while no query was running.

The fix: one executor per resource

The team split the shared pool into two executors. One runs database work. The other runs external API calls. Each stage now runs on the executor for the resource it blocks on.

@Configuration
class ExecutorConfig {
    // 18 = Hikari maximum-pool-size (20) minus 2 for request-path queries
    @Bean ThreadPoolTaskExecutor dbExecutor()  { return pool("db-", 18); }
    // 80 = ~200 calls/s x 0.3 s p95 latency = 60, plus headroom
    @Bean ThreadPoolTaskExecutor apiExecutor() { return pool("api-", 80); }

    private static ThreadPoolTaskExecutor pool(String prefix, int size) {
        var ex = new ThreadPoolTaskExecutor();
        ex.setCorePoolSize(size);
        ex.setMaxPoolSize(size);
        ex.setThreadNamePrefix(prefix);
        return ex;
    }
}

The DB executor was resized to align with the Hikari pool. A database thread without a connection can only wait, so threads beyond the connection count add waiting and nothing else. With the sizes aligned, extra database work waits in the executor queue, where it holds no thread and no connection. The executor got 18 threads rather than 20, because a few request-path queries use the same Hikari pool outside the executor.

The API executor needed a different method, and Little's Law gives it. In plain words, the calls in flight equal the calls started per second multiplied by the seconds each call takes. At peak, the service handled about 100 units of work per second, each making two API calls. That is about 200 calls per second. The API's p95 latency under load was around 300 ms, so about 60 calls are in flight at any moment. The team added headroom and set the executor to 80 threads.

The total thread count barely changed, from 100 to 98. What changed was which work could use which threads.

Latency is a live input here. If the API gets twice as slow, the same throughput needs twice as many threads. A call with no timeout breaks the math completely, because it can hold a thread for as long as the remote side likes.

The stages then hop between executors explicitly. A simplified version:

CompletableFuture<Void> process(Job job) {
    return CompletableFuture
        .supplyAsync(() -> externalApi.fetch(job), apiExecutor)
        .thenApplyAsync(data -> repository.save(job, data), dbExecutor)
        .thenApplyAsync(saved -> externalApi.confirm(saved), apiExecutor)
        .thenAcceptAsync(reply -> repository.markDone(job, reply), dbExecutor);
}

The Async variants matter. Plain thenApply can run the next stage on whatever thread completed the previous one. In this chain, that would quietly put database work back on an API thread.

The split also assumes each stage touches one resource. The transactional path broke that rule, so the team moved its API call out of the transaction. The transaction now covers only the writes. They also added dashboards and alerts on pending connections and on each executor's queue depth.

After the fix, p99 at peak load settled around 450 ms. Connection timeout errors stopped. Peak throughput went from about 70 units per second during the incident back to the expected 100.

It worked because the two kinds of waiting stopped sharing a budget. A slow API can fill its own executor, but it cannot take threads from database work. A busy database cannot stall API stages. A slow API still lowers overall throughput, since every unit needs both. It just no longer drags the database side down with it.

Would virtual threads have prevented this?

Java 21 made virtual threads a standard feature. They are cheap, and a service can run a very large number of them. When a virtual thread blocks on I/O, it usually releases the platform thread underneath it. Spring Boot 3.2 added a spring.threads.virtual.enabled property that moves several of its own components onto virtual threads on Java 21 or later.

The incident was a story about blocked threads, so the question is fair. The answer is no. Virtual threads make threads cheap. They do nothing about connections, and there would still be only 20 of those.

Picture the same service with one virtual thread per task, using Executors.newVirtualThreadPerTaskExecutor(). The 100-thread limit is gone, and every unit starts

Notify via email when an event occurs in a website

Gaddeti Rohit Babu — Wed, 20 May 2020 11:22:29 +0000

Ever wanted to know as soon as

a product comes into stock?
tickets are available for any event?

I think most of us have had those questions. After reading this, you'll know how to implement answers for the above questions/similar questions.

This implementation had paved ways to find answers to the above questions.

The project mentioned above was built in last summer(2019 April) during the IPL season. What is IPL? check here.

The main objective of this project is to notify the user when someone hits a SIX in the IPL match. With few modifications, you can build a system to notify yourself when an event occurs in a website.

Diving into how I did it, the first and core concept I should discuss about is Web Scraping. Web Scraping allows you to get data from websites even if the website owner is not providing any API kind of thing. I used Python language in this project and will be talking about python packages throughout this post. Python has web scraping libraries like BeautifulSoup, Scrapy etc. I used BeautifulSoup here because it is simple and fits our need.

BeautifulSoup consists of powerful parsers that can parse response objects into traversable objects. It also has several inbuilt methods to filter out the data extracted from the website. We can achieve a lot of things using them.

So, you first need to make a request to the website you wanted to scrape using its URL. This can be done with packages like urllib, urllib3, requests etc. You'll get a HttpResponse object from the server of the website you requested. Now you have to pass that response object to BeautifulSoup class specifying which parser to use. It will give a parsed bs4 object accordingly.

Now, you have to check for the required data from the parsed object using builtin methods or regular expressions.

You'll want to notify after checking for the data. To notify the user via email, you need two packages: smtplib, email.

email is used to frame email object which we will be using in sendmail method of smtplib.

Using threads to send notifications can prevent the pausing of main activity.