This post covers thread pools, ExecutorService, rejection policies,
scheduled tasks, thread states, and a ThreadLocal bug.
If you haven't read Part 1,
start here: Java Concurrency: From Threads to Thread Safety — Part 1
ExecutorService
A thread is not free. The JVM allocates a stack for every thread — about 512KB by default.
1000 threads = ~500MB just for stacks
10000 threads = ~5GB just for stacks
That is before your objects, your heap, or your application logic gets any memory. On top of that, Linux has a hard limit on threads per process — usually between 1000 and 4000. Cross that line and you get:
java.lang.OutOfMemoryError: unable to create native thread
Your server crashes.
Instead of creating a new thread for every task, keep a fixed pool of threads alive and let tasks queue up.
Raw threads = hire a new chef per order → restaurant goes bankrupt
ExecutorService = 10 permanent chefs + order queue → scalable
Order comes in → sits in queue → first free chef picks it up → finishes → picks next
Two things to know upfront about this model:
- When all threads are busy, tasks wait in a
BlockingQueueuntil a thread frees up - If a thread crashes mid-task, that task is lost by default. Java does not retry. No exception in your logs unless you handle it yourself.
ExecutorService executor = Executors.newFixedThreadPool(3);
for (int i = 0; i < 6; i++) {
int count = i;
executor.submit(() -> {
System.out.println("Task:" + count + " running on: "
+ Thread.currentThread().getName());
});
}
executor.shutdown();
Output:
Task:1 running on: pool-1-thread-2
Task:0 running on: pool-1-thread-1
Task:2 running on: pool-1-thread-3
Task:3 running on: pool-1-thread-2
Task:4 running on: pool-1-thread-3
Task:5 running on: pool-1-thread-1
Two things to notice here.
Only three thread names show up — pool-1-thread-1, pool-1-thread-2, pool-1-thread-3. Six tasks, three threads. Each thread picked up two tasks. That is the reuse happening.
Note: Task 0 was submitted first but printed second. Submission order is not execution order. First free thread wins. Not round robin, not ordered. If your business logic needs sequence, use
newSingleThreadExecutor.
ThreadPoolExecutor — what is actually running under the hood
All Executors.newXxx() factory methods are wrappers around ThreadPoolExecutor. When you call Executors.newFixedThreadPool(3), Java internally creates:
new ThreadPoolExecutor(
3, // corePoolSize
3, // maximumPoolSize
0L, TimeUnit.MILLISECONDS, // keepAliveTime
new LinkedBlockingQueue<>() // unbounded queue
);
In production, use ThreadPoolExecutor directly so you control the bounds:
new ThreadPoolExecutor(
10, // core threads — always alive
100, // max threads — ceiling under load
60L, TimeUnit.SECONDS, // kill extra threads after idle
new LinkedBlockingQueue<>(500) // bounded queue — 500 tasks max
);
How task submission works internally
1. Thread available in core pool? → assign immediately
2. Core pool full, queue not full? → add to queue
3. Queue full, below max pool size? → create new thread
4. Queue full, at max pool size? → REJECTION POLICY triggers
Rejection policies
When the queue is full and the pool is at maximum size, every new submit() triggers the rejection policy.
| Policy | What it does |
|---|---|
AbortPolicy |
Throws RejectedExecutionException — this is the default |
CallerRunsPolicy |
The calling thread runs the task itself |
DiscardPolicy |
Silently drops the task |
DiscardOldestPolicy |
Drops the oldest queued task, retries the new one |
I set up a pool with 2 threads and a queue of 2 to see the default behaviour:
ExecutorService service = new ThreadPoolExecutor(
2, 2, 0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<>(2)
);
for (int i = 0; i < 5; i++) {
service.submit(task);
}
2 threads busy + 2 queue slots full. Task 5 had nowhere to go:
java.util.concurrent.RejectedExecutionException
rejected from ThreadPoolExecutor
[Running, pool size = 2, active threads = 2, queued tasks = 2, completed tasks = 0]
That error message is a full status report of what the pool was doing at the moment of rejection.
CallerRunsPolicy
ExecutorService service = new ThreadPoolExecutor(
2, 2, 0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<>(2),
new ThreadPoolExecutor.CallerRunsPolicy()
);
When pool and queue are both full — the thread that called submit() runs the task itself:
Task:0 running on: pool-1-thread-1
Task:4 running on: main ← main thread ran this one
Task:2 running on: pool-1-thread-1
Task:3 running on: pool-1-thread-1
Task:1 running on: pool-1-thread-2
Main slows down. Pool catches up. No exception, no data loss, self-regulating.
Custom rejection handler
RejectedExecutionHandler handler = (rejectedTask, executor) -> {
System.out.println("Task rejected: " + rejectedTask.toString());
// options: dead letter queue, retry, alert ops team
};
ExecutorService serviceWithHandler = new ThreadPoolExecutor(
2, // core threads
2, // max threads
0L, TimeUnit.MILLISECONDS, // keep alive
new LinkedBlockingQueue<>(2), // bounded queue
handler // your custom handler
);
In production, a custom handler is almost always the right choice over the built-in policies.
Pitfall #1: DiscardPolicy
DiscardPolicy silently drops tasks with zero noise. No exception, no log, no trace. The task just disappears. Not recommended in production specifically in areas of payment processing, audit logging, archiving etc.,
If you are dropping tasks in production, at minimum log it.
shutdown() vs shutdownNow()
executor.shutdown();
// ✅ Running tasks → complete
// ✅ Queued tasks → complete
// ❌ New submissions → RejectedExecutionException
Graceful. Finish what is already submitted, close the door to new work.
executor.shutdownNow();
// ✅ Returns list of queued tasks that were NOT run
// ⚠️ Attempts to interrupt running tasks
// ❌ New submissions → RejectedExecutionException
I tried submitting a task after shutdown() to see what happens:
ExecutorService service = Executors.newFixedThreadPool(3);
service.shutdown();
service.submit(() -> System.out.println("Am I running?"));
java.util.concurrent.RejectedExecutionException
rejected from ThreadPoolExecutor
[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
Terminated, pool size = 0. The pool was already dead by the time the task hit it.
Pitfall #2: shutdownNow() does not guarantee threads stop
shutdownNow() sends an interrupt signal. A thread only actually stops if it is sleeping (throws InterruptedException) or checks Thread.currentThread().isInterrupted(). A thread doing pure CPU work ignores the signal and keeps running.
Always have a check in place.
executor.shutdown();
if (!executor.awaitTermination(5, TimeUnit.SECONDS)) {
executor.shutdownNow(); // force if graceful didn't finish in time
}
ThreadPool types
| Pool | Threads |
|---|---|
newFixedThreadPool(n) |
Always exactly n // steady load |
newCachedThreadPool |
0 to unlimited // for low volume only |
newSingleThreadExecutor |
Always exactly 1 // Strict sequential order |
newScheduledThreadPool(n) |
Fixed n // Fixed-rate or delayed tasks |
ThreadPoolExecutor directly |
You decide everything // Production — always |
Pitfall #3: newCachedThreadPool under load
newCachedThreadPool creates threads on demand and kills them after 60 seconds idle. Useful, but no upper bound.
10000 tasks come in. 10000 threads get created. Same original OOM problem.
The real answer for variable load:
new ThreadPoolExecutor(
10, // core — always ready
100, // max — ceiling at peak
60L, TimeUnit.SECONDS, // kill extras after idle
new LinkedBlockingQueue<>(500) // bounded — reject rather than OOM
);
Scheduled tasks — scheduleAtFixedRate vs scheduleWithFixedDelay
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
scheduler.scheduleAtFixedRate(task, 0, 30, TimeUnit.SECONDS);
scheduler.scheduleWithFixedDelay(task, 0, 30, TimeUnit.SECONDS);
If the task takes 10 seconds and the interval is 30 seconds:
scheduleAtFixedRate — fixed clock, ignores task duration:
T=0 task starts ──────── T=10 done
T=30 task starts ──────── T=40 done
T=60 task starts ──────── T=70 done
scheduleWithFixedDelay — gap measured AFTER task completes:
T=0 task starts ──────── T=10 done ── wait 30s ──
T=40 task starts ──────── T=50 done ── wait 30s ──
T=80 task starts
What if the task takes longer than the interval?
Let's take the case where task take more than the expected execution time. With scheduleAtFixedRate, if tasks take 45 seconds, interval 30 seconds
T=0 task starts ──────────────── T=45 done
T=30 should start — waits, does not overlap
T=45 starts immediately after previous finishes
ScheduledThreadPoolExecutor will not start the next run until the current one finishes. No pileup. No second thread spawned. It just runs back to back with no gap.
But with newScheduledThreadPool(n > 1) and multiple slow tasks — pileup can happen. For single recurring jobs, always use newScheduledThreadPool(1).
Stopping a scheduled task after N executions
The core issue: you need a counter inside the lambda to track the executions.
AtomicInteger solves both problems:
AtomicInteger count = new AtomicInteger(0);
// Object reference → effectively final
// incrementAndGet() → atomic via CAS
The lambda needs the ScheduledFuture to cancel itself. But the future is assigned after the lambda is created. AtomicReference fixes this — the lambda captures the box, not what is inside it:
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
AtomicInteger count = new AtomicInteger(0);
AtomicReference<ScheduledFuture<?>> future = new AtomicReference<>();
future.set(scheduler.scheduleAtFixedRate(() -> {
System.out.println("Health check: " + LocalDateTime.now());
if (count.incrementAndGet() >= 5) {
future.get().cancel(false);
scheduler.shutdown();
}
}, 0, 2, TimeUnit.SECONDS));
Output:
Health check: 2026-05-17T15:37:35.753357
Health check: 2026-05-17T15:37:37.748608
Health check: 2026-05-17T15:37:39.748212
Health check: 2026-05-17T15:37:41.748337
Health check: 2026-05-17T15:37:43.746872
Note:
cancel(false)means do not interrupt if the task is currently running.cancel(true)sends an interrupt signal to the running task.
Thread states
Every thread in Java is always in one of six states:
NEW → created, start() not called yet
RUNNABLE → running or ready to run
BLOCKED → waiting to acquire a synchronized lock
WAITING → waiting indefinitely — join(), wait()
TIMED_WAITING → waiting with a timeout — sleep(n), join(n)
TERMINATED → done
Check the below code:
Object lock = new Object();
Thread t1 = new Thread(() -> {
synchronized(lock) {
try { Thread.sleep(5000); }
catch (InterruptedException e) {}
}
});
Thread t2 = new Thread(() -> {
synchronized(lock) {
System.out.println("t2 got the lock");
}
});
t1.start();
Thread.sleep(100); // let t1 grab the lock first
t2.start();
Thread.sleep(200);
System.out.println("t1: " + t1.getState());
System.out.println("t2: " + t2.getState());
Output:
t1: TIMED_WAITING
t2: BLOCKED
t2 got the lock
t1 holds the lock and is sleeping — TIMED_WAITING because sleep() has a timeout.
t2 wants the lock but cannot get it — BLOCKED because the lock is held by t1.
BLOCKED vs WAITING
BLOCKED → fighting for a lock
cause: synchronized contention or deadlock
fix: find who holds the lock, reduce contention
WAITING → waiting for a signal
cause: join(), wait(), park()
fix: check if the thread it's waiting on is alive
If you see something similar in your application thread dump,
"pool-1-thread-2"
java.lang.Thread.State: BLOCKED
waiting to lock <0x000000076b572018>
held by "pool-1-thread-1"
"pool-1-thread-1"
java.lang.Thread.State: TIMED_WAITING
sleeping
Thread-1 sleeping while holding a lock, thread-2 stuck waiting for it.
ThreadLocal
The problem: A web server where each request runs on a thread from a pool. You need to store the current user or request ID so that any method in the call chain can access it, without passing it as a parameter to every method.
A static variable is the obvious first thought, but a strict NO.
ThreadLocal gives each thread its own private copy:
static ThreadLocal<String> requestId = new ThreadLocal<>();
// Thread 1
requestId.set("REQ-001");
requestId.get(); // → REQ-001
// Thread 2
requestId.set("REQ-002");
requestId.get(); // → REQ-002
// Thread 1 again
requestId.get(); // → REQ-001, not REQ-002
No sharing. No synchronization needed. Each thread sees only its own value.
How it works internally
ThreadLocal holds no data. It is just a key.
Every Thread object has a hidden field : ThreadLocalMap —> that belongs to that thread exclusively. The map is created lazily on the first set() call.
When you call set():
public void set(T value) {
Thread t = Thread.currentThread();
ThreadLocalMap map = t.threadLocals;
map.set(this, value);
// store with ThreadLocal as key
}
When you call get():
public T get() {
Thread t = Thread.currentThread();
ThreadLocalMap map = t.threadLocals;
return map.get(this);
// look up using ThreadLocal as key
}
Note: Spring uses this pattern heavily —
SecurityContextHolder,TransactionSynchronizationManager,RequestContextHolder. One thread per request, one context per thread.
Pitfall #4: The ThreadLocal memory leak
Thread pools reuse threads. A thread that handled Request 1 will handle Request 5000. The thread lives forever, until the pool shuts down.
If you set() a value and never clean it up, that value stays in the thread's map. Two things go wrong:
Memory leak: GC cannot collect it because the thread is alive and holding the reference. Thousands of requests, thousands of stale objects accumulating.
Data leak: The next request on that same thread calls get() without calling set() first — and gets the previous request's value.
I ran this with a single-thread pool:
static ThreadLocal<String> requestId = new ThreadLocal<>();
ExecutorService service = Executors.newFixedThreadPool(1);
// First batch — sets values
for (int i = 0; i < 3; i++) {
String value = "REQ-00" + (i + 1);
service.submit(() -> {
requestId.set(value);
System.out.println("first call: " + requestId.get());
});
}
// Second batch — only reads, no set
for (int i = 0; i < 3; i++) {
service.submit(() -> {
System.out.println("second call: " + requestId.get());
});
}
Without remove():
first call: REQ-001
first call: REQ-002
first call: REQ-003
second call: REQ-003
second call: REQ-003
second call: REQ-003
REQ-003 was the last value set by the first batch. The thread went back to the pool with that value still sitting in its map. Every subsequent task that only called get() received stale data from a previous request.
The fix:
With remove() in a finally block:
service.submit(() -> {
try {
requestId.set("REQ-001");
// do work
} finally {
requestId.remove(); // always in finally, on the pool thread
}
});
first call: REQ-001
first call: REQ-002
first call: REQ-003
second call: null
second call: null
second call: null
Note:
remove()must be inside the lambda — on the pool thread that actually set the value, otherwise it will be referrring tomainthread's map.
What's next?
I will be discussing Java memory model, volatile, CompletableFuture in upcoming posts.
Top comments (0)