DEV Community: Prathamesh Thakre

Redis + Circuit Breaker: The Real Defense Against Database Meltdowns in High-Throughput Systems

Prathamesh Thakre — Sat, 25 Apr 2026 11:31:16 +0000

Your system handles 10,000 requests per second. Everything is fine until 11 PM when your database connection pool exhausts itself.

What you see: Requests start failing with SQLException: Cannot get a Connection. Your monitoring alerts scream. Panic sets in.

What you do: You throw cache at it. Add Redis. Deploy. Response times drop, errors clear up. You declare victory.

Then, 48 hours later, it happens again. This time, Redis is also getting hammered. Cache misses climb to 80%. You're hitting the database anyway.

The uncomfortable truth? Redis alone doesn't protect you from a database failure. Neither does a circuit breaker alone. Together, they're unstoppable.

But most implementations treat them as separate concerns. They shouldn't be. In a high-throughput system, they need to work together, understanding what the other is doing, and degrading intelligently when things break.

This is where most Spring Boot applications fail under real load.

The Three Failure Modes of High-Throughput Systems

Before we talk about solutions, let's understand what actually kills your database:

Failure Mode 1: The Cache-Miss Avalanche

Your cache is working perfectly. Hit rate is 95%. Then, one operation fails (a network blip, a bad deploy). A cache key expires incorrectly, or your cache layer goes down.

Suddenly, every single request hits the database.

Normal state:
  100 req/s → Cache (95 hits) → 5 DB queries/s ✓

Cache fails:
  100 req/s → Cache (0 hits) → 100 DB queries/s ✗

Database connection pool: 20 connections
Queue: 80 requests waiting
Connections timeout: 50 failures
More requests arrive: Cascading failure

Your database was designed for 5 queries per second. It just got 100. Connection pools exhaust. Queries queue up and timeout. Your database suffers a slow, painful death.

And here's the cruel part: even after the cache recovers, the damage is done. Your database is already under water, responding slowly, triggering more cache misses, which triggers more database hits, which makes the database slower...

This is a cascade. And a circuit breaker is your only exit.

Failure Mode 2: The Slow-Database Trap

Your database becomes slow but doesn't fail. A missing index. A long-running migration. A query that's suddenly taking 500ms instead of 50ms.

Your thread pool has 20 threads. Each thread is now blocking for 500ms waiting for a database response. Within seconds, all 20 threads are busy. New requests queue up (remember Article 3? unbounded queues?). Threads are still blocking. Queue grows. Memory pressure increases.

Your application dies not because the database failed, but because it got slow.

A circuit breaker sees the slow responses, recognizes the pattern, and stops sending traffic to the database. Suddenly, your API returns 503 quickly instead of timing out after 30 seconds.

Failure Mode 3: The Resource Leak

Your connection pool is exhausted, but it's not because of traffic. It's because of connections that aren't being returned. A bug in your code, a timeout that doesn't close the connection, a third-party library that leaks resources.

Now you have:

20 available connections in the pool
50 connections checked out but never returned
100 new requests trying to acquire connections
80 requests timing out waiting

Your database is fine. Your application is broken.

A circuit breaker won't fix this directly, but combined with proper monitoring and backpressure (timeout handling), it ensures you fail fast instead of hanging.

The Architecture Most Teams Get Wrong

Here's what I see in most Spring Boot applications:

@Service
public class ProductService {

    @Autowired
    private ProductRepository repository;

    @Cacheable("products")
    public Product getProduct(Long id) {
        // Redis caching at the method level
        return repository.findById(id).orElseThrow();
    }
}

This has problems:

Cache layer is independent of database layer. If the database is slow, the cache layer doesn't know. It might serve stale data forever, or it might just let requests pile up.
No circuit breaker on the database. If the database fails, every cache miss attempts to hit it, potentially making things worse.
No timeout strategy. How long does the cache check wait for the database? If it waits forever, you've just turned a database failure into an application hang.
No degradation path. What happens when both cache and database are struggling? Do you return stale data? Return errors? Return nothing?

Here's what you actually need:

Request
  ↓
[Check Redis Cache]
  ├─→ CACHE HIT: Return immediately ✓
  ├─→ CACHE MISS: Try database
  │    ├─→ Circuit OPEN (DB is down): Return fallback/error
  │    ├─→ Circuit HALF-OPEN: Try DB, update cache if success
  │    ├─→ Circuit CLOSED: Query DB, update cache, return ✓
  │         └─→ Query TIMEOUT: Fail fast, don't retry
  └─→ CACHE ERROR: Try database with circuit breaker

This is the architecture that survives real-world chaos.

Implementation: Redis + Circuit Breaker in Spring Boot

Let's build this step by step.

Step 1: Basic Redis Configuration with Timeout

@Configuration
@EnableCaching
public class CacheConfig {

    @Bean
    public RedisCacheManagerBuilderCustomizer redisCacheManagerBuilderCustomizer() {
        return builder -> builder
            .withCacheConfiguration("products",
                RedisCacheConfiguration.defaultCacheConfig()
                    .entryTtl(Duration.ofMinutes(30))
                    .serializeValuesWith(
                        RedisSerializationContext.SerializationPair.fromSerializer(
                            new GenericJackson2JsonRedisSerializer()
                        )
                    ))
            .withCacheConfiguration("orders",
                RedisCacheConfiguration.defaultCacheConfig()
                    .entryTtl(Duration.ofMinutes(10)));
    }

    @Bean
    public LettuceConnectionFactory redisConnectionFactory() {
        return new LettuceConnectionFactory();
    }

    @Bean
    public RedisTemplate<String, Object> redisTemplate(
            RedisConnectionFactory connectionFactory) {
        RedisTemplate<String, Object> template = new RedisTemplate<>();
        template.setConnectionFactory(connectionFactory);
        return template;
    }
}

Notice the TTL is explicit. In a high-throughput system, you can't afford cache entries that live forever.

Step 2: Add Resilience4j Circuit Breaker

Add the dependency:

<dependency>
    <groupId>io.github.resilience4j</groupId>
    <artifactId>resilience4j-spring-boot3</artifactId>
    <version>2.1.0</version>
</dependency>
<dependency>
    <groupId>io.github.resilience4j</groupId>
    <artifactId>resilience4j-circuitbreaker</artifactId>
    <version>2.1.0</version>
</dependency>

Configure circuit breaker in application.yml:

resilience4j:
  circuitbreaker:
    configs:
      default:
        registerHealthIndicator: true
        slidingWindowSize: 100
        failureRateThreshold: 50.0
        slowCallRateThreshold: 50.0
        slowCallDurationThreshold: 2000ms
        permittedNumberOfCallsInHalfOpenState: 10
        automaticTransitionFromOpenToHalfOpenEnabled: true
        waitDurationInOpenState: 10s

    instances:
      productRepository:
        baseConfig: default
        slidingWindowType: COUNT_BASED

What this means:

Track the last 100 calls
If 50% fail OR 50% are slower than 2 seconds, OPEN the circuit
When OPEN, wait 10 seconds before trying again (HALF-OPEN)
In HALF-OPEN, allow 10 attempts before deciding if things are better

Step 3: The Real Implementation

Here's where Redis and circuit breaker come together:

@Service
@Slf4j
public class ProductService {

    @Autowired
    private ProductRepository repository;

    @Autowired
    private RedisTemplate<String, Object> redisTemplate;

    @Autowired
    private CircuitBreaker productCircuitBreaker;

    private static final String CACHE_KEY_PREFIX = "products:";
    private static final Duration CACHE_TTL = Duration.ofMinutes(30);
    private static final Duration QUERY_TIMEOUT = Duration.ofSeconds(5);

    public Product getProduct(Long id) {
        String cacheKey = CACHE_KEY_PREFIX + id;

        try {
            // Step 1: Check Redis first
            Object cached = redisTemplate.opsForValue().get(cacheKey);
            if (cached != null) {
                log.debug("Cache hit for product {}", id);
                return (Product) cached;
            }
        } catch (Exception e) {
            // Redis is down. Don't let this prevent us from trying the database.
            log.warn("Redis cache check failed for product {}: {}", id, e.getMessage());
        }

        // Step 2: Cache miss or cache error. Try database with circuit breaker.
        try {
            Product product = productCircuitBreaker.executeSupplier(() ->
                queryProductWithTimeout(id)
            );

            // Step 3: Update cache on successful database hit
            try {
                redisTemplate.opsForValue().set(
                    cacheKey,
                    product,
                    CACHE_TTL
                );
                log.debug("Cached product {} in Redis", id);
            } catch (Exception e) {
                log.warn("Failed to cache product {}: {}", id, e.getMessage());
                // Don't fail the request. Just return the product.
            }

            return product;

        } catch (CallNotPermittedException e) {
            // Circuit is OPEN. Database is down.
            log.error("Circuit breaker is OPEN for product {}", id);
            return getProductFallback(id);

        } catch (io.github.resilience4j.core.registry.EntryNotFoundConfiguredException e) {
            // This shouldn't happen, but handle gracefully
            log.error("Circuit breaker configuration error for product {}", id);
            throw new ServiceUnavailableException("Product service is degraded");
        }
    }

    private Product queryProductWithTimeout(Long id) {
        // The circuit breaker wraps this, monitoring for failures and slowness
        try {
            return CompletableFuture.supplyAsync(() ->
                repository.findById(id).orElseThrow(
                    () -> new ProductNotFoundException("Product " + id + " not found")
                )
            ).get(QUERY_TIMEOUT.toMillis(), TimeUnit.MILLISECONDS);

        } catch (TimeoutException e) {
            throw new DatabaseTimeoutException(
                "Database query timed out after " + QUERY_TIMEOUT.toMillis() + "ms"
            );
        } catch (InterruptedException | ExecutionException e) {
            throw new DatabaseException("Database query failed: " + e.getMessage(), e);
        }
    }

    private Product getProductFallback(Long id) {
        // Circuit is OPEN. Try to return stale data from Redis if available.
        try {
            Object cached = redisTemplate.opsForValue().get(CACHE_KEY_PREFIX + id);
            if (cached != null) {
                log.info("Returning stale cache for product {} (circuit is open)", id);
                return (Product) cached;
            }
        } catch (Exception e) {
            log.debug("Cannot access Redis fallback: {}", e.getMessage());
        }

        // No fallback available. Return error.
        throw new ServiceUnavailableException(
            "Product service is temporarily unavailable. Please try again."
        );
    }

    @CacheEvict(value = "products", key = "#id")
    @CircuitBreaker(name = "productRepository")
    public void updateProduct(Long id, ProductUpdateRequest request) {
        Product product = repository.findById(id).orElseThrow();
        product.setName(request.name());
        product.setPrice(request.price());
        repository.save(product);

        // Also explicitly clear Redis cache
        try {
            redisTemplate.delete(CACHE_KEY_PREFIX + id);
        } catch (Exception e) {
            log.warn("Failed to clear cache for product {}: {}", id, e.getMessage());
        }
    }

    @CircuitBreaker(name = "productRepository")
    public List<Product> searchProducts(String query) {
        // Use a different cache for search results with shorter TTL
        String cacheKey = "search:" + query.hashCode();

        try {
            Object cached = redisTemplate.opsForValue().get(cacheKey);
            if (cached != null) {
                return (List<Product>) cached;
            }
        } catch (Exception e) {
            log.warn("Search cache check failed: {}", e.getMessage());
        }

        List<Product> results = repository.findByNameContainingIgnoreCase(query);

        try {
            // Search results change frequently, shorter TTL
            redisTemplate.opsForValue().set(
                cacheKey,
                results,
                Duration.ofMinutes(5)
            );
        } catch (Exception e) {
            log.warn("Failed to cache search results: {}", e.getMessage());
        }

        return results;
    }
}

This is a real implementation. Let's break down what's happening:

Cache check is non-blocking. If Redis fails, we continue. Redis failures don't cascade.
Circuit breaker wraps database calls. If the database is slow or fails, the circuit opens automatically, preventing further hammering.
Timeout is enforced. Database queries can't block forever. After 5 seconds, we fail fast.
Fallback uses stale data. When the circuit is open, we return yesterday's data if available. A stale product is better than no product.
Cache invalidation is explicit. When data changes, we evict it.

Step 4: Monitoring and Metrics

You need visibility into what's happening:

@Component
@Slf4j
public class CircuitBreakerMetrics {

    @Autowired
    private CircuitBreakerRegistry circuitBreakerRegistry;

    @Autowired
    private CacheManager cacheManager;

    @Scheduled(fixedRate = 10000) // Every 10 seconds
    public void logMetrics() {
        CircuitBreaker cb = circuitBreakerRegistry.circuitBreaker("productRepository");

        log.info("Circuit Breaker Stats - State: {}, Failure Rate: {}%, Slow Call Rate: {}%",
            cb.getState(),
            cb.getMetrics().getFailureRate(),
            cb.getMetrics().getSlowCallRate()
        );

        Cache productsCache = cacheManager.getCache("products");
        if (productsCache instanceof RedisCache redisCache) {
            log.info("Redis Cache Stats for 'products' - Approximate size unknown, TTL: 30m");
        }
    }
}

Add to application.yml:

management:
  endpoints:
    web:
      exposure:
        include: health,metrics,circuitbreakers
  endpoint:
    health:
      show-details: always
  metrics:
    tags:
      application: product-service

Now you can check /actuator/health to see circuit breaker status:

{
  "status": "UP",
  "components": {
    "circuitBreakers": {
      "status": "UP",
      "details": {
        "productRepository": {
          "status": "UP",
          "details": {
            "state": "CLOSED",
            "failureRate": "0.0%",
            "slowCallRate": "5.2%"
          }
        }
      }
    }
  }
}

The Gotchas: Where This Implementation Can Still Break

Gotcha 1: Cache Stampede on Circuit Breaker Recovery

When the circuit transitions from OPEN to HALF-OPEN, suddenly 10 requests try the database simultaneously (your permittedNumberOfCallsInHalfOpenState). If many of these requests want the same product, they all miss the cache and query at once.

Circuit: OPEN
  All requests get fallback (stale data) ✓

Circuit: HALF-OPEN (10 permits)
  Request 1: Query product 42 → Hits DB
  Request 2: Query product 42 → Hits DB (cache miss, both in-flight)
  Request 3: Query product 42 → Hits DB (cache miss, all three in-flight)
  ...
  All 10 permits → Same 3 products queried 10 times total

Solution: Use a request-level cache (in-memory, per-request) for HALF-OPEN state:

@Service
@Slf4j
public class ProductService {

    private final ThreadLocal<Map<String, Product>> requestCache = 
        ThreadLocal.withInitial(HashMap::new);

    @Around("execution(public * com.example.service.ProductService.getProduct(..))")
    public Object cacheWithinRequest(ProceedingJoinPoint pjp) throws Throwable {
        String cacheKey = "req:" + pjp.getArgs()[0];
        Map<String, Product> cache = requestCache.get();

        if (cache.containsKey(cacheKey)) {
            return cache.get(cacheKey);
        }

        try {
            Product result = (Product) pjp.proceed();
            cache.put(cacheKey, result);
            return result;
        } finally {
            requestCache.remove();
        }
    }
}

Gotcha 2: Stale Data Divergence

Your circuit is open. You're serving stale product prices from Redis. Meanwhile, your admin panel is writing new prices to the database. When the circuit closes, users see the old price, then suddenly the new price changes. Confusion ensues.

Solution: Add a "stale" marker to cache entries:

public class CachedProduct {
    public Product product;
    public long cachedAtMillis;
    public boolean isStale;
}

private Product getProductFallback(Long id) {
    try {
        Object cached = redisTemplate.opsForValue().get(CACHE_KEY_PREFIX + id);
        if (cached instanceof CachedProduct cp) {
            long age = System.currentTimeMillis() - cp.cachedAtMillis;
            if (age > 60_000) {  // Older than 1 minute
                cp.isStale = true;
            }
            log.info("Returning {} cache for product {}", 
                cp.isStale ? "stale" : "fresh", id);
            return cp.product;
        }
    } catch (Exception e) {
        log.debug("Cannot access Redis fallback: {}", e.getMessage());
    }
    throw new ServiceUnavailableException("Product service unavailable");
}

Include staleness in the API response so clients know the data age.

Gotcha 3: Circuit Breaker State Not Visible to Load Balancer

Your application has 5 instances. Instance 1's circuit breaker is open (it detected the database is slow). Instances 2-5 still think the database is fine and send traffic to it. You get 80% of the benefit, not 100%.

Solution: Make circuit breaker state part of the health check. Your load balancer removes unhealthy instances:

@Component
public class CircuitBreakerHealthIndicator implements HealthIndicator {

    @Autowired
    private CircuitBreakerRegistry registry;

    @Override
    public Health health() {
        CircuitBreaker cb = registry.circuitBreaker("productRepository");
        if (cb.getState() == OPEN) {
            return Health.outOfService()
                .withDetail("reason", "Circuit breaker is OPEN")
                .build();
        }
        return Health.up().build();
    }
}

Now, when one instance's circuit opens, it reports UNHEALTHY and gets removed from the load balancer.

Gotcha 4: Cache Invalidation Across Instances

You have 5 instances. Instance 1 updates a product and clears the Redis cache. But Instance 2 is still serving the old value because it has it in local memory (if you're using multi-level caching).

Solution: Use a cache invalidation message broker:

@Service
@Slf4j
public class ProductService {

    @Autowired
    private RedisTemplate<String, String> redisTemplate;

    @Autowired
    private ApplicationEventPublisher eventPublisher;

    public void updateProduct(Long id, ProductUpdateRequest request) {
        Product product = repository.findById(id).orElseThrow();
        product.setName(request.name());
        repository.save(product);

        // Publish invalidation event
        String invalidationMessage = "product:" + id;
        redisTemplate.convertAndSend("cache-invalidation", invalidationMessage);

        eventPublisher.publishEvent(new ProductInvalidatedEvent(id));
    }
}

@Component
@Slf4j
public class CacheInvalidationListener {

    @Autowired
    private StringRedisTemplate redisTemplate;

    @Bean
    public MessageListenerAdapter messageListenerAdapter() {
        return new MessageListenerAdapter((message) -> {
            String key = new String((byte[]) message.getBody());
            log.info("Invalidating cache for {}", key);
            // Clear local caches
        });
    }
}

The Real-World Trade-Offs

You've now built a system that handles:

Cache misses without failing
Database failures without crashing
Slow databases without hanging
Circuit breaker recovery without stampeding

But it's more complex. More moving parts. More things that can fail.

Here's the honest assessment:

Scenario	Complexity	Reliability	Cost
Just database	Low	Low (high failure risk)	Low
Just cache	Low	Medium (cache failures)	Low
Database + cache	Medium	Medium (no circuit breaker)	Medium
Database + cache + circuit breaker	High	High (handles failures gracefully)	Medium

You pay in complexity. You gain in reliability. For a high-throughput system, it's always worth it.

The Mindset Shift

Here's what separates systems that survive 3 AM pages from those that don't:

Wrong approach: "Redis will make everything fast, and the database will always be there."

Right approach: "Redis handles the happy path. Circuit breaker handles the sad path. Together, they make the system survive chaos."

The circuit breaker isn't a performance optimization. It's a survival mechanism. It's saying: "If things are broken, fail fast and gracefully, not catastrophically."

Redis isn't just a cache. It's a fallback layer. It's saying: "Yesterday's data is better than no data."

When you implement both, you're not building a system that's 2x better. You're building a system that behaves fundamentally differently under stress. Instead of cascading failures, you get degradation. Instead of 10 seconds of errors, you get 10 seconds of slower responses with stale data.

Your users might not notice the difference. Your oncall engineer will.

One More Thing

This implementation assumes you control both the cache and circuit breaker. If you're using a managed cache (AWS ElastiCache, Azure Cache for Redis) and a managed database (RDS, Cloud SQL), you can't control the circuit breaker on their side.

What you can do: Put the circuit breaker in your application, not in the infrastructure. That's what this article shows.

And if you're wondering why you haven't had to think about this before: You probably haven't scaled to the point where it matters. Congratulations. But keep this pattern in your back pocket. When you do scale, you'll be ready.

Your 11 PM database will thank you.

Unbounded Queues: The Silent Killer of Production Services

Prathamesh Thakre — Sun, 19 Apr 2026 05:59:01 +0000

Your service runs fine at 2 PM.

At 6 PM, the database experiences a brief latency spike—nothing catastrophic, maybe 200ms slower than usual. Within minutes, your monitoring alerts start lighting up. Memory usage climbs 40%, then 60%. GC pauses increase. Users start timing out.

By 7 PM, you have an OutOfMemoryError.

You check the logs. Nothing unusual. The database recovered. The CPU is fine. The network is fine. So what killed you?

An unbounded queue in your ThreadPoolExecutor.

This is one of those bugs that feels like it shouldn't exist in 2026. It's well-known, thoroughly documented, yet somehow still sneaks past code reviews and deploys into production. The reason is simple: unbounded queues feel safe at first.

You define a thread pool with 10 threads, and you assume the queue is your safety net. When threads are busy, tasks wait. Seems reasonable. Until the queue has 100,000 tasks in it.

The Deceptive Logic of Unbounded Queues

Here's how the trap works:

ExecutorService executor = new ThreadPoolExecutor(
    10,                                    // corePoolSize
    10,                                    // maxPoolSize
    60, TimeUnit.SECONDS,                 // keepAliveTime
    new LinkedBlockingQueue<>()            // ← DANGER: unbounded queue
);

You've created a thread pool with 10 threads. When all 10 threads are busy, new tasks don't get rejected—they get queued. The queue can hold unlimited tasks.

Your mental model: "Threads are busy, tasks queue up, threads finish, queue drains."

The reality during latency: "Threads are busy waiting on slow database calls, tasks keep arriving and queueing, queue grows indefinitely, memory fills up, GC panics, everything crashes."

The core issue is that a queue is not a buffer—it's a pit.

A buffer should have boundaries. It should say "I can hold X items, then I stop accepting more." A queue with no bounds just keeps taking items until your JVM runs out of memory.

When This Goes Wrong

The sneaky part is that unbounded queues don't cause problems under normal load. They cause problems under the exact circumstances when you most need protection:

Scenario 1: Temporary Latency Spike

Your database experiences a brief slowdown. Queries that normally take 10ms now take 500ms. Your 10-thread pool fills up as threads block waiting for results.

Time 0:
  Thread 1-10: Processing requests (blocking on DB)
  Queue: Empty

Time 1 (DB latency spike):
  Thread 1-10: Still waiting for DB responses
  Queue: 100 pending requests

Time 2:
  Thread 1-10: Still waiting
  Queue: 1,000 pending requests

Time 3:
  Thread 1-10: Finally getting responses back
  Queue: 5,000 pending requests and growing

Time 5:
  Thread 1-10: Working through the backlog
  Queue: 100,000+ pending requests
  Memory: 2GB (heap is 4GB)

Time 7:
  Queue: OutOfMemoryError. Your service dies.

The database recovered at Time 3. But your service didn't. It spent the next 4 minutes executing stale work for requests that already timed out on the client side.

Scenario 2: Cascading Failures

Service A depends on Service B. Service B starts degrading. Service A's thread pool queues up requests waiting for responses. The queue grows. Memory spikes. Service A crashes, adding more load to Service B, which makes other services queuing up requests to Service B, which...

This is how a cascading failure happens. One slow service takes down three others.

The Root Cause: The Acceptance vs. Execution Mismatch

Here's the fundamental problem:

A ThreadPoolExecutor has two knobs:

Core threads — threads that always exist
Queue — where tasks wait when threads are busy

The issue: When the queue is unbounded, the executor accepts every single task regardless of capacity. Your thread pool can't say "no, I'm overloaded, reject this task." It just queues it.

This creates a mismatch between accepting work and executing work.

Request comes in → Added to unlimited queue → Task waits → More requests come in
                                               ↓
                                          Task still waiting
                                          Memory growing
                                          GC struggling
                                          JVM dying

The executor accepted the work (queued it), but never had capacity to execute it.

The Solution: Bounded Queues and Backpressure

The fix is to give your queue a hard limit:

ExecutorService executor = new ThreadPoolExecutor(
    10,                                    // corePoolSize
    20,                                    // maxPoolSize (grow if needed)
    60, TimeUnit.SECONDS,                 // keepAliveTime
    new LinkedBlockingQueue<>(1000),      // ← BOUNDED: max 1000 tasks
    new ThreadPoolExecutor.CallerRunsPolicy()  // Rejection policy
);

Now, when the queue reaches 1000 items, what happens to the 1001st task? It gets rejected.

By default, a rejected task throws a RejectedExecutionException. But that seems harsh—you want your service to degrade gracefully, not crash.

This is where rejection policies come in:

Rejection Policy 1: CallerRunsPolicy (My Favorite)

new ThreadPoolExecutor.CallerRunsPolicy()

When the queue is full, instead of rejecting the task, execute it in the caller's thread. This creates natural backpressure—the caller slows down, which slows down the request ingestion, which protects the thread pool.

Effect: Your API becomes slower under load instead of crashing. Users experience timeouts, not 503s.

Incoming requests → Thread pool queue (1000 items) → FULL
                                                      ↓
                              CallerRunsPolicy: Run in caller thread
                                                      ↓
                              Caller gets blocked → Slows down ingestion
                                                      ↓
                              Natural backpressure applied

Rejection Policy 2: AbortPolicy (Explicit Failure)

new ThreadPoolExecutor.AbortPolicy()

Throw an exception. The caller knows immediately that the system is overloaded. They can retry, circuit-break, or fail fast.

try {
    executor.submit(task);
} catch (RejectedExecutionException e) {
    log.warn("Executor is overloaded, backing off");
    return new ServiceUnavailableResponse();
}

This is more explicit but requires the caller to handle the rejection.

Rejection Policy 3: DiscardPolicy (Nuclear Option)

new ThreadPoolExecutor.DiscardPolicy()

Silently drop the task. Use this only for non-critical work where loss is acceptable (e.g., metrics collection, logging).

Tuning the Queue Capacity

How big should your queue be?

This is where it gets nuanced. Too small, and you reject valid requests during normal fluctuations. Too big, and you're back to the original problem.

A practical approach:

// Estimate based on your latency and throughput
int estimatedQueueSize = (int) (
    averageRequestsPerSecond *  // 1000 req/s
    maxAcceptableLatencySeconds  // 10 second wait
);

// Conservative estimate: 10,000 tasks
// This gives you 10 seconds of buffer at 1000 req/s

ExecutorService executor = new ThreadPoolExecutor(
    10,
    20,
    60, TimeUnit.SECONDS,
    new LinkedBlockingQueue<>(estimatedQueueSize),
    new CallerRunsPolicy()
);

Then monitor in production:

If queue hits capacity regularly, increase it (or increase core threads)
If queue rarely exceeds 10% capacity, you can reduce it

A Deeper Problem: Stale Task Execution

Even with bounded queues, there's another issue: stale tasks still get executed.

When a client times out waiting for a response, they've given up. But their task might still be sitting in the queue, waiting to execute. Hours later, when the queue drains, the thread pool dutifully executes it.

This is wasted work—your thread pool is doing something nobody cares about anymore.

One partial solution: Use Future with timeouts:

ExecutorService executor = Executors.newFixedThreadPool(10);

Future<Response> future = executor.submit(() -> {
    return expensiveOperation();
});

try {
    Response response = future.get(5, TimeUnit.SECONDS);
    return response;
} catch (TimeoutException e) {
    future.cancel(true);  // Cancel the task
    return timeoutResponse();
}

The cancel(true) flag attempts to interrupt the task. But this only works if your task respects interrupts. Many database drivers don't.

A better approach: Pass a deadline or timeout token to your task itself:

@FunctionalInterface
interface TimeoutAwareTask<T> {
    T execute(Deadline deadline) throws InterruptedException;
}

executor.submit(() -> {
    Deadline deadline = Deadline.ofMillis(System.currentTimeMillis() + 5000);
    if (deadline.isExpired()) {
        log.debug("Task timed out before execution, skipping");
        return;
    }
    expensiveOperation(deadline);
});

Now your task knows when the client gave up and can bail out early.

Additional Improvements

Beyond bounded queues, consider:

1. Thread Pool Size Optimization

// CPU-bound work
int coreThreads = Runtime.getRuntime().availableProcessors();

// IO-bound work (database calls, network)
int coreThreads = Runtime.getRuntime().availableProcessors() * 2;

IO-bound tasks spend time waiting, so more threads are useful.

2. Separate Thread Pools by Workload

Don't use one executor for everything. Separate concerns:

// Fast, non-blocking work
ExecutorService fastExecutor = new ThreadPoolExecutor(
    10, 10, 60, TimeUnit.SECONDS,
    new LinkedBlockingQueue<>(1000),
    new CallerRunsPolicy()
);

// Slow IO work (database queries)
ExecutorService slowExecutor = new ThreadPoolExecutor(
    20, 20, 60, TimeUnit.SECONDS,
    new LinkedBlockingQueue<>(5000),
    new CallerRunsPolicy()
);

One slow dependency doesn't starve the fast paths.

3. Spring Boot Configuration (If You Use Spring)

spring.task.execution.pool.core-size: 10
spring.task.execution.pool.max-size: 20
spring.task.execution.pool.queue-capacity: 1000
spring.task.execution.pool.allow-core-thread-timeout: true

Add a custom executor bean for control:

@Configuration
public class ExecutorConfig {

    @Bean(name = "taskExecutor")
    public ThreadPoolTaskExecutor taskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(10);
        executor.setMaxPoolSize(20);
        executor.setQueueCapacity(1000);
        executor.setRejectedExecutionHandler(
            new ThreadPoolTaskExecutor.CallerRunsPolicy()
        );
        executor.initialize();
        return executor;
    }
}

4. Monitoring and Alerting

Track these metrics religiously:

@Component
public class ExecutorMetrics {

    @Scheduled(fixedRate = 5000)
    public void logExecutorStats(ThreadPoolTaskExecutor executor) {
        log.info(
            "Executor stats - Active: {}, Queue: {}, Completed: {}",
            executor.getActiveCount(),
            executor.getThreadPoolExecutor().getQueue().size(),
            executor.getThreadPoolExecutor().getCompletedTaskCount()
        );
    }
}

Alert if:

Queue size consistently > 50% of capacity
Active threads = maxPoolSize (means you're at capacity)
Task rejection rate increases

The Mindset Shift

Here's what separates services that crash under load from those that degrade gracefully:

Dangerous mindset: "I'll use an unbounded queue as a buffer."

Safe mindset: "My queue has a limit. When I hit that limit, I stop accepting new work and return to the caller that I'm overloaded."

The second approach feels harsh—you're rejecting requests. But that's better than crashing. A rejection is honest; a crash is a lie.

One More Thing

Thread pool tuning is empirical, not theoretical. The "perfect" size for your executor depends on your latency profile, your hardware, and your workload.

Start with bounded queues and reasonable defaults. Deploy. Monitor. Adjust based on production behavior.

And if you see memory climbing during a latency spike, you already know what to look for: check your executor configuration. Odds are, somewhere there's an unbounded queue quietly queuing up requests until your JVM runs out of memory.

The fix is simple. The prevention is simpler. The cost of not doing it is expensive.

My First Open Source PR — Done During Hacktoberfest 2025!

Prathamesh Thakre — Thu, 02 Oct 2025 16:58:23 +0000

Hey devs!
After years of building full stack apps, I finally checked off something that was long overdue — my first OSS contribution!

This happened during Hacktoberfest 2025, and I wanted to share the real journey — with all the steps, wins, and lessons for beginners or even pros who haven’t yet jumped into open source.

Even though I’ve been coding professionally for years, I never actively contributed to open source.

This October, I wanted to be more than a consumer. So I searched for beginner-friendly Java issues, and I found this gem:

JEP 507 - Primitive Types in Patterns, instanceof, and switch

Goal: Add a demo for a cool upcoming Java 25 feature

My Step-by-Step OSS Journey

Here’s the exact workflow I followed, in case it helps you replicate it!

1. Find a Good Issue

Look for repos with:

hacktoberfest
help wanted
Your language of interest

2. Fork + Clone

git fork https://github.com/AloisSeckar/demos-java.git
git clone <your-fork-url>
cd demos-java

3. Understand the Code

Before writing anything, I read:

The existing demo structure

CONTRIBUTING.md
JEP 507 details (super useful!)

4. Code the Demo

I built PrimitiveTypesDemo.java, which included:

instanceof with primitives
Record pattern narrowing
Pattern-based switch blocks (with guards and fallbacks)

Also added print statements so devs could easily understand what’s happening at runtime.

5. Push and PR

git checkout -b add-jep507-demo
git add .
git commit -m "Add JEP 507 demo: instanceof and switch with primitive types"
git push origin add-jep507-demo

Then raised the Pull Request

6. Link to Issue & Thank the Maintainer

Got it linked to the issue. Said thanks to the maintainer.
Small gestures go a long way in OSS!

What I Learned

It’s not as intimidating as it looks
Reading the issue & docs carefully saves time
You don’t need to “fix bugs” to contribute — demos/docs/tests count too!
Most OSS maintainers are super welcoming

Your Turn?

Thinking of contributing? Just start. Really.

Even if you’re experienced like me, or just beginning, Hacktoberfest is the perfect excuse to finally hit that “Create Pull Request” button.

Hit me up if you’re contributing or looking for beginner-friendly Java OSS — happy to collaborate or cheer you on!