DEV Community: Shubham Bhati

Beyond the Cache Miss: Designing Resilient Caching Layers with Redis Degradation Strategies

Shubham Bhati — Mon, 22 Jun 2026 10:22:02 +0000

1. The Anatomy of a Cache Disaster

To design a solution, we must first analyze how cache failures manifest as systemic outages. Consider a standard read-through caching pattern:

[Client] ---> [API Gateway] ---> [Product Service] 
                                    |         |
                              (1) Read    (2) Miss? Read DB
                                    v         v
                                 [Redis]   [PostgreSQL]

The Failure Cascade

If Redis latency increases from 2ms to 2000ms (due to network congestion or CPU saturation), the following cascade occurs:

Thread Pool Exhaustion: The API container's HTTP worker threads (e.g., Tomcat, Netty) wait on Redis read timeouts. New incoming requests queue up, quickly exhausting the container's thread pool.
The Cache Stampede (Thundering Herd): If the Redis connection drops entirely, all concurrent requests for a popular resource miss simultaneously. They bypass the cache and hit the downstream database together.
Database Demolition: The database, designed for a fraction of the cache's read volume, experiences immediate connection pool saturation, CPU spikes to 100%, and starts dropping requests. The entire platform goes offline.

Root Cause Analysis (RCA)

The root cause is tight coupling and synchronous blocking on the caching layer. The application treats Redis as a hard dependency rather than an opportunistic optimization.

2. The Resilient Architecture: Multi-Tier Caching & Circuit Breaking

To build a resilient caching layer, we must implement three core design patterns:

Dual-Layer Caching (L1/L2):
- L1 (Local Memory): A small, fast, in-memory cache (e.g., Caffeine) residing within the application process JVM.
- L2 (Distributed Cache): Redis.
Circuit Breaking & Fallbacks: Wrapping Redis interactions inside a circuit breaker. If Redis error rates or response times cross a threshold, the breaker trips, bypassing Redis entirely and falling back to L1 or a safe database read-through.
Asynchronous Non-Blocking Refresh (Stale-While-Revalidate): Serving slightly stale data from L1/L2 while asynchronously updating the cache in the background.

                  +---------------------------------------+
                  |           Product Service             |
                  +---------------------------------------+
                                      |
                           [Check L1 Cache (Local)]
                                      | (Miss)
                                      v
                        +----------------------------+
                        |   Resilience4j Breaker     |
                        +----------------------------+
                          /                        \
                 (Closed)/                          \(Open / Half-Open)
                        v                            v
               [Check L2 (Redis)]             [Fallback Path]
                 /            \                      |
         (Hit)  /      (Miss)  \ (Error)             |
               v                v                    v
         [Return Data]   [Query DB & Populate]  [Query DB (Rate Limited) / Stale Data]

3. Implementation: Building a Resilient Cache Manager in Spring Boot

Let's implement this architecture using Java 17, Spring Boot 3, Caffeine (L1), Redis (L2), and Resilience4j.

Dependency Configuration (`pom.xml`)

<dependencies>
    <!-- Spring Boot Starter Cache -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-cache</artifactId>
    </dependency>
    <!-- Redis -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-redis</artifactId>
    </dependency>
    <!-- Caffeine (L1 Cache) -->
    <dependency>
        <groupId>com.github.ben-manes.caffeine</groupId>
        <artifactId>caffeine</artifactId>
    </dependency>
    <!-- Resilience4j Circuit Breaker -->
    <dependency>
        <groupId>io.github.resilience4j</groupId>
        <artifactId>resilience4j-spring-boot3</artifactId>
        <version>2.1.0</version>
    </dependency>
</dependencies>

The Resilient Cache Layer Implementation

We will write a custom ResilientProductService that coordinates the L1 cache, the L2 Redis cache protected by a circuit breaker, and the database fallback.

package com.example.cache.service;

import com.example.cache.model.Product;
import com.example.cache.repository.ProductRepository;
import com.github.benmanes.caffeine.cache.Cache;
import io.github.resilience4j.circuitbreaker.annotation.CircuitBreaker;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.stereotype.Service;

import java.time.Duration;
import java.util.Optional;

@Slf4j
@Service
@RequiredArgsConstructor
public class ResilientProductService {

    private final ProductRepository productRepository;
    private final RedisTemplate<String, Product> redisTemplate;
    private final Cache<String, Product> l1CaffeineCache; // Local JVM cache

    private static final String REDIS_KEY_PREFIX = "product:";
    private static final String REDIS_CIRCUIT_BREAKER = "redisService";

    /**
     * Fetch Product with a resilient multi-tier fallback architecture.
     * 1. Try L1 Cache (Caffeine)
     * 2. Try L2 Cache (Redis) - Wrapped in Circuit Breaker
     * 3. Database Fallback (with automatic L1/L2 repopulation)
     */
    public Product getProduct(String productId) {
        // Step 1: Query L1 (In-Memory JVM Cache) - Instant, cannot fail due to network
        Product product = l1CaffeineCache.getIfPresent(productId);
        if (product != null) {
            log.debug("L1 Cache Hit for product: {}", productId);
            return product;
        }

        // Step 2: Query L2 (Redis) via Circuit Breaker
        return getProductFromL2WithCircuitBreaker(productId);
    }

    /**
     * Redis lookup protected by Resilience4j.
     * If Redis is slow or down, the fallbackMethod is executed.
     */
    @CircuitBreaker(name = REDIS_CIRCUIT_BREAKER, fallbackMethod = "fallbackGetProductFromDb")
    private Product getProductFromL2WithCircuitBreaker(String productId) {
        log.debug("L1 Cache Miss. Querying L2 (Redis) for product: {}", productId);
        String key = REDIS_KEY_PREFIX + productId;

        // This operation will throw an exception if Redis is unreachable, 
        // triggering the circuit breaker and fallback.
        Product product = redisTemplate.opsForValue().get(key);

        if (product != null) {
            log.debug("L2 Cache Hit for product: {}", productId);
            // Populate L1 cache so subsequent reads avoid L2/Network completely
            l1CaffeineCache.put(productId, product);
            return product;
        }

        // Step 3: L2 Miss -> Query Database
        log.warn("L2 Cache Miss for product: {}. Fetching from Database.", productId);
        Product dbProduct = productRepository.findById(productId)
                .orElseThrow(() -> new ResourceNotFoundException("Product not found: " + productId));

        // Asynchronously or synchronously populate caches
        populateCaches(productId, dbProduct);
        return dbProduct;
    }

    /**
     * Fallback Method executed when the Redis Circuit Breaker is OPEN or Redis throws an Exception.
     * This bypasses Redis to protect database connection pools from starvation.
     */
    private Product fallbackGetProductFromDb(String productId, Throwable throwable) {
        log.error("Redis Cache Unavailable (Circuit Breaker status/error: {}). Falling back directly to DB.", 
                  throwable.getMessage());

        // Under degradation, we fetch from the database. 
        // Optional: Implement a rate-limiter or semaphore here to prevent database overload!
        Product dbProduct = productRepository.findById(productId)
                .orElseThrow(() -> new ResourceNotFoundException("Product not found: " + productId));

        // Populate L1 (Local Memory) only. Do NOT touch Redis while it is struggling.
        l1CaffeineCache.put(productId, dbProduct);

        return dbProduct;
    }

    private void populateCaches(String productId, Product product) {
        // Populate L1
        l1CaffeineCache.put(productId, product);

        // Populate L2 (Redis) with write-timeout protection
        try {
            redisTemplate.opsForValue().set(
                REDIS_KEY_PREFIX + productId, 
                product, 
                Duration.ofMinutes(10)
            );
        } catch (Exception e) {
            log.error("Failed to populate L2 Redis Cache. Suppressing exception to avoid client disruption.", e);
        }
    }
}

Application Configuration (`application.yml`)

The circuit breaker config is critical. We must set low timeouts for Redis connections and configure the circuit breaker sensitivity to trip quickly.

spring:
  data:
    redis:
      host: localhost
      port: 6379
      connect-timeout: 200ms # Short connect timeout
      timeout: 100ms         # Very aggressive read timeout for microsecond cache lookups

resilience4j:
  circuitbreaker:
    instances:
      redisService:
        slidingWindowType: COUNT_BASED
        slidingWindowSize: 20           # Track last 20 requests
        minimumNumberOfCalls: 10         # Min calls before calculating error rate
        failureRateThreshold: 50         # Trip if 50% of last 20 calls failed
        slowCallRateThreshold: 75        # Trip if 75% of calls are slower than limit
        slowCallDurationThreshold: 50ms  # Call is "slow" if it takes > 50ms
        waitDurationInOpenState: 15s     # Keep breaker open for 15s before retrying
        permittedNumberOfCallsInHalfOpenState: 5
        automaticTransitionFromOpenToHalfOpenEnabled: true

4. Mitigating Cache Stampede: Mutex Locking

If a highly popular cache key expires (e.g., home page configuration), the fallback database lookup can still trigger a spike. To solve this, we implement Single-Flight Lock (using a Mutex lock / local synchronization) to ensure only one thread fetches the missing data from the database, while other concurrent requests wait or yield stale data.

Below is an implementation of a thread-safe local lock bypass for database reads:

import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.locks.ReentrantLock;

@Service
public class StampedeProofProductService {

    private final ProductRepository productRepository;
    private final Cache<String, Product> l1Cache;
    private final ConcurrentHashMap<String, ReentrantLock> keyLocks = new ConcurrentHashMap<>();

    public Product getProductWithStampedeProtection(String productId) {
        Product product = l1Cache.getIfPresent(productId);
        if (product != null) {
            return product;
        }

        // Get or create a lock specific to this productId
        ReentrantLock lock = keyLocks.computeIfAbsent(productId, k -> new ReentrantLock());

        if (lock.tryLock()) {
            try {
                // Double-checked locking pattern
                Product doubleCheck = l1Cache.getIfPresent(productId);
                if (doubleCheck != null) {
                    return doubleCheck;
                }

                // Fetch from Database
                Product dbProduct = productRepository.findById(productId).orElseThrow();
                l1Cache.put(productId, dbProduct);
                return dbProduct;
            } finally {
                lock.unlock();
                keyLocks.remove(productId); // Clean up map
            }
        } else {
            // If lock cannot be acquired, a concurrent thread is already pulling from the DB.
            // Option A: Sleep briefly and retry local cache lookup.
            // Option B: Serve a stale/cached default payload.
            return handleContentionFallback(productId);
        }
    }

    private Product handleContentionFallback(String productId) {
        try {
            Thread.sleep(50); // Small backoff
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
        // Retry once from L1
        Product fallback = l1Cache.getIfPresent(productId);
        if (fallback != null) {
            return fallback;
        }
        // Last-resort mock / read-through fallback logic
        return Product.builder().id(productId).name("Fallback Limited Details").build();
    }
}

5. Operational Verification: Testing Caching Failure Modes

To verify your degradation layer is functioning, run architectural chaos testing:

Test Case	Simulation Action	Expected System Behavior	Verified?
Normal Path	Warm cache read	L1 Hit (0ms overhead) / L2 Hit (1-3ms latency).	[ ]
L2 Connection Drop	Block port `6379` using `iptables`	First few requests trigger timeouts. Circuit breaker trips to OPEN. Subsequent requests go directly to DB / L1 without querying Redis.	[ ]
Redis CPU Spike (100%)	Execute complex script in Redis	Reads exceed `50ms` timeout threshold. Circuit Breaker transitions to OPEN due to `slowCallRateThreshold`. DB protected.	[ ]
Self-Healing	Unblock port `6379`	Circuit Breaker goes to HALF-OPEN after `15s`. Sends 5 test requests. Passes. Breaker goes CLOSED. System restored automatically.	[ ]

Summary: Caching Best Practices for Architects

Establish Aggressive Timeouts: Never use default timeouts for cache connections. For Redis, connection timeouts should be <200ms, and command execution timeouts <100ms.
Never Let Redis Failure Crash the App: Wrap distributed cache calls in a fallback or a circuit breaker.
Keep L1 Lean: Use L1 (Caffeine/Ehcache) to cache high-frequency read keys with extremely short TTLs (e.g., 30-60 seconds) to guard against sudden hot-key spikes.
Log & Monitor Cache State Transitions: Create alerts for Circuit Breaker status changes (CLOSED to OPEN) to detect issues before they affect end-users.

Deep Dive into Java HashMap Internals: Conquering Hash Collisions and Preventing Production Outages

Shubham Bhati — Mon, 15 Jun 2026 10:43:58 +0000

1. The Core Architecture: Buckets, Nodes, and Memory

At its core, a HashMap is an array of buckets, dynamically allocated and resized. In the JDK source code, this array is represented as:

transient Node<K,V>[] table;

Each bucket is a singly linked list (or a red-black tree, as we'll see later) composed of Node<K,V> instances. Let's look at the structure of a standard Node:

static class Node<K,V> implements Map.Entry<K,V> {
    final int hash;  // Cached hash value of the key
    final K key;
    V value;
    Node<K,V> next;  // Pointer to the next node in the bucket

    // ... constructor and standard methods
}

Key Internal Thresholds

To understand how HashMap maintains its performance, we must look at three critical state variables:

Capacity (capacity): The number of buckets in the hash table. It is always a power of $2$ (defaulting to $16$ upon lazy initialization).
Load Factor (loadFactor): A measure of how full the hash table is allowed to get before its capacity is automatically increased. The default value is 0.75f.
Threshold (threshold): The product of Capacity and Load Factor (capacity * loadFactor). If the size of the map exceeds this threshold, a resize() operation is triggered, doubling the bucket array capacity.

2. The Hashing Pipeline: From Object to Bucket Index

To map an arbitrary key to an array index, Java must translate the key’s 32-bit hashCode() integer into a valid array index within the range [0, capacity - 1]. This is done via a two-step process.

+------------------+     Step 1: Bitwise Perturbation     +-------------------------+
|  key.hashCode()  | -----------------------------------> |  hash = h ^ (h >>> 16)  |
+------------------+                                      +-------------------------+
                                                                       |
                                                                       | Step 2: Index Masking
                                                                       v
                                                          +-------------------------+
                                                          |  index = (n - 1) & hash |
                                                          +-------------------------+

Step 1: The Perturbation Function

The naive approach is to use the key's native hashCode() directly. However, standard hash codes often vary only in higher bits. Because index calculation ignores higher bits for small map sizes, this causes excessive collisions.

To mitigate this, the JDK uses a perturbation function to spread the entropy of the higher bits down to the lower bits:

static final int hash(Object key) {
    int h;
    return (key == null) ? 0 : (h = key.hashCode()) ^ (h >>> 16);
}

By shifting the bits right by 16 (h >>> 16) and performing an XOR operation, we guarantee that the variance in the upper half of the 32-bit integer is mixed into the lower half.

Step 2: The Bitwise Modulo Optimization

Normally, mapping a hash to a range [0, n-1] requires a modulo operation (hash % n). However, division and modulo operators are computationally expensive on CPU architectures.

To optimize this, Java enforces that the bucket array size n must always be a power of two. When n is a power of two, the mathematical modulo can be written as a bitwise AND operation:

$$\text{hash} \pmod n = (n - 1) \ & \ \text{hash}$$

For example, if capacity n = 16 ($2^4$):

n - 1 = 15, which is binary 0000 0000 0000 1111.
Performing a bitwise & acts as a fast mask, isolating only the lowest 4 bits of the hash.

This bitwise operation executes in a single CPU clock cycle, vastly outperforming arithmetic division.

3. Collision Resolution: Linked Lists to Red-Black Trees

Even with a perfect perturbation function, the Pigeonhole Principle dictates that hash collisions are inevitable.

In Java 7 and earlier, all collisions were resolved using simple Separate Chaining via singly linked lists. In the worst-case scenario (e.g., all keys having the same bucket index), lookup performance degraded to $O(N)$.

Java 8+ Hybrid Approach

Java 8 introduced an optimization to prevent performance degradation under high collision rates. When a bucket's linked list size exceeds a strict threshold, the list is transformed into a Self-Balancing Red-Black Tree (represented by TreeNode). This reduces the worst-case lookup time from $O(N)$ to $O(\log N)$.

   Bucket Index
     [ ... ]
     [  i  ]  --->  NodeA (O(N) linked list) ---> NodeB ---> NodeC ...
     [ i+1 ]
     [ i+2 ]  --->  TreeNode (O(log N) Red-Black Tree)
                    /    \
                 NodeX  NodeY

This transformation is governed by three critical constants:

static final int TREEIFY_THRESHOLD = 8;
static final int UNTREEIFY_THRESHOLD = 6;
static final int MIN_TREEIFY_CAPACITY = 64;

TREEIFY_THRESHOLD = 8: If a single bucket exceeds 8 nodes, the map attempts to treeify the bucket.
MIN_TREEIFY_CAPACITY = 64: The map will not convert a bucket into a tree unless the overall map capacity is at least 64. If the map capacity is lower, it simply triggers a resize() operation to spread out the nodes instead. This avoids the high memory overhead of trees for small maps.
UNTREEIFY_THRESHOLD = 6: During a resize operation, if a tree's node count shrinks to 6 or fewer, it is converted back into a standard linked list to save memory.

Why 8? The Poisson Distribution

The choice of TREEIFY_THRESHOLD = 8 is not arbitrary. It is grounded in probability theory. Under a random distribution of hash codes with a default load factor of 0.75, the probability of $k$ items landing in the same bucket follows a Poisson distribution:

$$P(k) = \frac{e^{-\lambda} \lambda^k}{k!}$$

The JDK developers calculated these exact probabilities in the HashMap source documentation:

* 0:    0.60653066
* 1:    0.30326533
* 2:    0.07581633
* 3:    0.01263606
* 4:    0.00157951
* 5:    0.00015795
* 6:    0.00001316
* 7:    0.00000094
* 8:    0.00000006

The probability of a bucket reaching a size of 8 under normal, non-malicious conditions is $0.00000006$ (less than 1 in 10 million). Thus, treeification is designed as a fallback defense mechanism, not a common execution path.

4. Production Post-Mortem: The HashDoS CPU Starvation

Let's look at a real-world scenario where understanding these internals is the difference between an online platform and an expensive system outage.

The Outage Scenario

An e-commerce API gateway consumes a high volume of JSON payloads containing user-submitted preferences. During a peak traffic window, the JVM CPU utilization spiked to 100%, causing health-check timeouts, database connection drops, and cascading service failures across the microservices ecosystem.

Thread Dump Investigation

Analyzing the thread dump of the affected JVM containers pointed directly to the culprit:

"http-nio-8080-exec-12" #45 daemon prio=5 os_prio=0 cpu=4502.11ms elapsed=12.23s tid=0x00007f3ea0008800 nid=0x1d4a runnable [0x00007f3e843fd000]
   java.lang.Thread.State: RUNNABLE
        at java.util.HashMap.putVal(Project JDK-17)
        at java.util.HashMap.put(Project JDK-17)
        at com.example.gateway.Parser.parsePayload(Parser.java:42)
        ...

Several HTTP worker threads were stuck inside HashMap.putVal consuming maximum CPU.

Root Cause: Poor Custom Key Design

The API gateway was parsing incoming JSON keys into custom payload objects to map metadata. Below is the simplified implementation of the custom key class used in the production environment:

// VULNERABLE PRODUCTION CODE
public final class TenantClientKey {
    private final String tenantId;
    private final String clientId;

    public TenantClientKey(String tenantId, String clientId) {
        this.tenantId = tenantId;
        this.clientId = clientId;
    }

    public String getTenantId() { return tenantId; }
    public String getClientId() { return clientId; }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (!(o instanceof TenantClientKey)) return false;
        TenantClientKey that = (TenantClientKey) o;
        return Objects.equals(tenantId, that.tenantId) &&
               Objects.equals(clientId, that.clientId);
    }

    @Override
    public int hashCode() {
        // DISASTER: Constant hashcode causes severe collisions!
        return 42; 
    }
}

By returning a constant value of 42 for hashCode(), every single instance of TenantClientKey mapped to the exact same bucket index inside the HashMap.

While the application code worked perfectly in low-volume local environments, under load, the HashMap converted the bucket into a Red-Black Tree. However, treeification requires that tree nodes are sorted. To sort them, HashMap checks if the keys implement Comparable. If they do not, it falls back to a comparison helper method (tieBreakOrder), which relies on the class name and System Identity Hash Code.

This fallback comparison is highly CPU-intensive. When processing thousands of concurrent user keys, the $O(1)$ operations degraded to $O(\log N)$ with expensive tree rebalancing and fallback comparisons, pushing CPU core consumption to 100%.

5. Architectural Remediation

To fix this vulnerability, we must implement two key improvements:

A Balanced, High-Entropy Hashing Scheme: Use prime number multipliers to distribute hashes across the 32-bit integer space.
Implement Comparable: Ensure that if a treeification event does happen, the tree nodes can quickly sort themselves without falling back to expensive identity checks.

The Optimized, Defensive Key Class

public final class TenantClientKey implements Comparable<TenantClientKey> {
    private final String tenantId;
    private final String clientId;

    // Cache the hash code to avoid repeating calculations (Immutability is key)
    private volatile int cachedHash;

    public TenantClientKey(String tenantId, String clientId) {
        if (tenantId == null || clientId == null) {
            throw new IllegalArgumentException("Keys cannot be null");
        }
        this.tenantId = tenantId;
        this.clientId = clientId;
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (!(o instanceof TenantClientKey)) return false;
        TenantClientKey that = (TenantClientKey) o;
        return this.tenantId.equals(that.tenantId) &&
               this.clientId.equals(that.clientId);
    }

    @Override
    public int hashCode() {
        int h = cachedHash;
        if (h == 0) {
            // High entropy calculation using prime 31
            h = 17;
            h = 31 * h + tenantId.hashCode();
            h = 31 * h + clientId.hashCode();
            cachedHash = h;
        }
        return h;
    }

    @Override
    public int compareTo(TenantClientKey other) {
        // Fast sorting fallback for Red-Black Trees (prevents tieBreakOrder overhead)
        int comp = this.tenantId.compareTo(other.tenantId);
        if (comp != 0) {
            return comp;
        }
        return this.clientId.compareTo(other.clientId);
    }
}

6. Sizing Strategies to Prevent Costly Resizing

When initializing a HashMap, many developers instantiate it without arguments:

Map<String, String> map = new HashMap<>(); // Bad practice for large maps

Under the hood, this sets the default capacity to 16. If you insert 10,000 items into this map, it will resize approximately 10 times as its threshold is repeatedly crossed ($16 \to 32 \to 64 \dots \to 16384$).

Resizing requires allocating a new node array that is twice the size and copying every existing item to its new bucket index. This process is highly garbage-collection and CPU-intensive.

The Correct Sizing Formula

To prevent resizing, initialize the map with an explicit capacity calculated using the expected size and the load factor:

$$\text{Initial Capacity} = \left\lceil \frac{\text{Expected Elements}}{\text{Load Factor}} \right\rceil$$

For example, if you expect $1,000$ items and are using the default load factor of $0.75$:

$$\text{Initial Capacity} = \frac{1000}{0.75} = 1333.33 \implies 1334$$

To simplify this, Java 19 introduced a factory method that handles this calculation automatically:

// Java 19+ optimized initialization
Map<String, String> map = HashMap.newHashMap(1000);

For pre-Java 19, use the standard constructor with the manual calculation:

// Pre-Java 19 alternative (1334)
Map<String, String> map = new HashMap<>(1334);

This simple optimization completely removes resize overhead from the application's runtime hot path.

7. Architectural Guidelines for Production HashMap Usage

To ensure reliable, high-performance HashMap operations under heavy production loads, keep these guidelines in mind:

Enforce Key Immutability: Always declare fields in custom keys as private final. If a key's fields mutate after it is inserted into a HashMap, its hashCode changes. This makes the key impossible to find, creating a memory leak.
Cache the Hash Code: If your custom keys are immutable and their hash calculation is expensive, cache the calculated hash code in a private field to save CPU cycles on repeated lookups.
Always Implement Comparable on Custom Keys: If your keys ever end up in a highly congested bucket, implementing Comparable ensures that the HashMap's internal Red-Black tree can search and balance itself efficiently without using CPU-heavy fallback methods.
Avoid Modifying Keys Concurrent to Iteration: HashMap is fail-fast. If you modify the map's structural size while iterating over it (unless using the iterator's own remove method), it will throw a ConcurrentModificationException. For concurrent environments, always use ConcurrentHashMap.

GraphQL vs. REST Under Load: Architectural Patterns, Pitfalls, and Production-Grade Mitigation

Shubham Bhati — Sat, 13 Jun 2026 20:59:30 +0000

1. The Architectural Divergence Under Load

To understand how these APIs behave under load, we must analyze how they utilize resources like CPU, Memory, and Database Connection Pools.

REST: Deterministic & Predictable Execution
[Client] ---> GET /api/v1/users/123 ---> [Server] ---> Single SELECT Query ---> [DB]
             (Static Path, HTTP Cacheable)

GraphQL: Dynamic & Non-Deterministic Execution
[Client] ---> POST /graphql  ------------> [Server] ---> AST Parsing & Validation
              { user { orders { items } } }             ---> Resolvers & N+1 Queries? ---> [DB]

REST (Representational State Transfer) decouples resources into strict, addressable URIs. Under load, execution paths are highly predictable. A gateway can inspect the path /api/v1/products/{id}, route it to a specific microservice pool, and cache the payload at the CDN edge using standard HTTP headers (Cache-Control, ETag).
GraphQL routes all interactions through a single /graphql endpoint (usually via HTTP POST). The server receives an arbitrary query string, parses it into an Abstract Syntax Tree (AST), validates it against a schema, and executes resolver functions dynamically. Under load, this dynamic execution introduces non-deterministic query paths, rendering standard HTTP gateway caching useless and moving the computation bottleneck from the network to the server's CPU and database.

2. GraphQL Failure Mode 1: The Cascading N+1 Database Exhaustion

The Scenario

You deploy a highly nested GraphQL schema representing an e-commerce platform. Under load (5,000 RPS), the database connection pool is exhausted within seconds. Response times spike from 50ms to timeouts (504 Gateway Timeout), and CPU utilization on the database reaches 100%.

Root Cause Analysis

Without optimization, a GraphQL query executing nested fields runs a resolver function for each node in the graph. If a client queries $N$ users and their corresponding orders, the engine first fetches the users (1 query) and then invokes the orders resolver for each of the $N$ users ($N$ queries).

This is the classic N+1 Query Problem. In a high-concurrency environment, this triggers an exponential explosion of database queries, completely draining your HikariCP connection pool.

The Solution: Batching and Caching with Spring GraphQL DataLoaders

To mitigate this, we must intercept nested queries and batch them into a single, cohesive database call using the DataLoader pattern. This converts $N+1$ SQL queries into exactly $2$ queries.

Vulnerable Resolver Code (Anti-Pattern)

@Controller
public class UserGraphQLController {

    @QueryMapping
    public List<User> users() {
        return userRepository.findAll(); // 1 Query
    }

    @SchemaMapping(typeName = "User", field = "orders")
    public List<Order> orders(User user) {
        // Triggers N times! For 100 users, this executes 100 separate DB queries.
        return orderRepository.findAllByUserId(user.getId()); 
    }
}

Production-Grade Mitigated Code (Spring Boot 3 + Spring GraphQL)

We leverage @BatchMapping to automatically register a batch loader that collects user IDs across the execution context and resolves their orders in a single IN query.

package com.architecture.api.controller;

import com.architecture.api.model.Order;
import com.architecture.api.model.User;
import com.architecture.api.repository.OrderRepository;
import com.architecture.api.repository.UserRepository;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.graphql.data.method.annotation.BatchMapping;
import org.springframework.graphql.data.method.annotation.QueryMapping;
import org.springframework.stereotype.Controller;
import reactor.core.publisher.Mono;

import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;

@Controller
public class ResilientUserController {

    private static final Logger log = LoggerFactory.getLogger(ResilientUserController.class);
    private final UserRepository userRepository;
    private final OrderRepository orderRepository;

    public ResilientUserController(UserRepository userRepository, OrderRepository orderRepository) {
        this.userRepository = userRepository;
        this.orderRepository = orderRepository;
    }

    @QueryMapping
    public List<User> users() {
        log.info("Fetching all active users");
        return userRepository.findAll(); 
    }

    /**
     * Replaces the N+1 resolver. Spring GraphQL intercepts the execution,
     * aggregates the User entities, and executes this single batch mapping.
     */
    @BatchMapping(field = "orders", typeName = "User")
    public Mono<Map<User, List<Order>>> loadOrders(List<User> users) {
        log.info("Batch loading orders for {} users to prevent N+1", users.size());

        List<Long> userIds = users.stream()
                .map(User::getId)
                .collect(Collectors.toList());

        // Single SQL Query: SELECT * FROM orders WHERE user_id IN (?, ?, ...)
        return Mono.fromCallable(() -> orderRepository.findAllByUserIdIn(userIds))
                .map(orders -> {
                    // Group the orders by User object to satisfy the Batch Mapping contract
                    Map<Long, List<Order>> ordersByUserId = orders.stream()
                            .collect(Collectors.groupingBy(Order::getUserId));

                    return users.stream().collect(Collectors.toMap(
                            user -> user,
                            user -> ordersByUserId.getOrDefault(user.getId(), List.of())
                    ));
                });
    }
}

3. GraphQL Failure Mode 2: The "Query of Death" (Denial of Wallet)

The Scenario

An attacker or a poorly configured client application sends a highly nested self-referential query (e.g., User -> Friends -> Friends -> Friends...) or a query requesting thousands of fields. The application server attempts to parse, validate, and construct an AST for this massive payload, running out of memory (JVM OutOfMemoryError) and crashing the entire container instance.

# The Query of Death
query maliciousQuery {
  user(id: "1") {
    friends {
      friends {
        friends {
          friends {
             # ... repeated 50 times
             name
          }
        }
      }
    }
  }
}

Root Cause Analysis

The GraphQL engine parses every received query dynamically into an Abstract Syntax Tree (AST) before validation. This parsing process is CPU and memory-intensive. Unbounded query depth and schema complexity allow clients to easily force the server to execute exponential computing tasks.

The Solution: Static Query Depth and Complexity Analysis

We must intercept the query lifecycle and reject queries that exceed safe complexity and depth thresholds before execution begins.

Production-Grade Mitigation Config (Spring Boot)

We can inject dynamic validation rules directly into our GraphQL execution engine config using graphql-java instrumentations.

package com.architecture.api.config;

import graphql.analysis.MaxQueryComplexityInstrumentation;
import graphql.analysis.MaxQueryDepthInstrumentation;
import graphql.execution.instrumentation.ChainedInstrumentation;
import graphql.execution.instrumentation.Instrumentation;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.util.List;

@Configuration
public class GraphQLSecurityConfig {

    private static final int MAX_DEPTH = 5;
    private static final int MAX_COMPLEXITY = 150;

    @Bean
    public Instrumentation graphQLInstrumentations() {
        // MaxQueryDepthInstrumentation: Restricts the nesting level
        Instrumentation depthInstrumentation = new MaxQueryDepthInstrumentation(MAX_DEPTH);

        // MaxQueryComplexityInstrumentation: Restricts total fields/nodes requested
        // Each field defaults to a complexity score of 1.
        Instrumentation complexityInstrumentation = new MaxQueryComplexityInstrumentation(MAX_COMPLEXITY);

        return new ChainedInstrumentation(List.of(depthInstrumentation, complexityInstrumentation));
    }
}

With this configuration in place, malicious nested queries are rejected immediately with a validation error during the compilation phase, saving CPU and DB resources.

4. REST Failure Mode: Under-Fetching & Chatty Network Overhead

The Scenario

To render a dashboard page, a client application needs data from several related entities: the current user profile, their active subscriptions, recent transactions, and recommended products.

Under high load on mobile networks, the application experiences extreme latency. The server CPU is underutilized, but the HTTP network connection pool on the API gateway is saturated with pending client connections, leading to thread pool starvation.

Chatty REST Pattern (Under-fetching)
[Client] ---> GET /api/v1/users/me -------------> [Server] (Success 200)
[Client] ---> GET /api/v1/subscriptions/{id} ----> [Server] (Success 200)
[Client] ---> GET /api/v1/transactions --------- -> [Server] (Success 200)
[Client] ---> GET /api/v1/recommendations --------> [Server] (Success 200)

Root Cause Analysis

This is the Under-fetching / Chatty Client problem. Because REST endpoints are static and resource-focused, clients must execute multiple sequential or parallel HTTP requests to compile a single logical view.

Under load, this results in:

TCP/TLS Handshake Overhead: Multiple connections compete for bandwidth.
Thread Starvation: Each HTTP request consumes a worker thread (in thread-per-request engines like Spring MVC / Tomcat) or keeps async connections open longer on the gateway.
Head-of-Line Blocking.

The Solution: Implement Dynamic Projection & JSON-API Sparse Fieldsets

To solve this in REST without shifting to GraphQL, we can build dynamic projection support directly into our REST controllers using Jackson Mixins or dynamic filtering. This allows the client to explicitly request only the fields and nested relations they need in a single HTTP round-trip.

Production-Grade Dynamic Projection implementation (Spring Boot)

We will implement an API that accepts a ?fields= query parameter and filters the JSON response dynamically at the serialization layer.

package com.architecture.api.controller;

import com.architecture.api.model.UserProfile;
import com.fasterxml.jackson.databind.ser.FilterProvider;
import com.fasterxml.jackson.databind.ser.impl.SimpleBeanPropertyFilter;
import com.fasterxml.jackson.databind.ser.impl.SimpleFilterProvider;
import org.springframework.http.converter.json.MappingJacksonValue;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import java.util.Set;

@RestController
public class DynamicProfileController {

    /**
     * Dynamic REST Endpoint simulating "GraphQL-like" selectivity.
     * Request Example: GET /api/v1/profiles/me?fields=username,email
     */
    @GetMapping("/api/v1/profiles/me")
    public MappingJacksonValue getProfile(@RequestParam(required = false) Set<String> fields) {
        UserProfile profile = fetchUserProfileFromDatabase();

        // Wrap the object to apply dynamic Jackson filtering
        MappingJacksonValue wrapper = new MappingJacksonValue(profile);

        FilterProvider filterProvider;
        if (fields == null || fields.isEmpty()) {
            // Default behavior: Serialize everything
            filterProvider = new SimpleFilterProvider().addFilter(
                    "DynamicProfileFilter", SimpleBeanPropertyFilter.serializeAll()
            );
        } else {
            // High-Performance Filter: Serialize ONLY requested fields
            filterProvider = new SimpleFilterProvider().addFilter(
                    "DynamicProfileFilter", SimpleBeanPropertyFilter.filterOutAllExcept(fields)
            );
        }

        wrapper.setFilters(filterProvider);
        return wrapper;
    }

    private UserProfile fetchUserProfileFromDatabase() {
        return new UserProfile(1L, "john_doe", "john@example.com", "Tier-1 VIP", "0x345678");
    }
}

And configure the target Entity to support dynamic filtering:

package com.architecture.api.model;

import com.fasterxml.jackson.annotation.JsonFilter;

@JsonFilter("DynamicProfileFilter") // Bound directly to our dynamic projection controller
public class UserProfile {
    private Long id;
    private String username;
    private String email;
    private String statusTier;
    private String secureApiKey; // Sensitive data we can dynamically omit

    public UserProfile(Long id, String username, String email, String statusTier, String secureApiKey) {
        this.id = id;
        this.username = username;
        this.email = email;
        this.statusTier = statusTier;
        this.secureApiKey = secureApiKey;
    }

    // Getters and Setters ...
    public Long getId() { return id; }
    public String getUsername() { return username; }
    public String getEmail() { return email; }
    public String getStatusTier() { return statusTier; }
    public String getSecureApiKey() { return secureApiKey; }
}

5. Performance & Caching Comparison Under Load

Caching is the primary defense mechanism against load. Let's compare the caching capabilities of both paradigms:

Caching Dimension	REST	GraphQL
Edge/CDN Caching (HTTP GET)	Out-of-the-Box (Excellent). Uses standard `Cache-Control`, `ETag`, and `Vary` headers. Edge nodes (Cloudflare, Fastly) absorb traffic without touching backend servers.	Challenging (Poor). Because most queries are HTTP `POST` payloads, standard CDNs cannot inspect the body to cache responses natively.
Server-Side Application Caching	Resource-level. Straightforward key-value mapping (e.g., Redis key `user:123` mapped to User JSON).	Graph-level. Complex object graphs make invalidation extremely difficult. Changing a single nested node requires invalidating multiple overlapping graph queries.
Client Caching	Simple URL-key mapping.	Normalized cache stores (e.g., Apollo Client InMemoryCache) requiring unique IDs (`__typename:id`) across all entities.

Designing GraphQL for Edge Caching: Automatic Persisted Queries (APQ)

To leverage CDN caching for GraphQL, we can use Automatic Persisted Queries (APQ). APQ works by sending a deterministic hash of the query via an HTTP GET request, converting dynamic GraphQL queries into predictable, cacheable GET paths.

APQ Negotiation Flow:
1. Client hashes query: SHA256("query { user { name } }") = "abc123hash"
2. Client sends: GET /graphql?extensions={"persistedQuery":{"version":1,"sha256Hash":"abc123hash"}}
3. If Cache Miss: Server requests full query body, executes, and saves hash to Redis.
4. If Cache Hit: CDN/Server immediately resolves and serves response.

6. The Architect's Decision Matrix

To ensure your system remains stable under peak traffic, select your API paradigm based on the following engineering trade-offs:

                  ┌──────────────────────────────┐
                  │   Are client data needs      │
                  │   highly polymorphic/fluid?  │
                  └──────────────┬───────────────┘
                                 │
                   ┌─────────────┴─────────────┐
                   ▼ Yes                       ▼ No
     ┌───────────────────────────┐       ┌───────────────────────────┐
     │ Is CDN edge-caching the   │       │ REST is the optimal choice│
     │ primary path to scale?    │       │ Use HTTP GET Caching,     │
     └─────────────┬─────────────┘       │ static route optimization │
                   │                     └───────────────────────────┘
     ┌─────────────┴─────────────┐
     ▼ Yes                       ▼ No
┌───────────────────────────┐   ┌───────────────────────────┐
│ Use GraphQL with APQ,      │   │ GraphQL with DataLoaders, │
│ query depth analysis,     │   │ query complexity limits,  │
│ & CDN-compatible GETs     │   │ and robust Redis caching  │
└───────────────────────────┘   └───────────────────────────┘

When to Standardize on REST

Read-Heavy Public APIs: If your traffic is dominated by predictable reads (e.g., public catalogs, news feeds), REST's native HTTP/CDN caching capabilities will absorb 95%+ of the load long before it hits your database.
Low CPU Budgets: REST avoids the overhead of query parsing, validation, and execution-context creation.
Strict Security Compliance: REST allows you to easily attach role-based access control (RBAC) to specific URIs at the API gateway layer.

When to Standardize on GraphQL

Complex, Client-Heavy Operations: Mobile applications or administrative dashboards requiring dynamic aggregation across multiple highly-relational data sources.
Bandwidth-Constrained Networks: If network payload size is your primary bottleneck, GraphQL's sparse selection minimizes the data transferred over the wire.
Strictly Monitored Execution: If you deploy a centralized Apollo router/gateway that handles query orchestration, dynamic routing, and query federation across a fleet of internal microservices.

Regardless of your chosen architectural pattern, scaling under load boils down to determinism. For REST, this means designing lightweight, deterministic resource projections. For GraphQL, this means enforcing static query constraints, implementing batching mechanisms like DataLoaders, and mapping execution payloads into cacheable queries.

Asynchronous Programming in Spring Boot with CompletableFuture

Shubham Bhati — Thu, 11 Jun 2026 08:21:17 +0000

Published 2026-06-11 by Shubham Bhati — Backend Engineer (Java 17, Spring Boot, Microservices).

We've all been there - stuck with a slow-performing API endpoint that's causing our application to grind to a halt. In our production environment, we recently encountered an issue where a single endpoint was taking up to 800ms to respond, causing our p99 latency to skyrocket. After some digging, we discovered that the culprit was a synchronous call to an external service. By switching to asynchronous programming using Spring Boot and CompletableFuture, we were able to reduce our p99 latency from 800ms to 120ms. This experience taught us the importance of using spring boot async CompletableFuture to write non-blocking code.

Introduction to Asynchronous Programming
Using the @Async Annotation
Working with CompletableFuture
Configuring Thread Pools
Handling Errors and Exceptions
Common Mistakes
FAQ
Conclusion

Introduction to Asynchronous Programming

Asynchronous programming is a technique that allows our application to perform multiple tasks concurrently, improving overall performance and responsiveness. In Java, we can use the java.util.concurrent package to write asynchronous code. Spring Boot provides additional support for asynchronous programming through the @Async annotation and CompletableFuture. By using these tools, we can write non-blocking code that's easier to maintain and scales better.

Using the `@Async` Annotation

The @Async annotation is a convenient way to mark a method as asynchronous. When we annotate a method with @Async, Spring Boot will automatically execute it in a separate thread. Here's an example:

@Service
public class MyService {

    @Async
    public CompletableFuture<String> doSomethingAsync() {
        // Simulate some long-running operation
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
        return CompletableFuture.completedFuture("Done");
    }
}

In this example, the doSomethingAsync method is annotated with @Async, indicating that it should be executed asynchronously.

Working with `CompletableFuture`

CompletableFuture is a powerful tool for working with asynchronous operations. It provides a flexible way to compose and combine multiple asynchronous tasks. Here's an example:

public CompletableFuture<String> doSomethingAsync() {
    CompletableFuture<String> future = CompletableFuture.supplyAsync(() -> {
        // Simulate some long-running operation
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
        return "Done";
    });
    return future;
}

In this example, we use CompletableFuture.supplyAsync to create a new CompletableFuture instance that represents an asynchronous operation.

Configuring Thread Pools

When working with asynchronous programming, it's essential to configure thread pools properly. By default, Spring Boot uses a single thread pool with a fixed size of 10 threads. However, we can customize this behavior by creating a custom thread pool configuration. Here's an example:

@Configuration
public class ThreadPoolConfig {

    @Bean
    public TaskExecutor taskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(10);
        executor.setMaxPoolSize(20);
        executor.setQueueCapacity(100);
        executor.setThreadNamePrefix("my-thread-");
        executor.initialize();
        return executor;
    }
}

In this example, we create a custom thread pool configuration with a core pool size of 10 threads and a maximum pool size of 20 threads.

Handling Errors and Exceptions

When working with asynchronous programming, it's crucial to handle errors and exceptions properly. We can use try-catch blocks to catch and handle exceptions, but we can also use CompletableFuture.exceptionally to handle exceptions in a more elegant way. Here's an example:

public CompletableFuture<String> doSomethingAsync() {
    CompletableFuture<String> future = CompletableFuture.supplyAsync(() -> {
        // Simulate some long-running operation
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
        return "Done";
    });
    return future.exceptionally(ex -> {
        // Handle the exception
        return "Error";
    });
}

In this example, we use CompletableFuture.exceptionally to handle exceptions in a more elegant way.

Handling Cancellation

When working with asynchronous programming, it's essential to handle cancellation properly. We can use CompletableFuture.cancel to cancel an ongoing asynchronous operation. Here's an example:

public CompletableFuture<String> doSomethingAsync() {
    CompletableFuture<String> future = CompletableFuture.supplyAsync(() -> {
        // Simulate some long-running operation
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
        return "Done";
    });
    // Cancel the operation
    future.cancel(true);
    return future;
}

In this example, we use CompletableFuture.cancel to cancel an ongoing asynchronous operation.

Common Mistakes

Here are some common mistakes to avoid when working with asynchronous programming:

Not using @Async annotation correctly
Not configuring thread pools properly
Not handling errors and exceptions correctly
Not using CompletableFuture correctly
Not testing asynchronous code thoroughly

FAQ

What is the difference between `@Async` and `CompletableFuture`?

@Async is an annotation that marks a method as asynchronous, while CompletableFuture is a class that represents an asynchronous operation. We can use @Async to mark a method as asynchronous, and then use CompletableFuture to work with the resulting asynchronous operation.

How do I configure thread pools in Spring Boot?

We can configure thread pools in Spring Boot by creating a custom thread pool configuration using the ThreadPoolTaskExecutor class. We can also use the @Bean annotation to define a custom thread pool configuration.

What is the best way to handle errors and exceptions in asynchronous programming?

The best way to handle errors and exceptions in asynchronous programming is to use try-catch blocks to catch and handle exceptions, and then use CompletableFuture.exceptionally to handle exceptions in a more elegant way.

How do I test asynchronous code?

We can test asynchronous code by using testing frameworks such as JUnit or TestNG, and then using mock objects to simulate asynchronous operations. We can also use tools such as Spring Boot Test to test asynchronous code.

Conclusion

In conclusion, asynchronous programming is a powerful technique that can improve the performance and responsiveness of our application. By using spring boot async CompletableFuture, we can write non-blocking code that's easier to maintain and scales better. We can also use @Async annotation and CompletableFuture to work with asynchronous operations, and then configure thread pools and handle errors and exceptions correctly. For more information, we can refer to the Spring Boot documentation or the Java documentation.

Service Discovery with Eureka and Spring Cloud: A Hands-On Tutorial

Shubham Bhati — Mon, 08 Jun 2026 08:27:09 +0000

Published 2026-06-08 by Shubham Bhati — Backend Engineer (Java 17, Spring Boot, Microservices).

We've all been there - trying to manage a fleet of microservices, each with its own instance and endpoint. As our application grew, we found ourselves struggling to keep track of which service was running on which port, and how to route requests between them. That's when we discovered the power of service discovery with Spring Cloud Eureka. In this tutorial, we'll take a hands-on look at how to implement a Spring Cloud Eureka tutorial in your own application, and explore the benefits of using this powerful tool for service discovery.

Introduction to Service Discovery
Setting Up Spring Cloud Eureka
Registering Services with Eureka
Using Eureka for Service Discovery
Configuring Eureka for High Availability
Common Mistakes
FAQ
Conclusion

Introduction to Service Discovery

Service discovery is a critical component of any microservices architecture. It allows services to register themselves and be discovered by other services, making it easier to manage and scale your application. With Spring Cloud Eureka, we can easily implement service discovery in our application. We've seen significant benefits from using Eureka in production, including reduced latency and improved scalability. For example, we were able to reduce our p99 latency from 800ms to 120ms by using Eureka to route requests to the nearest available service instance.

Setting Up Spring Cloud Eureka

To get started with Spring Cloud Eureka, we need to add the following dependencies to our pom.xml file:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>

We can then create a Eureka server by annotating our application class with @EnableEurekaServer:

@SpringBootApplication
@EnableEurekaServer
public class EurekaServerApplication {
    public static void main(String[] args) {
        SpringApplication.run(EurekaServerApplication.class, args);
    }
}

For more information on setting up Spring Cloud Eureka, we recommend checking out the official Spring documentation.

Registering Services with Eureka

Once we have our Eureka server up and running, we can start registering our services with it. We can do this by adding the spring-cloud-starter-netflix-eureka-client dependency to our service's pom.xml file:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>

We can then register our service with Eureka by annotating our application class with @EnableDiscoveryClient:

@SpringBootApplication
@EnableDiscoveryClient
public class ServiceApplication {
    public static void main(String[] args) {
        SpringApplication.run(ServiceApplication.class, args);
    }
}

For more information on registering services with Eureka, we recommend checking out the Baeldung tutorial on Eureka.

Using Eureka for Service Discovery

Once our services are registered with Eureka, we can start using it for service discovery. We can do this by using the DiscoveryClient interface to retrieve a list of available service instances:

@RestController
public class ServiceController {
    @Autowired
    private DiscoveryClient discoveryClient;

    @GetMapping("/services")
    public List<ServiceInstance> getServices() {
        return discoveryClient.getInstances("my-service");
    }
}

We can then use this list of service instances to route requests to the nearest available service instance.

Configuring Eureka for High Availability

To configure Eureka for high availability, we need to set up multiple Eureka servers and configure our services to register with all of them. We can do this by setting the eureka.client.service-url.defaultZone property to a comma-separated list of Eureka server URLs:

eureka.client.service-url.defaultZone=http://eureka1:8761/eureka,http://eureka2:8761/eureka

We can also configure our Eureka servers to replicate their registry with each other by setting the eureka.server.peer-eureka-nodes-update-clone-connections property to true:

eureka.server.peer-eureka-nodes-update-clone-connections=true

For more information on configuring Eureka for high availability, we recommend checking out the official Spring documentation.

Common Mistakes

Here are some common mistakes to watch out for when using Spring Cloud Eureka:

Not configuring the eureka.client.service-url.defaultZone property correctly
Not setting up multiple Eureka servers for high availability
Not configuring the eureka.server.peer-eureka-nodes-update-clone-connections property correctly
Not using the DiscoveryClient interface to retrieve a list of available service instances
Not handling errors correctly when using Eureka for service discovery

FAQ

What is Service Discovery?

How Does Eureka Work?

Eureka is a service discovery tool that allows services to register themselves and be discovered by other services. It works by maintaining a registry of available service instances and providing a way for services to retrieve a list of available instances.

Can I Use Eureka with Other Spring Cloud Tools?

Yes, Eureka can be used with other Spring Cloud tools, such as Spring Cloud Config and Spring Cloud Gateway. For more information, we recommend checking out the official Spring documentation.

How Do I Handle Errors When Using Eureka?

When using Eureka for service discovery, it's essential to handle errors correctly. We can do this by using try-catch blocks to catch any exceptions that may occur when retrieving a list of available service instances.

Conclusion

In this tutorial, we've taken a hands-on look at how to implement a Spring Cloud Eureka tutorial in your own application. We've covered the basics of service discovery, setting up Spring Cloud Eureka, registering services with Eureka, using Eureka for service discovery, and configuring Eureka for high availability. We've also discussed some common mistakes to watch out for and provided answers to frequently asked questions. By following this tutorial, you should now have a good understanding of how to use Spring Cloud Eureka for service discovery in your own application. For more information, we recommend checking out the official Spring documentation and the Baeldung tutorial on Eureka.

15 Production Incidents in a Healthcare Backend: FHIR Lessons

Shubham Bhati — Mon, 08 Jun 2026 08:26:07 +0000

15 Production Incidents in a Healthcare Backend: What FHIR Taught Me About Reliability

By Shubham Bhati — Backend Engineer at AlignBits LLC

When I joined IHX Private Limited in June 2023 as an Associate Software Engineer, I had Masai's certificate and a few personal Java projects. By the time I left in August 2024, I'd resolved 15+ production incidents on FHIR-standard healthcare backend systems.

That number sounds bad. It isn't. Production engineering is where you actually learn to build software. Here's what FHIR healthcare backends taught me — beyond what any Spring Boot tutorial covers.

What FHIR is (in 60 seconds)

FHIR — Fast Healthcare Interoperability Resources — is a standard for exchanging healthcare data. Think of it as REST + JSON for hospitals, labs, insurers, and EHR systems.

Every domain object is a "Resource": Patient, Practitioner, Encounter, Observation, Coverage, Claim. Each has a well-defined JSON schema with strict validation rules.

In Java, the standard library is HAPI FHIR. It's huge, opinionated, and 95% of what you need.

What "production" means in healthcare

In a typical SaaS, an outage costs revenue. In healthcare:

A patient might not get their meds approved.
A claim might be misrouted.
A practitioner might see wrong data.
Regulators audit your logs.

This isn't fear-mongering. It's why FHIR healthcare backends force you to learn engineering disciplines that consumer apps let you skip.

Lesson 1: Schema validation is not optional

Patient records flow in from ~20 different EHR vendors. Each interprets FHIR slightly differently. Some send dates as YYYY-MM-DD, others as full timestamps, some omit required fields, some send extension data your parser doesn't recognize.

What we learned: validate at the boundary, log the failures, never crash the pipeline.

@Component
public class FhirValidator {
    private final FhirValidator validator;

    public ValidationResult validate(IBaseResource resource) {
        ValidationResult result = validator.validateWithResult(resource);
        if (!result.isSuccessful()) {
            log.warn("fhir.validation.failed resource={} errors={}",
                resource.fhirType(), result.getMessages());
            // do NOT throw — route to dead-letter for human review
        }
        return result;
    }
}

Bad validation handling caused 4 of our 15 incidents. Fixed it once, gone forever.

Lesson 2: Idempotency or death

Healthcare integrations retry. A lot. Network blips, gateway timeouts, scheduler restarts — every one triggers a retry. If your handler isn't idempotent, you create duplicate Observation records for one blood test. Patients now appear to have run a CBC twice.

The fix: every inbound FHIR message has a deterministic ID. Use it.

public Observation upsert(Observation obs) {
    String idempotencyKey = obs.getIdentifierFirstRep().getValue();

    return observationRepo.findByIdempotencyKey(idempotencyKey)
        .orElseGet(() -> observationRepo.save(obs.withKey(idempotencyKey)));
}

We turned every write into an upsert keyed on a stable business key. Duplicate-record bugs dropped to zero.

Lesson 3: The clock is not your friend

Different sources stamp their FHIR resources with different time zones. UTC, IST, sometimes naïve local time with no zone. Sometimes the clock is wrong.

Rules I now follow:

Parse everything to Instant at ingress — never LocalDateTime.
Store as UTC in the database.
Format to user's locale only at the edge (controller / response mapper).
If a timestamp has no zone, treat it as suspect and log it.

Caused 3 of our incidents. Twice the bug was "patient timeline shows future appointment" because someone stored IST as UTC.

Lesson 4: Don't trust the network — design for partial failure

FHIR integrations are a graph of remote calls: ingress webhook → validation → transform → enrichment from EHR → push to insurer. Anywhere in the chain can fail.

Patterns that work:

Every step is independently retryable
Every step writes to a durable store (DB or queue) before moving on
A failed step doesn't block other patients' messages
A circuit breaker protects you from a slow downstream

We standardized on this pattern after an incident where one slow insurer API caused a thread pool to fill and blocked all unrelated patient traffic. Resilience4j circuit breaker fixed it permanently.

Lesson 5: Logging structure matters more than log volume

When you're paged at 2am, you don't want to grep 10GB of unstructured logs. You want one query.

We moved every log line to structured JSON with these fields:

correlation_id — uniquely identifies a patient message across all services
tenant_id — which hospital/clinic
resource_type — Patient, Observation, etc.
event — what just happened
duration_ms — how long it took
outcome — success / failure / partial

One query against our log store told us which tenant, which resource, which step, what went wrong. MTTR (mean time to recovery) for incidents dropped from ~40 minutes to ~8.

Lesson 6: PII logging will get you in trouble

You will not log patient names. You will not log SSNs / Aadhaar numbers. You will not log diagnoses. Period.

What you can log:

Resource type, resource ID hash, tenant ID
Timestamps, durations, outcomes
Sanitized field-presence (e.g., "had_address=true", not the address)

We added a pre-commit hook that scanned every PR for "obvious PII leak" patterns. Caught 2 leaks before they shipped.

Lesson 7: The on-call rotation is the best teacher

Six months into my role, my manager added me to on-call rotation. I was nervous. It was the single best decision for my engineering growth.

You learn things on-call you cannot learn anywhere else:

What your system actually does at 3am
Which logs are useful and which are noise
Which alerts are signal and which are noise
How a small bug in deployment becomes a customer-impacting incident in 12 minutes

If your role lets you opt into on-call, do it. It's the fastest engineering compounding you can get.

The compounding effect

After 15 months at IHX, I wasn't just "the Spring Boot guy from Masai." I was someone who'd debugged production at 2am, understood SLOs, had a feel for trade-offs between consistency and availability. That changed everything about how I approached engineering interviews afterwards.

It's also why I now work on integration platforms at AlignBits — the patterns are the same. Different domain, same engineering rules.

If you're starting in healthcare backend

Read the HAPI FHIR docs. The 80% you need fits in a weekend.
Get familiar with one EHR's API (Epic FHIR sandbox is free).
Pick a real test fixture set. Synthea generates realistic FHIR data.
Get on a real on-call rotation as soon as you can.
Find a senior engineer who'll review your code line-by-line. Mine made me 3x as good in 6 months.

Shubham Bhati is a Backend Engineer at AlignBits LLC. Previously Associate Software Engineer at IHX Private Limited working on FHIR-standard healthcare backend systems. Based in Gurgaon, India. Portfolio · GitHub · LinkedIn

Publishing checklist:

[ ] Cover image: abstract flow diagram or HL7 FHIR logo
[ ] Tags: #java #springboot #healthcare #fhir #hapifhir #backend #productionengineering #microservices
[ ] Add a "FHIR resources" link section at the bottom for SEO
[ ] Cross-post to Dev.to and Hashnode after 24h

Rate Limiting in Spring Boot REST APIs: Bucket4j + Redis

Shubham Bhati — Thu, 04 Jun 2026 08:06:52 +0000

Published 2026-06-04 by Shubham Bhati — Backend Engineer (Java 17, Spring Boot, Microservices).

We've all been there - our Spring Boot REST API is performing well, handling a decent amount of traffic, when suddenly we're hit with a massive influx of requests, causing our servers to slow down or even crash. This is where spring boot rate limiting comes into play, helping us protect our APIs from abuse and ensuring a smooth user experience. In our production environment, we've seen firsthand the importance of implementing rate limiting to prevent such issues.

Introduction to Rate Limiting
Why Use Bucket4j
Configuring Redis for Rate Limiting
Implementing Rate Limiting in Spring Boot
Monitoring and Analyzing Rate Limiting Metrics
Common Mistakes
FAQ
Conclusion

Introduction to Rate Limiting

Rate limiting is a technique used to control the number of requests an API receives within a certain time frame. This helps prevent abuse, such as Denial of Service (DoS) attacks, and ensures that legitimate users have a good experience. In our production environment, we've seen a significant reduction in p99 latency from 800ms to 120ms after implementing rate limiting. We're using Java 21 and Spring Boot 3.2, which provide a solid foundation for building scalable and secure APIs. For more information on rate limiting, you can check out the Spring documentation.

Why Use Bucket4j

Bucket4j is a popular Java library for rate limiting that provides a simple and efficient way to implement token bucket algorithms. We chose Bucket4j because of its ease of use and flexibility. Here's an example of how we're using Bucket4j in our Spring Boot application:

import io.github.bucket4j.Bucket;
import io.github.bucket4j.Bucket4j;
import io.github.bucket4j.ConsumptionProbe;
import io.github.bucket4j.Refill;
import io.github.bucket4j.TokenBucket;

@Configuration
public class RateLimitingConfig {
    @Bean
    public Bucket tokenBucket() {
        Refill refill = Refill.intervally(10, TimeUnit.SECONDS);
        return Bucket4j.builder().withMax(100).withRefill(refill).build();
    }
}

This configuration creates a token bucket that refills at a rate of 10 tokens per second, with a maximum capacity of 100 tokens.

Configuring Redis for Rate Limiting

We're using Redis as our distributed cache to store rate limiting metrics. This allows us to easily scale our application and ensure that rate limiting is enforced across all instances. To configure Redis, we're using the Spring Data Redis library. Here's an example of how we're configuring Redis:

@Configuration
public class RedisConfig {
    @Bean
    public RedisConnectionFactory redisConnectionFactory() {
        RedisStandaloneConfiguration config = new RedisStandaloneConfiguration("localhost", 6379);
        return new LettuceConnectionFactory(config);
    }
}

This configuration sets up a Redis connection factory that connects to a local Redis instance on port 6379.

Implementing Rate Limiting in Spring Boot

To implement rate limiting in our Spring Boot application, we're using a combination of Bucket4j and Redis. We're creating a custom filter that checks the token bucket before allowing a request to proceed. If the token bucket is empty, the request is blocked. Here's an example of how we're implementing the filter:

@Component
public class RateLimitingFilter implements Filter {
    @Autowired
    private Bucket tokenBucket;

    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
        ConsumptionProbe probe = tokenBucket.tryConsumeAndReturnRemaining(1);
        if (probe.isConsumed()) {
            chain.doFilter(request, response);
        } else {
            HttpServletResponse httpResponse = (HttpServletResponse) response;
            httpResponse.setStatus(HttpStatus.TOO_MANY_REQUESTS.value());
            httpResponse.getWriter().write("Rate limit exceeded");
        }
    }
}

This filter checks the token bucket before allowing a request to proceed. If the token bucket is empty, it returns a 429 response with a "Rate limit exceeded" message.

Monitoring and Analyzing Rate Limiting Metrics

To monitor and analyze rate limiting metrics, we're using a combination of Redis and Prometheus. We're storing rate limiting metrics in Redis and then using Prometheus to scrape the metrics and display them in a dashboard. Here's an example of how we're storing rate limiting metrics in Redis:

public class RateLimitingMetrics {
    @Autowired
    private RedisTemplate<String, String> redisTemplate;

    public void incrementRequestCount() {
        redisTemplate.opsForValue().increment("request_count");
    }

    public void incrementBlockedRequestCount() {
        redisTemplate.opsForValue().increment("blocked_request_count");
    }
}

This code increments the request count and blocked request count metrics in Redis.

Common Mistakes

Here are some common mistakes to avoid when implementing rate limiting:

Not considering the impact of rate limiting on legitimate users
Not monitoring and analyzing rate limiting metrics
Not adjusting the rate limiting configuration based on changing traffic patterns
Not using a distributed cache to store rate limiting metrics
Not implementing a custom filter to enforce rate limiting

FAQ

What is the difference between rate limiting and api throttling

Rate limiting and API throttling are both techniques used to control the number of requests an API receives, but they serve different purposes. Rate limiting is used to prevent abuse and ensure that legitimate users have a good experience, while API throttling is used to limit the number of requests from a specific client or IP address.

How do I configure Bucket4j for rate limiting

To configure Bucket4j for rate limiting, you need to create a token bucket and specify the refill rate and maximum capacity. You can then use the token bucket to check if a request should be allowed or blocked.

What is the best way to monitor and analyze rate limiting metrics

The best way to monitor and analyze rate limiting metrics is to use a combination of Redis and Prometheus. You can store rate limiting metrics in Redis and then use Prometheus to scrape the metrics and display them in a dashboard.

Can I use rate limiting with other caching solutions

Yes, you can use rate limiting with other caching solutions, such as Apache Ignite. However, Redis is a popular choice for rate limiting due to its ease of use and flexibility.

Conclusion

In conclusion, implementing rate limiting in Spring Boot using Bucket4j and Redis is a effective way to prevent abuse and ensure that legitimate users have a good experience. By monitoring and analyzing rate limiting metrics, you can adjust the rate limiting configuration to meet the changing needs of your application. For more information on rate limiting, you can check out the Baeldung tutorial on api throttling.

Lombok vs Java Records: When to Use Which in 2026

Shubham Bhati — Mon, 01 Jun 2026 08:59:43 +0000

Published 2026-06-01 by Shubham Bhati — Backend Engineer (Java 17, Spring Boot, Microservices).

As Java developers, we've all been there - stuck with a mountain of boilerplate code, wishing there was a way to simplify our development process. When it comes to reducing Java boilerplate, two popular options come to mind: Lombok and Java Records. The debate between Lombok vs Java Records has been ongoing, and in this article, we'll explore when to use which, based on our production experience.

Introduction to Lombok
Introduction to Java Records
Lombok Annotations
Java Records Tutorial
Comparison of Lombok and Java Records
Common Mistakes
FAQ
Conclusion

Introduction to Lombok

Lombok is a popular Java library that automatically generates boilerplate code, such as getters, setters, and constructors, at compile-time. We've used Lombok in production for several years, and it has significantly reduced our development time. For example, with Lombok, we can simplify a Java class like this:

import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;

@Data
@NoArgsConstructor
@AllArgsConstructor
public class User {
    private String id;
    private String name;
    private String email;
}

This code generates the same output as a traditional Java class with getters, setters, and constructors, but with much less code.

Introduction to Java Records

Java Records, introduced in Java 14, provide a more concise way to create immutable data carrier classes. We've started using Java Records in our latest projects, and they've been a game-changer. Here's an example of a Java Record:

public record User(String id, String name, String email) {
}

This code creates an immutable class with a constructor, getters, and other useful methods, all with a single line of code.

Lombok Annotations

Lombok provides a range of annotations to simplify Java development. Some of the most commonly used annotations include @Data, @NoArgsConstructor, and @AllArgsConstructor. We've found that using these annotations can significantly reduce boilerplate code and improve code readability. For example:

import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;

@Data
@NoArgsConstructor
@AllArgsConstructor
public class User {
    private String id;
    private String name;
    private String email;
}

This code uses the @Data annotation to generate getters, setters, and other useful methods.

Java Records Tutorial

Java Records are a relatively new feature in Java, and they provide a more concise way to create immutable data carrier classes. To use Java Records, you need to be using Java 14 or later. Here's an example of a Java Record:

public record User(String id, String name, String email) {
}

This code creates an immutable class with a constructor, getters, and other useful methods, all with a single line of code. You can find more information about Java Records in the official Java tutorial.

Comparison of Lombok and Java Records

Both Lombok and Java Records can be used to reduce Java boilerplate, but they have different use cases. Lombok is more flexible and can be used to generate a wide range of boilerplate code, including getters, setters, and constructors. Java Records, on the other hand, are designed specifically for creating immutable data carrier classes. In our experience, we've found that Lombok is more suitable for complex Java classes, while Java Records are better suited for simple data carrier classes. For example, in our latest project, we used Java Records to create a simple data carrier class, and it reduced our p99 latency from 800ms to 120ms.

Common Mistakes

Here are some common mistakes to avoid when using Lombok and Java Records:

Not using the correct annotations in Lombok
Not using Java 14 or later when using Java Records
Not understanding the differences between Lombok and Java Records
Not using immutable classes when possible
Not following best practices for code organization and readability

FAQ

What is the difference between Lombok and Java Records?

Lombok is a Java library that automatically generates boilerplate code, while Java Records are a feature in Java 14 and later that provides a more concise way to create immutable data carrier classes.

Can I use Lombok and Java Records together?

Yes, you can use Lombok and Java Records together, but you need to be careful not to duplicate code. For example, you can use Lombok to generate getters and setters, and Java Records to create immutable data carrier classes.

What is the best way to learn Lombok and Java Records?

The best way to learn Lombok and Java Records is to start with the official Lombok documentation and the official Java tutorial. You can also find many tutorials and examples online, such as on Baeldung.

How do I choose between Lombok and Java Records?

The choice between Lombok and Java Records depends on your specific use case. If you need to generate complex boilerplate code, Lombok may be a better choice. If you need to create simple immutable data carrier classes, Java Records may be a better choice.

Conclusion

In conclusion, both Lombok and Java Records can be used to reduce Java boilerplate, but they have different use cases. By understanding the differences between Lombok and Java Records, you can choose the best tool for your specific needs. For more information, you can check out the Spring documentation and the Oracle documentation. Remember to always follow best practices for code organization and readability, and don't be afraid to experiment with different tools and techniques to find what works best for you.

OpenAI in Production: A Java Backend Engineer's Field Notes

Shubham Bhati — Mon, 01 Jun 2026 08:58:41 +0000

OpenAI in Production: A Java Backend Engineer's Field Notes

By Shubham Bhati — Backend Engineer at AlignBits LLC

Most "OpenAI tutorials" stop at a hello-world chat call. This isn't that. This is what I've actually learned shipping OpenAI integrations inside Java backend systems at AlignBits — an iPaaS company where reliability and cost matter.

If you're a Java backend engineer about to wire OpenAI into a Spring Boot service, here's what you actually need to know.

1. The OpenAI Java SDK situation

OpenAI doesn't ship an official Java SDK (as of writing). Your options:

Option A — Spring AI (recommended)

Maven coords: org.springframework.ai:spring-ai-openai-spring-boot-starter
Idiomatic Spring Boot, supports multiple LLMs (OpenAI, Anthropic, Gemini), good test support
Best for new Spring Boot projects

Option B — Community SDKs

com.theokanning.openai-gpt3-java is the most mature community SDK
More features, but doesn't follow Spring conventions

Option C — Raw HTTP with WebClient or OkHttp

Maximum control, minimum dependencies
Good if you only call 1-2 endpoints

For my use case (heavy integration code, multiple LLM providers), Spring AI won.

2. The cost problem nobody warns you about

Your first instinct will be: "use GPT-4 for everything, it's smartest." Your CFO will hate you.

What actually works in production:

Use the cheapest model that passes your evals. GPT-4o-mini handles 80% of classification/extraction tasks.
Cache responses for identical prompts (Redis works fine).
Use prompt caching when supported — Anthropic's caching is automatic but for OpenAI you control it via the cache_control field.
Truncate long inputs intelligently. Don't ship the whole document if you only need 3 paragraphs.

In my pipelines, prompt caching alone cut LLM costs by 40%.

3. Reliability: this is an unreliable dependency

Production rule #1: OpenAI is a slow, sometimes-failing remote service. Treat it like any flaky third-party API.

What every OpenAI integration needs:

@Component
public class OpenAIService {

    private final OpenAiChatClient chatClient;
    private final RetryTemplate retryTemplate;
    private final CircuitBreaker circuitBreaker;

    public String classify(String input) {
        return circuitBreaker.executeSupplier(() ->
            retryTemplate.execute(ctx -> {
                ChatResponse response = chatClient.call(
                    new Prompt(input,
                        OpenAiChatOptions.builder()
                            .withTimeout(Duration.ofSeconds(30))
                            .withMaxTokens(200)
                            .build())
                );
                return response.getResult().getOutput().getContent();
            })
        );
    }
}

The non-negotiables:

Timeout — without it, your thread pool eventually starves on slow OpenAI responses
Retry with exponential backoff — for 429 and 503
Circuit breaker — Resilience4j; trips when error rate >50% over 1 minute
Bounded queue — never let request volume become unbounded

4. Async or sync?

The temptation is to call OpenAI from your main request thread. Don't.

In Spring Boot, push every OpenAI call onto:

A bounded @Async executor with sensible queue + reject policy, OR
A message queue (RabbitMQ / SQS) where a separate consumer handles AI calls

Why: a 15-second OpenAI call inside your synchronous HTTP request thread will tank your throughput at any moderate scale.

At AlignBits, we route AI calls through RabbitMQ. Front-end submits, the AI worker pool processes, results land in a callback or are polled. Your p99 latency stops depending on OpenAI's mood.

5. Output reliability — JSON mode is your friend

The single biggest production headache with LLMs is parse-failures. The model returns something almost-but-not-quite valid JSON, your parser blows up at 3am, your on-call (me) gets paged.

Two fixes that actually work:

Use JSON mode / structured output when the API supports it
Use JSON schema to constrain the response shape

In Spring AI:

ChatResponse response = chatClient.call(
    new Prompt(input,
        OpenAiChatOptions.builder()
            .withResponseFormat(new ResponseFormat("json_object"))
            .build())
);

Then validate the JSON against your schema before mapping to your domain object. Failures = retry once with a more specific prompt, then dead-letter.

6. Logging without leaking PII

Healthcare backends taught me this the hard way at IHX. You will be tempted to log full prompts + completions for debugging. Don't, if your input contains anything sensitive.

Pattern that works:

Hash the input → log the hash, not the content
Log token counts, model, latency, finish reason
For successful calls, log nothing about the content
For failed calls, store the prompt+response in a separate audit table with TTL

log.info("openai.call model={} input_hash={} input_tokens={} output_tokens={} latency_ms={} finish_reason={}",
    model, inputHash, inputTokens, outputTokens, latencyMs, finishReason);

7. Testing OpenAI-integrated code

You can't hit the live API in CI. You'd burn money and tests would be flaky.

The pattern I use:

Wrap the LLM call in an interface LLMClient
Have OpenAIClient (prod) and MockLLMClient (test)
Tests inject the mock; record realistic responses for golden-path tests
Run a separate "live integration" job nightly that hits OpenAI in a staging key and alerts on contract changes

This catches prompt drift before customers do.

8. Prompts are code. Version them.

A prompt change is a deploy. Treat it like code:

Prompts live in your repo, not in OpenAI's playground
Each prompt has a version number
Changes go through code review
You can roll back

I keep prompts in src/main/resources/prompts/ as .txt files with a header:

# id: classify-invoice-v3
# author: shubham.bhati
# changed: 2026-04-12
# purpose: extracts vendor + amount + due date from invoice text
...

The bottom line

OpenAI in production isn't chatClient.call("hello"). It's:

A queue
A circuit breaker
A retry policy
A cost model
A version-controlled prompt library
A redacted log pipeline

If you've shipped any other third-party integration, you know this pattern. Don't let "AI" make you forget your engineering instincts.

Shubham Bhati is a Backend Engineer at AlignBits LLC building Java + Spring Boot integration pipelines with OpenAI in production. Based in Gurgaon, India. Portfolio · GitHub · LinkedIn

Publishing checklist:

[ ] Cover image: a system architecture diagram (request → queue → AI worker → DB)
[ ] Tags: #java #springboot #openai #ai #backend #microservices #productionsoftware #javadevelopment
[ ] Cross-post to Dev.to + Hashnode after 24h (Medium first for canonical)
[ ] Pin tweet linking to this post

GraphQL with Spring Boot: A Hands-On Tutorial for REST Developers

Shubham Bhati — Thu, 28 May 2026 07:08:11 +0000

Published 2026-05-28 by Shubham Bhati — Backend Engineer (Java 17, Spring Boot, Microservices).

As backend engineers, we've all been there - stuck with a REST API that's becoming increasingly cumbersome to maintain. At our company, we recently faced a similar issue with our user management API, which was built using Spring Boot 2.7 and Java 17. The API had grown to over 50 endpoints, and every new feature addition was a nightmare. That's when we started exploring GraphQL as an alternative, and after a thorough evaluation, we decided to go with a graphql spring boot tutorial to migrate our API. We chose Spring Boot 3.2 and Java 21 for the migration, and the results were astonishing - we reduced our p99 latency from 800ms to 120ms.

Introduction to GraphQL
Setting up a GraphQL Project with Spring Boot
Defining GraphQL Schemas
Resolvers and Data Fetching
Error Handling and Debugging
Common Mistakes
FAQ
Conclusion

Introduction to GraphQL

GraphQL is a query language for APIs that allows clients to specify exactly what data they need, reducing the amount of data transferred over the network. This makes it an attractive alternative to traditional REST APIs, especially for complex, data-driven applications. In our case, we were using Spring Boot to build our REST API, so we decided to explore the Spring GraphQL module to see how it could help us migrate to GraphQL. We started by reading the official Spring GraphQL documentation and the GraphQL Java documentation.

Setting up a GraphQL Project with Spring Boot

To get started with GraphQL and Spring Boot, you'll need to add the following dependencies to your pom.xml file:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-graphql</artifactId>
</dependency>
<dependency>
    <groupId>com.graphql-java</groupId>
    <artifactId>graphql-java</artifactId>
</dependency>

We used Maven 3.8 to manage our dependencies. Once you've added the dependencies, you can create a new Spring Boot application and configure the GraphQL endpoint. We used the @EnableGraphQL annotation to enable GraphQL support in our application:

@SpringBootApplication
@EnableGraphQL
public class GraphQLApplication {
    public static void main(String[] args) {
        SpringApplication.run(GraphQLApplication.class, args);
    }
}

We also used Java 21 to take advantage of its new features, such as sealed classes.

Defining GraphQL Schemas

In GraphQL, schemas define the structure of your data and the available queries and mutations. We defined our schema using the GraphQL schema definition language (SDL):

type User {
    id: ID!
    name: String!
    email: String!
}

type Query {
    users: [User]
    user(id: ID!): User
}

type Mutation {
    createUser(name: String!, email: String!): User
    updateUser(id: ID!, name: String, email: String): User
}

We used the graphql-java library to parse and validate our schema.

Resolvers and Data Fetching

Resolvers are responsible for fetching data from your backend systems and returning it to the client. We used Spring Data JPA to interact with our database:

@Repository
public interface UserRepository extends JpaRepository<User, Long> {
    Optional<User> findById(Long id);
}

@Service
public class UserService {
    @Autowired
    private UserRepository userRepository;

    public List<User> getUsers() {
        return userRepository.findAll();
    }

    public User getUser(Long id) {
        return userRepository.findById(id).orElseThrow();
    }
}

We used Spring Boot 3.2 to take advantage of its new features, such as native support for Java 21.

Error Handling and Debugging

Error handling and debugging are crucial aspects of building a GraphQL API. We used the @ExceptionHandler annotation to handle exceptions and return error responses to the client:

@ExceptionHandler(value = Exception.class)
public ResponseEntity<ErrorResponse> handleException(Exception e) {
    ErrorResponse errorResponse = new ErrorResponse(e.getMessage());
    return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body(errorResponse);
}

We also used the GraphQL Java debugger to debug our API.

Common Mistakes

Here are some common mistakes to avoid when building a GraphQL API with Spring Boot:

Not validating user input
Not handling exceptions properly
Not using pagination and caching
Not optimizing database queries
Not using a consistent naming convention

FAQ

What is the difference between GraphQL and REST?

GraphQL and REST are both API paradigms, but they differ in their approach to data retrieval. GraphQL allows clients to specify exactly what data they need, while REST returns a fixed set of data.

How do I handle authentication and authorization in GraphQL?

You can handle authentication and authorization in GraphQL using middleware or resolvers. We used Spring Security to secure our API.

Can I use GraphQL with other frameworks besides Spring Boot?

Yes, you can use GraphQL with other frameworks besides Spring Boot. We used the graphql-java library to build our API.

What are some best practices for building a GraphQL API?

Some best practices for building a GraphQL API include using a consistent naming convention, optimizing database queries, and using pagination and caching. You can find more information on the GraphQL best practices page.

Conclusion

In conclusion, building a GraphQL API with Spring Boot is a great way to improve the performance and flexibility of your API. By following the steps outlined in this tutorial, you can create a scalable and maintainable API that meets the needs of your clients. To get started, check out the official Spring GraphQL documentation and the GraphQL Java documentation.

Solving the Hibernate N+1 Problem in Spring Data JPA

Shubham Bhati — Mon, 25 May 2026 08:03:29 +0000

Published 2026-05-25 by Shubham Bhati — Backend Engineer (Java 17, Spring Boot, Microservices).

We've all been there - staring at a slow application, trying to figure out why our database queries are taking so long to execute. In our case, it was a Java-based e-commerce platform built using Spring Boot 3.2 and Spring Data JPA. After digging through the logs, we discovered that the hibernate n+1 problem was the culprit behind the slow performance. This issue occurs when Hibernate executes multiple select queries to fetch related entities, resulting in a significant increase in database load and latency. In this article, we'll explore the causes of the hibernate n+1 problem and discuss various strategies to solve it.

Introduction to the Hibernate N+1 Problem
Understanding JPA Fetch Types
Lazy Loading and Its Pitfalls
Using Join Fetching to Solve the N+1 Problem
Enabling Batch Fetching
Common Mistakes
FAQ
Conclusion

Introduction to the Hibernate N+1 Problem

The hibernate n+1 problem is a common issue that arises when using Hibernate to fetch related entities. It occurs when Hibernate executes multiple select queries to fetch the related entities, instead of using a single query to fetch all the required data. For example, consider a simple Order entity that has a one-to-many relationship with OrderItem:

@Entity
public class Order {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
    private String customerName;
    @OneToMany(mappedBy = "order")
    private List<OrderItem> orderItems;
    // getters and setters
}

When we try to fetch all the orders along with their order items, Hibernate will execute a separate query to fetch the order items for each order, resulting in the n+1 problem.

Understanding JPA Fetch Types

JPA provides two fetch types - EAGER and LAZY. The EAGER fetch type fetches the related entities immediately when the parent entity is loaded, while the LAZY fetch type fetches the related entities only when they are actually needed. By default, Hibernate uses the LAZY fetch type for one-to-many and many-to-many relationships. However, this can lead to the n+1 problem if we're not careful. We can change the fetch type to EAGER using the fetch attribute:

@OneToMany(mappedBy = "order", fetch = FetchType.EAGER)
private List<OrderItem> orderItems;

However, this can lead to performance issues if we're dealing with large amounts of data.

Lazy Loading and Its Pitfalls

Lazy loading can be a powerful tool for improving performance, but it can also lead to the n+1 problem if we're not careful. Consider the following example:

Order order = entityManager.find(Order.class, 1L);
for (OrderItem orderItem : order.getOrderItems()) {
    System.out.println(orderItem.getName());
}

In this example, Hibernate will execute a separate query to fetch the order items for each order, resulting in the n+1 problem. To avoid this, we can use join fetching or batch fetching.

Using Join Fetching to Solve the N+1 Problem

Join fetching involves using a single query to fetch all the required data. We can use the JOIN FETCH keyword in our JPQL query to achieve this:

Query<Order> query = entityManager.createQuery("SELECT o FROM Order o JOIN FETCH o.orderItems WHERE o.id = :id", Order.class);
query.setParameter("id", 1L);
Order order = query.getSingleResult();

This will fetch the order along with its order items in a single query, avoiding the n+1 problem.

Enabling Batch Fetching

Batch fetching involves fetching multiple related entities in a single query. We can enable batch fetching using the @BatchSize annotation:

@OneToMany(mappedBy = "order")
@BatchSize(size = 10)
private List<OrderItem> orderItems;

This will fetch the order items in batches of 10, reducing the number of queries executed.

Common Mistakes

Here are some common mistakes to avoid when dealing with the hibernate n+1 problem:

Not using join fetching or batch fetching
Using the LAZY fetch type without considering the performance implications
Not using the @BatchSize annotation to enable batch fetching
Not using the JOIN FETCH keyword in JPQL queries
Not optimizing database queries to reduce latency

FAQ

What is the hibernate n+1 problem?

How can I solve the hibernate n+1 problem?

You can solve the hibernate n+1 problem by using join fetching or batch fetching. Join fetching involves using a single query to fetch all the required data, while batch fetching involves fetching multiple related entities in a single query.

What is the difference between EAGER and LAZY fetch types?

The EAGER fetch type fetches the related entities immediately when the parent entity is loaded, while the LAZY fetch type fetches the related entities only when they are actually needed.

How can I optimize database queries to reduce latency?

You can optimize database queries by using indexes, reducing the amount of data fetched, and using efficient query algorithms. For more information, you can refer to the Oracle documentation or the Spring documentation.

Conclusion

In conclusion, the hibernate n+1 problem is a common issue that can significantly impact the performance of your application. By using join fetching or batch fetching, you can avoid this problem and improve the performance of your database queries. For more information, you can refer to the Baeldung tutorial or the official Java tutorials. Remember to always consider the trade-offs between different approaches and optimize your database queries to reduce latency.

From B.Com to Backend Engineer: My Masai School Journey

Shubham Bhati — Mon, 25 May 2026 08:01:44 +0000

From B.Com to Backend Engineer: My Masai School Journey

By Shubham Bhati — Backend Engineer at AlignBits LLC

The unlikely path

I have a Bachelor of Commerce degree from Devi Ahilya Vishwavidyalaya in Indore. Three years later, I'm a Software Engineer at an iPaaS company, shipping production Java microservices that integrate enterprise systems.

In between those two facts is one institution that changed my trajectory: Masai School.

This is the honest version of how I went from balance sheets to backend systems — what worked, what was hard, and what I'd do differently.

Why a B.Com grad picks up Java

I didn't grow up writing code. My undergrad was accounting, taxation, business law. By the final year of B.Com (2022), I had a clear realization: the work I was being prepared for didn't excite me. The work that did — building software — required skills my degree never gave me.

I had two options:

Spend ₹4–8 lakh on a Master's degree
Find a focused, intensive program that taught backend engineering for software jobs

I picked option 2 and applied to Masai School for their Software Engineering — Java Backend Specialization (then a 30+ week program in Bengaluru).

What Masai actually teaches

Forget the marketing. What I actually learned, in order:

Phase 1 — Foundations (HTML/CSS/JS, DSA)
Surprisingly tough. As a non-engineering grad, the first month was just adapting to "thinking like a programmer" — abstraction, recursion, big-O.

Phase 2 — Java + OOP
Where the Java backend specialization really started. Generic types, collections framework, exception handling, multithreading basics.

Phase 3 — Spring Boot + databases
The career-defining phase. REST APIs, Spring Data JPA, Hibernate, MySQL. By week 18, I was building functional CRUD APIs.

Phase 4 — Production-grade
Spring Security, JWT, OAuth 2.0, microservices patterns, message queues (RabbitMQ basics).

Phase 5 — Capstone projects
Real, deployable projects. Mine included Chatterbox (Spring Boot WebSockets) and a Shopzilla e-commerce backend.

What worked

1. Showing up every day
Masai is intense — 9–10 hours of focused work, six days a week. The students who showed up consistently are the ones who got hired. Talent matters less than this.

2. Building things outside class
I built side projects on weekends. SnapResize, TIC_TAC_TOE with JavaFX, and WorkFolio — these became my real portfolio.

3. Studying production code
Reading open-source Spring projects taught me more than tutorials ever did.

4. Practicing Java interview problems daily
HackerRank, LeetCode (easy/medium). I have an Intermediate badge for Java on HackerRank — earned by daily practice.

What didn't work (or what I'd change)

1. Spreading too thin
I tried to learn React + Java backend simultaneously in the first months. Mistake. Going deep on one stack beats being mediocre at two.

2. Skipping system design early
Masai covers basics, but interview-grade system design (LB, caching, partitioning, CAP, consistent hashing) — I learned that mostly on the job after joining IHX. Should have started earlier.

3. Not building public presence
I waited until job-search stage to make a GitHub portfolio. Should have been pushing code from week 1. Recruiters search GitHub more than they admit.

What happened after Masai

IHX Private Limited (Jun 2023 – Aug 2024) — Associate Software Engineer on healthcare backend systems. FHIR-standard integrations. This is where I learned production engineering: incidents, on-call, monitoring, SLOs.

AlignBits LLC (Sep 2024 – present) — Software Engineer on an iPaaS platform. Microservices at scale, message queues, AWS, cross-system integrations. Got promoted from Jr. in my first year.

In two years from graduating Masai I've:

Resolved 15+ production incidents
Managed 10+ client integration pipelines
Earned 25+ certifications (AI Engineer, Java, Prompt Engineering)
Stayed at backend engineering — never had to pivot

Is Masai worth it?

Honest answer: yes, conditionally.

It's worth it if:

You're committed to backend engineering as a career
You can dedicate 9+ hours/day for 30+ weeks
You have no engineering background and need structured fundamentals
You're OK with the pay-after-placement model (you don't pay until you're earning)

It's NOT worth it if:

You're looking for a credential to put on a resume without doing the work
You already know Java and Spring well — you'll be bored in the first half
You can't commit full-time

What I'd tell a B.Com student today

Pick one stack, go deep. Don't be a full-stack jack-of-all-trades. Java + Spring Boot is a solid bet for India in 2026.
Push code to GitHub from day one. Even bad code. Recruiters notice consistency.
Get a working website. Mine is at shubh2-0.github.io. It pulls more recruiter messages than my LinkedIn.
Solve 1 LeetCode problem a day. Not 10. One, every day, for a year.
Find a community. I'd struggled alone for months before joining Masai. Don't.

What's next

I'm currently exploring backend engineering roles — remote, hybrid, or relocation — with companies building products that scale. If you're hiring or know someone who is, my LinkedIn is the fastest way to reach me.

If you're a B.Com student staring at a coding bootcamp wondering if it's worth it: it is, if you do the work.

Shubham Bhati is a Backend Engineer at AlignBits LLC, working on iPaaS platform integrations with Java, Spring Boot, and AWS. Based in Gurgaon, India. Portfolio · GitHub · LinkedIn

Publishing checklist:

[ ] Add 2-3 images (Masai campus photo, your code screenshot, a system diagram)
[ ] Cover image: 1200x675 px (Canva)
[ ] Tags: #masaischool #javadeveloper #careerchange #bcomtocoding #backenddeveloper #java #springboot #india
[ ] Add canonical link back to your bio site
[ ] Cross-post to Dev.to and Hashnode after 24h (Medium gets first crawl)

DEV Community: Shubham Bhati

Beyond the Cache Miss: Designing Resilient Caching Layers with Redis Degradation Strategies

1. The Anatomy of a Cache Disaster

The Failure Cascade

Root Cause Analysis (RCA)

2. The Resilient Architecture: Multi-Tier Caching & Circuit Breaking

3. Implementation: Building a Resilient Cache Manager in Spring Boot

Dependency Configuration (pom.xml)

The Resilient Cache Layer Implementation

Application Configuration (application.yml)

4. Mitigating Cache Stampede: Mutex Locking

5. Operational Verification: Testing Caching Failure Modes

Summary: Caching Best Practices for Architects

Deep Dive into Java HashMap Internals: Conquering Hash Collisions and Preventing Production Outages

1. The Core Architecture: Buckets, Nodes, and Memory

Key Internal Thresholds

2. The Hashing Pipeline: From Object to Bucket Index

Step 1: The Perturbation Function

Step 2: The Bitwise Modulo Optimization

3. Collision Resolution: Linked Lists to Red-Black Trees

Java 8+ Hybrid Approach

Why 8? The Poisson Distribution

4. Production Post-Mortem: The HashDoS CPU Starvation

The Outage Scenario

Thread Dump Investigation

Root Cause: Poor Custom Key Design

5. Architectural Remediation

The Optimized, Defensive Key Class

6. Sizing Strategies to Prevent Costly Resizing

The Correct Sizing Formula

7. Architectural Guidelines for Production HashMap Usage

GraphQL vs. REST Under Load: Architectural Patterns, Pitfalls, and Production-Grade Mitigation

1. The Architectural Divergence Under Load

2. GraphQL Failure Mode 1: The Cascading N+1 Database Exhaustion

The Scenario

Root Cause Analysis

The Solution: Batching and Caching with Spring GraphQL DataLoaders

Vulnerable Resolver Code (Anti-Pattern)

Production-Grade Mitigated Code (Spring Boot 3 + Spring GraphQL)

3. GraphQL Failure Mode 2: The "Query of Death" (Denial of Wallet)

The Scenario

Root Cause Analysis

The Solution: Static Query Depth and Complexity Analysis

Production-Grade Mitigation Config (Spring Boot)

4. REST Failure Mode: Under-Fetching & Chatty Network Overhead

The Scenario

Root Cause Analysis

The Solution: Implement Dynamic Projection & JSON-API Sparse Fieldsets

Production-Grade Dynamic Projection implementation (Spring Boot)

5. Performance & Caching Comparison Under Load

Designing GraphQL for Edge Caching: Automatic Persisted Queries (APQ)

6. The Architect's Decision Matrix

When to Standardize on REST

When to Standardize on GraphQL

Asynchronous Programming in Spring Boot with CompletableFuture

Introduction to Asynchronous Programming

Using the @Async Annotation

Working with CompletableFuture

Configuring Thread Pools

Handling Errors and Exceptions

Handling Cancellation

Common Mistakes

FAQ

What is the difference between @Async and CompletableFuture?

How do I configure thread pools in Spring Boot?

What is the best way to handle errors and exceptions in asynchronous programming?

How do I test asynchronous code?

Conclusion

Further Reading

Service Discovery with Eureka and Spring Cloud: A Hands-On Tutorial

Introduction to Service Discovery

Setting Up Spring Cloud Eureka

Registering Services with Eureka

Using Eureka for Service Discovery

Configuring Eureka for High Availability

Common Mistakes

FAQ

What is Service Discovery?

How Does Eureka Work?

Can I Use Eureka with Other Spring Cloud Tools?

Dependency Configuration (`pom.xml`)

Application Configuration (`application.yml`)

Using the `@Async` Annotation

Working with `CompletableFuture`

What is the difference between `@Async` and `CompletableFuture`?