Building Resilient Distributed Systems with Springboot

When building distributed systems, many teams concentrate on making individual services strong and reliable. While that's important, the real challenge lies in how these services interact, especially when the system is under heavy stress. 

A resilient system isn't just about tough services - it's about ensuring all parts of the system work together smoothly when things go wrong.

Now, if you're a Spring Boot developer like me, you're in luck.

In this article, I'll not only break down four essential strategies for building resilient systems, but I'll also provide sample code snippets, showing you how to implement them with Spring Boot libraries and tools. 

Let's get started.

1. Prioritize Performance with Caching

When your application is flooded with requests, where does it break first? Often, it’s the database or an external API. Caching is the go-to solution to ease this bottleneck by storing frequently accessed data closer to your application.

Why Caching Matters

1. Speed: A cache retrieves data faster than a database or an API call. Faster responses mean happier users.
2. Reliability: Even if your database or API is temporarily unavailable, a cache can keep things running smoothly.
3. Scalability: Reducing database hits means your system can handle more traffic without crumbling under pressure.

Common Use Cases for Caching

1. Static Data: Product catalogs, user preferences, or site configurations that don’t change frequently.
2. Computed Data: Expensive calculations, like recommendation engines or data aggregations.
3. Session Management: Tracking user sessions or authentication tokens.

Implementing Caching in Spring Boot

Step 1: Add Caching Dependencies

For basic caching, include this in your pom.xml:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-cache</artifactId>
</dependency>

If you want distributed caching with Redis:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>

Step 2: Enable Caching

Enable caching in your application by adding @EnableCaching:

@SpringBootApplication
@EnableCaching
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
}

Step 3: Define Cacheable Methods

Use @Cacheable to specify which methods should cache their results:

@Service
public class WeatherService {

    @Cacheable("weather")
    public String getWeather(String location) {
        // Simulate slow external API call
        slowApiCall();
        return "Sunny in " + location;
    }

    private void slowApiCall() {
        //api call code
    }
}

Step 4: Configure Cache Settings

For Redis, add this to application.properties:

spring.cache.type=redis
spring.redis.host=localhost
spring.redis.port=6379

2. Graceful Recovery with Circuit Breakers

Failures are inevitable in distributed systems. The trick is to isolate those failures and prevent them from dragging down the rest of the system. This is where circuit breakers shine.

What is a Circuit Breaker?

A circuit breaker monitors your service calls. If too many failures occur, it trips, temporarily stopping further calls. This prevents overloading a struggling service and allows it time to recover.

When to Use Circuit Breakers

1. When calling external APIs or microservices that are prone to latency or failures.
2. To protect critical parts of your system from cascading failures.

Using Circuit Breakers with Resilience4j in Spring Boot

Step 1: Add Dependencies

Add Resilience4j to your project:

<dependency>
    <groupId>io.github.resilience4j</groupId>
    <artifactId>resilience4j-spring-boot2</artifactId>
    <version>1.7.1</version>
</dependency>

Step 2: Configure Circuit Breakers

Define your circuit breaker settings in application.properties:

resilience4j.circuitbreaker.instances.myService.failure-rate-threshold=50
resilience4j.circuitbreaker.instances.myService.sliding-window-size=10
resilience4j.circuitbreaker.instances.myService.wait-duration-in-open-state=10000

Step 3: Wrap Calls with Circuit Breakers

Use @CircuitBreaker to wrap risky service calls:

@RestController
public class ApiController {

    @CircuitBreaker(name = "myService", fallbackMethod = "fallbackResponse")
    @GetMapping("/data")
    public String fetchData() {
        // Simulating an external API call
        return webClient.get()
                .uri("http://external-service.com/api")
                .retrieve()
                .bodyToMono(String.class)
                .block();
    }

    public String fallbackResponse(Throwable t) {
        return "Service is unavailable, please try again later.";
    }
}

3. Prevent Overload with Throttling

Imagine your app gets flooded with requests—maybe because you went viral, or someone’s intentionally hitting your endpoints nonstop. Without some kind of control, your servers could get overwhelmed, leading to crashes or performance issues.

That’s where throttling comes in. It limits how often users (or systems) can access your app, helping you protect your backend and keep things running smoothly.

What’s Throttling?

Throttling is like placing a speed limit on a highway. It ensures no one can send too many requests at once, giving your app breathing room to handle other users.

When Should You Use Throttling?

1. When your app serves a public API or has user-facing endpoints that could get spammed.
2. To prevent abuse from automated bots or malicious users.
3. To handle traffic surges gracefully, especially during sales or promotions.

How to Implement Throttling in Spring Boot

Option 1: Use Spring Web Filters for Simple Throttling

For basic throttling, you can use a filter to track request counts and delay or block users who exceed the limit.

@Component  
public class ThrottleFilter implements Filter {  

    private final Map<String, Integer> requestCounts = new ConcurrentHashMap<>();  
    private final int MAX_REQUESTS_PER_MINUTE = 100;  

    @Override  
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {  
        HttpServletRequest httpRequest = (HttpServletRequest) request;  
        String clientIP = httpRequest.getRemoteAddr();  

        requestCounts.merge(clientIP, 1, Integer::sum);  

        if (requestCounts.get(clientIP) > MAX_REQUESTS_PER_MINUTE) {  
            ((HttpServletResponse) response).setStatus(HttpServletResponse.SC_TOO_MANY_REQUESTS);  
            response.getWriter().write("Too many requests. Please try again later.");  
            return;  
        }  

        chain.doFilter(request, response);  
    }  

    @Scheduled(fixedRate = 60000)  
    public void resetCounts() {  
        requestCounts.clear();  
    }  
}  

This example tracks requests per client IP and blocks further requests if they exceed a set limit within a minute.

Option 2: Use a Library Like Bucket4j for Advanced Throttling

For more control, such as rate-limiting based on API keys or user roles, use Bucket4j.

Add Dependency
Include the Bucket4j library:

<dependency>  
    <groupId>com.github.vladimir-bukhtoyarov</groupId>  
    <artifactId>bucket4j-core</artifactId>  
    <version>8.0.0</version>  
</dependency>  

Set Up Throttling Logic
Here’s an example of rate-limiting requests:

@RestController  
@RequestMapping("/api")  
public class RateLimitController {  

    private final Map<String, Bucket> buckets = new ConcurrentHashMap<>();  

    @GetMapping("/data")  
    public ResponseEntity<String> getData(HttpServletRequest request) {  
        String clientIP = request.getRemoteAddr();  
        Bucket bucket = buckets.computeIfAbsent(clientIP, k -> createNewBucket());  

        if (bucket.tryConsume(1)) {  
            return ResponseEntity.ok("Here's your data!");  
        } else {  
            return ResponseEntity.status(HttpStatus.TOO_MANY_REQUESTS).body("Rate limit exceeded. Try again later.");  
        }  
    }  

    private Bucket createNewBucket() {  
        return Bucket4j.builder()  
                .addLimit(Bandwidth.simple(50, Duration.ofMinutes(1)))  
                .build();  
    }  
}  

Here, each client can make 50 requests per minute. If they exceed the limit, they’ll get a 429 Too Many Requests response.

Option 3: Fine-Tune with Spring Cloud Gateway

If you’re using a gateway, Spring Cloud Gateway makes it easy to add rate-limiting globally.

Add Dependencies
Include these in your pom.xml:

<dependency>  
    <groupId>org.springframework.cloud</groupId>  
    <artifactId>spring-cloud-starter-gateway</artifactId>  
</dependency>  
<dependency>  
    <groupId>org.springframework.boot</groupId>  
    <artifactId>spring-boot-starter-data-redis</artifactId>  
</dependency>  

Configure Rate-Limiting in application.yml
Set up Redis-based rate-limiting for API routes:

spring:  
  cloud:  
    gateway:  
      routes:  
        - id: data-service  
          uri: http://localhost:8080  
          predicates:  
            - Path=/api/data  
          filters:  
            - name: RequestRateLimiter  
              args:  
                redis-rate-limiter:  
                  replenishRate: 10  
                  burstCapacity: 20  

By adding throttling, you not only protect your system from abuse but also ensure a smoother experience for all users, even during traffic spikes.

4. Use Retry Strategies: Because Nobody Likes a Nag

When dealing with temporary failures, retries can be a lifesaver, but overdoing them can overwhelm your system and make minor issues worse. It’s essential to implement retries thoughtfully to strike a balance between resilience and system stability.

Introducing Spring Retry

Spring Retry is a powerful library that simplifies implementing retry mechanisms in Java applications. It integrates seamlessly with Spring, providing declarative and programmatic approaches to handle retries effectively.

Spring Retry allows you to:

1. Retry operations based on specific exceptions.
2. Define policies like fixed intervals, exponential backoff, and more.
3. Customize maximum retry attempts and delays.

Adding the Dependency

To use Spring Retry in your project, add the following dependency to your pom.xml :

<dependency>
    <groupId>org.springframework.retry</groupId>
    <artifactId>spring-retry</artifactId>
    <version>1.3.4</version> <!-- Use the latest version -->
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-aop</artifactId>
</dependency>

The spring-boot-starter-aop dependency is required for handling annotations like @Retryable.

Configuring Retry Logic

Spring Retry provides annotations like @Retryable to handle retries declaratively.

1. Enable Retry Support: Add the @EnableRetry annotation to your configuration class to enable retry functionality.

import org.springframework.context.annotation.Configuration;
import org.springframework.retry.annotation.EnableRetry;

@Configuration
@EnableRetry
public class RetryConfig {
}

1. Add Retry Logic to a Service: Use the @Retryable annotation to define retry policies.

import org.springframework.retry.annotation.Retryable;
import org.springframework.retry.annotation.Backoff;
import org.springframework.stereotype.Service;

@Service
public class ApiService {
    @Retryable(
        value = {RuntimeException.class},
        maxAttempts = 3,
        backoff = @Backoff(delay = 2000, multiplier = 2)
    )
    public String callExternalService() {
        System.out.println("Attempting to call external service...");
        throw new RuntimeException("Temporary failure");
    }
}

• value: Specifies the exceptions to trigger a retry.
• maxAttempts: Limits the number of retry attempts.
• backoff: Configures the delay between retries (e.g., exponential backoff with a multiplier).

1. Fallback for Graceful Failure: Handle failures gracefully by defining a fallback method using @Recover.

import org.springframework.retry.annotation.Recover;
import org.springframework.stereotype.Service;

@Service
public class ApiService {
    // Retryable method as shown above

    @Recover
    public String recover(RuntimeException e) {
        System.out.println("Fallback executed due to: " + e.getMessage());
        return "Fallback response";
    }
}

The Bottom Line: Focus on Interactions

A truly resilient system isn't just about robust services; it’s about ensuring those services interact seamlessly and effectively.

By caching wisely, preventing cascading failures, designing fallbacks, and managing retries, you can create systems that effortlessly bounce back from issues.

Failures are inevitable, but what truly defines a resilient system is how gracefully it responds to those challenges.

Best of luck in building your resilient system, may it handle every hiccup like a pro!