TL;DR: When a Spring Boot application depends on an external API, failures are inevitable. In this article, I'll walk through how to think about timeouts, retries, backoff, rate limits, and fallback handling when building a resilient backend.
When I was building an AI-powered GitHub Pull Request reviewer, one thing became very clear:
The AI API is not under my control.
My application can be perfectly healthy, but an external service can still:
Take too long to respond
Return a 500
Temporarily become unavailable
Reject requests because of rate limits
Return an unexpected response
Have a network failure
So this raises an important backend question:
What should my Spring Boot application do when an external API fails?
Simply calling the API and returning the exception to the user isn't enough.
Let's look at a better approach.
The Problem
Imagine this simple flow:
Client
|
v
Spring Boot
|
v
External API
|
X
Failure
If the external API fails, our application needs to decide:
Should we retry?
How many times?
How long should we wait?
What if the API is still unavailable?
What response should our client receive?
These are resilience questions.
- Always Set a Timeout
One of the first mistakes that can happen with external API calls is forgetting about timeouts.
Imagine:
Spring Boot
|
| Request
v
External API
|
| .............
| .............
| .............
If the external service doesn't respond, you don't want your application waiting indefinitely.
A timeout gives your application a clear boundary.
Conceptually:
Request
|
v
External API
|
|---- Response within timeout → Continue
|
|---- Timeout → Handle failure
The exact timeout depends on the API and the operation.
A request that normally takes 200 ms shouldn't necessarily have a 5-minute timeout.
The important idea is:
Every external network call should have an intentional timeout strategy.
- Should We Retry?
Not every failure should be retried.
For example, suppose the external API returns:
500 Internal Server Error
A retry might make sense because the failure could be temporary.
But imagine the API returns:
400 Bad Request
Retrying the exact same request probably won't fix anything.
The request itself is invalid.
So we need to distinguish between failures.
Potentially retryable
Connection failure
Timeout
Temporary 5xx response
429 Too Many Requests
Usually not retryable
400 Bad Request
401 Unauthorized
403 Forbidden
404 Not Found
Invalid request data
The exact retry policy depends on the API and the operation.
- Don't Retry Immediately
Suppose an external API is temporarily unavailable.
A naive implementation might do:
Request
↓
Failure
↓
Retry immediately
↓
Failure
↓
Retry immediately
↓
Failure
This can make the situation worse.
Instead, we can introduce a delay between attempts.
This is called backoff.
For example:
Attempt 1 → Failure
↓
Wait 1 sec
↓
Attempt 2 → Failure
↓
Wait 2 sec
↓
Attempt 3 → Failure
A common approach is exponential backoff, where the delay increases after each failed attempt.
In production systems, adding some jitter to the delay can also help prevent many clients from retrying at exactly the same time.
- Example Using Spring Retry
One way to implement retries in a Spring application is with Spring Retry.
For example:
@Retryable(
retryFor = ExternalApiException.class,
maxAttempts = 3,
backoff = @Backoff(delay = 1000)
)
public String callExternalApi() {
return externalApiClient.call();
}
The basic idea is:
Maximum attempts = 3
Initial delay = 1000 ms
So if the call fails with the configured exception, the application can attempt the operation again according to the retry configuration.
But there's an important question:
What happens after all retry attempts fail?
That's where fallback handling becomes useful.
- Fallback Handling
Suppose all retry attempts fail:
Attempt 1 → Failure
Attempt 2 → Failure
Attempt 3 → Failure
At this point, continuing to retry isn't useful.
The application needs to fail gracefully.
Conceptually:
Client
|
v
Spring Boot
|
v
External API
|
X
Failure
|
v
Retry
|
X
Failure
|
v
Fallback / Error Response
Depending on the application, the fallback could:
Return a meaningful error
Use cached data
Queue the request for later processing
Mark the operation as pending
Use an alternative service
The correct fallback depends heavily on the business requirement.
- Handling Rate Limits
Another interesting failure scenario is:
429 Too Many Requests
This usually means the client has exceeded the API's allowed request rate.
Immediately sending more requests isn't a good strategy.
Instead, the application should respect the API's rate-limit behavior.
Some APIs provide information such as:
Retry-After
which can indicate when the client should try again.
This is particularly important when building applications that make frequent calls to third-party APIs.
- What About the AI Code Reviewer?
This became particularly relevant to the AI code reviewer I was building.
The workflow looks roughly like:
GitHub
|
v
Webhook
|
v
Spring Boot
|
v
Get PR Diff
|
v
AI API
|
X
Possible failure
What happens if the AI service doesn't respond?
I don't want the entire application to simply crash.
A better design would be something like:
GitHub Webhook
|
v
Spring Boot
|
v
Create Review Job
|
v
Process Review
|
v
AI API
|
+---+---+
| |
Success Failure
| |
v v
Review Retry
|
v
Backoff
|
v
Try Again
|
v
Still failing?
/ \
Yes No
| |
v v
Mark Review
Failed
This is where resilience becomes more than just adding a retry annotation.
It becomes an architectural decision.
- Synchronous vs Asynchronous Processing
Suppose the GitHub webhook directly waits for the AI response:
GitHub
|
v
Webhook
|
v
Spring Boot
|
v
AI API
|
v
Response
The webhook request remains open while the AI request is being processed.
A more scalable approach could be:
GitHub
|
v
Webhook
|
v
Spring Boot
|
v
Queue
|
v
Worker
|
v
AI API
Now the webhook can be acknowledged quickly while the actual review happens asynchronously.
This also makes it easier to implement:
Retries
Delayed processing
Failure tracking
Dead-letter queues
Multiple workers
Horizontal scaling
- Don't Retry Everything
This is probably the most important lesson.
Adding:
retry = 5
doesn't automatically make an application resilient.
Imagine five application instances all calling the same unavailable API.
If every instance retries aggressively, we could end up with:
Application 1 → Retry
Application 2 → Retry
Application 3 → Retry
Application 4 → Retry
Application 5 → Retry
The external service is already struggling.
Our retries could make the situation worse.
This is sometimes called a retry storm.
That's why retry policies need to consider:
Which errors are retryable
Maximum attempts
Backoff
Jitter
Rate limits
Request volume
Idempotency
Overall system behavior
- What Is Idempotency and Why Does It Matter?
This is another important consideration when implementing retries.
Suppose we have:
POST /payments
and the first request actually succeeds, but the response is lost because of a network problem.
Our application doesn't know whether the operation succeeded.
If we blindly retry:
POST /payments
we could potentially create the operation twice.
That's why retrying a request isn't simply a technical decision.
We also need to understand whether the operation is safe to repeat.
For operations that can have side effects, idempotency mechanisms may be necessary.
- Circuit Breakers
Retries aren't always enough.
Imagine an external service has been unavailable for several minutes.
Our application keeps sending requests.
Each request waits for a timeout.
Then it retries.
Then another request does the same thing.
We're wasting resources while the dependency is clearly unhealthy.
This is where a circuit breaker can help.
Conceptually:
Normal
|
v
+--------------+
| CLOSED |
+--------------+
|
Too many failures
|
v
+--------------+
| OPEN |
+--------------+
|
Wait / test
|
v
+--------------+
| HALF-OPEN |
+--------------+
/ \
Success Failure
| |
v v
CLOSED OPEN
The circuit breaker temporarily stops calls to an unhealthy dependency.
This prevents our application from continuously hammering an unavailable service.
Libraries such as Resilience4j can be used to implement patterns like:
Retry
Circuit breaker
Rate limiter
Bulkhead
Time limiter
A Resilient External API Flow
Putting the concepts together:
Client
|
v
Spring Boot
|
v
External API
|
+---------+---------+
| |
Success Failure
| |
v v
Continue Is it
retryable?
/ \
Yes No
| |
v v
Backoff Fail
|
v
Retry
|
Still failing?
/ \
Yes No
| |
v v
Fallback Success
This is the kind of flow I now think about whenever my backend depends on an external service.
What I Learned
Before working on projects involving external APIs, it was easy to think:
"I'll just call the API and handle the response."
But real backend systems aren't that simple.
External dependencies can fail.
Networks can fail.
Services can become slow.
Rate limits can be reached.
Responses can change.
And your application needs to behave predictably when those things happen.
The goal isn't to make failures impossible.
The goal is to make failures manageable.
My Spring Boot Resilience Checklist
When I integrate an external API now, these are some of the questions I ask:
✓ Do I have a connection timeout?
✓ Do I have a response timeout?
✓ Which failures are retryable?
✓ How many retries should I allow?
✓ Do I need exponential backoff?
✓ Should I add jitter?
✓ How do I handle 429 responses?
✓ Is the operation safe to retry?
✓ Do I need a circuit breaker?
✓ Should this operation be asynchronous?
✓ What happens after all retries fail?
✓ How will I monitor failures?
Thinking about these questions early can prevent a lot of problems later.
Final Thoughts
While building my AI-powered GitHub Pull Request reviewer, I initially focused mostly on the AI part.
But the more I thought about the architecture, the more I realized:
The AI call is just another external dependency.
The same principles apply whether you're calling:
An AI API
A payment gateway
A notification service
A database service
A third-party REST API
Your application needs to assume that external dependencies can fail.
That's one of the differences between a backend that works in a demo and one that is designed to handle real-world conditions.
What would you add?
If your Spring Boot application depends heavily on an external API, what resilience strategy do you usually use?
Retries? Circuit breakers? Async processing? Queues? Caching?
I'd love to hear how other developers approach this.
Top comments (0)