Building Resilient .NET Applications with Polly

Retry and Circuit Breaker Patterns Explained with Practical Examples

Modern applications rarely live in isolation. They constantly communicate with databases, third-party APIs, microservices, and cloud resources.
Because of this, failure is not an exception — it is an expectation.

Transient network issues, temporary service outages, or brief timeouts can easily break an otherwise well-designed system if resilience is not considered.

In this article, we will explore:

• What resilience means in software systems
• Common resilience patterns
• How to implement Retry and Circuit Breaker patterns using Polly in .NET

---

What Is Resilience in Software Systems?

Resilience is the ability of an application to:

• Handle failures gracefully
• Recover automatically from transient issues
• Maintain availability and responsiveness

Instead of crashing or blocking users when something goes wrong, a resilient system adapts and continues operating.

Think of resilience mechanisms as safety nets — they catch your application when things inevitably fail.

---

Understanding Transient Failures

A transient failure is a temporary issue that usually resolves itself after a short period of time.

Common examples include:

• Temporary network glitches
• API timeouts
• Short-lived database connection failures
• Cloud service throttling

These failures should not be treated as permanent errors.
Retrying the operation after a short delay often succeeds.

---

The Retry Pattern

What Is the Retry Pattern?

The Retry Pattern automatically re-executes a failed operation a limited number of times before giving up.

When Should You Use Retry?

• Network calls
• External APIs
• Message queues
• Cloud resources

When NOT to Use Retry?

• Validation errors
• Authentication failures
• Business rule violations

---

Retry Example with Polly

var retryPolicy = Policy
    .Handle<HttpRequestException>()
    .WaitAndRetryAsync(
        retryCount: 3,
        sleepDurationProvider: attempt => TimeSpan.FromSeconds(2),
        onRetry: (exception, timeSpan, retryCount, context) =>
        {
            Console.WriteLine($"Retry {retryCount} after {timeSpan.TotalSeconds}s");
        });

What this does:

• Retries the request up to 3 times
• Waits 2 seconds between retries
• Only retries when a HttpRequestException occurs

This approach significantly increases success rates for transient failures without impacting user experience.

---

The Circuit Breaker Pattern

The Problem Circuit Breaker Solves

Imagine calling a third-party service that is completely down:

• Requests keep failing
• Threads remain blocked
• System resources are exhausted
• The failure cascades through your system

This is how small failures turn into system-wide outages.

---

What Is a Circuit Breaker?

A Circuit Breaker prevents an application from executing operations that are likely to fail.

It works similarly to an electrical circuit breaker:

• Detects repeated failures
• Stops sending requests temporarily
• Allows the system to recover

---

Circuit Breaker States

1. Closed
   Requests flow normally.

2. Open
   Requests are immediately rejected (fail-fast).

3. Half-Open
   A limited number of test requests are allowed to check recovery.

---

Circuit Breaker Example with Polly

var circuitBreakerPolicy = Policy
    .Handle<HttpRequestException>()
    .CircuitBreakerAsync(
        exceptionsAllowedBeforeBreaking: 5,
        durationOfBreak: TimeSpan.FromSeconds(30),
        onBreak: (exception, duration) =>
        {
            Console.WriteLine("Circuit opened");
        },
        onReset: () =>
        {
            Console.WriteLine("Circuit closed");
        },
        onHalfOpen: () =>
        {
            Console.WriteLine("Circuit half-open");
        });

What this does:

• Opens the circuit after 5 consecutive failures
• Blocks requests for 30 seconds
• Automatically attempts recovery

---

Combining Retry and Circuit Breaker

Using Retry alone can overload a failing service.
Using Circuit Breaker alone can be overly aggressive.

The real power comes from combining them.

var resiliencePolicy = Policy.WrapAsync(
    retryPolicy,
    circuitBreakerPolicy
);

Execution Example

await resiliencePolicy.ExecuteAsync(async () =>
{
    var response = await httpClient.GetAsync("https://external-api.com/data");
    response.EnsureSuccessStatusCode();
});

This setup ensures:

• Transient failures are retried
• Persistent failures trigger circuit breaking
• System stability is preserved

---

Why Polly?

Polly is the de-facto resilience library in the .NET ecosystem because:

• Fluent and expressive API
• Supports async and sync operations
• Easily integrates with HttpClientFactory
• Widely used in production systems

Supported patterns include:

• Retry
• Circuit Breaker
• Timeout
• Fallback
• Bulkhead Isolation

---

Real-World Use Cases

Polly is especially useful for:

• Microservices communication
• Financial and trading systems
• Cloud-native applications
• API gateways
• SignalR and real-time systems

Any place where IO or remote calls exist, resilience should be a first-class concern.

---

Key Takeaways

• Failures are inevitable in distributed systems
• Resilience prevents small issues from becoming outages
• Retry handles transient faults
• Circuit Breaker protects system stability
• Polly makes resilience practical and production-ready in .NET

---

Final Thoughts

Resilience is not an optional feature — it is a design requirement for modern applications.

By leveraging proven patterns and libraries like Polly, you can build systems that are:

• Stable
• Scalable
• Fault-tolerant
• User-friendly

If your application talks to the outside world, it must be resilient.

---

I’m Morteza Jangjoo and “Explaining things I wish someone had explained to me”