DEV Community

izma anwar
izma anwar

Posted on

Implementing the Retry Pattern Using Polly in .NET Applications

The Problem — Why Your Microservices Fail Under Pressure
Picture this: you have an email microservice responsible for retrieving emails from a database, processing them, and delivering them to users.
Suddenly, the processing service becomes temporarily unavailable — a high traffic spike, a fleeting network glitch. Without a resilience strategy, your users immediately receive error messages.
This is where the Retry Pattern comes in.

What Is the Retry Pattern?
The Retry pattern is a resilience strategy designed to handle transient failures by automatically re-attempting an unsuccessful operation a defined number of times — instead of instantly marking it as a failure.
It empowers your application to gracefully navigate:

🌐 Network timeouts
⚡ Sporadic service unavailability
🔄 Temporary database connection drops
📡 Intermittent external API failures

Introducing Polly
Polly is a .NET resilience and transient-fault-handling library that lets you express policies like Retry, Circuit Breaker, and Timeout in a fluent, thread-safe manner.

Let's explore different policies.

🔁 Wait and Retry Policy
The most commonly used policy. It retries a failed operation with an optional delay between attempts.

Basic Retry on Exception
csharpRetryPolicy retryIfException =
    Policy.Handle<Exception>().Retry(3);

retryIfException.Execute(
    someBusinessLogic.DoSomethingThatMightThrowException);
Retry When Result Is Unexpected
csharpRetryPolicy<bool> retryPolicyNeedsTrueResponse =
    Policy.HandleResult<bool>(b => b != true).Retry(3);

bool result = retryPolicyNeedsTrueResponse.Execute(
    () => someBusinessLogic.GetBool());
Enter fullscreen mode Exit fullscreen mode

Re-authenticate Before Retrying
Useful for APIs that return 401 Unauthorized with expired tokens:

csharpRetryPolicy<HttpResponseMessage> httpRetryWithReauthorizationPolicy =
    Policy.HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
        .RetryAsync(3, onRetry: (response, retryCount) =>
        {
            if (response.Result.StatusCode == HttpStatusCode.Unauthorized)
            {
                PerformReauthorization(); // Refresh your token here
            }
        });
Enter fullscreen mode Exit fullscreen mode

Wait and Retry with Exponential Backoff
This is the one you'll use most in production. It introduces an exponential delay between retries — giving the failing service time to recover:

csharpRetryPolicy<HttpResponseMessage> httpRetryPolicy = Policy
    .HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
    .Or<HttpRequestException>()
    .WaitAndRetryAsync(3, retryAttempt =>
        TimeSpan.FromSeconds(Math.Pow(2, retryAttempt) / 2));

HttpResponseMessage httpResponseMessage = await
    httpRetryPolicy.ExecuteAsync(
        () => httpClient.GetAsync(remoteEndpoint));
Enter fullscreen mode Exit fullscreen mode

💡 Pro tip: Exponential backoff (2s → 4s → 8s) prevents your retry storm from hammering an already struggling service.

Circuit Breaker Policy
The Circuit Breaker is like an electrical fuse for your microservices.
When too many failures are detected, the circuit "opens" and immediately rejects all requests — preventing cascading failures across your entire system. After a defined break duration, it allows one test request through. If that succeeds, the circuit "closes" again.
Basic Circuit Breaker
Opens after 2 consecutive failures, stays open for 60 seconds:

csharpCircuitBreakerPolicy<HttpResponseMessage> basicCircuitBreakerPolicy = Policy
    .HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
    .CircuitBreakerAsync(2, TimeSpan.FromSeconds(60));

HttpResponseMessage response =
    await basicCircuitBreakerPolicy.ExecuteAsync(
        () => _httpClient.GetAsync(remoteEndpoint));
Enter fullscreen mode Exit fullscreen mode

Advanced Circuit Breaker ⭐ (Used in Production)
More intelligent — breaks based on a failure rate threshold within a time window:

csharpCircuitBreakerPolicy<HttpResponseMessage> advancedCircuitBreakerPolicy = Policy
    .HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
    .AdvancedCircuitBreakerAsync(
        0.01,                      // 1% failure rate threshold
        TimeSpan.FromSeconds(60),  // Sampling window
        1000,                      // Minimum throughput before evaluating
        TimeSpan.FromSeconds(10)   // How long to stay open
    );

HttpResponseMessage response =
    await advancedCircuitBreakerPolicy.ExecuteAsync(
        () => _httpClient.GetAsync(remoteEndpoint));
Enter fullscreen mode Exit fullscreen mode

How it works: If there's a 1% failure rate within a 60-second window (with at least 1,000 requests), the circuit opens for 10 seconds. After that, one test request is allowed through.

✅ Real-world use case: At LexCheck, this Advanced Circuit Breaker prevented a cascade failure during a traffic spike — stopping one struggling upstream service from taking down our entire email processing pipeline.

🛡️ Fallback Policy
Sometimes retries simply won't fix the problem. The Fallback policy lets you define a graceful degradation path — returning a default value or executing an alternative action:

csharpFallbackPolicy fallback = Policy.Handle<Exception>()
    .Fallback(() => PageAnAdmin()); // Notify admin if all else fails

fallback.Execute(() => GetInventory());
Fallback actions could include:
Enter fullscreen mode Exit fullscreen mode

📧 Sending an alert to your on-call engineer
📊 Returning cached/stale data
🔃 Triggering a service restart
📝 Logging the failure for analysis

🔗 Policy Wraps — Combining Multiple Policies
The real power of Polly comes from combining policies. A Policy Wrap lets you stack them together:

csharpvar wrapPolicy = Policy.Wrap(
    fallbackPolicy,   // Outermost: catch everything
    retryPolicy,      // Middle: retry before giving up
    timeoutPolicy     // Innermost: enforce time limits
);

wrapPolicy.Execute(() => SomeMethod());
Enter fullscreen mode Exit fullscreen mode

Recommended wrap order for HTTP calls:
Fallback → Circuit Breaker → Retry → Timeout
This ensures timeouts trigger retries, retries trigger the circuit breaker, and if all fails — the fallback kicks in.

⏱️ Timeout Policy
Not all HTTP clients or services have built-in timeouts. The Timeout policy lets you define the maximum acceptable duration for any operation:

csharpTimeoutPolicy timeoutPolicy =
    Policy.Timeout(1, TimeoutStrategy.Pessimistic, OnTimeout);

var result = timeoutPolicy.Execute(() => ComplexAndSlowCode());
Enter fullscreen mode Exit fullscreen mode

TimeoutStrategy.Pessimistic uses a cancellation token to gracefully terminate long-running operations.

💾 Cache Policy
Polly's Cache policy stores the results of previous requests — either in-memory or in a distributed cache (like Redis). On a duplicate request, Polly returns the cached result instead of making another call:

csharpvar memoryCache = new MemoryCache(new MemoryCacheOptions());
var memoryCacheProvider = new MemoryCacheProvider(memoryCache);

CachePolicy<int> cachePolicy =
    Policy.Cache<int>(memoryCacheProvider, TimeSpan.FromSeconds(10));

var result = cachePolicy.Execute(
    context => QueryRemoteService(id),
    new Context($"QRS-{id}"));
Enter fullscreen mode Exit fullscreen mode

Polly supports multiple TTL strategies:

Relative — 10 seconds from when it was cached
Absolute — expires at a fixed point in time
Sliding — resets on each access
Result — TTL extracted from the result itself (perfect for auth tokens!)

🧱 Bulkhead Isolation
Named after the watertight compartments in ship hulls — Bulkhead Isolation limits how many concurrent requests a specific operation can consume, preventing one slow upstream service from starving your entire application:

csharp// Max 3 concurrent executions, queue up to 6 waiting
BulkheadPolicy bulkheadPolicy = Policy.Bulkhead(3, 6);

var result = bulkheadPolicy.Execute(() => ResourceHeavyRequest());
Enter fullscreen mode Exit fullscreen mode

Once the execution slots (3) and queue slots (6) are full, additional requests are rejected immediately — keeping your application responsive for everything else.

When Should You Use the Retry Pattern?
✅ Use Retry for:

HTTP calls to external APIs
Database connection attempts
Network socket operations
Message queue interactions
Any operation where failures are transient and idempotent

Avoid Retry for:

Non-idempotent operations — inserting a payment record twice = duplicate charges
Permanent failures — a 404 Not Found won't be fixed by retrying
Validation errors — bad input data won't become valid after 3 retries

Complete Resilience Pipeline Example
Here's a production-ready pipeline combining all key policies for an HTTP service:

csharp// 1. Timeout — abort if takes more than 10 seconds
var timeoutPolicy = Policy.TimeoutAsync<HttpResponseMessage>(10);

// 2. Retry — try up to 3 times with exponential backoff
var retryPolicy = Policy
    .HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
    .Or<HttpRequestException>()
    .WaitAndRetryAsync(3, attempt =>
        TimeSpan.FromSeconds(Math.Pow(2, attempt)));

// 3. Circuit Breaker — open on 50% failure rate
var circuitBreakerPolicy = Policy
    .HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
    .AdvancedCircuitBreakerAsync(0.5,
        TimeSpan.FromSeconds(30), 20, TimeSpan.FromSeconds(15));

// 4. Fallback — return default on total failure
var fallbackPolicy = Policy
    .HandleResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
    .Or<BrokenCircuitException>()
    .FallbackAsync(new HttpResponseMessage(HttpStatusCode.OK)
    {
        Content = new StringContent("{ \"fallback\": true }")
    });

// 5. Wrap them all together
var resiliencePolicy = Policy.WrapAsync(
    fallbackPolicy,
    circuitBreakerPolicy,
    retryPolicy,
    timeoutPolicy
);

// Execute your HTTP call with full resilience
HttpResponseMessage response = await resiliencePolicy.ExecuteAsync(
    () => httpClient.GetAsync("/api/resource"));
Enter fullscreen mode Exit fullscreen mode

Conclusion
Building resilient .NET applications doesn't have to be complicated. Polly gives you a clean, expressive API to handle the messy reality of distributed systems — where things will fail.
Here's a quick summary of when to use each policy:

Policy Use When
Retry Transient, self-healing failures
Circuit Breaker Upstream service is consistently failing
Fallback You need graceful degradation
Policy Wrap Combining multiple strategies
Timeout Operations have no built-in time limit
Cache Repeated read requests to slow services
Bulkhead Isolating resource-heavy operations

What's Next?

📖 Official Polly Documentation
🔗 Microsoft's guide on HTTP resilience with Polly

Have questions or want to share how you're using Polly in production? Drop a comment below! 👇
If this helped you, consider sharing it with your team — resilience patterns save production systems at the worst possible moments.

About the Author
I'm Izma Anwar, a Senior .NET & Full Stack Engineer with 6+ years building enterprise-grade microservices, cloud-native applications, and scalable backend systems. I write about .NET, C#, AWS, Azure, and software architecture.
🔗 LinkedIn | 🌐 izmaanwar.com | 💻 GitHub

Top comments (0)