What every ASP.NET Core Web API project needs - Part 5 - Polly

The 4th part of this series was about Error Message Reusability and Localization and in this article, I'm going to show you how to add resilience and transient fault handling to HttpClient with Polly.

When you create an ASP.NET Core Web API project, there is a WeatherForecastController that returns a list of weather forecasts. I'm going to use external weather API to get real weather forecasts instead of returning static data.

Step 1 - External API

• Go to the weatherapi.com
• Create an account
• After login, you find the API key at the dashboard

Step 2 - Typed HTTP Client

There are several ways to use HttpClient class in a project:

• Basic usage
• Named clients
• Typed clients
• Generated clients

Most of the time I Prefer to use typed client because of:

• Provide the same capabilities as named clients without the need to use strings as keys
• Provides IntelliSense and compiler help when consuming clients
• Provide a single location to configure and interact with a particular HttpClient. For example, a single typed client might be used:
  • For a single backend endpoint.
  • To encapsulate all logic dealing with the endpoint
• Work with DI and can be injected where required in the app

Let's create a typed client for weather external API:

• Create a new folder HttpClients inside Infrastructure folder
• Add new class WeatherHttpClient.cs to HttpClients folder
• Open appsettings.json file and add the following key/values:

"WeatherSettings": {
  "ApiKey": "YOURKEY"
  "BaseUrl": "https://api.weatherapi.com",
  "NoDaysForecast": 5
} 

• Open WeatherHttpClient.cs file and create a class for API settings:

public class WeatherSettings
{
    public string ApiKey { get; set; }

    public string BaseUrl { get; set; }

    public int NoDaysForecast { get; set; }
}

• Open Startup.cs class and inside ConfigureServices method bind and register weather settings:

var weatherSettings = new WeatherSettings();
Configuration.GetSection("WeatherSettings").Bind(weatherSettings);
services.AddSingleton(weatherSettings);

• Inside WeatherHttpClient.cs file create an interface:

public interface IWeatherHttpClient
{
    Task<IEnumerable<WeatherForecast>> GetForecastAsync(string cityName);
}

• Let's implement the above interface:

public class WeatherHttpClient : IWeatherHttpClient
{
    private readonly HttpClient _client;
    private readonly WeatherSettings _settings;

    public WeatherHttpClient(HttpClient client, WeatherSettings settings)
    {
        _client = client;
        _settings = settings;
        _client.BaseAddress = new Uri(_settings.BaseUrl);
        _client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
    }

    public async Task<IEnumerable<WeatherForecast>> GetForecastAsync(string cityName)
    {
        var url = $"v1/forecast.json?key={_settings.ApiKey}&q={cityName}&days={_settings.NoDaysForecast}";
        var response = await _client.GetAsync(url);
        response.EnsureSuccessStatusCode();
        var content = await response.Content.ReadAsStringAsync();
        ...
    }
}

The response of API is really verbose and I'm not going to create an object for deserializing JSON instead I use an anonymous object to extract only data that I need:

public async Task<IEnumerable<WeatherForecast>> GetForecastsAsync(string cityName)
{
    ...
    var days = JsonSerializerExtensions.DeserializeAnonymousType(content, new
    {
        forecast = new
        {
            forecastday = new[]
            {
               new
               {
                   date = DateTime.Now,
                   day = new { avgtemp_c = 0.0, condition = new { text = "" } }
               }
            }
        }
    }).forecast.forecastday;

    return days.Select(d => new WeatherForecast
    {
        Date = d.date,
        Summary = d.day.condition.text,
        TemperatureC = (int)d.day.avgtemp_c
    });

    // Other way to deserialize json without creating anonymous object
    // To get more information see https://docs.microsoft.com/en-us/dotnet/api/system.text.json.jsonelement?view=net-5.0
    //dynamic result = JsonSerializer.Deserialize<ExpandoObject>(content);
    //var days = result.forecast.GetProperty("forecastday").EnumerateArray();
    //foreach (var day in days)
    //{
    //    var date = day.GetProperty("date").GetDateTime();
    //    var temp = day.GetProperty("day").GetProperty("avgtemp_c").GetDouble();
    //    var condition = day.GetProperty("day").GetProperty("condition").GetProperty("text").GetString();
    //}
}

• Open Startup.cs class and register weather HTTP client service:

services.AddHttpClient<IWeatherHttpClient, WeatherHttpClient>();

• Open WeatherForecastController and inject IWeatherHttpClient:

namespace CoolWebApi.Apis.V1.Controllers
{
    [ApiController]
    [ApiVersion("1.0")]
    [Route("api/v{version:apiVersion}/[controller]")]
    public class WeatherForecastController : ControllerBase
    {
        private readonly IWeatherHttpClient _weatherClient;

        public WeatherForecastController(IWeatherHttpClient weatherClient)
        {
            _weatherClient = weatherClient;
        }

        [HttpGet]
        [ProducesResponseType(StatusCodes.Status200OK)]
        [ProducesResponseType(StatusCodes.Status400BadRequest)]
        public async Task<IEnumerable<WeatherForecast>> Get(string city = "London")
        {
            return await _weatherClient.GetForecastsAsync(city);
        }

It's time to simulate an error when we call weather API. Hence we have no control over external API, I use Fiddler Everywhere to capture and fiddle HTTP request and response.

Step 3 - Using Fiddler

• Download and install Fiddler Everywhere
• Run Fiddler and call the api/v1/weatherforecast API As you can see Fiddler captured HTTP traffics but we cannot see the request/response of HTTPS calls because they are encrypted, however, Fiddler has a mechanism to decrypt HTTPS traffics.
• On the menu click View->Preferences or hit ctrl+, to open Settings
• In the Settings window click on the HTTPS tab and click on Trust root certificate (You can remove the certificate anytime by clicking on Remove root certificate)
• Call the API again and now you can see decrypted request/response Let's filter captured traffic to get rid of other requests.
• On the URL column click on vertical dots to open the filter window and enter localhost:5001 and api.weatherapi.com URLs and change And to Or: Now it's time to change the response of api.weatherapi.com API.
• On the Live Traffic tab right-click on api.weatherapi.com row and from the menu click on Add new rule(1). In the Auto Responder tab click on the switch button to enable it(2) then click on the edit icon(3)
• On the Rule Editor window clear Raw input and the following text then click on the Save button:

HTTP/1.1 503

Service is unavailable

Let's test the rule. Again right-click on api.weatherapi.com row and from the menu click on Reply->Reissue requests and now API returns 503 instead of 200 status code

Try to call API with the Swagger and you will receive an Internal Server Error

Step 4 - Installing Polly

• Install Microsoft.Extensions.Http.Polly nuget package
• Open Startup.cs class and in ConfigureServices method modify WeatherHttpClient registration to this:

services.AddHttpClient<IWeatherHttpClient, WeatherHttpClient>()
    .AddTransientHttpErrorPolicy(policy => policy.WaitAndRetryAsync(3, _ => TimeSpan.FromSeconds(4)));

Now WeatherHttpClient retries 3 times and waits 4 seconds between each retry. Let's try again and see how many API calls will be made by WeatherHttpClient:

Let's try one more time and during API call I will turn off Fiddler Auto Responder:

This time after turning off Auto Responder API returns the status code 200 and forecast result.
If you hover your mouse on the AddTransientHttpErrorPolicy method, you get noticed that PolicyBuilder handles the following error categories:

• Network failures (as System.Net.Http.HttpRequestException)
• HTTP 5XX status codes (server errors)
• HTTP 408 status code (request timeout) For example, no retry will be made on HTTP error code 400.

Polly offers multiple resilience policies:

• Retry
• Circuit-breaker
• Timeout
• Bulkhead Isolation
• Cache
• Fallback
• PolicyWrap

Failing fast is better than making users/callers wait. If the external service is down or seriously struggling, it's better to give that system a break. In this case we can chain multiple policies in Polly to give a break. Let's chain retry policy with circuit breaker policy.

services.AddHttpClient<IWeatherHttpClient, WeatherHttpClient>()
    .AddTransientHttpErrorPolicy(policy => 
        policy.WaitAndRetryAsync(2, _ => TimeSpan.FromSeconds(2)))
    .AddTransientHttpErrorPolicy(policy => 
        policy.CircuitBreakerAsync(2, TimeSpan.FromSeconds(5)));

We want to retry 2 times and wait 2 seconds between each retry and after 4 failed retries, stop for 5 seconds.

More examples:

• Different retry times:

services.AddHttpClient<IWeatherHttpClient, WeatherHttpClient>()
    .AddTransientHttpErrorPolicy(policy => policy.WaitAndRetryAsync(new[]
        {
            TimeSpan.FromSeconds(2),
            TimeSpan.FromSeconds(6),
            TimeSpan.FromSeconds(10)
        }));

This policy will delay 2 seconds before the first retry, 6 seconds before the second retry, and 10 seconds before the third retry.

• Handle other status codes

var policy = HttpPolicyExtensions
  .HandleTransientHttpError()
  .OrResult(response => (int)response.StatusCode == 417) // RetryAfter
  .WaitAndRetryAsync(...);

As I mentioned earlier by default HandleTransientHttpError handles HttpRequestException, 5XX and 408 errors. The above policy can handle the 429 status code too.

• Custom error handling logic:

services.AddHttpClient<IWeatherHttpClient, WeatherHttpClient>()
    .AddTransientHttpErrorPolicy(policy => policy.RetryAsync(3, onRetry: (exception, retryCount) =>
    {
        //Add logic to be executed before each retry
    }));

• Retry forever until it succeeds

services.AddHttpClient<IWeatherHttpClient, WeatherHttpClient>()
    .AddTransientHttpErrorPolicy(policy => policy.RetryForeverAsync());

• Advanced Circuit Breaker

services.AddHttpClient<IWeatherHttpClient, WeatherHttpClient>()
    .AddTransientHttpErrorPolicy(policy => policy.AdvancedCircuitBreakerAsync(
        failureThreshold: 0.5,
        samplingDuration: TimeSpan.FromSeconds(10),
        minimumThroughput: 8,
        durationOfBreak: TimeSpan.FromSeconds(30)
    ));

Breaks the circuit for 30 seconds if 50% of above of incoming requests fails or a minimum of 8 faults occur within a 10 second period. The circuit is reset/closed after 30 seconds.

• Fallback

services.AddHttpClient<IWeatherHttpClient, WeatherHttpClient>()
    .AddTransientHttpErrorPolicy(policy => 
        policy.FallbackAsync(new HttpResponseMessage(HttpStatusCode.RequestTimeout)));

The fallback technique helps to return a fallback value instead of an exception re-throw when faults keep occurring. This helps the system to ensure that it gracefully tries to keep the system stable when it detects a fallback value coming by.

• Timeout

var timeoutPolicy = Policy.TimeoutAsync<HttpResponseMessage>(10);
services.AddHttpClient<IWeatherHttpClient, WeatherHttpClient>()
    .AddPolicyHandler(timeoutPolicy);

Timeout policy allows you to specify how long a request should take to respond and if it doesn’t respond in the time period you specify, the request will be canceled.

• Dynamic selection strategy

var timeoutPolicy = Policy.TimeoutAsync<HttpResponseMessage>(10);
var noOpPolicy = Policy.NoOpAsync().AsAsyncPolicy<HttpResponseMessage>();
services.AddHttpClient<IWeatherHttpClient, WeatherHttpClient>()
    .AddPolicyHandler(request => request.Method == HttpMethod.Get 
        ? retryPolicy 
        : noOpPolicy
);

You may want to define a strategy that will only apply to GET requests but not other HTTP verbs. In the above example, the timeout policy will only be used for GET requests.

• Policy Registry

var registry = services.AddPolicyRegistry();
registry.Add("DefaultRetryStrategy", HttpPolicyExtensions.HandleTransientHttpError().WaitAndRetryAsync(...));
registry.Add("DefaultCircuitBreaker", HttpPolicyExtensions.HandleTransientHttpError().CircuitBreakerAsync(...));

// Or

var registry = new PolicyRegistry()
{
    { "DefaultRetryStrategy", HttpPolicyExtensions.HandleTransientHttpError().WaitAndRetryAsync(...) },
    { "DefaultCircuitBreaker", HttpPolicyExtensions.HandleTransientHttpError().CircuitBreakerAsync(...) }
};
services.AddSingleton<IReadOnlyPolicyRegistry<string>>(registry);

services.AddHttpClient<IWeatherHttpClient, WeatherHttpClient>()
    .AddPolicyHandlerFromRegistry("DefaultCircuitBreaker")
    .AddPolicyHandlerFromRegistry("DefaultCircuitBreaker");

Polly provides a policy registry, it is equivalent to the strategy storage center, the registered strategy allows you to reuse it in multiple locations in the application.

You can find the source code for this walkthrough on Github.

Comments

[Ricardo]

Wow. What a complete article. Thanks for sharing!

Awesome sir.