Polly is a .NET library that helps to increase the resiliency of your application. It offers a myriad of strategies such as Retry, Circuit Breaker, Timeout, Rate Limiter, Fallback, and Hedging to manage unexpected behaviors. Polly also offers chaos engineering features, enabling you to introduce unexpected behaviors into your app, allowing you to test your resilience setup without waiting for a real incident. In this article, we will focus on practical implementation strategies to add Polly to your .NET app.
We will describe the following approaches:
-
Using built-in support in the client API, typically in
HttpClient. - Adding Polly to your business code, doing things manually.
- Using the Type Decorator pattern to reduce boilerplate.
- Reducing boilerplate with an aspect featuring Metalama as another, more general approach to boilerplate reduction.
- Adding Polly to ASP.NET inbound requests using ASP.NET middleware.
1. Using built-in support in the client API
Certain components are designed with Polly in mind. One of them is the HttpClient class that comes with .NET. Microsoft provides the Microsoft.Extensions.Http.Resilience and Microsoft.Extensions.Resilience libraries. These libraries are built with Polly and simplify the integration of resilience into the .NET application.
Here’s a straightforward example that retrieves data from an HTTP endpoint and retries when there’s a failure. It calls the AddStandardResilienceHandler method to inject pre-configured resilience policies into the HttpClient.
const string httpClientName = "MyClient";
var services = new ServiceCollection()
.AddLogging( b => b.AddConsole().SetMinimumLevel( LogLevel.Debug ) )
.AddHttpClient( httpClientName )
.AddStandardResilienceHandler()
.Services
.BuildServiceProvider();
var clientFactory = services.GetRequiredService<IHttpClientFactory>();
var client = clientFactory.CreateClient( httpClientName );
var response = await client.GetAsync(
"http://localhost:52394/FailEveryOtherTime" );
Console.WriteLine( await response.Content.ReadAsStringAsync() );
The full source code for this article is available from blog.postsharp.net.
You can customize the resilience policies by passing options as an argument to the AddStandardResilienceHandler method. You can learn more about the resilience strategies in the Polly documentation.
Avoid creating HttpClient instances yourself
The AddStandardResilienceHandler method will work only if you get your HttpClient instances by using the IHttpCientFactory you get from the service provider. It won’t handle the case where an HttpClient instance is created using its constructor. It might be challenging to remember this rule. Luckily, software architecture validation tools, like Metalama, can prevent developers from writing code that breaks this rule. Adding the following code to your project would trigger warnings wherever the HttpClient’s constructor is used explicitly:
internal class AvoidInstantiatingHttpClientFabric : ProjectFabric
{
public override void AmendProject( IProjectAmender amender )
{
amender
.Verify()
.SelectTypes( typeof(HttpClient) )
.SelectMany( t => t.Constructors )
.CannotBeUsedFrom(
r => r.Always(),
$"Use {nameof(IHttpClientFactory)} instead." );
}
}
With this code in your project, code using the HttpClient constructor will immediately be reported with a warning.
2. Adding Polly to your business code
There are a few cases where you might need to call Polly directly from your business code or data access layer. One of those is when there’s no component-specific API for managing application resilience. The second is when the business logic involves several steps that must be retried as a whole.
Let’s consider a method that executes SQL commands on a cloud database service. This is a money transfer operation, where both account updates must be performed atomically in a transaction.
public async Task TransferAsync(
int sourceAccountId,
int targetAccountId,
int amount,
CancellationToken cancellationToken = default )
{
var transaction = await connection.BeginTransactionAsync( cancellationToken );
try
{
await using ( var command = connection.CreateCommand() )
{
command.CommandText =
"UPDATE accounts SET balance = balance - $amount WHERE id = $id";
command.AddParameter( "$id", sourceAccountId );
command.AddParameter( "$amount", amount );
await command.ExecuteNonQueryAsync( cancellationToken );
}
await using ( var command = connection.CreateCommand() )
{
command.CommandText =
"UPDATE accounts SET balance = balance + $amount WHERE id = $id";
command.AddParameter( "$id", targetAccountId );
command.AddParameter( "$amount", amount );
await command.ExecuteNonQueryAsync( cancellationToken );
}
await transaction.CommitAsync( cancellationToken );
}
catch
{
await transaction.RollbackAsync( cancellationToken );
throw;
}
}
Occasionally, an influx of requests may temporarily overload the database. In such cases, we might want to retry the whole transaction.
The best way to add Polly to your services is to use dependency injection and to add a resilience policy to the IServiceCollection. We configure the policy with a Retry Strategy that reacts when the database commands fail on the DbException, waits initially 1 second, followed by an exponential backoff strategy, and retries no more than 3 times:
services.AddResiliencePipeline(
"db-pipeline",
pipelineBuilder =>
{
pipelineBuilder.AddRetry(
new RetryStrategyOptions
{
ShouldHandle = new PredicateBuilder()
.Handle<DbException>(),
Delay = TimeSpan.FromSeconds( 1 ),
MaxRetryAttempts = 3,
BackoffType = DelayBackoffType.Exponential
} )
.ConfigureTelemetry( LoggerFactory.Create(
loggingBuilder => loggingBuilder.AddConsole() ) );
} );
With the pipeline in place, we can consume it from the Accounts service.
internal class Accounts(
DbConnection connection,
[FromKeyedServices( "db-pipeline" )]
ResiliencePipeline resiliencePipeline )
We can wrap the method to be retried with a call to the policy’s Execute method.
public async Task TransferAsync(
int sourceAccountId,
int targetAccountId,
int amount,
CancellationToken cancellationToken = default )
{
await resiliencePipeline.ExecuteAsync(
async t =>
{
var transaction = await connection.BeginTransactionAsync( t );
try
{
await using ( var command = connection.CreateCommand() )
{
command.CommandText =
"UPDATE accounts SET balance = balance - $amount WHERE id = $id";
command.AddParameter( "$id", sourceAccountId );
command.AddParameter( "$amount", amount );
await command.ExecuteNonQueryAsync( t );
}
await using ( var command = connection.CreateCommand() )
{
command.CommandText =
"UPDATE accounts SET balance = balance + $amount WHERE id = $id";
command.AddParameter( "$id", targetAccountId );
command.AddParameter( "$amount", amount );
await command.ExecuteNonQueryAsync( t );
}
await transaction.CommitAsync( t );
}
catch
{
await transaction.RollbackAsync( t );
throw;
}
},
cancellationToken );
}
3. Using the Type Decorator pattern
Instead of editing all code locations that use a DbCommand, an alternative approach is to inject the Polly logic into the DbCommand itself. Since DbCommand is an abstract class, we can implement a Type Decorator pattern and wrap the call to the real database client with a call to Polly.
Some database connectors, like the SqlConnection class, already have their retry mechanism and would not benefit from an additional decorator.
Here is a partial implementation of ResilientDbCommand that follows the Type Decorator pattern:
public partial class ResilientDbCommand(
DbCommand underlyingCommand,
ResiliencePipeline resiliencePipeline ) : DbCommand
{
public override int ExecuteNonQuery()
=> resiliencePipeline.Execute( underlyingCommand.ExecuteNonQuery );
public override object? ExecuteScalar()
=> resiliencePipeline.Execute( underlyingCommand.ExecuteScalar );
protected override DbDataReader ExecuteDbDataReader(
CommandBehavior behavior )
=> resiliencePipeline.Execute( () =>
underlyingCommand.ExecuteReader( behavior ) );
public override void Prepare()
=> resiliencePipeline.Execute( underlyingCommand.Prepare );
public override void Cancel()
=> resiliencePipeline.Execute( underlyingCommand.Cancel );
}
And here is how the connection is initialized.
await using var connection = new UnreliableDbConnection( new SqliteConnection( "Data Source=:memory:" ) );
var resiliencePipeline = CreateRetryOnDbExceptionPipeline();
var resilientConnection = new ResilientDbConnection(
connection, resiliencePipeline );
services.AddSingleton<DbConnection>( resilientConnection );
In this manner, the data layer code remains unchanged.
There is a fundamental difference between the two previous approaches: while the type-decorator approach retries an individual DbCommand, the data-layer approach retries the whole transaction. This difference can be significant if the DbCommand executes a non-transactional, multi-step operation, such as a stored procedure.
4. Reducing boilerplate with an aspect
When the Type Decorator pattern is not possible or convenient, there is still a better approach than using Polly directly in the business code. Imagine that your business code does not have a single method as in this simplistic example, but hundreds or thousands. Do you really want to repeat the Polly boilerplate for each of them? Probably not.
Fortunately, there are tools that allow you to add features to methods without modifying their source code, thus keeping the code readable. One such tool is Metalama. Metalama allows for moving the wrapping logic to a custom attribute, called an aspect. You can compare an aspect to a code template.
Without going into details, here is the source code of the aspect, where OverrideMethod is the template for non-async methods.
public partial class RetryAttribute : OverrideMethodAspect
{
private readonly string _pipelineName;
[IntroduceDependency]
private readonly ResiliencePipelineProvider<string> _resiliencePipelineProvider;
public RetryAttribute( string pipelineName = "default" )
{
this._pipelineName = pipelineName;
}
public override dynamic? OverrideMethod()
{
var pipeline = this._resiliencePipelineProvider
.GetPipeline( this._pipelineName );
return pipeline.Execute( Invoke );
object? Invoke( CancellationToken cancellationToken = default )
{
return meta.Proceed();
}
}
}
If we add the aspect to our method, the code template will be expanded at compile time. As a bonus, we also implemented transaction handling as an aspect:
[Retry]
[DbTransaction]
public async Task TransferAsync(
int sourceAccountId,
int targetAccountId,
int amount,
CancellationToken cancellationToken = default )
{
await using ( var command = this._connection.CreateCommand() )
{
command.CommandText =
"UPDATE accounts SET balance = balance - $amount WHERE id = $id";
command.AddParameter( "$id", sourceAccountId );
command.AddParameter( "$amount", amount );
await command.ExecuteNonQueryAsync( cancellationToken );
}
await using ( var command = this._connection.CreateCommand() )
{
command.CommandText =
"UPDATE accounts SET balance = balance + $amount WHERE id = $id";
command.AddParameter( "$id", targetAccountId );
command.AddParameter( "$amount", amount );
await command.ExecuteNonQueryAsync( cancellationToken );
}
}
As you can see, we got rid of most of the boilerplate code in this code.
5. Adding Polly to ASP.NET inbound requests
We have seen approaches to add Polly to client endpoints (both HTTP and database), and approaches to add Polly in your business code. A third possibility is to add Polly at your server endpoint, that is, to wrap your entire request processing in a Polly policy.
Indeed, in ASP.NET Core apps, Polly can be easily introduced as a middleware. To illustrate this approach, let’s consider a microservice that processes data of a public API endpoint. Since we have no control over the endpoint, we need to handle any transient failures on our app’s side.
Any class with an InvokeAsync method of the proper signature can be an ASP.NET Core middleware. Here is ours. It consumes the Polly policy named middleware, which we need to configure exactly as in the above examples. The only difficulty to overcome is that we need to supply a restartable HttpContext to the downstream handler because an exception could happen in the middle of writing the HTTP response, and retrying the whole operation could cause a duplication of the output that has been written before the exception.
// Noted that keyed service does not seem available for middleware.
public class ResilienceMiddleware( RequestDelegate next, ResiliencePipelineProvider<string> pipelineProvider )
{
public async Task InvokeAsync( HttpContext httpContext )
{
var pipeline = pipelineProvider.GetPipeline( "middleware" );
var bufferingContext = new RestartableHttpContext( httpContext );
await bufferingContext.InitializeAsync( httpContext.RequestAborted );
await pipeline.ExecuteAsync(
async _ =>
{
bufferingContext.Reset();
await next( bufferingContext );
},
httpContext.RequestAborted );
await bufferingContext.AcceptAsync();
}
}
To add the middleware in the ASP.NET Core pipeline, use the UseMiddleware method:
var app = builder.Build();
app.UseMiddleware<ResilienceMiddleware>();
Now all the requests served by our microservice are handled by Polly. The microservice then behaves more reliably, even when depending on services that experience transient failures.
Summary
Polly is a useful .NET library that helps make our .NET app resilient using various strategies. Polly can be added using a component-specific API, directly to your code, or using the decorator pattern, either by creating a wrapping type, or by moving the resiliency logic to method decorators (aspect-oriented), to keep your code clean, maintainable, and scalable, without losing the resiliency power of Polly.
This article was first published on a https://blog.postsharp.net under the title 5 Practical Ways to Add Polly to Your C# Application [2024].
Top comments (0)