An outbound integration can be perfectly healthy and still fail under load because your client exceeds the provider's request quota. Retrying every 429 Too Many Requests response often makes the burst worse; the client needs admission control before the request leaves the process.
This article compares SemaphoreSlim, System.Threading.RateLimiting, and Polly resilience pipelines for rate-limited HttpClient calls. The focus is not only on staying below a requests-per-second limit, but also on configuration, dependency injection, rejection behavior, retries, and unit-testability.
Client-Side Rate Limiting vs. Throttling
The general term for deliberately restricting the rate of work is throttling. In this scenario, the client admits outbound requests according to a policy that reflects the provider's quota.
Throttling is not the same as retrying. A limiter controls when new work may start; a retry policy decides what to do after an attempt fails. Production integrations often need both, configured so retries also consume permits instead of bypassing the quota.
Action games often require players to press buttons rapidly to perform actions such as shooting or striking. Players usually press them much more often than necessary, perhaps because they get caught up in the action. A player may press the attack button 10 times in five seconds, while the character can attack no more than once per second. Throttling the attack event lets the game ignore repeated presses during that second.
How do we implement a similar limit?
Rate-Limiting Algorithms in .NET
Several algorithms implement rate limiting by controlling how much traffic an object admits.
Many rate-limiting algorithms can control a flow of requests. .NET 7 introduced four of them.
Concurrency Limit
A concurrency limiter restricts the number of simultaneous requests that can access a resource. If the limit is 10, then 10 requests can access the resource at once and the 11th is rejected. When one request finishes, one permit becomes available; when two finish, two become available, and so on. This happens when Dispose is called on the RateLimitLease instance, which we will discuss later.
Token Bucket Limit
The token bucket algorithm takes its name from how it works. Imagine a bucket filled to the brim with tokens. An incoming request takes a token and keeps it forever. At intervals, someone adds a predefined number of tokens to the bucket, never exceeding its capacity. If the bucket is empty, an incoming request is denied access to the resource.
Here is a concrete example. Suppose the bucket can hold 10 tokens, and 2 tokens are added every minute. The first request takes 1 token, leaving 9. Three more requests take 3 tokens, leaving 6. A minute later, 2 new tokens arrive, bringing the total to 8. Eight requests consume the remaining tokens and empty the bucket. Any request after that cannot access the resource until more tokens are available. In this example, tokens are replenished every minute. After five minutes without requests, the bucket contains all 10 tokens again, and no more are added until new requests begin consuming them.
Fixed Window Limit
The fixed window algorithm uses the concept of a window, which also appears in the next algorithm. A window is the period during which a limit applies before the algorithm moves to the next window. With a fixed window, moving to the next one resets the limit to its initial state.
Imagine a movie theater with one 100-seat auditorium showing a two-hour film. When the film begins, people may queue for the next showing two hours later. Up to 100 people can join the queue before newcomers are told to return later. After two hours, the film ends and anywhere from 0 to 100 people can enter the auditorium, allowing a new queue to form. That is equivalent to moving the window in the fixed-window algorithm.
Sliding Window Limit
The sliding window algorithm resembles the fixed window algorithm but divides the window into segments. A segment is one part of the window. If we split the two-hour window from the previous section into four segments, we get four 30-minute segments. The algorithm also tracks the current segment index, which always points to the newest segment in the window. Requests during a 30-minute period enter the current segment, and every 30 minutes the window advances by one segment. If the segment that slides out of the window contained requests, those requests expire and the available limit increases by that amount. If it contained none, the limit remains unchanged.
Three Ways to Rate-Limit HttpClient Calls
| Approach | Best fit in this article |
|---|---|
SemaphoreSlim |
A simple concurrency cap with minimal dependencies |
System.Threading.RateLimiting |
An explicit, configurable rate-limiting algorithm |
| Polly resilience pipeline | Rate limiting composed with retries, telemetry, and other resilience policies |
That is enough theory for now. The manager is still waiting for the integration, so we need to start writing code as soon as possible.
This is what the manager looks like, by the way:
Suppose we have a DataObject that contains some Content.
We can retrieve a DataObject by calling an IDataObjectExternalApiService, where an HttpClient instance sends the request under the hood:
record DataObject(string Content);
interface IDataObjectExternalApiService
{
Task<DataObject> GetByIdAsync(int id, CancellationToken ct = default);
}
We have thousands of identifiers and need to download the corresponding content for each one:
interface IDataObjectCollectionProvider
{
Task<IReadOnlyCollection<DataObject>> GetByIdsAsync(
IReadOnlyCollection<int> ids,
CancellationToken ct = default);
}
There is one problem: the external API allows no more than 10 RPS.
SemaphoreSlim
If the requirement means no more than 10 simultaneous requests, we can try to implement a concurrency limiter with the SemaphoreSlim synchronization primitive.
SemaphoreSlim is a lightweight alternative to Semaphore that limits the number of threads that can access a resource or pool of resources at the same time. It also works with async/await.
The result looks roughly like this:
class DataObjectCollectionProvider : IDataObjectCollectionProvider
{
private readonly IDataObjectExternalApiService _externalApiService;
public DataObjectCollectionProvider(IDataObjectExternalApiService externalApiService) =>
_externalApiService = externalApiService;
public async Task<IReadOnlyCollection<DataObject>> GetByIdsAsync(
IReadOnlyCollection<int> ids,
CancellationToken ct = default)
{
if (ids.Count == 0)
return [];
var semaphoreSlim = new SemaphoreSlim(
initialCount: 10,
maxCount: 10);
ConcurrentBag<DataObject> dataObjects = [];
var tasks = ids.Select(async id =>
{
await semaphoreSlim.WaitAsync(ct);
try
{
var dataObject = await _externalApiService.GetByIdAsync(id, ct);
dataObjects.Add(dataObject);
}
finally
{
semaphoreSlim.Release();
}
});
await Task.WhenAll(tasks);
return dataObjects;
}
}
In my view, this looks fairly ad hoc. We have to embed the synchronization primitive in the invocation and iteration logic, mixing infrastructure code with business logic.
What happens if we need a different algorithm?
System.Threading.RateLimiting
As mentioned above, .NET 7 introduced the System.Threading.RateLimiting NuGet package, which implements the algorithms discussed in this article.
They all derive from the abstract RateLimiter class:
public abstract class RateLimiter : IAsyncDisposable, IDisposable
{
public abstract int GetAvailablePermits();
public abstract TimeSpan? IdleDuration { get; }
public RateLimitLease Acquire(int permitCount = 1);
public ValueTask<RateLimitLease> WaitAsync(
int permitCount = 1,
CancellationToken cancellationToken = default);
public void Dispose();
public ValueTask DisposeAsync();
}
Each derived class accepts dedicated configuration options that control the algorithm's behavior.
We now have a choice. In my opinion, a fixed window is the most intuitive solution to this problem.
We can rewrite the code like this:
class DataObjectCollectionProvider : IDataObjectCollectionProvider
{
private readonly IDataObjectExternalApiService _externalApiService;
public DataObjectCollectionProvider(IDataObjectExternalApiService externalApiService) =>
_externalApiService = externalApiService;
public async Task<IReadOnlyCollection<DataObject>> GetByIdsAsync(
IReadOnlyCollection<int> ids,
CancellationToken ct = default)
{
if (ids.Count == 0)
return [];
var limiter = new FixedWindowRateLimiter(
new FixedWindowRateLimiterOptions
{
Window = TimeSpan.FromSeconds(1),
PermitLimit = 10,
QueueLimit = 10
});
ConcurrentBag<DataObject> dataObjects = [];
var tasks = ids.Select(async id =>
{
using var lease = await limiter.AcquireAsync(cancellationToken: ct);
if (lease.IsAcquired)
{
var dataObject = await _externalApiService.GetByIdAsync(id, ct);
dataObjects.Add(dataObject);
}
});
await Task.WhenAll(tasks);
return dataObjects;
}
}
This is better: the algorithm is replaceable, and the options can be injected through DI.
Notice that the limiter uses PermitLimit = 10 and QueueLimit = 10. This means no more than 10 requests enter a one-second window, while queued WaitAsync calls may request at most 10 permits in total.
But what happens if a request cannot obtain a permit and the limiter rejects it? How should we build the error-handling logic—and keep it unit-testable?
Polly.RateLimiting
This is where Polly helps. It wraps System.Threading.RateLimiting in the Polly.RateLimiting package.
The tool changed significantly when it introduced pipelines. A pipeline wraps the call you need to make after you attach all the necessary bells and whistles. See this documentation section for details.
Rate limiters can be added as pipeline stages, and the pipeline itself can be passed through DI to a particular consumer.
Injection uses a dedicated provider that retrieves the pipeline by key. For simplicity, I will use the consumer type name as the key.
We can decorate the external service and execute its call through the pipeline:
class DataObjectServiceRateLimiterDecorator : IDataObjectExternalApiService
{
private readonly IDataObjectExternalApiService _decorated;
private readonly ResiliencePipeline<DataObject> _pipeline;
public DataObjectServiceRateLimiterDecorator(
IDataObjectExternalApiService decorated,
ResiliencePipelineProvider<string> pipelineProvider)
{
_decorated = decorated;
_pipeline = pipelineProvider.GetPipeline<DataObject>(
key: nameof(DataObjectServiceRateLimiterDecorator));
}
public async Task<DataObject> GetByIdAsync(int id, CancellationToken ct = default) =>
await _pipeline.ExecuteAsync(async token => await _decorated.GetByIdAsync(id, token), ct);
}
class DataObjectCollectionProvider : IDataObjectCollectionProvider
{
private readonly IDataObjectExternalApiService _externalApiService;
public DataObjectCollectionProvider(IDataObjectExternalApiService externalApiService) =>
_externalApiService = externalApiService;
public async Task<IReadOnlyCollection<DataObject>> GetByIdsAsync(
IReadOnlyCollection<int> ids,
CancellationToken ct = default)
{
if (ids.Count == 0)
return [];
var tasks = ids.Select(id => _externalApiService.GetByIdAsync(id, ct));
return await Task.WhenAll(tasks);
}
}
Suppose the RPS value comes from options and we want to configure a retry policy alongside the limiter. The configuration would look roughly like this:
services.AddResiliencePipeline<string, DataObject>(
nameof(DataObjectServiceRateLimiterDecorator),
(builder, pollyContext) =>
{
var allowedRps = pollyContext.ServiceProvider
.GetRequiredService<IOptions<IDataObjectApiOptions>>()
.Value.RequestsPerSecond;
builder
.ConfigureTelemetry(NullLoggerFactory.Instance)
.AddRetry(
new RetryStrategyOptions<DataObject>
{
Delay = TimeSpan.FromSeconds(1),
MaxRetryAttempts = 5
})
.AddRateLimiter(
new FixedWindowRateLimiter(
new FixedWindowRateLimiterOptions
{
Window = TimeSpan.FromSeconds(1),
PermitLimit = allowedRps,
QueueLimit = allowedRps / 3 + 10,
}));
});
Finally, we can write unit tests that verify pipeline behavior in several situations. For example:
public class DataObjectServiceRateLimiterDecoratorTests
{
[Theory]
[InlineData(1)]
[InlineData(2)]
[InlineData(3)]
[InlineData(4)]
[InlineData(5)]
public async Task GetByIdAsync_ApiReturnedError_RetryCallHappened(int retryCount)
{
// arrange
var response = new DataObject(Content: Guid.NewGuid().ToString());
var apiService = new Mock<IDataObjectExternalApiService>();
var sequentialResult = apiService.SetupSequence(
x => x.GetByIdAsync(
It.IsAny<int>(),
It.IsAny<CancellationToken>()));
for (var i = 0; i < retryCount - 1; i++)
sequentialResult = sequentialResult.ThrowsAsync(new Exception());
sequentialResult.ReturnsAsync(response);
var pipelineProvider = new Mock<ResiliencePipelineProvider<string>>();
pipelineProvider
.Setup(
x => x.GetPipeline<DataObject>(
nameof(DataObjectServiceRateLimiterDecorator)))
.Returns(
new ResiliencePipelineBuilder<DataObject>()
.AddRetry(
new RetryStrategyOptions<DataObject>
{
MaxRetryAttempts = retryCount,
Delay = TimeSpan.FromMilliseconds(1)
})
.Build());
var decorator = new DataObjectServiceRateLimiterDecorator(
apiService.Object, pipelineProvider.Object);
// act
var dataObject = await decorator.GetByIdAsync(id: default, ct: default);
// assert
dataObject.Should().BeEquivalentTo(response);
apiService
.Verify(
x => x.GetByIdAsync(
It.IsAny<int>(),
It.IsAny<CancellationToken>()),
Times.Exactly(retryCount));
}
[Theory]
[InlineData(5, 1)]
[InlineData(100, 50)]
[InlineData(60, 60)]
[InlineData(10, 11)]
[InlineData(20, 40)]
[InlineData(30, 100)]
public async Task GetByIdAsync_IfRpsRateLimitExceeded_ThenExceptionIsThrown(int rps, int amount)
{
// arrange
var apiService = new Mock<IDataObjectExternalApiService>();
apiService
.Setup(
x => x.GetByIdAsync(
It.IsAny<int>(),
It.IsAny<CancellationToken>()))
.ReturnsAsync(new DataObject(Content: string.Empty));
var pipelineProvider = new Mock<ResiliencePipelineProvider<string>>();
pipelineProvider
.Setup(
x => x.GetPipeline<DataObject>(
nameof(DataObjectServiceRateLimiterDecorator)))
.Returns(
new ResiliencePipelineBuilder<DataObject>()
.AddRateLimiter(
new FixedWindowRateLimiter(
new FixedWindowRateLimiterOptions
{
PermitLimit = rps,
Window = TimeSpan.FromSeconds(1)
}))
.Build());
var decorator = new DataObjectServiceRateLimiterDecorator(
apiService.Object, pipelineProvider.Object);
// act
var tasks = Enumerable.Range(0, amount)
.Select(id => decorator.GetByIdAsync(id, ct: default));
var ex = await Record.ExceptionAsync(() => Task.WhenAll(tasks));
// assert
if (amount > rps)
ex.Should().BeOfType<RateLimiterRejectedException>();
else
ex.Should().BeNull();
}
[Fact]
public async Task GetByIdAsync_HappyPath()
{
// arrange
var apiService = new Mock<IDataObjectExternalApiService>();
apiService
.Setup(
x => x.GetByIdAsync(
It.IsAny<int>(),
It.IsAny<CancellationToken>()))
.ReturnsAsync(new DataObject(Content: Guid.NewGuid().ToString()));
var pipelineProvider = new Mock<ResiliencePipelineProvider<string>>();
pipelineProvider
.Setup(
x => x.GetPipeline<DataObject>(
nameof(DataObjectServiceRateLimiterDecorator)))
.Returns(
new ResiliencePipelineBuilder<DataObject>()
.AddRetry(
new RetryStrategyOptions<DataObject>
{
MaxRetryAttempts = 5,
Delay = TimeSpan.FromSeconds(1)
})
.AddRateLimiter(
new FixedWindowRateLimiter(
new FixedWindowRateLimiterOptions
{
PermitLimit = 10,
Window = TimeSpan.FromSeconds(1)
}))
.Build());
var decorator = new DataObjectServiceRateLimiterDecorator(
apiService.Object, pipelineProvider.Object);
// act
var tasks = Enumerable.Range(0, 100)
.Select(id => decorator.GetByIdAsync(id, ct: default));
var result = await Task.WhenAll(tasks);
// assert
result.Length.Should().Be(100);
}
}
Choosing a .NET Rate-Limiting Approach
For a small, local concurrency cap, SemaphoreSlim may be enough. Use System.Threading.RateLimiting when the rate-limiting algorithm itself should be explicit and configurable. Use a Polly pipeline when rate limiting must compose with retries, telemetry, and other resilience policies behind a testable service boundary.
A client-side limiter is still only one part of the contract. Multiple application instances do not share an in-memory quota, and the provider may change limits dynamically. Account for deployment topology, honor server feedback such as Retry-After, and decide explicitly whether excess work should wait, fail fast, or be dropped.
Related .NET Guides
- C# Concurrent Collections: A Practical Guide
- Polymorphic Dependency Injection in .NET: The Complete Guide
Follow StepOne on GitHub for open-source C#/.NET projects, compiler experiments, production-focused examples, and new releases.
Top comments (0)