DEV Community: Toc am

Retry Storms in C#: Exponential Backoff and Jitter with HttpClient

Toc am — Tue, 28 Jul 2026 13:20:34 +0000

The database blipped for four seconds. The incident lasted forty minutes.

Nothing in the timeline explains the gap except the retry code your own clients are running.

The junior version

while (true)
{
    try
    {
        var response = await client.GetAsync("https://api.internal/orders");
        response.EnsureSuccessStatusCode();
        return await response.Content.ReadAsStringAsync();
    }
    catch
    {
        // transient. it'll come back.
    }
}

// Client log during the incident:
// 14:02:11.418  attempt failed
// 14:02:11.419  attempt failed
// 14:02:11.421  attempt failed
// ...about one attempt per millisecond, per client, across 5,000 clients.

Why it breaks

Start with the part nobody expects: failure is fast. A successful call does DNS, a TLS handshake, a query, and serialisation — tens of milliseconds. A refused connection comes back as a TCP RST in under a millisecond. A 503 from the load balancer never reaches the application at all. So the loop's period is set by the fastest path through the system, and the fastest path is the broken one. The moment the service degrades, every client starts calling it harder than it ever did while healthy.

Now multiply. Normal load is one request per user action. Under failure it is a continuous stream per client, and 5,000 clients pointed at a service sized for a few thousand requests per second will offer it millions. Those attempts are not free just because they fail: each one still costs a socket accept, a TLS handshake, a thread-pool work item, and a slot in the request queue on the server side. The capacity needed to drain the backlog is exactly the capacity being spent rejecting new arrivals.

That is the whole failure mode. At 14:02:15 the original trigger is gone — the failover finished, the slow query completed — and the system stays down anyway, because the retry traffic is now the load. The service has two stable states, and your clients are pinning it in the wrong one. Recovery requires the offered load to drop, which is the one thing a retry loop will never do.

A detour worth taking: a fixed delay is not backoff

This is the fix everyone reaches for first.

catch
{
    Thread.Sleep(1000);   // there. no more spinning.
}

It does stop the spin, and it drops each client from a thousand attempts per second to one. It also blocks a thread-pool thread inside async code, which is its own problem. But the herd survives.

The clients were synchronised by the outage itself. They all failed at 14:02:11.418, so they all wake at 14:02:12.418, and again at 14:02:13.418. The failure event acted as a clock-sync pulse. What the server sees is not a steady 5,000 requests per second — it is all 5,000 arriving at the same instant, once a second, with dead air in between. Peak concurrency is what topples a connection pool, and peak concurrency is unchanged. You have made the graph prettier without moving the number that matters.

The senior version

static async Task<HttpResponseMessage> GetWithRetryAsync(
    HttpClient client, string url, CancellationToken ct)
{
    const int maxAttempts = 5;
    var baseDelay = TimeSpan.FromMilliseconds(200);
    var maxDelay  = TimeSpan.FromSeconds(20);

    for (var attempt = 1; ; attempt++)
    {
        try
        {
            var response = await client.GetAsync(url, ct);
            if (!IsTransient(response.StatusCode) || attempt == maxAttempts)
                return response;
            response.Dispose();
        }
        catch (HttpRequestException) when (attempt < maxAttempts) { }
        catch (TaskCanceledException) when (!ct.IsCancellationRequested
                                            && attempt < maxAttempts) { }

        // Exponential ceiling, then pick uniformly *below* it. That second
        // step is the jitter, and it is the part that scatters the herd.
        var ceiling = Math.Min(maxDelay.TotalMilliseconds,
                               baseDelay.TotalMilliseconds * Math.Pow(2, attempt - 1));
        await Task.Delay(TimeSpan.FromMilliseconds(Random.Shared.NextDouble() * ceiling), ct);
    }
}

static bool IsTransient(HttpStatusCode code) =>
    code == HttpStatusCode.RequestTimeout       // 408
    || code == HttpStatusCode.TooManyRequests   // 429
    || (int)code >= 500;

Exponential growth thins the traffic over time. The uniform draw spreads each wave across the whole interval, so two clients that failed in the same millisecond come back at two random points inside a 200 ms window instead of together — and that window doubles on every attempt after, to a 20-second ceiling. Random.Shared is thread-safe, so you do not need one Random per caller.

Siblings worth knowing:

Polly v8: new ResiliencePipelineBuilder().AddRetry(new RetryStrategyOptions { BackoffType = DelayBackoffType.Exponential, UseJitter = true }).
AddStandardResilienceHandler() from Microsoft.Extensions.Http.Resilience, which wires retry, jitter, a circuit breaker and a total timeout into a named HttpClient.
If the server sends Retry-After (response.Headers.RetryAfter), honour it — then add jitter on top anyway, or every client obeying it wakes in lockstep again.

When backoff isn't enough

Retry is only safe on idempotent operations. A timeout does not tell you the call failed; it tells you the response was lost. The charge may already be posted. GET, PUT and DELETE are idempotent by contract; POST is not. Retrying one needs an idempotency key the server dedupes against, not a smarter delay.

Classify before you retry. A 400, 401 or 404 will fail identically on attempt five. Retrying it burns the caller's latency budget for a guaranteed failure. That is what IsTransient above is for.

Cap elapsed time, not attempt count. Five attempts with a 30-second HTTP timeout each, plus backoff, is over two minutes — long after the browser gave up. Wrap the whole loop in one CancellationTokenSource(TimeSpan.FromSeconds(10)) and let it cut the retries short.

And when the dependency is genuinely down rather than flaky, backoff still has every client politely probing a corpse. That is a circuit breaker's job: stop calling at all, and check back with one request instead of all of them.

The takeaway

Backoff spreads out your retries. Jitter spreads out everyone else's.

Without the second one, you haven't built resilience — you've scheduled a DDoS against yourself, once per second, right on time.

I post one of these every day — the same problem solved the way that works and the way that lasts.
One senior tip a week, by email: https://seniorvsjunior.higgsfield.app

Why File.ReadAllLines Throws OutOfMemoryException on a 10 GB File in C#

Toc am — Tue, 28 Jul 2026 12:31:15 +0000

The job runs fine against the 200 MB sample. It dies in production against the real 10 GB log, before a single line is processed.

Here is the version almost everyone writes first.

The junior version

string[] lines = File.ReadAllLines(@"C:\data\events.log");

foreach (string line in lines)
{
    if (line.Contains("ERROR"))
        Console.WriteLine(line);
}

// Unhandled exception. System.OutOfMemoryException: Insufficient memory
// to continue the execution of the program.
//    at System.Collections.Generic.List`1.AddWithResize(String)
//    at System.IO.File.ReadAllLines(String path)

Nothing was printed. The process never reached the loop.

Why it breaks

File.ReadAllLines is not a reading primitive, it is a materialising one. Internally it opens a StreamReader, appends every ReadLine() result to a List<string>, and returns list.ToArray(). The method only returns once the last line of the file is in memory.

Three things multiply the cost:

UTF-16. .NET strings hold two bytes per character. Your 10 GB of ASCII on disk becomes ~20 GB of character data on the heap.
Per-object overhead. Each line is a separate object: 8 bytes of header, 8 for the method table pointer, 4 for the length, a null terminator, padded to an 8-byte boundary — call it ~24 bytes per line — plus 8 bytes for its slot in the array. At 100-byte lines that's ~107 million objects and roughly 23 GB of heap for a 10 GB file.
The doubling. The backing array of the List<string> grows by repeated doubling, and every array above 85,000 bytes lands on the Large Object Heap, which is not compacted by default. You allocate, abandon and fragment your way up through 100 MB, 200 MB, 400 MB of reference arrays, then ToArray() allocates one more full-size copy alongside the last one.

The GC cannot help. Every one of those strings is reachable from the array you asked for, so nothing is garbage. The allocation that finally fails is arbitrary — the exception surfaces wherever the heap happened to run out.

A detour worth taking: `foreach` does not make it lazy

This is the most common wrong fix:

foreach (string line in File.ReadAllLines(path))   // still OOMs
{
    Process(line);
}

A foreach evaluates its collection expression once, in full, before the first iteration. File.ReadAllLines(path) runs to completion and hands back a finished string[]; only then does enumeration start. The loop shape tells you nothing. The return type does: string[] is a result, IEnumerable<string> is a plan.

The other reach is File.ReadAllText(path).Split('\n'), which is strictly worse. A string is hard-capped at 1,073,741,791 characters — roughly 1.07 billion, no matter how much RAM the box has — so the 10 GB read fails on the string itself, before Split runs. If it did run, Split allocates every substring on top of the original, and on a CRLF file it leaves a stray \r on the end of every line.

And "just give the box more RAM" is not a fix, it is a subscription. It makes your memory ceiling a function of your input size, so you pay more every quarter and still fall over the day someone hands you a bigger file.

The senior version

foreach (string line in File.ReadLines(@"C:\data\events.log"))
{
    if (line.Contains("ERROR"))
        Console.WriteLine(line);
}

File.ReadLines returns a lazy iterator over a StreamReader. Each MoveNext() decodes exactly one line and yields it; the foreach disposes the enumerator — and with it the reader and the file handle — when the loop ends. Peak memory is one line plus the reader's buffers, whether the file is 10 MB or 10 TB. The 10 GB case now runs in a few megabytes.

Because it is a real IEnumerable<string>, LINQ composes over it without buffering:

var firstTen = File.ReadLines(path)
                   .Where(l => l.Contains("ERROR"))
                   .Take(10);

Siblings worth knowing:

File.ReadLines(path, Encoding.UTF8);           // explicit encoding

await foreach (string line in File.ReadLinesAsync(path))   // .NET 7+
    Process(line);

// Full control: bigger buffer, sequential-scan hint to the OS.
using var reader = new StreamReader(
    new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read,
                   bufferSize: 1 << 16, FileOptions.SequentialScan));
while (reader.ReadLine() is { } line)
    Process(line);

When streaming isn't enough

File.ReadLines opens the file when you call it, not on the first MoveNext() — the handle is live before the loop starts. So don't return File.ReadLines(...) out of a method that owns the path and expect the handle to be closed: it stays open until someone finishes enumerating or disposes the enumerator, a sequence nobody ever enumerates holds it until the finalizer runs, and with the default FileShare.Read it blocks writers on Windows for the whole run.

Enumerating twice reads the file twice. That is 10 GB of I/O per pass, and if the file changed in between, the two passes disagree. And the moment you call .OrderBy(...), .ToList() or .GroupBy(...) on the sequence, you have re-materialised the whole thing and bought the original bug back. .Count() is the one that looks like those and isn't: it walks the sequence and keeps nothing, so it costs you a full 10 GB pass in time while memory stays flat.

If you genuinely need random access or several passes, streaming alone won't carry you: build an index of line offsets on one pass, sort externally, or load only the projection you need. Streaming buys you constant memory over a single forward pass — that is the whole contract.

The takeaway

ReadAllLines gives you an array; ReadLines gives you a promise. Only one of them fits in RAM.

I post one of these every day — the same problem solved the way that works and the way that lasts.
One senior tip a week, by email: https://seniorvsjunior.higgsfield.app

How LLMs Generate Text — one token at a time

Toc am — Tue, 28 Jul 2026 11:00:38 +0000

One token at a time — a very well-read autocomplete.

A large language model doesn't plan a whole answer up front. It predicts the next token from everything so far, appends it, and repeats — with attention letting it weigh which earlier words matter most.

How each token appears

Tokenize. Text is split into subword tokens and mapped to numbers.
Embed. Each token becomes a vector encoding meaning and position.
Attention. Every token looks at the others and decides what to focus on.
Predict. The model outputs a probability for every possible next token.
Sample. Temperature and top-p pick one; append it and feed the whole thing back in.

Why it feels coherent

Context window. The model sees thousands of prior tokens at once, keeping track of the thread.

Scale of training. Patterns from vast text let it continue in-style and on-topic.

It's still prediction. No lookup of facts — which is why it can sound confident yet be wrong.

The one-line mental model

Generation is autocomplete with attention: predict the next token, append, repeat.

This is part of LearningTechBasics — one tech idea a day, each with an animated diagram and a 60-second narrated video.

📊 Animated version with the live diagram

Follow @amtocbot · #LearningTechBasics

DEV Community: Toc am

Retry Storms in C#: Exponential Backoff and Jitter with HttpClient

The junior version

Why it breaks

A detour worth taking: a fixed delay is not backoff

The senior version

When backoff isn't enough

The takeaway

Why File.ReadAllLines Throws OutOfMemoryException on a 10 GB File in C#

The junior version

Why it breaks

A detour worth taking: foreach does not make it lazy

The senior version

When streaming isn't enough

The takeaway

How LLMs Generate Text — one token at a time

How each token appears

Why it feels coherent

The one-line mental model

A detour worth taking: `foreach` does not make it lazy