Hello, I'm Maneshwar, and I'm building LiveReview — a blast-radius aware AI code review built for your business-critical systems. Star us to help devs discover the project, give it a try, and share your feedback to help improve the product.
Picture an e-commerce store.
Checkout calls the payment service on every order. One call, one arrow on the architecture diagram, the least interesting line in the codebase.
Now break it in two different ways and watch which one hurts.
Take the payment service completely down. Every call to it is refused in about three milliseconds. Checkout returns a clean failure for that order, and every other page on the site keeps serving.
Now leave it running and answering every call correctly, only slower. 40 milliseconds becomes 30 seconds.
Checkout has nothing to react to, because from its point of view nothing failed.
On paper the second case is the healthier one. Every call is answered, correctly, with the right data.
It is also the one that takes the whole site down.
The reason is not subtle once you see it. Your service holds a worker for the entire length of every call it makes.
When the dependency is down, that worker comes back in milliseconds and moves straight on to the next request.
A slow dependency keeps the same worker for 30 seconds, and you only have so many of them.
So this whole article is about a pattern that protects you, not the service you are calling.
It works by stopping your own service from waiting.
The bill for a slow call, itemised
Every call your service makes takes a small pile of things with it while it runs.
A thread. A database connection. Some memory. A port.
It holds all of that until the call finishes, one way or the other. A slow call just holds it for longer.
You have a fixed number of each, and nothing new can start once they are all in use.
That is why the product page died in the opening scene, even though the product page never calls payment. The blocked requests were sitting on threads and connections that the rest of your system shares.
One slow dependency can saturate every resource on every one of your services in seconds.
And this is not a rare situation, because dependencies multiply. Say your service calls 30 of them, and each one is up 99.99% of the time. You need all 30.
Multiply those together and you land at about 99.7%.
That is 3 million failed requests in every billion, and over two hours of downtime a month, with all 30 dependencies behaving exactly as advertised.
The fix is to fail fast. Turn down a call you already expect to fail, instead of waiting for it to time out.
But nothing frees a slot until a call ends. So something has to decide when a call has taken too long.
Link one: the timeout
That decision is the timeout, and it is the one rule here with no exception.
Set a timeout on every remote call. Honestly, on any call that leaves your process, even one going to something on the same machine.
And there are two of them, not one. The connection timeout covers getting the socket open. The request timeout covers waiting for the answer.
Picking the number is not a vibe. AWS's Builders' Library has the method: decide what rate of false timeouts you can live with, meaning calls you cut off that would have succeeded, then set the timeout at the matching latency percentile of the service you are calling.
Their worked example accepts one call in a thousand being cut off wrongly, so they use the P99.9 latency.
Set it too high and it barely counts as a timeout at all. You are still holding the thread, the connection and the memory for the entire time you wait for a number in a config file to tell you that you are protected.
The percentile method has two places it breaks down.
It struggles with clients that carry real network latency, like calls coming in over the internet.
And it struggles with services where P99.9 sits close to P50, because then the timeout lands almost on top of the normal case.
Both of those need padding on top of the measured number.
There is one case a timeout alone does not cover, and it is the sneaky one.
A call that is slow and still succeeds never counts as a failure. It never moves a failure counter anywhere, so nothing downstream ever learns that the dependency has gone bad.
That is why a breaker needs a separate slow call rule. Resilience4j ships one: a slowCallDurationThreshold and a slowCallRateThreshold, where a call only counts toward opening the breaker once it crosses that duration.
Link two: retries, the powerful medicine
A timeout turns a hang into an error. And the first thing anyone does with an error is try it again.
That second attempt costs the server another slice of its time, and it buys that time for exactly one client.
AWS has a blunt word for this: retries are selfish.
That trade is a good one when the failure was a one-off, because the server had time to spare anyway.
It is a terrible one when the failure came from overload in the first place. You are adding load to something that is already buried, which makes it worse and holds it down long after the original problem is gone.
AWS calls retries a powerful medicine, which is a good way to hold it in your head. The right dose helps. The same substance in a larger dose does the real damage.
Backing off between attempts is the first fix, and on its own it does not go far enough.
All the calls that failed, failed at the same moment, because the dependency went bad at one instant. Back every one of them off by the same amount and they come back together too, overloading the thing a second time.
What breaks the pattern is jitter, a random delay on top of the backoff, so the attempts spread out instead of arriving in a block.
Notice what has happened to the original cause here.
Whatever knocked the dependency over is gone, and the system is still down, held there by its own traffic.
Researchers call this a metastable failure. A trigger pushes the system into a bad state, the bad state feeds itself, and removing the trigger changes nothing.
Retries make you more vulnerable to it, because they turn a small outage into an internal storm.
The part worth remembering is what this looks like from inside your own monitoring.
Clients give up waiting before the answer arrives, then send the request again.
Meanwhile your system is still finishing old calls that nobody is listening for any more.
Your throughput graph looks excellent, because work genuinely is being completed.
The number that matters is goodput, the work that reaches a caller who is still waiting, and that has gone to zero.
Link three: the breaker itself
Backoff and jitter slow the storm down, and they are worth having.
But nothing we have put on that wire so far actually stops the call.
That is the breaker's job, and the pattern is not new. It comes from Michael Nygard's Release It!, popularised precisely to prevent the cascade we have spent three sections building.
Everyone can draw the three states. Far fewer can say what number theirs trips at, or whether it can trip at all.
stateDiagram-v2
[*] --> Closed
Closed --> Open: failure rate crosses the threshold<br/>over a real number of calls
Open --> HalfOpen: wait duration has elapsed<br/>AND a call actually arrives
HalfOpen --> Closed: the permitted probe calls succeed
HalfOpen --> Open: ONE probe fails, timer restarts
Closed: Closed<br/>calls pass through, failures counted
Open: Open<br/>call not attempted, fails in microseconds
HalfOpen: Half-Open<br/>a few real calls allowed through as probesClosed is normal, and where the breaker spends almost all of its life. Calls pass straight through, failures are counted as they happen.
Open means it has tripped. The call is not attempted at all, it fails on the spot, and your application gets an exception back in microseconds instead of waiting 30 seconds. That is the fail fast idea from earlier, turned into something you can point at.
Half-open is the trial run, and it is how the breaker finds its way back. Once the reset timeout expires, it lets a limited number of real requests through to see whether the problem is fixed.
People mix this up with the retry pattern constantly, and the two are doing opposite jobs.
A retry assumes the call will succeed eventually, so it keeps trying.
A breaker assumes it will not, and stops you making a call that is likely to fail.
Three states is the teaching model. Resilience4j actually exposes six, adding METRICS_ONLY, DISABLED and FORCED_OPEN, which a person puts it into rather than a failure rate.
What actually pulls the lever
Ask most engineers what trips a breaker and they will say a count. Something like five failures in a row.
The libraries you are most likely to have installed do not work that way at all.
They measure a failure rate, and when that rate reaches your threshold, the breaker opens.
That rate is measured over a sliding window, and the window comes in two shapes. A count-based window covers the last N calls, whatever length of time those took. A time-based window covers the calls from the last N seconds, however many arrived.
A rate over a handful of calls is meaningless, because two failures out of three is 67% and also nothing.
So the rate is gated. It can only be calculated once a minimum number of calls has been recorded.
Hystrix called that gate the request volume threshold: the minimum number of requests that have to arrive in the rolling window before the circuit is allowed to trip at all.
Set it to 20. Nineteen requests come in over 10 seconds and every one of them fails. The circuit does not trip, because 19 is short of the 20 it was told to wait for.
Counting breakers do still exist. gobreaker trips by default once consecutive failures go above five. And a counter in the closed state does not pile up forever either. Azure's guidance is that the closed-state failure count is time based and resets at intervals, so the odd failure scattered across a quiet afternoon never adds up into a trip.
So four separate settings decide whether your breaker ever opens: the threshold, the shape of the window, the size of the window, and the minimum call volume.
Every one of those is a value somebody chose for you.
The defaults do not agree with each other
Start with the one most tutorials still reach for.
Hystrix is no longer in active development. It is in maintenance mode, which in Netflix's own words means they will not review issues, merge pull requests or release new versions. The last release was 1.5.18.
Netflix's own recommendation for new work is to use an active project instead, and they name Resilience4j. Internally they moved toward adaptive implementations that react to real-time performance rather than to thresholds you pick in advance.
Here are four sets of stock defaults, read the same way each time. What trips it, how long it stays open, how many calls it lets back through.
The disagreement is material rather than cosmetic.
The trip threshold runs from 10% in Polly, up to 50% in Hystrix and Resilience4j, over to six consecutive failures in gobreaker.
Time in the open state runs from 5 seconds at one end to 60 at the other.
And the half-open probe count runs from 1 to 10, which is a tenfold difference in how hard each library leans on a service that is still recovering.
Two of those defaults quietly switch the breaker off, and this is the part nobody checks.
Resilience4j needs 100 recorded calls before it evaluates a rate at all, measured over a window of the last 100 calls. Put that on a low traffic internal endpoint and it may never accumulate enough calls to evaluate anything. That breaker simply never opens.
Resilience4j also does not move from open to half-open on a timer. The transition happens when a call arrives after the wait duration has elapsed, not at the moment it elapses. On an endpoint that has gone quiet, the breaker sits open until something finally asks.
Reading your own config out loud is a genuinely useful five minutes:
resilience4j:
circuitbreaker:
instances:
payment:
slidingWindowType: COUNT_BASED
slidingWindowSize: 100 # the last 100 calls
minimumNumberOfCalls: 100 # <- can your endpoint even reach this?
failureRateThreshold: 50 # percent, not a count
slowCallDurationThreshold: 2s # slow successes count as failures
slowCallRateThreshold: 100 # percent
waitDurationInOpenState: 60s
permittedNumberOfCallsInHalfOpenState: 10 # <- the recovery dial
Half-open is where systems break a second time
Half-open exists for one reason. A service that has just come back is not at full strength, and if you hand it everything at once it falls over again.
So the breaker does not go straight from open back to closed. It opens a gate only a few calls fit through, and you choose how many.
Resilience4j describes those calls as permitted calls, and their job is to find out whether the backend is still unavailable or has become available again.
Watch what happens if one of them comes back red.
If any probe fails, the breaker assumes the fault is still there, drops straight back to open, and restarts its timer.
A single bad probe undoes the whole cooldown and puts you back at the start.
There is a way to get this wrong in the other direction too. Set the cooldown too short and the breaker keeps trying before anything has changed, so it swings between open and half-open over and over. That flapping shows up in your application as worse response times.
Put those together and you can see why half-open is where systems break twice. The first incident knocked the dependency down. The second one is your breaker's doing, because a dependency that is only halfway back got handed full traffic before it could carry it.
Link four: what do you actually return?
Every call the breaker turns away has to return something to whoever asked.
That is the fourth link, and this one is your job rather than the breaker's. The breaker only decides whether to make the call. What comes back when it says no is application logic you write.
Martin Fowler's two examples are the ones worth keeping in your head. A credit card authorization does not have to happen right now, so it can go onto a queue and be dealt with later. And missing data can often be covered by showing something slightly out of date.
Azure lists four realistic options while the breaker is open:
- Degrade the feature, so the page still loads without that section.
- Call something else, an alternative operation that does not depend on the broken thing.
- Return a default value that means something to your application.
- Report the exception to the user, which is what you get if you write no fallback at all, and is a real option rather than a failure to decide.
Think about what actually changed when you added the breaker. Your caller was going to get an error either way. The breaker changed which error it is, and how long the caller waited to receive it.
Amazon's position runs the other way, and it is worth hearing.
They avoid fallbacks for two reasons. The effectiveness of a fallback is hard to prove and hard to test. And a fallback is a mode a system only enters at the most chaotic possible moment, when things are already breaking, and switching modes right then increases the chaos.
The mechanism behind that objection is staleness in the code path itself.
Think about a fallback you wrote two years ago that almost never triggers. If there is a bug in it, or a side effect that makes the whole problem worse, nobody has looked since, and the people who wrote it have long forgotten how it worked.
Their preference turns into something you can act on this week: favour code paths that run in production continuously over ones that run rarely. If a fallback really is essential, exercise it in production as often as you can, so it behaves as predictably as the primary path.
The uncomfortable part: AWS is not sold on breakers
If fallback code is that risky, it is fair to ask whether the breaker in front of it is worth having at all.
AWS puts the objection in writing, in the Builders' Library, word for word:
Circuit breakers, where calls to a downstream service are stopped entirely when an error threshold is exceeded, are widely promoted to solve this problem. Unfortunately, circuit breakers introduce modal behavior into systems that can be difficult to test, and can introduce significant additional time to recovery.
There are two separate complaints packed in there.
The first is modal behavior, and it is everything we just spent this whole article building, described as a cost instead of a feature. Closed, open and half-open are three modes. Two of them are states your system is almost never in, which makes them exactly the states hardest to test.
The second is added recovery time, which is the cooldown seen from the other side. The dependency can be perfectly healthy again while your breaker is still refusing to call it, because its timer has not finished running.
What AWS ships instead, inside its own SDK, is a retry token bucket. Retries cost tokens, the bucket refills over time, and while there are tokens in it everything retries freely. Once the tokens run out, retries do not stop, they continue at a fixed rate. That behaviour went into the AWS SDK back in 2016.
The structural difference is that the bucket is continuous where the breaker is modal.
No tripped state, no cooldown, no trial run, so no rarely exercised mode for you to test. Retry capacity gets thinner as the bucket empties instead of switching off.
The bucket also never stops the original call. It limits the retries only, so the caller keeps giving the dependency a chance on every request rather than blocking a whole dependency at once.
flowchart TD
A[Your call to a dependency<br/>is failing] --> B{What are you<br/>actually protecting<br/>against?}
B -->|My own retries are<br/>amplifying the outage| C[Retry token bucket]
B -->|My workers pile up on a<br/>dependency that stopped<br/>being useful| D[Circuit breaker]
C --> E[Continuous, no modes,<br/>easy to test<br/>never blocks the first call]
D --> F{Can it actually trip<br/>on your traffic volume?}
F -->|No, too few calls<br/>for minimumNumberOfCalls| G[You have a decoration,<br/>not a breaker]
F -->|Yes| H[You now own a mode<br/>you must test and alarm on]
classDef decision fill:#f4d35e,stroke:#b8991f,color:#1a1a1a
classDef start fill:#e9ecef,stroke:#6c757d,color:#1a1a1a
classDef good fill:#5ee6c8,stroke:#1f9c86,color:#1a1a1a
classDef warn fill:#ff9a5c,stroke:#c25b23,color:#1a1a1a
class B,F decision
class A start
class C,D,E good
class G,H warnSo the rule comes down to what you are actually protecting against.
If the worry is amplification from your own retries, the bucket handles that continuously and is easier to test.
If the worry is your workers piling up against a dependency that has stopped being useful, the breaker is the thing that stops the waiting.
Choose it knowing you have added a mode you now have to test and alarm on.
Failures neither of them should ever act on
Your own bugs. A 400 is the clearest case. That is a broken request, and it will fail identically on every host you send it to. Count those and you can trip a breaker against a dependency that is completely healthy. A good breaker sorts failures by type, and can require more timeouts before tripping than outright "service unavailable" responses, because those two mean different things.
Writes with side effects. A write is only safe to retry if it is idempotent, meaning the side effect happens once no matter how many times the same request arrives. Read-only APIs usually get that for free. Resource creation APIs often do not, and that is what makes retries and fallbacks dangerous on a write path with no idempotency key.
Scoping, which is the one people get wrong quietly. One breaker per resource type stops working the moment that resource has independent providers behind it. Take a sharded data store. One shard can be completely fine while another has a temporary problem, and if you merge their errors into one breaker, it blocks calls to healthy shards while still letting calls through to the failing one.
Azure also lists five cases where a breaker only adds overhead: local in-memory resources with no network to protect, anything an ordinary retry already handles, cases where waiting out a reset introduces a delay you cannot accept, event-driven systems where failed work already lands in a dead letter queue, and systems where the platform underneath already handles recovery.
That last one deserves its own section.
Something may already be doing this for you
Everything so far has lived inside your own process, and that is not the only place this happens.
A service mesh can run a breaker for you, as a sidecar or as a capability of the platform underneath.
What it does is not quite the same thing though. Your library breaker stops calling a dependency. Envoy ejects a host, which means it pulls one bad instance out of the load balancing pool and lets the other four keep serving.
It also arrives with its own defaults, and they look nothing like the library numbers.
Envoy ejects after five consecutive 5xx responses, sweeping and re-evaluating every 10 seconds. It ejects a host for a base of 30 seconds, and it will never eject more than 10% of the fleet.
That 30 seconds is not fixed, because ejection backs itself off. The duration is the base time multiplied by how many times that host has been ejected in a row, so 30 seconds becomes 60, then 90.
Coming back is not only a matter of waiting either. A successful active health check un-ejects the host and clears its outlier counters, and clearing those resets the backoff too.
The setting that matters most mid-incident is that maximum ejection percentage. Once enough of the fleet is already out, the mesh stops ejecting, which is why you can watch a host that is clearly failing stay in the pool.
So before you tune anything in your own process, find out which of these layers is already acting on your traffic.
Possibly more than one. And all of them can be doing their job without telling you.
The part everyone skips: alarm on the state change
Fowler's requirement is straightforward. Any change in breaker state gets logged, and the breaker reveals its state so something can monitor it.
The state change is the thing you alarm on, not the errors underneath it.
Skip that wire and an open breaker becomes a silent partial outage.
Think about what your dashboards see. Every call the breaker turns away comes back fast and looks like a clean response. Your latency graph improves. Your error rate stays flat.
A good fallback does exactly the same thing to your monitoring.
The device needs a handle a person can reach, too. Operations staff should be able to trip or reset a breaker by hand, so you can force one closed when you know the dependency is back, or force it open when you know it is down.
And the person on call needs to tell those two apart. Polly makes that visible by throwing a different exception when a breaker was deliberately isolated, so a human decision never looks like the system tripping on its own.
The whole chain, one question per link
Timeout. How long are you willing to wait before you call it a failure?
Retry. How many times do you try again, and with how much jitter so your retries do not all arrive together?
Breaker, which is really three questions, because that is what a breaker is. What share of a real number of calls has to fail before you stop trying? How long do you stay stopped? How many probes do you let through on the way back?
Fallback. What do you actually return while you are stopped?
Two of those numbers carry more weight than the rest, and both are usually sitting at whatever the library shipped.
The minimum call volume decides whether your failure rate means anything at all.
The half-open probe count decides whether your recovery survives contact with real traffic.
The most useful thing you can do after reading this is open your own config and read those two lines. Then check that something alarms when the breaker changes state.
Set your timeout from a percentile rather than a round number. Jitter your retries so they do not all arrive together. Trip on a rate measured over a real number of calls. And decide what you return before the breaker ever opens.
Those four choices are the whole chain, and every number on it is yours rather than the library's.
Your team's attention is limited, and the deluge of AI-generated code is making it harder to keep production secure and reliable without slowing you down.
I'm building LiveReview, a blast-radius aware AI code review built for your business-critical systems.
Instead of presenting every diff with equal emphasis, LiveReview scores each change by blast radius — how far its impact reaches through your call graph — so you can focus attention where it actually matters.
Spend code review effort where business risk is highest — not spread evenly across every diff.
⭐ Star it on GitHub:
HexmosTech
/
LiveReview
Blast-Radius Aware AI Code Review for Business-Critical Systems
LiveReview: Blast-Radius Aware AI Code Review for Business-Critical Systems
LiveReview is an AI code reviewer that scores every hunk of a diff by blast radius: how far a change reaches through your call graph, how much persistent state it touches, and how well-tested it is. A 3-line change to a shared auth check can outrank a 300-line UI tweak. Your team's attention goes to the highest-risk code first, not spread evenly across every diff.
blast-radius-demo.mp4
LiveReview's Blast Radius & Review Priority scoring, live in the diff viewer.
The exact math, not a black box
Visualize blast radius at a glance
Every factor that feeds the score
Here's the goal:
- A 3-line fix in a function used by 40 other files, that also writes to a database, should score high.
- A 300-line UI change in one file, fully covered by…
Click below to try LiveReview with your codebase:
Top comments (0)