Circuit breakers

The problem it solves

A dependency stops working. The bad case is not that it returns errors — errors are fast, and a fast error is a manageable event. The bad case is that it hangs: connections are accepted, requests are sent, and nothing comes back until the timeout fires three seconds later.

Now count. Your service has 64 workers and receives 400 requests a second. Each one that touches the slow dependency occupies a worker for the full three seconds. Within a fraction of a second every worker is parked, and from that moment your service is down — not degraded, down — including every endpoint that has nothing whatsoever to do with the failing dependency. The health check itself will time out, so the load balancer will pull you out, and you will look like the broken component to everyone downstream.

A circuit breaker exists to prevent that, and it is important to be precise about what it protects: your own threads, not the dependency.

The mechanism

The breaker is a small state machine wrapped around calls to one dependency.

Closed is normal: calls pass through, and outcomes are recorded in a rolling window. If the failure rate in that window exceeds a threshold — and the window has enough traffic to be meaningful, which is what the minimum-throughput setting is for — the breaker trips.

Open is the useful state: every call fails immediately, without touching the dependency. No worker is occupied. The failure takes microseconds instead of seconds. Your other endpoints keep working, your health check passes, and the caller gets a fast, honest error it can act on — fall back to a cached value, degrade the feature, or tell the user.

Half-open is where breakers are usually got wrong. After a cool-down, the breaker must find out whether the dependency has recovered, which requires sending real traffic. The question is how much. Let a small, fixed number of probes through and the test is cheap. Let everything through and you have re-created the original failure — the entire waiting herd hits the still-broken dependency and refills the worker pool in one gulp, which is exactly what the breaker was installed to prevent.

What the simulation shows

The dependency hangs rather than failing, which is the realistic and much worse case. Run with the breaker off and watch the worker-pool panel: within a second of the fault, every worker is held. The outcomes panel shows the whole service failing, not just the calls that needed the dependency. That is the failure this pattern exists for, and it is worth watching once at full width.

Now turn the breaker back on. The pool fills briefly — the breaker needs failures before it can trip, so the first timeout period is always paid — and then the breaker opens and the workers are released all at once. Latency for the affected calls drops from three seconds to effectively zero, because a fast failure is fast. The dependency is still broken; your service is not.

Then set the half-open policy to admit all traffic. Watch the worker-pool panel oscillate: every time the breaker tests recovery, the herd refills the pool, the breaker trips again, and the cycle repeats for as long as the fault lasts. The breaker is present, configured, and providing a fraction of its value. Half-open must be narrow.

The numbers worth carrying

Time to exhaustion = workers / (rps × dependency_share). With 64 workers, 400 requests a second, and every request touching the dependency, that is 0.16 seconds. You do not have minutes to react; you have a fraction of a second, which is why this must be automatic.

The breaker’s own cost is one timeout period per open cycle — you cannot trip on failures you have not yet observed. Shorter timeouts on the dependency reduce that cost directly, which is why timeouts and breakers are complementary rather than alternative.

Threshold, window and minimum throughput are the three settings that matter. A threshold of 50% over a 2-second window with a minimum of 10 requests per second is a reasonable starting point: sensitive enough to trip inside a second of a hard failure, insensitive enough that three failures on a quiet endpoint do not trip it.

Where it breaks down

A breaker does not fix the dependency. It converts a hang into an error. If the calling code has no meaningful fallback, the user still sees a failure — a faster one. That is genuine progress (fast failures do not cascade) but it is not a working feature, and the fallback is the part of the design that actually preserves the user experience.

Shared breakers hide per-instance reality. If a breaker’s state is per-process, fifty instances each learn separately, which costs fifty timeout periods of pool exhaustion but avoids one instance’s bad luck tripping the fleet. If state is shared, one flaky instance can open the circuit for everyone. Per-process with a short window is the usual and better default.

Granularity. One breaker per dependency is standard; one per endpoint of that dependency is often better, because a slow /search should not open the circuit for a healthy /health.

Breakers plus retries interact badly. Retries inside a closed breaker accelerate tripping (which is fine) and retries against an open breaker are pointless work. Configure them as one policy, not two — see retry storms.

What people get wrong

“The breaker protects the dependency.” It protects you. Load shedding at the dependency protects the dependency. Say this in an interview and you will be one of the few who does.

“Trip on error count.” Trip on error rate, with a minimum-throughput floor. A count threshold trips on quiet endpoints and never trips on busy ones.

“Half-open lets traffic through to test.” How much? The oscillation scenario above is the reason this is the interesting question.

“We have timeouts, so we do not need a breaker.” Timeouts bound each call. The breaker bounds the number of calls in flight into a known-bad dependency. With a 3-second timeout and 400 requests a second, timeouts alone still let 1,200 requests into the hole before the first one returns.

In production

Netflix Hystrix popularised the pattern and is now in maintenance; Resilience4j is the standard JVM successor, with a rolling window, a permitted-calls-in-half-open setting, and a slow-call threshold — that last one matters, because a dependency that returns successfully but slowly should trip a breaker too, and error-rate-only breakers miss it. Polly does the same for .NET. Envoy implements the pattern as outlier detection plus circuit-breaker limits on concurrent requests and pending requests, which is the same idea expressed as bounds rather than as a state machine.

The complementary pattern is on the bulkhead page: a breaker limits how long you are exposed, a bulkhead limits how much of you is exposed. Real systems use both, and the combination is what makes a slow dependency a non-event.

The follow-up questions

“A dependency starts taking three seconds. What happens to your service?” — The exhaustion arithmetic. Then the breaker, then the fallback.

“How do you know when to close it again?” — Half-open with a small number of probes, and say what happens if a probe fails.

“What is your fallback?” — Cached value, default, degraded feature, or an honest error. “There is no fallback” is an acceptable answer if you say what the user sees.

“Why not just retry?” — Because the dependency is not transiently unavailable, it is saturated, and retries add load. The breaker exists to stop sending.