Bulkheads

The problem it solves

The name comes from shipbuilding. A hull divided into sealed compartments can take a breach in one and stay afloat; a hull that is one open space sinks from a single hole. The engineering translation is that shared, unpartitioned resources propagate failures perfectly — and the most commonly shared resource in a service is its thread pool or its connection pool.

Here is the situation the simulation models. Your service has three endpoints. One of them calls a dependency that becomes slow. That endpoint is 20% of your traffic. The other two are fine — one of them touches nothing but a local cache and could serve from memory forever. All three share a pool of 60 threads.

Within seconds, all three are failing. The endpoint that reads a local cache in five milliseconds is timing out, because there is no thread available to run it on. A fifth of your traffic has taken down all of it.

The mechanism

Whichever resource is shared and finite becomes the coupling. Slow requests hold threads longer, so their share of the pool grows without their share of traffic growing at all. Little’s Law makes the arithmetic exact: threads held equals arrival rate times holding time. At 120 requests a second (20% of 600) and a 2-second holding time, the slow endpoint needs 240 threads. There are 60. It takes all of them, permanently, and everything else starves.

Notice that the slow endpoint’s traffic never increased. Only its duration did, and duration is a multiplier on occupancy.

A bulkhead partitions the resource. Give each endpoint its own slice — its own pool, or a semaphore capping its concurrency within a shared pool — and the arithmetic is confined. The slow endpoint exhausts its own 20 threads and then rejects further requests. The other two endpoints have their slices untouched and never notice.

The essential and slightly uncomfortable point: the slow endpoint still fails. A bulkhead does not repair anything. It converts “everything is down” into “one feature is down”, which is a choice about which thing breaks, made in advance, deliberately.

What the simulation shows

Run the default with bulkheads off. The threads-held panel makes the mechanism visible: at the moment of the fault, the slow endpoint’s band grows until it fills the chart, and the other two are squeezed to nothing. The dedicated panel for the endpoint that has nothing to do with any of this — the one reading a local cache — shows it failing along with everything else. That panel is the whole argument.

Now turn bulkheads on. Same fault, same duration, same seed. The slow endpoint’s band is capped at its slice. The local-cache endpoint runs at full success throughout. The blast radius has gone from “the service” to “one feature”, and nothing about the dependency changed.

Then push it: drop the slow endpoint to 5% of traffic with bulkheads off. It still takes everything, just a little more slowly — because 5% of 600 requests a second held for two seconds is still 60 threads, which is the entire pool. Small traffic share is no protection. Holding time is what matters.

The numbers worth carrying

Threads held = rps × holding time. That single expression explains every incident on this page, and it is the reason a 20% endpoint can consume 100% of a pool.

Sizing a bulkhead: give each partition enough concurrency to serve its normal load at normal latency with headroom — rps × normal_latency × 1.5 or so — and no more. Over-provisioning a partition defeats the isolation, since the whole point is that a misbehaving partition hits its ceiling quickly. Under-provisioning rejects healthy traffic during ordinary bursts.

Leave headroom unallocated rather than dividing 100% of the pool. If the partitions sum to exactly the pool size, a burst on one healthy endpoint has nowhere to go even though the pool is half idle.

Where it breaks down

Isolation costs utilisation. Partitioned capacity is less efficient than pooled capacity — this is the utilisation curve in reverse. Sixteen threads reserved for an endpoint sit idle when that endpoint is quiet, whereas a shared pool would have lent them out. You are paying for the isolation in hardware, and that is the trade.

Too many partitions. Partition by dependency or by criticality tier, not by endpoint, or you end up with forty pools of three threads each, every one of which is too small to absorb a normal burst.

The bulkhead has to be where the resource is. A semaphore in your application does nothing about a shared database connection pool, a shared file descriptor limit, or a shared event loop. Find the actual constrained resource. In Node.js or any single-threaded event-loop runtime, “threads” is the wrong unit entirely and the bulkhead must cap in-flight operations.

Memory is the pool you forget. Requests parked on a slow dependency hold their buffers. Concurrency limits bound that too, which is a second reason to have them.

What people get wrong

“Timeouts are enough.” A 2-second timeout with 120 requests a second arriving still means 240 threads’ worth of demand against 60. Timeouts bound how long each failure lasts; bulkheads bound how many can be in flight. Both are needed, and the circuit breaker is the third member of the set.

“It is only 5% of traffic.” Traffic share is irrelevant; occupancy is rate times duration. Show the arithmetic.

“Bulkheads fix the slow endpoint.” They do not, and saying so plainly is the strongest version of this answer: you have chosen which thing breaks.

“We use async, so we do not block threads.” Then your finite resource is in-flight operations, sockets, or memory, and it will exhaust in exactly the same way. Async raises the ceiling; it does not remove it.

In production

Resilience4j’s Bulkhead (semaphore) and ThreadPoolBulkhead implement both variants directly. Envoy’s circuit-breaker settings are per-cluster limits on connections, pending requests and concurrent retries — bulkheads by another name, at the right place in the stack. Kubernetes resource requests and limits are bulkheads at the container level, and a pod without limits is a shared thread pool at cluster scale.

The most valuable production version is often the crudest: separate connection pools per downstream dependency, and separate deployments for critical and non-critical traffic. Running the checkout path on its own fleet means a recommendation-service outage cannot touch it, and no amount of in-process configuration gives you an isolation guarantee as strong as a different set of machines.

The follow-up questions

“One dependency gets slow. Which of your endpoints are affected?” — With a shared pool, all of them. Show the occupancy arithmetic and then propose the partition.

“How do you size each partition?” — Little’s Law at normal load plus headroom, and less than the total so healthy bursts have somewhere to go.

“What is the downside?” — Lower utilisation. Isolated capacity is idle capacity when the partition is quiet.

“You have bulkheads and a breaker. What does each do?” — The bulkhead bounds how much of you is exposed; the breaker bounds how long. Different axes, both needed.