Pools and parallelism — every question, written out
Sizing a thread pool, the deadlock it invites, the ceiling Amdahl puts on all of it, and the cache line two unrelated counters accidentally share.
Every thread in a pool is waiting for a task submitted to that same pool. What is this?
Code diagnosis
Deadlock — the pool is the resource, and the dependency runs back onto itself.
The only threads that could run the queued work are the ones waiting for it. No lock appears anywhere, and the four Coffman conditions are all satisfied with "a worker" as the held resource.
See every interleaving — 6 of 2,500 interleavings deadlock; the rest complete perfectly.
Does doubling the pool size fix a pool-waiting-on-itself deadlock?
Trade-off & selection
No — it survives more callers and fails once enough of them nest at the same time.
Sizing changes how much traffic closes the cycle and moves the failure to a load you have not tested. The structural fix is a separate pool for the inner work, which removes the cycle for any size.
See every interleaving — Four workers, four callers: 38 of 2,500 deadlock.
How many threads for a CPU-bound pool?
Complexity derivation
About one per core — more only adds context switches.
CPU-bound work keeps its thread busy, so a thread beyond the core count cannot add throughput and does add switching. The interesting sizing question is the I/O-bound one.
How many threads for an I/O-bound pool?
Complexity derivation
Roughly cores × (1 + wait ÷ compute) — for 90% waiting, about ten times the core count.
Each thread is only using a core for the compute fraction, so you need more of them to keep the cores busy. If that number is large, async I/O is usually a better answer than a bigger pool.
A program is 95% parallel. What is the maximum speedup?
Complexity derivation
20× — one divided by the serial fraction, and the core count does not appear in it.
Speedup is 1 / (s + (1−s)/n), which approaches 1/s as n grows. At 5% serial that is 20×, and 64 cores already deliver 15.4× of it.
See every interleaving — Drag the serial fraction and watch the ceiling move.
You doubled the cores and gained 10%. What should you conclude?
Code diagnosis
You are near the ceiling; estimate the serial fraction from the two data points.
Two measurements are enough to solve for s, and the fitted value includes everything you forgot — the allocator, the GC, the shared queue, the memory bus. Then decide whether reducing it is worth more than adding hardware.
Why is reducing the serial fraction the highest-leverage optimisation available?
Trade-off & selection
The ceiling is one over the serial fraction, so halving it doubles what the hardware can ever deliver.
Going from 10% to 5% serial takes the ceiling from 10× to 20×. No amount of making the parallel part faster does that, which is why profiling the sequential section usually wins.
Two threads increment two separate counters and it is slower than one thread. Why?
Code diagnosis
The counters share a cache line, so every write invalidates the other core’s copy.
Coherence works in 64-byte lines, not variables. Two independent counters in one line serialise on a resource neither thread was told about, at roughly 80ns per transfer against 1ns for an L1 hit.
See every interleaving — Two threads on a shared line are slower than one.
How do you fix false sharing, and what does it cost?
Trade-off & selection
Pad each variable onto its own cache line — 64 bytes each instead of 8.
Use the platform mechanism — `@Contended`, `alignas(64)`, `#[repr(align(64))]` — rather than dummy fields, because the compiler and allocator decide layout. The cost is memory, and at eight threads the payoff is orders of magnitude.
Which of these most commonly causes false sharing by accident?
Edge case reasoning
A `long[]` of per-thread counters indexed by thread id.
Eight 64-bit counters fit in one line, so eight threads all contend. It looks like the obvious way to write per-thread statistics, which is why it is the classic instance.
Why does one shared work queue stop scaling?
Comparison
Every task costs one contended operation on one structure, so the queue caps the pool.
The queue becomes the serial fraction. Per-worker deques make the common case local and uncontended, and charge coordination only to a worker that has run dry.
See every interleaving — The shared-queue line goes flat; the stealing line keeps climbing.
Why does a thief take from the opposite end of the deque to the owner?
Invariant identification
It avoids colliding with the owner and takes the oldest, usually largest, task.
Owner and thief work at opposite ends, so they rarely touch the same slot. Taking the oldest transfers a large subtree in one steal, while the owner keeps working on the newest task whose data is still warm.
When does work stealing fail to help?
Edge case reasoning
When one task is long and indivisible — nothing can be stolen from a running task.
Stealing balances queues, not tasks. If one task takes ten seconds, the granularity of your decomposition is the floor, and no scheduler can improve on it.
Why should blocking work stay off a fork-join pool?
Trade-off & selection
A blocked worker is neither working nor available to steal, so its queue stalls.
The pool is sized on the assumption that a worker is either working or stealing. A blocked one is doing neither, and enough of them starve the pool — which is the thread-pool deadlock in a different guise.
Why is using the default common pool for blocking work particularly dangerous?
Code diagnosis
It is shared process-wide, so unrelated code can exhaust each other’s capacity.
Parallel streams, `CompletableFuture` defaults and library internals all land on it, so two components that never heard of each other can deadlock or starve one another. Pass an explicit executor for anything that blocks.
What does giving each stage of a pipeline its own pool buy you?
Explain it plainly
It is the bulkhead pattern. Each stage has bounded, private capacity, so a slow or stuck stage degrades itself rather than everything. It also makes the pool-waiting-on-itself deadlock structurally impossible, because the dependency now runs from one resource to another rather than back onto the same one. The cost is lower utilisation — idle capacity in one pool cannot be lent to another.
Isolation: a stage that saturates or blocks cannot consume the capacity another stage needs. It also removes the cycle that causes a pool to wait on itself, for any pool size.