Little's Law

The problem it solves

Almost every capacity question in an interview, and most of them in production, reduce to a relationship between three numbers: how fast work arrives, how long each piece stays, and how many pieces are in the system at once. Little’s Law says those three numbers are not independent. Any two determine the third:

L = λW — items in the system equals arrival rate times time in the system.

That is the entire law. Its value is that it is almost impossible to misuse, because it assumes almost nothing: no distribution, no queue discipline, no independence, no steady state beyond the system not growing without bound over the interval you measure. It holds for a bank branch, a thread pool, a Kafka partition, a warehouse, and the whole internet. When someone gives you two of the numbers, you have the third, and you can check their claim against it in your head.

The mechanism

The proof is a picture rather than an algebra. Draw a horizontal bar for each item, starting when it arrives and ending when it leaves. Over an interval of length T, the total area of those bars is the sum of every item’s residence time. Now count that area two ways. Slice it vertically and you are counting how many items are present at each instant, so the area is L × T. Slice it horizontally and you are counting each item’s stay once, so the area is (number of arrivals) × W, which is λT × W. The same area computed two ways gives LT = λTW, and L = λW.

Nothing in that argument mentions how arrivals are distributed or how service is scheduled. That is why the law survives conditions that break every closed-form queueing result: bursty arrivals, heavy-tailed service times, priority scheduling, preemption. It is a conservation identity, not a model.

The useful trick is that “the system” is whatever box you draw. Draw it around one thread pool and L is threads busy. Draw it around the whole request path and L is concurrent requests. Draw it around a queue only, excluding service, and W is queueing delay alone. Each box gives a different, valid equation, and the art is choosing the box where two of the numbers are already instrumented.

What the simulation shows

Two lines are drawn: the number of items in the system counted directly, and the same number computed as λ × W from arrivals and residence times measured independently at the edges. They sit on top of each other.

That is the whole demonstration, and it is worth making it try to fail. Make arrivals violently bursty and the lines still coincide. Widen the service-time spread until the distribution is nothing like the mean and they still coincide. Change the server count, change the rate — the two curves track. Every other formula in queueing theory would have broken by now; this one does not, because it is not a formula about queues, it is a fact about areas.

The second use of the simulation is the practical one. Turn any single dial and watch the other two numbers move to satisfy the identity. If you double λ and W stays put, L doubles. If W rises because the servers are saturating, L rises with it even though arrivals never changed — which is precisely how a thread pool fills up during an incident that started somewhere else.

The numbers worth carrying

The three forms are worth having ready in each direction:

• Threads needed = throughput × latency. 1,000 requests per second at 200 ms each needs 200 concurrent workers. Not 1,000; not 20.
• Latency implied = concurrency ÷ throughput. A pool of 50 threads serving 500 requests a second is spending 100 ms per request, and you never had to instrument latency to know that.
• Throughput implied = concurrency ÷ latency. A 200-connection pool against a database averaging 10 ms tops out at 20,000 queries a second, and no amount of application-side tuning moves that ceiling.

Every one of those is a sanity check you can run on someone else’s design in five seconds, and interviewers use it on you. When a candidate says “we will run 20 instances with 100 threads each” and separately says “p50 is 300 ms”, Little’s Law says the design is claiming 6,600 requests a second. If the requirement was 50,000, the design is off by an order of magnitude and the arithmetic said so before any code was written.

Where it breaks down

It does not break, but it can be misapplied, in three ways.

Not steady state. The law holds over intervals where the system is not systematically accumulating. During a spike, arrivals exceed departures and L grows; the identity still holds if you measure W for the items that actually completed, but a naive computation over a window that begins and ends mid-transient will be off. In practice: measure over a long enough window, or accept the error and know its sign.

Wrong box. Half of misuse is computing λ at one boundary and W at another. If W includes queueing time but λ counts only admitted requests after a limiter rejected some, the numbers refer to different systems.

Averages are not percentiles. The law is about means. It tells you nothing about p99, and a design sized from mean concurrency will be under-provisioned for the tail. Little’s Law tells you the pool needs 200 threads; the utilisation curve tells you why 200 is not enough.

What people get wrong

“L = λW only works for M/M/1.” No. That is the mistake this page exists to correct, and the simulation exists to make un-arguable. Bursty, heavy-tailed, multi-server, priority-scheduled — the identity holds.

“It gives me my p99.” It gives you a mean. The variance is somewhere else entirely.

“Add threads to reduce latency.” Adding threads increases L, and if the bottleneck is downstream, W rises to match: the same throughput, more things waiting, longer waits. Concurrency is not speed. A thread pool larger than λW at the bottleneck’s real capacity is a queue with extra steps.

“We do not have the numbers.” You have two of them. That is the point of the law.

In production

The most valuable everyday use is deriving the metric you did not instrument. Connection pools report in-use connections and query rates; that gives you mean query latency for free. Kafka reports consumer lag and throughput; that gives you the time a message waits. A thread pool’s active count and its completion rate give you mean task duration without a timer anywhere.

It is also the correct way to size a pool. Take target throughput, take measured service time at that throughput — not at idle, because service time rises with load — multiply, then add headroom for variance. Doing it the other way round, picking a round number and hoping, is how services end up with 500-thread pools in front of a database that can serve 40 concurrent queries.

The follow-up questions

“You need 10,000 requests per second and each takes 50 ms. How many workers?” — 500. Then the harder half: at what utilisation, and therefore how many more than 500.

“Your queue depth is climbing but throughput is flat. What does that tell you?” — Arrivals exceed departures, so W is growing linearly. It is a capacity problem, not a latency problem, and the queue is hiding it — see backpressure.

“Can you use this to check my design?” — Yes, and you should, out loud. It is the fastest credibility you can earn in a system design interview.

“Does it hold during an incident?” — Over a window that includes the whole transient, yes. Instant-by-instant during accumulation, L is growing and the mean W you measure from completions lags reality.