System design, simulated
Every system design resource teaches with static boxes and arrows. A rectangle labelled “Cache” between two other rectangles tells you nothing about what happens when it goes cold at ten thousand requests a second — and that gap is why candidates can recite architectures and freeze when an interviewer asks what breaks.
These 31 concepts are not diagrams. Each one is a discrete-event simulation with real latency numbers, seeded so that the same parameters always produce the same run, and every sentence a page puts on screen is paired with a predicate the build checks. Turn a dial and the run re-executes from the same seed, so the past you were watching stays the past.
Start here
- The signature demo: one parameter, opposite outcomes.Retry stormsRetries are load. Immediate retries turn a two-second fault into a sustained outage; exponential backoff with jitter turns the same fault into a blip.
- The most important graph in system design, drawn.The utilisation curveWaiting time rises with 1/(1 - ρ). The last few percent of capacity cost more than all the rest put together, which is why 95% utilised is a different world from 70%.
- Easier to watch than to read.Consistent hashingAdding a node to a ring of n moves about 1/n of the keys. Under modulo hashing it moves nearly all of them — and virtual nodes are what make the ring balanced rather than merely stable.
Load and traffic
Where requests go and what decides. Balancing strategies under uneven service times, hash rings that survive a node change, limiters that hold their limit, and the scaling lag no policy can outrun.
- Load balancingasked constantlySampling two nodes at random and taking the lighter is nearly as good as a global scan and costs almost nothing — and no strategy at all can split a single hot key.
- Consistent hashingasked constantlyAdding a node to a ring of n moves about 1/n of the keys. Under modulo hashing it moves nearly all of them — and virtual nodes are what make the ring balanced rather than merely stable.
- Rate limitingasked constantlyA fixed window admits up to twice its stated limit across a boundary, and its own metrics will never show it.
- AutoscalingintermediateReactive scaling cannot beat its own boot time, and a utilisation signal saturates at 100% so it cannot tell you how far behind you are.
Latency and queueing
The arithmetic of waiting. Little’s Law made turnable, the utilisation curve drawn rather than claimed, fan-out turning a rare tail into the common case, and what a queue is really for.
- Little's LawfoundationalThe law holds for any arrival pattern, any service distribution and any queue discipline. Any two of the three numbers give you the third.
- The utilisation curveasked constantlyWaiting time rises with 1/(1 - ρ). The last few percent of capacity cost more than all the rest put together, which is why 95% utilised is a different world from 70%.
- Tail latency amplificationasked constantlyAt a fan-out of 100, the backend p99 is the user median. Two thirds of user requests contain at least one slow call.
- BackpressureintermediateAn unbounded queue does not absorb overload, it converts an error you would have noticed into a latency you will not.
Caching
Hit ratio as a curve rather than a line, five eviction policies on one trace, the stampede that takes a database down, and what each write strategy loses in a crash.
- Cache hit ratioasked constantlyCaching 1% of a skewed keyspace delivers most of the hit ratio that caching 40% would. Read the number as origin load, not as a percentage.
- Eviction policiesintermediateA single scan over a table larger than the cache evicts every useful entry in exactly the order it will next be needed — which is why scan resistance is a feature databases advertise.
- Cache stampedeasked constantlyRequest coalescing turns thousands of identical database calls into one, and it is a few lines of code in any real client library.
- Write strategiesintermediateWrite-back makes writes as fast as memory and puts a number on your data loss: everything since the last flush.
Reliability and failure
How a two-second fault becomes a one-minute outage, and the handful of patterns that stop it. Retry storms, circuit breakers, bulkheads, timeout budgets and exactly-once effects.
- Retry stormsasked constantlyRetries are load. Immediate retries turn a two-second fault into a sustained outage; exponential backoff with jitter turns the same fault into a blip.
- Circuit breakersasked constantlyA breaker does not fix the dependency. It converts a three-second hang into a microsecond error, which is the difference between a degraded feature and a dead service.
- BulkheadsintermediateA fifth of your traffic hitting a slow dependency can take down every other endpoint, because they share a pool. Isolation is the only thing that stops it.
- Timeout cascadesintermediateTimeouts must decrease as you go inward, and the honest version is that they should not be independent numbers at all: pass a deadline and subtract what you have spent.
- Idempotency and exactly-onceasked constantlyExactly-once delivery is not available. Exactly-once effect is, and it is a unique constraint in the same transaction as the effect.
Data
Replication lag you can watch produce a stale read, partitioning schemes and the hot shard each one makes, CAP with the network actually cut, and the storage engines underneath.
- Replication lagasked constantlyEventual consistency is a promise about the limit, not about the next request. Read-your-writes costs a token and a small share of leader reads.
- Shardingasked constantlyA hot partition key is one key. Hashing spreads keys, and there is only one of it — so the answer is a compound key, a dedicated shard, or a different store.
- CAP in practiceasked constantlyPartitions are not chosen, they happen. The choice is what to do during one, and there is no third option.
- B-tree against LSM treeadvancedThe trade is not which is faster, it is where you would rather pay — and whether you can tolerate the p99 spike when a compaction runs.
- Bloom filtersintermediateTen bits per key buys about a 1% false-positive rate. False positives cost a wasted lookup; false negatives cannot happen, and that asymmetry is the whole point.
Distributed coordination
Agreement without a coordinator. Raft elections, the quorum arithmetic that prevents split brain, fencing tokens, and what a clock cannot tell you about causality.
- Raft leader electionasked constantlyOne vote per term plus a majority requirement makes two leaders impossible. Randomised timeouts are what make the algorithm terminate at all.
- Split brainintermediateTwo majorities of the same set always overlap, so only one side can hold one. Split brain in production is almost always a configuration that forgot to grow.
- Distributed locks and fencingadvancedA lock that expires cannot guarantee mutual exclusion. A fencing token moves the correctness to the resource, which is the only place it can live.
- Vector clocks and causalityadvancedA timestamp cannot tell a concurrent write from an ordered one. A vector clock can, and concurrent is an answer the application has to handle.
Asynchronous architecture
What you gain and what you give up by decoupling. Ordering against parallelism, the dual-write problem, compensating transactions, projection lag and connection fan-out.
- Queues and orderingasked constantlyA partitioned log guarantees order within a partition, not across the topic. Ordering costs you parallelism and makes one slow consumer everybody’s problem.
- The outbox patternasked constantlyThe dual-write problem cannot be retried away. Put the event in the same transaction as the row and everything after it becomes retryable.
- Sagas and compensationadvancedA saga is not a rollback. Every compensation must be idempotent and must eventually succeed, because there is nothing to compensate a compensation with.
- CQRS projection lagintermediateSplitting the read model creates a window in which a user cannot see their own write, and it widens exactly when traffic is heaviest.
- WebSocket fan-outintermediateThe publish rate is not the load; the fan-out multiplier is. And a bounded per-connection queue is the difference between dropping one client and dropping all of them.
The rest of the section
- Napkin mathEstimate a system’s load with the working shown, then push that number into a design and find out whether it holds.
- The design canvasCompose the pieces, press run, turn the dials while it runs, and be told which component is the bottleneck and why.
- Mock interviewFive phases, a timer you can ignore, and feedback that tells you what a strong candidate would have covered.