System design interviews feel intimidating because they're open-ended - there's no single right answer. But every design is assembled from a small set of reusable building blocks. Learn the blocks and the trade-offs between them, and any prompt becomes a matter of composition.
What system design tests
The interviewer isn't checking whether you've memorised an architecture. They're watching how you handle ambiguity: do you ask about scale before drawing boxes? Do you name trade-offs instead of pretending there's a free lunch? Do you know where the bottleneck will be? The building blocks below are the vocabulary you use to show it.
Anatomy of a request
Almost every web system is a variation on this path: a client hits a load balancer, which spreads traffic across stateless app servers, which read from a cache first and fall back to a database that replicates for read scale. Master this skeleton and you can grow any design from it.
Scaling: vertical vs horizontal
When load grows you have two options: a bigger machine (vertical) or more machines (horizontal). Vertical is simple but has a hard ceiling and a single point of failure. Horizontal scales almost without limit but forces your servers to be stateless and introduces coordination problems.
| Vertical (scale up) | Horizontal (scale out) | |
|---|---|---|
| How | Bigger CPU / RAM / disk | Add more machines behind a load balancer |
| Ceiling | Hardware limit | Practically unlimited |
| Failure | Single point of failure | Tolerates node loss |
| Cost | Cheap early, steep later | Linear, predictable |
| Catch | Simple | Requires stateless services + coordination |
Caching
Caching is the highest-leverage optimisation in most systems: keep frequently-read data in fast memory so you don't recompute or re-fetch it. The hard part is never the cache - it's invalidation and staleness.
Databases, sharding & replication
Two independent decisions: replication (copies of the data for read scale and failover) and sharding (splitting the data across machines for write scale and size). And, upstream of both, the SQL-vs-NoSQL choice.
- Replication - a primary takes writes; read replicas serve reads. Great for read-heavy systems; introduces replication lag (eventual consistency on the replicas).
- Sharding - partition rows across machines by a shard key (e.g. user id). Scales writes and storage; makes cross-shard queries and re-balancing hard. Choose the shard key carefully to avoid hot spots.
- SQL vs NoSQL - SQL for strong consistency, relations and transactions; NoSQL for massive scale, flexible schemas and simple access patterns. Justify the choice from the requirements, never by fashion.
Queues & async processing
Not everything needs to happen inside the request. A message queue (Kafka, SQS, RabbitMQ) lets you accept work fast and process it later, smoothing traffic spikes and decoupling services so a slow consumer can't take the whole system down.
Consistency & CAP
The CAP theorem says that when the network partitions (and it will), a distributed system must choose between consistency (every read sees the latest write) and availability (every request gets a response). You can't have both during a partition - so you choose per use case.
| Choice | Behaviour | Good for |
|---|---|---|
| CP - consistency | Rejects/blocks rather than serve stale data | Payments, inventory, bookings |
| AP - availability | Always answers, may be briefly stale | Feeds, likes, presence, analytics |
The full building-block map
The blocks above cover most interviews, but the full landscape is broader. This is the map we teach at TopCoding - nine modules from fundamentals to observability - so you can see where each concept fits and what to study next.
Sources & further reading
- 1Designing Data-Intensive Applications — Martin Kleppmann
- 2The System Design Primer — GitHub (donnemartin)
- 3CAP Theorem, twelve years later — Eric Brewer / InfoQ
- 4AWS Well-Architected Framework — Amazon Web Services