System Design Interview by Alex Xu (ByteByteGo, 2020; Volume 2, 2022) carries a title that undersells its value. Despite the interview framing, the two volumes are among the most accessible and practically useful introductions to large-scale system design available to software engineers.
The books work through a series of concrete design problems — URL shorteners, rate limiters, notification systems, distributed key-value stores, social feeds, search autocomplete — and explain not just what to build but how to think about the problem. That thinking process is the real product.
Part of the Top 5 Books Every Software Engineer Should Read series.
What the Books Are About
Each chapter presents a design problem, walks through a structured approach to solving it, and explains the reasoning behind key decisions.
The structure is consistent:
- Understand the problem — clarify requirements, constraints, and scale
- High-level design — identify the major components and their relationships
- Deep dive — examine specific components in detail
- Wrap-up — review bottlenecks, trade-offs, and potential improvements
This framework is transferable. The same approach applies whether the problem is designing a chat system, a distributed cache, or a content delivery network.
Core Concepts
1. Requirements First
Every chapter begins with requirements gathering. This is not a procedural nicety; it fundamentally shapes the design.
The book distinguishes between:
Functional requirements: what the system does
- Users can upload photos
- The system returns shortened URLs
- Notifications are delivered within 10 seconds
Non-functional requirements: how the system performs
- 100 million daily active users
- 99.99% availability
- Reads must complete within 100ms at p99
The non-functional requirements drive nearly all architectural decisions. A system handling 1,000 requests per second is designed differently from one handling 100,000. The choice between SQL and NoSQL, the need for caching, the replication strategy — all of these follow from the scale and reliability requirements, not from personal preference.
One of the book’s most useful habits is explicit back-of-envelope estimation. How much storage is needed? What is the read/write ratio? How much bandwidth? Engineers who develop comfort with these calculations make better early architectural decisions.
2. Horizontal Scaling and Statelessness
A recurring pattern across the chapters is the move from a single-server architecture to one that can scale horizontally — adding more machines rather than making each machine larger.
The prerequisite for horizontal scaling is statelessness: the application tier should not store session data or user state locally. Any request should be serviceable by any server instance.
When state is required, it is stored in a shared backend:
- Session store (Redis, Memcached) for ephemeral state
- Relational or NoSQL database for persistent state
- Object storage (S3, GCS) for large files
A load balancer distributes requests across the stateless application tier. Because no instance holds unique state, adding or removing instances has no effect on correctness.
3. Caching
Caching is one of the most frequently used techniques in the book, and Xu covers it with appropriate depth.
A cache stores frequently accessed data in a faster storage tier (usually in-memory) to reduce latency and reduce load on the underlying storage system. The tradeoffs are:
Read strategies:
- Cache-aside (lazy loading): the application checks the cache first; on a miss, fetches from the database and populates the cache. Simple but can result in stale data.
- Read-through: the cache is interposed in front of the database; misses are automatically populated. Cleaner but the cache layer must understand the data model.
- Refresh-ahead: items are proactively refreshed before expiry. Reduces latency spikes but requires predicting which items will be needed.
Write strategies:
- Write-through: every write goes to both cache and database. Consistent but higher write latency.
- Write-behind (write-back): writes go to the cache immediately; the database is updated asynchronously. Lower write latency but higher risk of data loss on failure.
Eviction policies:
- LRU (Least Recently Used): evicts the item not accessed most recently. Good general default.
- LFU (Least Frequently Used): evicts the item accessed least often. Better for workloads with stable popularity distributions.
- TTL-based expiration: items expire after a fixed duration regardless of access patterns.
Cache invalidation — ensuring the cache does not serve stale data — is one of the genuinely hard problems in distributed systems. The book is honest about this: there is no universally correct invalidation strategy. The right approach depends on how stale data is acceptable for the specific use case.
4. Rate Limiting
The rate limiting chapter is one of the most technically detailed in the first volume.
Rate limiting controls the number of requests a client can make in a given time window. It serves several purposes:
- Protects against abuse and DDoS attacks
- Prevents a single client from consuming disproportionate resources
- Enforces API quotas for external consumers
The book covers several algorithms:
Token bucket Each client has a bucket of tokens that refills at a fixed rate. Each request consumes one token. Requests are rejected when the bucket is empty. Allows short bursts as long as the average rate is within the limit.
Leaking bucket Requests are added to a queue and processed at a fixed rate, regardless of arrival pattern. Provides a smoother, more predictable traffic pattern but cannot handle bursts.
Fixed window counter The time axis is divided into fixed windows (e.g., one-minute intervals). Request counts are tracked per window. Simple but has an edge case at window boundaries: clients can make twice the allowed requests by clustering requests around the boundary.
Sliding window log Maintains a log of request timestamps. Counts only requests within the rolling window. Accurate but memory-intensive.
Sliding window counter Approximates the sliding window using the current and previous fixed window counts weighted by position. A practical balance of accuracy and efficiency.
In a distributed system, rate limiting must be coordinated across multiple service instances. Centralized counters in Redis are a common approach, with Lua scripts ensuring atomicity.
5. Consistent Hashing
The consistent hashing chapter addresses a fundamental problem in distributed caching and storage: how to distribute data across multiple nodes such that adding or removing a node requires minimal data movement.
With naive modular hashing (key % n), adding or removing a node changes the mapping for almost all keys, requiring a massive redistribution of data.
Consistent hashing places both keys and nodes on a virtual ring. Each key is assigned to the nearest node clockwise on the ring. When a node is added, it takes ownership only of the keys between it and the previous node. When a node is removed, its keys move to the next node. The number of remapped keys is proportional to 1/n rather than nearly all keys.
Virtual nodes (vnodes) improve balance: each physical node is represented by multiple points on the ring, distributing load more evenly.
Consistent hashing is used in production by systems including Amazon DynamoDB, Apache Cassandra, and distributed CDNs.
6. Database Choices
The book consistently returns to the question of which data store to use, and consistently resists simple answers.
The general guidance:
Relational databases are appropriate when:
- Data has complex relationships
- ACID transactions are required
- The query access patterns are not fully known at design time
NoSQL databases are appropriate when:
- Very high write throughput is required
- The data model is simple and access patterns are well-defined
- Schema flexibility is valued
- Horizontal scaling is a hard requirement from the start
The practical answer is that most large systems use both. A primary relational database handles transactional data; a NoSQL store handles high-volume writes or flexible document structures; a cache sits in front of both.
7. Over-Engineering
One of the book’s most valuable warnings comes in its framing of trade-offs:
“Over-engineering is a real disease of many engineers as they delight in design purity and ignore tradeoffs.”
This is worth dwelling on. System design discussions and interview preparation can lead engineers to default toward complex, sophisticated architectures — microservices, distributed caches, message queues — even when simpler designs would serve the requirements better.
The pragmatic test is always: does the added complexity solve a real problem at the scale and reliability requirements specified? A URL shortener for 100 daily users does not need a distributed key-value store. A social network for 100 million daily users probably does.
Good system design begins with understanding the requirements precisely enough to know which problems are real.
Selected Design Patterns
The books walk through many concrete systems. A few patterns appear repeatedly:
Read/Write Separation
As read traffic grows, it is often more efficient to separate read and write paths. Writes go to a primary database; reads are served from read replicas. This trades some consistency (replicas may lag) for significantly higher read throughput.
Message Queues for Asynchrony
Operations that do not need to complete synchronously — sending email, processing images, logging events — can be placed in a message queue (Kafka, RabbitMQ, SQS). This decouples the producer from the consumer, provides buffering, and makes the system more resilient to downstream failures.
CDNs for Static Content
Content delivery networks cache static assets (images, videos, JS, CSS) at edge locations geographically close to users. This reduces latency and offloads bandwidth from origin servers.
Database Sharding
When a single database cannot handle the write load or data volume, the dataset is partitioned across multiple database instances. Each shard handles a subset of the data. The sharding key must be chosen to distribute load evenly and avoid hotspots.
Limitations of the Books
The System Design Interview volumes are designed for accessibility. This means:
- Explanations are visual and example-driven rather than formally rigorous
- Some topics are introduced without covering failure modes and edge cases in depth
- The interview framing occasionally shapes the presentation (time-boxed exploration, emphasis on completeness over depth)
For deeper treatment of distributed systems fundamentals, Designing Data-Intensive Applications provides the theoretical grounding that System Design Interview leaves implicit.
Who Should Read It
System Design Interview is valuable for:
- Engineers moving from feature development to architecture: the framework for decomposing large systems is directly applicable
- Mid-level to senior engineers preparing for technical discussions with cross-functional teams
- Any engineer who has never had formal exposure to system design and wants a practical, visual starting point
The books are explicitly accessible — they do not assume prior distributed systems knowledge. This is a deliberate choice: they are designed to build intuition before depth.
Summary
System Design Interview provides a practical, example-driven introduction to the patterns and trade-offs of large-scale system design. Its real value is not the specific systems it covers but the structured problem-solving process it teaches.
The engineer who works through both volumes will approach architectural problems differently: with clearer questions about scale, consistency, availability, and cost; with a repertoire of battle-tested patterns to draw on; and with an appreciation that good design is about choosing the right trade-offs rather than maximizing complexity.
Related posts in this series:
