System Design Patterns

A framework-agnostic system design fieldbook built around production contracts, protocol mechanics, failure recovery, capacity models, and evidence from papers and operating systems.

Read the interactive fieldbook · Download PDF or EPUB

"Most system design resources are unorganized and overly simple. This repository aims to change that."

Philosophy

1. Contracts before components - Start with invariants and observable promises
2. Depth over breadth - Follow mechanisms through overload, failure, recovery, and migration
3. Single-owner chapters - Cross-reference neighboring concepts instead of repeating generic checklists
4. Framework-agnostic reasoning - Use products as evidence, not as the architecture
5. Quantitative honesty - Source reported figures and label illustrative assumptions
6. Verifiable decisions - Include failure traces, observability, rollout, and tests

Evidence labels used throughout the fieldbook:

• Documented - A cited primary source states the behavior for its named version or publication scope.
• Inference - The conclusion follows from stated mechanisms and assumptions; it is not attributed to an undisclosed deployment.
• Reference design - A reusable architecture proposed by this book; thresholds and worked numbers remain illustrative until measured.

Table of Contents

Part 1: Foundations

• ACID Transactions
• Isolation Levels
• CAP Theorem
• Consistency Models
• Distributed Time
• Failure Semantics, Detection, and Recovery Boundaries
• Idempotency
• Distributed Locks
• Capacity Planning and Estimation

Part 2: Distributed Databases

• Single-Leader Replication
• Multi-Leader Replication
• Leaderless Replication
• Conflict Resolution
• Partitioning Strategies
• Secondary Indexes
• Distributed Transactions
• Consensus Algorithms
• Leader Election
• Workload-Driven Data Modeling and Derived Projections

Part 3: Storage Engines

• B-Trees
• LSM Trees
• SSTables and Compaction
• Write-Ahead Logging
• Bloom Filters
• Column-Oriented Storage
• Data Encoding
• Object Storage Architecture and Commit Protocols

Part 4: Caching

• Cache Semantics and Economics
• Cache Invalidation and Coherence
• Distributed Cache Internals
• Cache Stampede, Cold Start, and Warming

Part 5: Messaging

• Message Queue Architecture
• Publish-Subscribe Architecture
• Message Ordering
• Delivery Guarantees and Effect Boundaries
• Event Sourcing and Domain Logs
• CQRS and Projection Architecture
• Transactional Outbox, Inbox, and CDC Publication
• Poison-Message Quarantine and Redrive

Part 6: Scaling

• Load Balancing
• Horizontal vs Vertical
• Database Sharding
• CDN Architecture
• Rate Limiting
• Circuit Breakers
• Backpressure
• Auto-Scaling
• Multi-Region Architecture
• Retries, Timeouts, and Hedging
• Cell-Based Architecture and Shuffle Sharding
• Multi-Tenant Isolation and Tenant Lifecycle Architecture
• DNS and Connection Management
• Network Transport Internals

Part 7: Real-Time Systems

• Client Delivery Transports
• WebRTC
• Presence Systems
• Collaborative Document Replication and CRDT Sync Engines

Part 8: Case Studies

• Twitter Timeline
• Instagram Feed
• Uber Ride Matching
• Netflix Streaming
• Slack Messaging
• Stripe Payments
• Dropbox Sync
• Discord Voice
• Google Maps
• WhatsApp Messaging
• Figma
• Cloudflare
• LLM Inference Platforms

Part 9: Whitepapers

• MapReduce (2004)
• Dynamo (2007)
• BigTable (2006)
• Spanner (2012)
• TAO (2013)
• Kafka (2011)
• Raft (2014)
• Chubby (2006)
• Aurora (2017)
• CockroachDB (2020)
• Zanzibar (2019)
• Monarch (2020)
• FoundationDB (2021)
• DynamoDB (2022)
• Attention Is All You Need (2017)

Part 10: Security

• Authentication Systems
• OAuth 2.0 and OpenID Connect
• JOSE and JSON Web Token Verification
• API Threat Boundaries and Abuse Resistance
• Zero-Trust Service and Workload Architecture
• Cryptographic Key and Data-Protection Architecture
• Authorization at Scale

Part 11: Observability and Operations

• Distributed Tracing and Telemetry Pipelines
• Metrics Systems and Monitoring
• Production Logging Architecture
• Alert Evaluation and Notification
• SLOs and Error-Budget Control
• FinOps and Cost Engineering
• Incident Command and Learning

Part 12: Service Connectivity and APIs

• Service Discovery and Control-Plane State
• Edge Gateway and API Mediation
• Service Mesh Data and Control Planes
• API Design and Evolution

Part 13: Data Pipelines

• Batch Execution: DAGs, Shuffle, and Safe Reprocessing
• Stream Execution: Time, State, Recovery, and Backpressure
• Change Data Capture: Snapshot, Tail, Apply, and Repair
• Lakehouse Table Formats: Snapshots, Commits, and Maintenance

Part 14: Search Systems

• Search Index Architecture and Internals
• Lexical Query Execution
• Vector Retrieval Systems
• Ranking and Evaluation Systems
• Typeahead and Autocomplete

Part 15: Deployment

• Progressive Delivery and Deployment Strategies
• Feature-Flag Control Planes
• Database Schema Migrations
• Software Delivery Control Planes and GitOps Reconciliation
• Disaster Recovery and Data Reconstruction
• Service and Platform Migration

Part 16: ML Systems

• ML System Fundamentals: control planes, training/serving parity, maturity model
• Feature Stores: point-in-time correctness, offline/online parity, freshness SLOs
• Model Serving: latency budgets, batching, routing, degradation ladders
• Model Monitoring: drift detection, label delay, slice guardrails, triage flow
• Training Pipelines: TFX/Kubeflow/Airflow DSLs, reproducibility, promotion gates, distributed training
• Model Deployment and Rollouts: canary sizing, shadow isolation, kill switches, threshold migration, rollback playbooks
• Recommendation Systems: two-tower retrieval, Wide & Deep, ANN indexing, DPP diversity, exploration, cold start
• Online Experiments: power analysis, SRM checks, CUPED, sequential testing, interleaving
• ML Risk and Governance: risk tiers, model cards, proxy detection, audit logs, incident response, lifecycle
• Label and Ground-Truth Systems: label delay, selective labels, human review, weak supervision, append-only label stores
• Dataset Management and Versioning: immutable snapshots, manifests, split reproducibility, lineage, privacy deletion
• Offline Evaluation and Metric Design: metric selection, thresholds, calibration, leakage, slices, uncertainty
• Model Registry and ML Metadata: artifact identity, serving contracts, lifecycle states, promotion gates, rollback metadata
• ML Capacity and Cost Planning: GPU sizing, batching, feature fanout, training cost, queueing, headroom
• Distributed Training Internals: parallelism topologies, collectives, ZeRO/FSDP, MFU, failure math, checkpointing

Part 17: LLM Systems

• Agent Fundamentals: durable loops, action identity, authority, evidence, and bounded autonomy
• Orchestration Patterns: deterministic workflows versus adaptive branches, typed state graphs, latency and cost composition
• Multi-Agent Systems: delegation, shared state, message semantics, scheduling, and correlated failure
• RAG Patterns: corpus publication, authorization-aware retrieval, evidence packets, grounded generation, and citation provenance
• LLM Infrastructure: gateway and model control planes, regional admission/routing/streaming, provider and self-hosted fleets
• Prompt Engineering: versioned request compilation, structured outputs, tool schemas, prompt-injection test boundaries
• Fine-Tuning Patterns: method choice, dataset/release identity, serving adapters, privacy and evaluation
• Context Management: context revisions, compaction, memory, token budgets, and concurrency
• Harness Engineering: action transactions, permission boundaries, sandbox/secret broker, recovery and rollout
• LLM Evaluation and Observability: estimands, datasets, calibrated evaluators, uncertainty, traces and production feedback
• GPU Inference Internals: phase rooflines, KV/device memory, kernels, parallelism, and accelerator isolation
• Agent Inference: multi-turn session scheduling, KV residency, cache-aware routing, fan-out and task-level goodput

Part 18: Workflow & Job Systems

• Workflow System Fundamentals: workload taxonomy, durable state model, ownership and engine selection
• Background Jobs and Worker Pools: queue/worker contracts, acknowledgement, concurrency and overload
• Distributed Scheduling and Timer Services: timer state, claiming, misfires, recurrence, sharding and clock semantics
• Durable Execution and Workflow Engines: event histories, deterministic replay, commands, signals and versioning
• DAG Orchestration: dependency graphs, artifact readiness, backfills and critical-path execution
• Effect Commit Protocols for Workflows: ambiguous effects, inbox/outbox, idempotency, reconciliation and compensation
• Priority Queues, Fairness, and Backpressure: admission, quotas, starvation bounds and tenant scheduling
• Leases, Heartbeats, and Recovery: attempt ownership, fencing, timeout detection and reassignment
• Workflow Observability and Replay: history/attempt correlation, stuck-state diagnosis, replay and forensic evidence

Part 19: Engineering Systems for Coding Agents

• Coding Agent Platform Fundamentals
• Tool and Runtime Contracts for Coding Agents
• Repository Context and Policy Plane
• Repository Architecture for Safe Agentic Change
• Verification and Governance for Agentic Change
• Parallel Development and Integration

Notation

Symbol	Meaning
N	Total nodes/replicas
W	Write quorum size
R	Read quorum size
f	Failures tolerated

References

Books

Book	Author	Topics
Designing Data-Intensive Applications	Martin Kleppmann	Replication, partitioning, transactions, distributed systems
System Design Interview Vol. 1	Alex Xu	Rate limiting, consistent hashing, key-value stores
System Design Interview Vol. 2	Alex Xu	Real-world systems, proximity services, stock exchange
Database Internals	Alex Petrov	B-trees, LSM trees, storage engines, distributed databases
Understanding Distributed Systems	Roberto Vitillo	Networking, coordination, scalability, resiliency
Building Microservices	Sam Newman	Service decomposition, integration, deployment

Original Papers

Distributed Systems Foundations

• Time, Clocks, and the Ordering of Events - Lamport, 1978
• Impossibility of Distributed Consensus with One Faulty Process (FLP) - Fischer, Lynch, Paterson, 1985
• The Part-Time Parliament (Paxos) - Lamport, 1998
• Paxos Made Simple - Lamport, 2001

Storage & Databases

• The Google File System - Ghemawat et al., 2003
• MapReduce: Simplified Data Processing on Large Clusters - Dean & Ghemawat, 2004
• Bigtable: A Distributed Storage System for Structured Data - Chang et al., 2006
• Dynamo: Amazon's Highly Available Key-value Store - DeCandia et al., 2007
• Spanner: Google's Globally-Distributed Database - Corbett et al., 2012
• Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases - Verbitski et al., 2017

Consensus & Coordination

• The Chubby Lock Service for Loosely-Coupled Distributed Systems - Burrows, 2006
• ZooKeeper: Wait-free coordination for Internet-scale systems - Hunt et al., 2010
• In Search of an Understandable Consensus Algorithm (Raft) - Ongaro & Ousterhout, 2014

Messaging & Streaming

• Kafka: a Distributed Messaging System for Log Processing - Kreps et al., 2011
• The Log: What every software engineer should know about real-time data's unifying abstraction - Jay Kreps, 2013

Social & Web Scale

• TAO: Facebook's Distributed Data Store for the Social Graph - Bronson et al., 2013
• Scaling Memcache at Facebook - Nishtala et al., 2013
• F4: Facebook's Warm BLOB Storage System - Muralidhar et al., 2014

NewSQL

• CockroachDB: The Resilient Geo-Distributed SQL Database - Taft et al., 2020
• TiDB: A Raft-based HTAP Database - Huang et al., 2020

Engineering Blogs

Company	Notable Posts
Netflix Tech Blog	Microservices, chaos engineering, streaming
Uber Engineering	Real-time systems, geospatial, scaling
Meta Engineering	TAO, distributed systems, ML infrastructure
Stripe Engineering	API design, idempotency, payments
Cloudflare Blog	Edge computing, DNS, DDoS mitigation
Discord Engineering	Real-time messaging, voice, scaling
Slack Engineering	Messaging architecture, search, reliability
Dropbox Tech Blog	Sync, storage, infrastructure
Pinterest Engineering	Recommendations, search, scaling
LinkedIn Engineering	Kafka, data infrastructure, ML
Twitter Engineering	Timeline, real-time, graph processing
Spotify Engineering	Streaming, personalization, microservices
GitHub Engineering	Git internals, availability, scaling
Shopify Engineering	E-commerce, flash sales, payments

Online Resources

Resource	Description
High Scalability	Architecture case studies
The Morning Paper	CS paper summaries (archived)
Papers We Love	Community for reading CS papers
Distributed Systems Reading List	Curated paper collection
System Design Primer	Popular GitHub resource
ByteByteGo	Visual system design explanations
Awesome Distributed Systems	Curated resources list

Video Lectures

Course	Institution	Topics
MIT 6.824: Distributed Systems	MIT	MapReduce, Raft, Spanner, distributed transactions
CMU 15-445: Database Systems	CMU	Storage, indexing, query processing, concurrency
CMU 15-721: Advanced Database Systems	CMU	In-memory databases, query optimization
Stanford CS244B: Distributed Systems	Stanford	Consensus, replication, distributed storage

Additional Books

Architecture & Design Patterns

Book	Author	Topics
Clean Architecture	Robert C. Martin	Dependency rule, boundaries, components, frameworks
Patterns of Enterprise Application Architecture	Martin Fowler	Domain logic, data source, web presentation patterns
Domain-Driven Design	Eric Evans	Bounded contexts, aggregates, ubiquitous language
Implementing Domain-Driven Design	Vaughn Vernon	Practical DDD patterns and techniques
Release It!	Michael Nygard	Stability patterns, capacity, networking
Software Architecture: The Hard Parts	Ford, Richards, Sadalage, Dehghani	Trade-off analysis, modularity, decomposition
Fundamentals of Software Architecture	Mark Richards, Neal Ford	Architecture styles, characteristics, decisions
A Philosophy of Software Design	John Ousterhout	Complexity, modules, abstractions, comments

Distributed Systems & Reliability

Book	Author	Topics
Site Reliability Engineering	Google	SLOs, error budgets, toil, monitoring, on-call
The Site Reliability Workbook	Google	Practical SRE implementation
Distributed Systems	Tanenbaum & Van Steen	Processes, communication, naming, coordination
Designing Distributed Systems	Brendan Burns	Patterns for scalable, reliable services
Database Reliability Engineering	Campbell & Majors	Database operations, infrastructure, recovery
Web Scalability for Startup Engineers	Artur Ejsmont	Practical scaling strategies

Data & Streaming

Book	Author	Topics
Streaming Systems	Akidau, Chernyak, Lax	Watermarks, windows, triggers, exactly-once
Kafka: The Definitive Guide	Shapira, Palino, et al.	Kafka internals, producers, consumers, operations
Making Sense of Stream Processing	Martin Kleppmann	Event sourcing, change capture, stream processing
Data Mesh	Zhamak Dehghani	Decentralized data architecture
The Data Warehouse Toolkit	Ralph Kimball	Dimensional modeling, ETL, BI

Performance & Optimization

Book	Author	Topics
Systems Performance	Brendan Gregg	Linux, observability, methodologies, tools
BPF Performance Tools	Brendan Gregg	Linux BPF observability and tracing
High Performance MySQL	Silvia Botros, Jeremy Tinley	Query optimization, replication, scaling
High Performance Browser Networking	Ilya Grigorik	TCP, UDP, TLS, HTTP/2, WebSocket, WebRTC

More Papers

Consistency & Transactions

• Linearizability: A Correctness Condition for Concurrent Objects - Herlihy & Wing, 1990
• A Critique of ANSI SQL Isolation Levels - Berenson et al., 1995
• Large-scale Incremental Processing Using Distributed Transactions and Notifications (Percolator) - Peng & Dabek, 2010
• Calvin: Fast Distributed Transactions for Partitioned Database Systems - Thomson et al., 2012
• Highly Available Transactions: Virtues and Limitations - Bailis et al., 2013
• Serializable Snapshot Isolation in PostgreSQL - Ports & Grittner, 2012

Distributed Data Structures

• Consistent Hashing and Random Trees - Karger et al., 1997
• CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data - Weil et al., 2006
• A Comprehensive Study of Convergent and Commutative Replicated Data Types (CRDTs) - Shapiro et al., 2011
• The Phi Accrual Failure Detector - Hayashibara et al., 2004

Search & Indexing

• The Anatomy of a Large-Scale Hypertextual Web Search Engine - Brin & Page, 1998
• Web Search for a Planet: The Google Cluster Architecture - Barroso et al., 2003
• Elasticsearch: The Definitive Guide (online) - Clinton Gormley, Zachary Tong

Machine Learning Systems

• Scaling Distributed Machine Learning with the Parameter Server - Li et al., 2014
• TensorFlow: A System for Large-Scale Machine Learning - Abadi et al., 2016
• Hidden Technical Debt in Machine Learning Systems - Sculley et al., 2015
• TFX: A TensorFlow-Based Production-Scale Machine Learning Platform - Baylor et al., 2017
• Data Validation for Machine Learning - Breck et al., 2019
• TensorFlow Serving: Flexible, High-Performance ML Serving - Olston et al., 2017
• Deep Neural Networks for YouTube Recommendations - Covington et al., 2016
• Wide & Deep Learning for Recommender Systems - Cheng et al., 2016
• Sampling-Bias-Corrected Neural Modeling for Large Corpus Item Recommendations - Yi et al., 2019
• Diversity-Promoting Recommendation with Determinantal Point Processes - Chen et al., 2018
• ZeRO: Memory Optimizations Toward Training Trillion Parameter Models - Rajbhandari et al., 2019
• Trustworthy Online Controlled Experiments - Kohavi et al., 2020
• Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-Experiment Data (CUPED) - Deng et al., 2013
• Overlapping Experiment Infrastructure: More, Better, Faster Experimentation - Tang et al., 2010
• Model Cards for Model Reporting - Mitchell et al., 2019
• Datasheets for Datasets - Gebru et al., 2018
• A Unified Approach to Interpreting Model Predictions (SHAP) - Lundberg & Lee, 2017
• NIST AI Risk Management Framework - NIST, 2023

Container & Orchestration

• Borg, Omega, and Kubernetes - Burns et al., 2016
• Large-scale cluster management at Google with Borg - Verma et al., 2015
• Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center - Hindman et al., 2011

License

MIT License