Users Keep Getting Logged Out
Application State & Sessions
Sticky Sessions Under Failure
Application State & Sessions
The Growing Session Store
Shared session storage
Stateful WebSocket Gateways
Application State & Sessions
The Query Gets Slower Every Month
Relational Databases
Too Many Connections
Relational Databases
Checkout vs Analytics
Relational Databases
Choose the Relational Model
Relational Databases
Reads Are Crushing the Primary
Database Scaling & Replication
The Replica Is Stale
System Design
Replica Lag During Import
Database Scaling & Replication
Fail the Primary
Database Scaling & Replication
The Write Ceiling
Partitioning & Sharding
Shard 7 Is Melting
Partitioning & Sharding
The Newest Shard Is Always Hot
Partitioning & Sharding
Cross-Shard Checkout
Partitioning & Sharding
Stop Putting Videos in PostgreSQL
Object Storage & File Systems
Millions of Tiny Files
Object Storage & File Systems
Shared Filesystem Bottleneck
Object Storage & File Systems
Durability vs. Temporary Data
Object Storage & File Systems
The Same Read 50,000 Times
Caching
Inventory Is Wrong
Caching
Midnight Cache Explosion
Caching
Celebrity Hot Key
Caching
Australia Is 600ms Away
CDNs & Edge Delivery
The CDN Serves Old JavaScript
CDNs & Edge Delivery
Personalized Pages at the Edge
CDNs & Edge Delivery
Origin Shield Under Viral Traffic
CDNs & Edge Delivery
Checkout Takes Eight Seconds
Queues & Background Processing
The Backlog Keeps Growing
Queues & Background Processing
The Poison Job
Queues & Background Processing
Priority Matters
Queues & Background Processing
Campaign Capacity
Servers & Compute
CPU Isn't the Problem
Servers & Compute
Background Jobs Are Starving the API
Servers & Compute
Burst Without Waste
Servers & Compute
Uneven Fleet
Load Balancers & Traffic Routing
The Dead Server Still Gets Traffic
Load Balancers & Traffic Routing
Long Requests Break Round Robin
Load Balancers & Traffic Routing
Route by Workload
Load Balancers & Traffic Routing
Users Keep Getting Logged Out
Application State & Sessions
Sticky Sessions Under Failure
Application State & Sessions
The Growing Session Store
Shared session storage
Stateful WebSocket Gateways
Application State & Sessions
The Query Gets Slower Every Month
Relational Databases
Too Many Connections
Relational Databases
Checkout vs Analytics
Relational Databases
Choose the Relational Model
Relational Databases
Reads Are Crushing the Primary
Database Scaling & Replication
The Replica Is Stale
System Design
Replica Lag During Import
Database Scaling & Replication
Fail the Primary
Database Scaling & Replication
The Write Ceiling
Partitioning & Sharding
Shard 7 Is Melting
Partitioning & Sharding
The Newest Shard Is Always Hot
Partitioning & Sharding
Cross-Shard Checkout
Partitioning & Sharding
Stop Putting Videos in PostgreSQL
Object Storage & File Systems
Millions of Tiny Files
Object Storage & File Systems
Shared Filesystem Bottleneck
Object Storage & File Systems
Durability vs. Temporary Data
Object Storage & File Systems
The Same Read 50,000 Times
Caching
Inventory Is Wrong
Caching
Midnight Cache Explosion
Caching
Celebrity Hot Key
Caching
Australia Is 600ms Away
CDNs & Edge Delivery
The CDN Serves Old JavaScript
CDNs & Edge Delivery
Personalized Pages at the Edge
CDNs & Edge Delivery
Origin Shield Under Viral Traffic
CDNs & Edge Delivery
Checkout Takes Eight Seconds
Queues & Background Processing
The Backlog Keeps Growing
Queues & Background Processing
The Poison Job
Queues & Background Processing
Priority Matters
Queues & Background Processing
Campaign Capacity
Servers & Compute
CPU Isn't the Problem
Servers & Compute
Background Jobs Are Starving the API
Servers & Compute
Burst Without Waste
Servers & Compute
Uneven Fleet
Load Balancers & Traffic Routing
The Dead Server Still Gets Traffic
Load Balancers & Traffic Routing
Long Requests Break Round Robin
Load Balancers & Traffic Routing
Route by Workload
Load Balancers & Traffic Routing
One Order, Five Consumers
Pub/Sub & Event-Driven Systems
The Slow Subscriber
Pub/Sub & Event-Driven Systems
Too Many Event Types
Pub/Sub & Event-Driven Systems
Event-Driven Workflow Challenge
Pub/Sub & Event-Driven Systems
The Payment Ran Twice
Idempotency, Ordering & Delivery Guarantees
Cancelled Before Created
Idempotency, Ordering & Delivery Guarantees
Exactly Once Requirement
Idempotency, Ordering & Delivery Guarantees
Commit Succeeded, Publish Failed
Idempotency, Ordering & Delivery Guarantees
The Chatty Mobile App
APIs & Service Communication
Internal Calls Waste Capacity
APIs & Service Communication
The Public API Needs Protection
APIs & Service Communication
Version Without Breaking Clients
APIs & Service Communication
The Monolith's Reporting Module
Service Boundaries & Microservices
Shared Database Coupling
Service Boundaries & Microservices
Too Many Tiny Services
Service Boundaries & Microservices
Synchronous Dependency Chain
Service Boundaries & Microservices
Profile Update Looks Old
Consistency & Distributed Data
Two Regions Edit the Same Record
Consistency & Distributed Data
Availability or Strong Consistency
Consistency & Distributed Data
Materialized View Is Behind
Consistency & Distributed Data
Recommendation Takes Down Product Page
System Design
Retry Storm
System Design
One Pool Consumes Everything
System Design
Recover Without a Herd
System Design
Scale on the Right Signal
System Design
Cold Start During a Flash Crowd
System Design
Workers Scale, Database Doesn't
System Design
Capacity Plan for Launch
System Design
Average Latency Looks Fine
System Design
Where Did Four Seconds Go?
System Design
Queue Is Quiet, Customers Aren't
System Design
Production Diagnostic Challenge
System Design
Release Breaks Everyone
System Design
Zero-Downtime Version Swap
System Design
Database Change Without Downtime
System Design
Feature Launch Without Redeploy
System Design
The Architecture Costs 2x
System Design
Storage Bill Never Stops Growing
System Design
Cross-Region Bill Shock
System Design
Final Foundation Optimization Challenge
System Design
One Order, Five Consumers
System Design
The Slow Subscriber
System Design
Too Many Event Types
System Design
Event-Driven Workflow Challenge
System Design
The Payment Ran Twice
System Design
Cancelled Before Created
System Design
Exactly Once Requirement
System Design
Commit Succeeded, Publish Failed
System Design
The Chatty Mobile App
System Design
Internal Calls Waste Capacity
System Design
The Public API Needs Protection
System Design
Version Without Breaking Clients
System Design
The Monolith's Reporting Module
System Design
Shared Database Coupling
System Design
Too Many Tiny Services
System Design
Synchronous Dependency Chain
System Design
Profile Update Looks Old
System Design
Two Regions Edit the Same Record
System Design
Availability or Strong Consistency
System Design
Materialized View Is Behind
System Design
Recommendation Takes Down Product Page
System Design
Retry Storm
System Design
One Pool Consumes Everything
System Design
Recover Without a Herd
System Design
Scale on the Right Signal
System Design
Cold Start During a Flash Crowd
System Design
Workers Scale, Database Doesn't
System Design
Capacity Plan for Launch
System Design
Average Latency Looks Fine
System Design
Where Did Four Seconds Go?
System Design
Queue Is Quiet, Customers Aren't
System Design
Production Diagnostic Challenge
System Design
Release Breaks Everyone
System Design
Zero-Downtime Version Swap
System Design
Database Change Without Downtime
System Design
Feature Launch Without Redeploy
System Design
The Architecture Costs 2x
System Design
Storage Bill Never Stops Growing
System Design
Cross-Region Bill Shock
System Design
Final Foundation Optimization Challenge
System Design
URL Shortener
System Design
Notification Platform
System Design
Distributed Rate Limiter
System Design
Dropbox
System Design
System Design
Twitter/X Feed
System Design
System Design
YouTube
System Design
Netflix
System Design
Uber
System Design
Ticketmaster
System Design
Stripe
System Design
Slack
System Design
Google Drive
System Design
Search Autocomplete
System Design
Full-Text Search
System Design
Collaborative Editor
System Design
Webhook Platform
System Design
Distributed Scheduler
System Design
Metrics Platform
System Design
Logging Platform
System Design
Feature Flag Platform
System Design
Distributed Cache
System Design
Object Storage Service
System Design
Kafka-Like Streaming Platform
System Design
From Notebook to Production
ML System Architecture Fundamentals
Batch vs. Online, Not Both at Once
ML System Architecture Fundamentals
A Safe Path to Raw Data
ML System Architecture Fundamentals
A Small, Complete ML Lifecycle
ML System Architecture Fundamentals
Sampling Bias in Event Collection
Data Collection & Labeling
Joining Late-Arriving Labels
Data Collection & Labeling
Human Labeling Throughput
Data Collection & Labeling
Filtering Sensitive Data Before Training
Data Collection & Labeling
Outgrowing the Nightly Window
Data Pipelines & Preprocessing
A Realtime Feature Arrives Too Late
Data Pipelines & Preprocessing
Duplicate Events Corrupt Training Examples
Data Pipelines & Preprocessing
Backfills Overwhelm the Pipeline
Data Pipelines & Preprocessing
Training-Serving Skew
Feature Engineering & Feature Stores
Online Feature Lookups Under Peak Load
Feature Engineering & Feature Stores
Freshness SLAs Differ by Feature
Feature Engineering & Feature Stores
Six Teams, One Feature
Feature Engineering & Feature Stores
Shrinking a 46-Hour Training Run
Training Infrastructure
GPU Utilization Bottlenecked by Data Loading
Training Infrastructure
Multiple Teams Competing for GPUs
Training Infrastructure
Checkpoint and Resume After Failure
Training Infrastructure
Same Code, Different Results
Experiment Tracking & Reproducibility
Untraceable Hyperparameters
Experiment Tracking & Reproducibility
Hundreds of Untraceable Experiments
Experiment Tracking & Reproducibility
Reproducing a Six-Month-Old Model
Experiment Tracking & Reproducibility
Aggregate Accuracy Hides a Segment Regression
Model Evaluation & Validation
Undeployable Despite Good Offline Metrics
Model Evaluation & Validation
Release Without a Champion Comparison
Model Evaluation & Validation
Data Leakage Inflates the Eval Score
Model Evaluation & Validation
Deploying Straight From the Training Machine
Model Registry & Versioning
Rolling Back in Minutes, Not Hours
Model Registry & Versioning
Different Regions, Different Versions
Model Registry & Versioning
Candidate to Staging to Production
Model Registry & Versioning
Score Everyone Before 6 AM
Batch Inference
One Partition, All the Stragglers
Batch Inference
Rescore Only What Changed
Batch Inference
Publishing a Half-Finished Run
Batch Inference
Behind a Production API, Finally
Online Inference & Model Serving
Cold Starts Make New Servers Slow to Help
Online Inference & Model Serving
Feature Retrieval Dominates Latency
Online Inference & Model Serving
One Slow Dependency Blocks a Healthy Request
Online Inference & Model Serving