Scalability — Cheat Sheet

Tags: scalability, system-design, load-balancing, caching, sharding, microservices

Level: advanced

TL;DR: Scalability is handling growth gracefully via horizontal expansion, stateless application tiers, distributed caching, and partitioned persistence.

Key concepts

Horizontal scaling adds more machines; vertical scaling upgrades existing machine capacity.

Stateless service architecture enables elastic auto-scaling behind load balancers.

Read replicas offload read-heavy database traffic from the primary write node.

Database sharding partitions data across multiple database instances horizontally.

Content Delivery Networks cache static and edge content close to end users.

In-memory data stores like Redis prevent repetitive, costly database lookups.

Asynchronous message queues decouple compute-intensive operations from critical request paths.

Eventual consistency balances availability and latency in distributed data architectures.

Connection pooling prevents exhaustion of database sockets under high concurrency.

Circuit breakers isolate downstream failures to preserve overall platform availability.

Typical questions and ready answers

Q: How do you scale a monolithic application struggling under peak traffic?R: I first profile bottlenecks, offload static assets to a CDN, extract state to Redis, and scale application instances horizontally behind a load balancer before considering database read replicas.

Q: How do you choose between horizontal and vertical database scaling?R: I vertical scale initially for simplicity and minimal operational overhead; when hardware limits or single-point-of-failure risks emerge, I adopt read replicas, then table partitioning or sharding.

Q: What strategies mitigate the thundering herd problem during cache expirations?R: I apply probabilistic early cache expiration (XFetch), lock-based cache regeneration, or pre-warmed background cache refreshes to ensure only a single worker queries the database on miss.

Pitfalls and common mistakes

Storing user session state in local server memory instead of centralized distributed stores.

Premature sharding before exhausting caching, indexing, and read replication optimizations.

Missing distributed database transactions and foreign key enforcement challenges after partitioning.

Failing to set timeouts and backoff strategies on external downstream service calls.

Over-caching dynamic data without a rigorous cache invalidation protocol.

Snippets and commands

# Monitor connection states and backlog on Linux
ss -s

# Test concurrency throughput using Apache Benchmark
ab -n 10000 -c 100 -k https://api.example.com/health

Nginx

# Upstream load balancing with least connections strategy
upstream backend_cluster {
    least_conn;
    server 10.0.0.10:8080 max_fails=3 fail_timeout=10s;
    server 10.0.0.11:8080 max_fails=3 fail_timeout=10s;
    keepalive 32;
}

To review

[ ] Consistent hashing implementation for distributed cache and storage distribution

[ ] CAP theorem trade-offs in distributed data consistency models

[x] Read/write splitting with primary-replica database topologies

[x] Stateless JWT authentication across multi-instance clusters

References

Designing Data-Intensive Applications by Martin Kleppmann

System Design Interview – An Insider's Guide by Alex Xu

The Art of Scalability by Martin L. Abbott and Michael T. Fisher