Design a Distributed Cache
Design a distributed caching system like Redis or Memcached that can handle millions of requests per second.
Use consistent hashing for distribution, LRU eviction, and replication for availability.
Research consistent hashing
How do you handle hot keys?
Consider cache invalidation strategies
functional
- •GET/SET/DELETE operations
- •TTL support
- •Atomic operations
- •Pub/Sub (optional)
non functional
- •Sub-millisecond latency
- •High throughput
- •Horizontal scalability
- •Fault tolerance
Architecture diagram visualization
(Diagram generation coming soon)
client
Smart client with consistent hashing
server
In-memory hash table with eviction
cluster
Gossip protocol for membership
hot keys
Replicate hot keys to multiple nodes
thundering herd
Probabilistic early expiration, locking
memory management
Slab allocation, avoid fragmentation
strategy
Replicate to N successor nodes on ring
consistency
Async replication for performance, sync option for consistency
virtual nodes
Each physical node owns multiple positions on ring
consistent hashing
benefits
- •Minimal redistribution on node change
- •Even distribution with virtual nodes
description
Hash keys to virtual ring, assign ranges to nodes
LFU
Least Frequently Used - track access count
LRU
Least Recently Used - track access time
TTL
Time-based expiration