Design Twitter/X
Design a microblogging platform like Twitter with features like posting tweets, following users, timeline, and trending topics.
Use hybrid fan-out approach for timeline, pre-compute trends with streaming analytics, and heavy caching.
Consider the celebrity problem for fan-out
How do you detect trending topics in real-time?
Think about eventual consistency
functional
- •Post tweets
- •Follow users
- •Home timeline
- •Search
- •Trending topics
- •Notifications
non functional
- •High read availability
- •Eventually consistent
- •Low latency timeline
- •Handle viral content
timeline reads
200M × 10 reads = 2B timeline reads/day
tweets per day
500M tweets
daily active users
200M DAU
media
Store in S3, reference in tweet metadata
search
Elasticsearch with inverted index on tweet text
rate limiting
Token bucket per user for API calls
pipeline
- •Stream tweets to Kafka
- •Count hashtags with sliding window
- •Rank by velocity of growth
- •Filter by quality/spam
algorithm
Exponential decay to favor recent trends
hybrid
Combine both approaches based on follower count
fan out on read
how
Merge celebrity tweets at read time
when
Celebrities (> 10K followers)
fan out on write
how
On tweet, push to all followers timelines in Redis
when
Regular users (< 10K followers)