HARD
Design a Metrics Collection System
Design a metrics collection and monitoring system like Datadog or Prometheus handling millions of time-series data points.
Estimated Time: 45 minutes
#Time Series#Aggregation#Alerting#Visualization
Solution Overview
Use time-series database optimized for write-heavy workloads. Implement pre-aggregation for common queries. Support flexible alerting rules.
Solution Overview
Use time-series database optimized for write-heavy workloads. Implement pre-aggregation for common queries. Support flexible alerting rules.
Hints to Get Started
1
Time-series compression techniques
2
Cardinality explosion handling
3
Push vs pull model
Storage
- •Hot storage (recent)
- •Warm storage (aggregated)
- •Cold storage (raw archive)
Data Model
metric_name{labels} value timestamp
Query Patterns
- •Range queries
- •Aggregations
- •Top-K