HardData Engineering
Data Pipeline Orchestrator
Design a data pipeline orchestrator with dependency management, retry logic, and data quality checks.
Constraints
Support 1000+ daily jobs. Handle backfills efficiently.
Examples & Scenarios
Design a data pipeline orchestrator with dependency management, retry logic, and data quality checks.
Support 1000+ daily jobs. Handle backfills efficiently.
This is an Arena challenge - solutions are not shown publicly.
Work on this problem and submit your solution to contribute!