Email 1 of 5
How pipelines multiply
Nobody plans for two hundred pipelines. Everybody ends up there.
No data team plans for two hundred pipelines. They plan for ten — and then a stakeholder needs a join the warehouse doesn't have, a deadline arrives, and the eleventh pipeline gets built point-to-point because it's Tuesday and the dashboard ships Thursday.
Repeat that for three years and you have a graph nobody can draw. Pipelines that feed pipelines. Two transformations computing 'revenue' with definitions that diverge in the third decimal. A nightly job that exists only because deleting it is scarier than paying for it.
The sprawl isn't a failure of discipline. It's the rational result of optimizing each pipeline locally and the system never.
The way out isn't a heroic rewrite. It's changing what's cheap — which we'll get to later this week.
Tomorrow: the failure mode that turns sprawl from expensive into dangerous.