Why Data Engineering Is the Oldest Profession in Infrastructure

Not that one. The other one: keeping water flowing to people who need it.

The comparison between data engineers and plumbers gets made a lot, usually as a light joke about unglamorous work. But if you actually follow the metaphor further than the punchline, it stops being a joke and starts being a pretty useful mental model for why data systems are built the way they are.

Aqueducts Were Just Early DAGs

Roman engineers did not build a single pipe from mountain spring to city fountain. They built a network of channels, settling tanks, distribution castles (castella divisiorum), and branch lines, each one flowing in one direction, each one handling a specific stage of the journey from source to consumer.

That is a directed acyclic graph. The Romans did not call it that, but the engineering constraint is identical: water flows downhill, and you cannot have it flowing in a circle without losing pressure and introducing contamination. Data engineers rediscovered this constraint independently when they started building ETL pipelines at scale. You route, filter, transform, and deliver. Backflow is a bug.

The specialist tooling maps too. Roman aqueduct crews had lead workers, stone cutters, surveyors, and hydraulic engineers. A modern data platform team has people who work exclusively on ingestion, on transformation logic, on orchestration, on the serving layer. Nobody does all of it well, and the people who try to usually end up with a system that leaks.

What "The Plumber" Got Right in Scala

There is a Scala series called "Data Plumber" that uses this metaphor not as decoration but as actual architecture. The design choices it makes are worth looking at directly.

The central idea is that a pipeline should be resumable. A plumber who installs a pipe does not assume the building will never need maintenance. They put in access panels, cleanouts, and shutoff valves. The Scala equivalent is offset tracking: storing where you are in a stream so that when a process dies (and it will), you can restart from a known position instead of from the beginning.

def processWithOffset[F[_]: Async](
  stream: Stream[F, Record],
  saveOffset: Offset => F[Unit],
  loadOffset: F[Option[Offset]]
): Stream[F, Unit] =
  Stream.eval(loadOffset).flatMap { maybeOffset =>
    val resumed = maybeOffset.fold(stream)(stream.drop(_))
    resumed.evalMap { record =>
      processRecord(record) >> saveOffset(record.offset)
    }
  }

That pattern, resuming from a checkpoint rather than replaying from the top, is not a Scala-specific insight. It is what every serious streaming system does, whether you are using FS2, Kafka Streams, or Flink. The plumber metaphor earns its keep here because it makes the requirement obvious: a pipe that can only ever be installed once, never inspected, and never repaired is not infrastructure. It is a liability.

Continuous streaming with FS2 reinforces the same principle. Water does not arrive in batches. Neither does most real-world data, and designing a system that pretends it does will eventually force you to either introduce latency or build a layer that converts streams into fake batches and then back again. Better to model the stream as a stream from the start.

The Part Nobody Puts in the Job Posting

Here is the thing about plumbers that the metaphor usually skips: most of their actual work is not greenfield installation. It is diagnosing someone else's system, often a system with no documentation, built across three different decades, with fittings that are technically compatible but practically a nightmare.

Data engineering is the same. The exciting job description talks about building real-time pipelines and designing event-driven architectures. The actual job, especially in the first year at any established company, is figuring out why a Spark job that ran fine for eighteen months started OOMing last Tuesday, or why a column that was supposed to be a timestamp is sometimes a Unix epoch in milliseconds and sometimes a string formatted as "DD-MM-YYYY HH24:MI:SS" depending on which upstream team wrote the row.

Senior plumbers know this before they take a job. They ask to see the building plans and they know that if no building plans exist, the estimate goes up significantly. Data engineers are slowly learning to do the same thing: asking for data contracts, lineage documentation, and schema registries before committing to delivery timelines.

Documentation as building plans is the part of the plumber metaphor that data engineering has adopted the most slowly and has paid for the most dearly. A plumber who does not leave a diagram of where the pipes run behind the wall is creating future problems for every person who works on that building afterward. A data engineer who does not document what a pipeline does, where it reads from, and what transformations it applies is doing the same thing, just with consequences measured in incorrect dashboards and misfired marketing emails instead of flooded kitchens.

The Takeaway

The plumber comparison is not a consolation prize for unsexy work. It is an accurate description of a discipline that has real engineering depth, a long historical lineage, and a clear set of failure modes that come from ignoring its own principles.

One-directional flow, specialist tooling at scale, resumable processes, and documentation that survives the original builder: these are not best practices invented by some framework. They are the same requirements that kept Roman cities from running out of water, and they are still the requirements today. The medium changed. The problem did not.