Data platform teams are becoming exception handlers

An r/dataengineering thread about losing interest in the field described a data platform engineer who knew the full route from source systems to business value. They maintained Snowflake, dbt, Airflow, ingestion code, an internal library, and a collection of BI tools. They were also pulled into dbt failures, model performance problems, and Snowflake troubleshooting.

That breadth sounds valuable. It is also how a platform engineer becomes the human exception handler for an entire data organization.

Every problem that does not have an obvious owner lands with the person who understands the whole system. They fix the pipeline, explain the metric, repair the framework, and help the next team use it. Because they solve the problem, the organization sends them the next one.

The platform becomes more important while its engineers have less time to improve it.

I would treat this as an architecture problem. Team capacity, support boundaries, and ownership are part of the platform design.

The platform sits at every boundary

A data platform connects systems that are owned by different groups:

source application
        ↓
ingestion service
        ↓
storage and processing
        ↓
domain models
        ↓
BI, ML, applications, and agents

When a number is wrong, any arrow could be responsible. The source may have changed its deletion behavior. Ingestion may have skipped a partition. A transformation may have duplicated records. A dashboard may apply another filter. The platform team has visibility across the route, so asking it for help is reasonable.

The problem begins when visibility is confused with ownership.

The platform team should own the shared capabilities it provides: execution environments, deployment paths, observability conventions, access patterns, templates, and supported interfaces. It should not silently inherit every pipeline's business rules, every source system's semantics, and every consumer's interpretation.

Without that distinction, the platform is a shared service in the diagram and a catch-all team in practice.

Support work has to return information to the platform

Google's SRE guidance calls repetitive, manual, service-related work that scales with growth toil. It also warns that excessive toil slows engineering progress, confuses the team's role, and can drive experienced engineers away.

A support request is not automatically toil. Diagnosing a new failure mode can teach the team something important. Helping an early adopter can reveal a missing interface. A production incident may deserve immediate collaboration across every boundary.

The work becomes toil when the same class of issue returns without improving the system.

I would require every recurring support case to take one of four exits:

Exit	Example
Platform fix	The shared retry behavior is wrong
Paved-road improvement	A template needs a default freshness check
Owner runbook	The domain team needs a documented backfill procedure
Explicit exception	A temporary legacy workflow needs named support and an end date

Closing a ticket is not enough. The ticket should leave behind a safer interface, better diagnostic evidence, clearer ownership, or a conscious decision to keep paying the support cost.

Google's chapter on operational overload recommends identifying the sources of stress, separating avoidable toil from legitimate operational work, and using service-level objectives to restore focus. The same approach works for a data platform. Count the interruptions. Group them by cause. Find the few categories consuming most of the time.

The first useful dashboard may be about the platform team rather than the platform:

support requests by producing team
repeated requests by failure class
hours spent on unplanned work
requests missing an owner or runbook
issues converted into platform improvements

If the team cannot see its operational load, it will plan as though every engineer is available for roadmap work. The roadmap will slip, support will feel like an interruption, and the people doing both will carry the discrepancy.

Define a support contract before adding a portal

Platform engineering is often presented as a self-service portal. The portal is the visible part. The important part is the contract behind it.

The CNCF Platforms White Paper describes an internal platform as a collection of capabilities shaped around user needs, with consistent experiences such as APIs, templates, documentation, and support. It also emphasizes that platform teams are responsible for the interfaces and user experience, while the underlying capability may be run by another provider or team.

That separation is useful for data platforms. A support contract can define three areas.

What the platform owns

• the supported pipeline runtime and deployment path
• authentication and access interfaces
• shared monitoring and lineage integration
• platform upgrades and compatibility windows
• documented recovery controls offered by the platform
• defects in templates, libraries, and managed services the team publishes

What the workload owner owns

• source and business semantics
• transformation correctness
• data-quality rules for its domain
• schema-change coordination with consumers
• backfill approval and validation
• first response to alerts for the workload

What requires collaboration

• failures that cross several platform services
• incidents where the cause is still unknown
• capacity changes that affect shared infrastructure
• migrations that change a supported interface
• security or data-loss events

The collaborative area matters. A boundary should route work, not provide an excuse to throw a ticket back over the wall.

The contract should also name service levels. A production outage, a failing scheduled pipeline, a design consultation, and a request for a new connector do not need the same response time. When everything arrives through a direct message marked urgent, the platform team cannot make a sensible trade-off.

Require evidence at the support boundary

Many data incidents begin with a sentence like "the numbers look wrong." That is a valid symptom, but it is not enough information for efficient triage.

The support interface should collect a small diagnostic envelope:

affected dataset or job
business impact
first known bad time or partition
expected result and observed result
recent deployment or source change
run identifier and relevant logs
checks already performed
named workload owner

This is not bureaucracy for its own sake. It prevents the platform engineer from spending the first hour discovering which table, date, environment, and dashboard the request refers to.

The platform should make the evidence easy to obtain. A job page can expose the run identifier, input versions, row counts, quality results, owner, and runbook. A template can register the support channel and escalation policy when a new pipeline is created. An alert can link directly to the failed partition and recovery action.

Self-service is credible when it includes failure, not only creation.

The happy path may let a team create a pipeline in five minutes. If recovery still requires finding the one platform engineer who remembers a private command, the platform has automated provisioning while preserving dependency.

Route data meaning to the people who know it

Central platform teams often accumulate business logic because they are the only team with the skills or access to change the pipeline. That may be necessary during an early migration. It should not become invisible permanent ownership.

Google Cloud's overview of data mesh describes domain-oriented ownership and a self-service data platform as separate principles. The useful part is not the label. It is the recognition that the people closest to a domain understand its semantics, while a shared platform can remove the infrastructure burden.

Distributed ownership still costs something. Domain teams need training, access, time, and an escalation path. Giving a team a repository and declaring it the owner does not create operational capability.

I would transfer ownership in stages:

1. The platform team operates the workflow while documenting failure modes.
2. The domain team joins incident review and validates business outcomes.
3. The platform provides a runbook, dashboards, and a safe recovery action.
4. The domain team becomes first responder for its workload.
5. The platform remains responsible for shared-service defects and difficult cross-boundary failures.

This is slower than changing an owner field. It is also much more likely to survive the first incident.

For critical datasets, record ownership in the same place engineers see operational state. A catalog entry that names a business owner but does not connect to the runbook, alert route, and deployment repository is useful documentation, but it is not an operating model.

Protect enough capacity to improve the platform

Google's SRE model uses a 50% upper bound for operational work as a forcing function, not as a universal staffing formula. A data platform team can choose a different threshold. The important part is to choose and measure one.

If unplanned support consumes most of the team's week, it cannot build the self-service features expected to reduce support. That feedback loop is the dangerous part:

more unsupported workloads
        ↓
more manual help
        ↓
less platform engineering
        ↓
more missing capabilities
        ↓
more manual help

Breaking the loop may require temporarily slowing onboarding. It may require a support rotation so interruptions do not reach every engineer. It may require deleting a rarely used feature whose maintenance cost exceeds its value. It may require one quarter focused on observability and recovery rather than another set of integrations.

I would reserve capacity for the top recurring sources of support and review the balance each month. If the limit is exceeded, leadership should decide what work stops. The answer cannot be an invisible extension of the team's working day.

The platform should remove dependency, not collect it

A strong platform team will still be asked difficult questions. That is part of serving many workloads across shared infrastructure. The goal is not zero support.

The goal is for each support interaction to strengthen the system. A repeated request becomes an API, a template, an alert, a runbook, a training session, or a clearer boundary. Workload owners gain enough context to handle ordinary failures. Platform engineers keep responsibility for the capabilities only they can change.

For the engineer in the Reddit discussion, knowing the entire data route was an asset. The operating model turned that asset into a dependency.

I would fix the return path. Name who owns data meaning, workload operation, and shared infrastructure. Collect evidence at the support boundary. Measure recurring interruption. Protect enough engineering capacity to remove the next class of tickets.

A platform is successful when capable teams can use it without borrowing one of its engineers for every exception.