Managed Services vs Self-Hosted: When to Hand Over the Keys

☁ Cloud Without the Chaos — Article 02

You've placed the workload in the cloud. That decision is done. Now comes the one nobody gives you a clean framework for: do you run the infrastructure yourself, or do you hand the keys to a managed service?

I worked with a customer who answered this the hard way — twice. Once on their own platform, and once after they'd already made the switch. Both incidents involved the same underlying failure mode. Neither one turned out the way anyone in the room expected.

The warehouse analogy

In supply chain, this decision has a name: make vs. buy. Do you run your own warehouse, or use a third-party logistics provider (3PL)?

Running your own warehouse means you control everything — layout, staffing, hours, how fast you respond when a shipment needs to move at 2am. It also means you own every forklift breakdown and every understaffed shift.

A 3PL means someone else owns the building, the labor, the equipment. You focus on what you sell instead of how it's stored and moved. But you're also dependent on their systems, their SLAs, and their priorities when your shipment competes with someone else's for dock space.

Cloud infrastructure is the same trade in different clothes. Self-hosted is your own warehouse. Managed platforms are the 3PL. What follows is what that trade actually looked like for one team, in production, with real numbers attached.

The renewal that forced the question

The customer had a lean platform team self-managing Kubernetes — patching, logging, monitoring, upgrades, all of it in-house. Their platform renewal was coming up, expensive, and stacked against a team that was already stretched. The maintenance burden that made the renewal conversation painful had a specific shape: every major Kubernetes upgrade meant validating deprecated and removed APIs across their entire application estate, inside a fixed window, before the old control plane version went out of support.

That window was 30 days. The estate was roughly 1,400 microservices across 120+ namespaces. And on one upgrade cycle, something got missed.

The incident that made the case for managed

During a post-upgrade hotfix rollout, a Helm chart still referencing a removed Ingress API version hit the CD pipeline. The deployment failed. So the team did what you're trained to do — they rolled back.

The rollback failed too, and it failed for a specific, mechanical reason worth understanding rather than waving past. Helm doesn't recompute a prior release from source when you roll back — it retrieves the immutable historical manifest stored in a cluster Secret and re-applies it, which means that manifest still has to pass live API discovery validation on the way back in. Kubernetes 1.22 had removed the v1beta1 Ingress API group entirely. The API server had no way to accept a manifest written against an API version that no longer existed on the cluster. Helm halted before applying anything.

With the rollback path closed, the Ingress controller hit a state-sync failure. Payments traffic started returning 502s. Engineers resolved it by hand — removing orphaned resources, rewriting the manifests to the current API schema, and re-running the deployment.

Forty-five minutes of active, customer-facing outage on payments, before service was restored.

The team built a deprecation-scanning guardrail afterward. It was the right fix for the immediate problem. But it didn't touch the actual bottleneck, which wasn't technical — it was a small team with too many competing priorities trying to own an entire platform lifecycle by hand. That's what pushed the renewal conversation toward evaluating ROSA HCP and EKS as replacements, rather than just renewing what they had.

Why ROSA HCP over EKS

The deciding factor wasn't a feature checklist. It was a question of who owns which lifecycle. EKS would have meant the customer still assembling and operating most of the surrounding components themselves. ROSA HCP's integrated stack meant Red Hat owned Day 2 operations on the platform — the control plane, the surrounding components, the patching cadence — while the customer kept ownership of the application layer. For a team whose whole problem was "we can't keep operating all of this ourselves," that division of labor was the actual point — the same operational-ownership calculus that plays out anywhere a team decides what to stop managing directly.

The surprise: managed doesn't mean hands-off

Here's where the assumption cracked. The team expected that ROSA's managed Day 2 operations meant node draining, cordoning, and worker upgrades would happen the way a serverless abstraction behaves — invisibly, without touching running workloads.

That's not how it works. On ROSA HCP, control plane upgrades are decoupled and handled automatically by Red Hat. Worker node pool upgrades are not automatic in the same way — they require customer initiation, and once initiated, they're still governed by whatever workload configuration is sitting on top of those nodes.

A critical CVE forced the question. Red Hat's SREs patched the managed control plane within 24 hours of disclosure — the response time this engagement actually observed, not a contractual SLA figure. When the customer then initiated the worker pool upgrade, ROSA's automated drain process ran into a misconfigured PodDisruptionBudget (PDB) on a legacy service. The PDB was blocking pod eviction, so the node drain stalled.

The contrast that matters

This is the moment worth sitting with, because it's the clearest evidence for how this trade actually plays out.

On the self-hosted side, the team discovered their outage the way most self-hosted teams discover outages: customers saw 502s first. Forty-five minutes of active, visible damage before anyone had it under control.

On ROSA HCP, Red Hat's SRE automation caught the stalled drain and alerted the customer's team directly — before any workload was affected. The Machine Config Operator's drain process retried against the exponential backoff pattern it's designed to use rather than failing outright, so once the customer's application team fixed their own PDB configuration, the rollout resumed and completed on its own. No support ticket. No manual cluster-operator intervention from Red Hat.

That episode took 2 to 2.5 hours from alert to resolution — longer than the payments outage. But customer-facing impact was zero, because the misconfigured PDB had done its one useful job: it prevented running pods from being prematurely terminated while the drain was stuck.

Managed didn't mean faster. It meant the failure mode moved from "customer-facing outage you discover yourself" to "silent stall you get alerted to before anyone outside notices."

The exposure window nobody puts in the vendor deck

The CVE episode had a second layer worth naming plainly, because it's the part every "just move to managed" pitch tends to skip.

Red Hat's SREs patched the control plane in 24 hours. But because worker upgrades required customer initiation, and the customer held off starting that upgrade until their standard maintenance window — partly out of exactly the fear that then materialized as the PDB stall — the worker nodes running actual application workloads stayed unpatched for roughly four days. The 2.5-hour PDB delay was a small piece of that gap. The bigger piece was hesitation.

Managed infrastructure narrowed the attack surface. It didn't eliminate the scheduling risk sitting on the customer's side of the line.

What actually closes the gap

The fix wasn't anything Red Hat needed to change. It was proactive PodDisruptionBudget hygiene at the manifest level, applied consistently, so that node pool upgrades stop being a feared, manually-delayed event and become a routine background operation the team can trust. Once that trust exists, there's no reason to hold a CVE-driven worker upgrade for a scheduled maintenance window. The team's target shifted from a hesitant four-day window down to an automated 24-hour response following any control-plane security release — closing the exposure gap by removing the fear that created it.

Where this breaks at scale

At 120+ namespaces and 1,400+ microservices, PDB hygiene enforced only in CI/CD isn't enough. Pipeline linting has a blind spot: it can't see PodDisruptionBudgets generated at runtime by Operators, bundled inside third-party Helm charts, or applied directly via kubectl outside any pipeline at all. A policy that only lives in CI will pass clean while the cluster quietly accumulates exactly the kind of misconfiguration that stalled the worker upgrade in the first place. State drift on managed platforms follows the same pattern — the enforcement point that looked sufficient in isolation turns out to have blind spots once the estate gets large enough.

The customer's answer was to move enforcement to the cluster's front door — deploying Kyverno as a ValidatingWebhookConfiguration at the Kubernetes API server itself, so no PDB, regardless of where it came from, gets accepted onto the cluster without passing policy. It's deployed today. It hasn't yet been tested against a live attempted violation in production, so I'd call it the right architecture rather than a proven one — worth watching, not yet worth treating as settled.

The takeaway that doesn't show up in vendor documentation

The lesson here isn't "avoid managed services," and it isn't "always choose managed services" either. It's this: handing off infrastructure management doesn't reduce your governance burden. It relocates it, and intensifies it, at the API layer.

Once a vendor owns everything below the API — control plane, patching, node lifecycle — your job stops being "keep the cluster running." It becomes enforcing strict, API-level tenant policy so that the vendor's automation doesn't hit an invisible wall it has no way to see past. The vendor owns the warehouse. You still own what's on the shelves, and if what's on the shelves is stacked wrong, the automated forklift stops moving no matter how good the warehouse is.

A framework, not a rule

Three questions do most of the real work in this decision:

1. Is the scarce resource time, or is it control?
If the team is small and stretched, and the system isn't your competitive differentiator, lean managed. If it's core to what makes your product actually work, the operational knowledge from running it yourself has value beyond uptime.

2. Can you operate it today, under pressure — not eventually, today?
If not, that's not an argument for self-hosting. It's a sign you need training or headcount before this decision is even yours to make cleanly.

3. When it breaks, whose policy is actually holding it together?
Managed services don't remove your governance obligations. They just move the point of failure from "the platform" to "whatever policy your team enforces on top of it." Know which one you're actually testing before you sign the renewal.

None of these have a universal answer. They have an answer for your team, this quarter, with the staffing and policy discipline you actually have — not the version you're hoping to build eventually.

---

GitHub Repo

No companion repository for this article — this is a decision-framework piece about managed vs. self-hosted infrastructure, not an implementation walkthrough. Article 3 in this series returns to concrete implementation, where API-level governance becomes its own architecture decision.

What's Next

Cloud Without the Chaos, Article 03 picks up where this one leaves off: once you've handed the keys to a managed platform and enforced policy at the API layer, connectivity becomes the next architecture decision — VPNs, Direct Connect, ExpressRoute, and what actually breaks when two environments have to talk to each other reliably.