If you are trying to decide whether it is finally time to run a cluster, the honest answer is that request volume is almost never the trigger. The triggers are structural: per-tenant isolation you cannot express in your database, workloads you do not trust, a compliance boundary that requires network-level segmentation with an audit trail, or a pile of homegrown scripts that have quietly become a bad orchestrator. Before any of those, you should be able to prove you have run out of headroom on the managed platform you already pay for.
A reader on an earlier post about boring stacks made the sharpest version of this point: the real failure mode that pushes teams to Kubernetes is not high traffic, it is complex multi-tenant stateful workloads and compliance boundaries that a managed platform cannot express. They then asked the question this post exists to answer — what specific metrics and team thresholds should sit in the decision table. Here is the one I use.
Why is "we're getting too much traffic" almost never the real trigger?
Because managed platforms scale vertically and horizontally long past where most teams assume they stop. A single large instance running a well-tuned application server handles a volume of traffic that surprises people who have only ever read about scale, and every managed runtime worth using will run several identical copies of your container behind a load balancer without you learning a new config language.
Before you accept "we need to scale" as an argument, measure your actual headroom. The database is usually the real ceiling, and it is the one people misread most often — an app that falls over at 300 concurrent requests is frequently exhausting Postgres connections, not CPU:
SELECT
(SELECT setting::int FROM pg_settings WHERE name = 'max_connections') AS max_connections,
count(*) AS in_use,
round(
100.0 * count(*) / (SELECT setting::int FROM pg_settings WHERE name = 'max_connections'),
1
) AS pct_used
FROM pg_stat_activity;
If that comes back at 95% while your instance CPU sits at 20%, Kubernetes solves nothing for you. A connection pooler and a smaller per-process pool size solve it in an afternoon. The same applies to the app tier: if your peak CPU utilization never crosses 40% on the instance size you are paying for, you do not have a scaling problem, you have a latency or concurrency problem hiding behind one.
If you cannot point at a specific resource you have saturated after tuning, you do not have a scaling argument — you have a discomfort.
What are the structural signals that actually justify a cluster?
These are the four I treat as genuine. Each one has a test you can apply today, and each one has a boring-stack workaround you should exhaust first.
| Signal | How to test it | Try this first | If that fails |
|---|---|---|---|
| Per-tenant stateful isolation | Can one tenant's data or load reach another's, in a way a customer contract forbids? | Row-level security, or a schema/database per tenant on managed Postgres | Per-tenant namespaces with resource quotas |
| Untrusted or arbitrary workloads | Are you executing code you did not write — customer plugins, build jobs, notebooks? | Managed sandbox/build services, hard per-job timeouts | Real pod-level isolation and admission control |
| Compliance boundary | Does an auditor need to see enforced network segmentation and deploy provenance? | Separate managed environments per boundary, VPC peering rules | Network policies, RBAC, signed image admission |
| Deploy coordination | Count the lines of custom bash that restart, health-check, and roll out services | A managed runtime with health checks and rolling deploys | Declarative orchestration |
The multi-tenancy row is where most teams misdiagnose themselves, so it deserves the concrete version. "Multi-tenant stateful workload" usually means shared rows in shared tables, and that is a database problem with a database answer:
ALTER TABLE invoices ENABLE ROW LEVEL SECURITY;
ALTER TABLE invoices FORCE ROW LEVEL SECURITY;
CREATE POLICY tenant_isolation ON invoices
USING (tenant_id = current_setting('app.tenant_id')::uuid);
Then set the tenant once per transaction, from your connection checkout or request middleware:
BEGIN;
SET LOCAL app.tenant_id = '3f1c2a9e-0b47-4f8a-9d21-5c6e8f0a1b23';
SELECT * FROM invoices WHERE status = 'open';
COMMIT;
FORCE ROW LEVEL SECURITY matters because policies are skipped for the table owner by default, which is exactly the role most applications connect as — that omission is the single most common way an RLS setup looks correct and enforces nothing. SET LOCAL scopes the setting to the transaction so a pooled connection cannot leak one tenant's context into the next request.
If that satisfies your isolation requirement, you did not need an orchestrator, you needed twelve lines of SQL. If your requirement is instead "tenant A's batch job must never consume tenant B's CPU," you have a genuine scheduling problem and the case for Kubernetes gets real.
Isolation requirements that live in your data model are database work; isolation requirements that live in the kernel are orchestrator work.
What team size can actually carry a cluster?
The number I use is not a headcount, it is an ownership commitment: someone whose job description includes the cluster, with roughly half a day a week of unglamorous maintenance budgeted, and at least one other person who can do a node upgrade when that person is on vacation. On a team of five engineers that is 10% of your engineering capacity gone, permanently, to something your customers never see.
The recurring work is real and it does not go away with a managed control plane: node group upgrades on the provider's deprecation schedule, ingress controller and cert-manager version bumps, CNI plugin compatibility, RBAC drift, and the periodic afternoon spent discovering that a pod is pending because of a resource request nobody remembers writing. Managed offerings like EKS and GKE remove the control plane from your worry list and genuinely lower the floor, but the worker nodes, networking, and deploy pipeline are still yours.
There is also a middle tier that most decision tables skip entirely. If your only real need is "run these containers, restart them when they die, roll them out without downtime," AWS Fargate gives you container scheduling with no nodes to patch, at the cost of slower cold starts and a weaker local development story. If your workloads are HTTP services that can tolerate scale-to-zero, Google Cloud Run handles request-driven autoscaling and TLS termination without exposing you to any cluster concepts at all, though it constrains you to its request lifecycle. If you want orchestration primitives without the Kubernetes ecosystem surface area, HashiCorp Nomad schedules containers and plain binaries with a config format a new engineer can read in an afternoon, with the tradeoff of a much smaller community and fewer off-the-shelf integrations.
Adopt Kubernetes when you need what only Kubernetes gives you, not when you need what any scheduler gives you.
How do you make the switch without a big-bang migration?
Move one stateless service first, and pick the least important one you have — an internal admin tool, a metrics exporter, a webhook receiver. It should be something whose outage costs you an apology, not revenue. That first workload is how you discover the parts nobody writes tickets for: image pull secrets, DNS resolution inside the cluster, log shipping, and how you actually get a shell when something breaks.
Keep state outside the cluster for as long as you can. Managed Postgres, managed object storage, and managed queues stay exactly where they are; running your own database inside Kubernetes is a separate project with its own operator, backup, and failover story, and taking both on at once is how migrations stall for a quarter.
Set an explicit rollback condition before you start — something like "if the admin tool is not stable on the cluster within three weeks, it goes back." Migrations without a stated failure condition tend to continue on sunk cost alone.
Your first cluster workload should be chosen for how little it matters, not how well it demonstrates the platform.
FAQ
How many services do you need before Kubernetes is worth it?
There is no clean count, but the useful proxy is deploy coordination, not service count. If you are running fewer than roughly five services and your deploys are independent, a managed runtime handles it; once services must be rolled out in a specific order, share service discovery, and are held together by custom scripts you are afraid to edit, the orchestrator is doing work you are currently doing by hand.
Is Kubernetes required for SOC 2 or HIPAA compliance?
No. Neither framework names any orchestrator. What auditors ask for is enforced access control, network segmentation between environments, encryption, and evidence of change management — all of which managed platforms can satisfy with separate environments and provider-level controls. Kubernetes becomes relevant when your segmentation requirements are finer-grained than the boundaries your platform lets you draw.
Can you run a multi-tenant SaaS without Kubernetes?
Yes, and most do. Tenant isolation at the data layer with row-level security or a database per tenant covers the majority of contractual requirements. You need kernel-level isolation only when tenants can trigger workloads that compete for CPU and memory, or when you execute code the tenant supplied.
Bottom line
If you are on a managed platform and shopping for a reason to leave it, measure your headroom first — connection saturation, CPU utilization at peak, and how much of your latency is your own code. Adopt Kubernetes when you hit a structural wall: untrusted workloads, isolation that must be enforced below your application, or an audit boundary your platform cannot draw. If you need scheduling but not the full ecosystem, Fargate, Cloud Run, and Nomad are the honest middle, and each costs you flexibility in exchange for the operational load it removes. And if nobody on your team can name the person who owns node upgrades six months from now, the answer for today is still no.
Top comments (0)