Your Kubernetes Cluster Is Not Overprovisioned by Accident, It Is Overprovisioned by Incentive

A Kubernetes fleet running at roughly 25 percent CPU utilization may look like a straightforward technical optimization problem. Lower inflated resource requests, improve bin packing, remove excess nodes, and reduce the bill.

The discussion showed why that almost never happens so cleanly.

Every application team padded requests to feel safe. Those values stayed untouched for a year. The platform team could see the waste, but changing another team’s configuration created an asymmetric risk: if the change saved money, the result was expected; if any latency incident happened later, the person who changed the request became responsible.

That is the real blocker.

The cluster was not simply overprovisioned. It was governed by a system where the cost was shared, the risk was personal, and nobody received meaningful credit for fixing it.

Tools can identify waste. They cannot repair that incentive model.

The hardest Kubernetes problem is often political

The highest-supported response in the discussion was blunt: the hardest part is not the technology, it is the politics.

That is accurate because the technical path is well understood.

Teams can collect historical usage, compare requests with real demand, adjust values gradually, and observe the result. Platform engineers can use VPA recommendations, Goldilocks, KRR, cost dashboards, or internal scripts. Kubernetes already provides the scheduler, autoscalers, and metrics needed to act.

The missing component is ownership.

The platform team sees a cluster-wide problem, but every application team sees a local risk. Lowering one service’s request may save only a small amount for that team. The engineer doing the work must coordinate, test, review, deploy, monitor, and potentially explain any incident.

The organization receives the aggregated savings. The individual team receives the operational burden.

This creates rational inaction.

A platform team may identify hundreds of small opportunities, but each application team has higher-priority product work. A saving of less than $50 per month may not justify an engineering meeting. Across dozens or hundreds of workloads, however, those ignored savings become a large infrastructure bill.

The economics work at platform scale, while the effort is assigned at team scale.

That mismatch must be corrected before rightsizing becomes routine.

Requests and limits are different risk conversations

One of the most useful technical points in the discussion was to separate CPU requests from CPU limits.

Requests influence scheduling and bin packing.

Limits influence how much CPU a container can consume and whether it experiences throttling.

These settings are often configured together, sometimes with the same value, and then discussed as if changing either one creates the same risk. That confusion makes rightsizing harder.

Lowering an inflated CPU request does not directly cap the application’s CPU usage. It tells the scheduler that the pod needs less guaranteed capacity when deciding where to place it.

If the node has spare CPU and there is no restrictive CPU limit, the workload can still use more CPU than its request.

This is why several commenters argued that teams could reclaim scheduling capacity without touching limits. That removes much of the latency fear associated with changing CPU constraints.

The risk is not zero.

Requests affect placement and the amount of capacity Kubernetes considers committed. If they are lowered too aggressively, the scheduler may pack more workloads onto a node. Under contention, pods with lower requests may receive a smaller proportional share of CPU.

Still, that is a different failure mode from CPU throttling caused by a limit.

A sensible rightsizing program should treat these as separate phases:

• Adjust requests to improve scheduling and bin packing
• Evaluate limits independently using throttling and latency data
• Avoid changing both at once unless the workload has been tested
• Preserve rollback values in version control
• Monitor node-level contention after packing density increases

This makes the change easier to reason about and easier to defend.

Twenty-five percent CPU does not prove the fleet is wasteful

Some participants challenged the assumption that 25 percent CPU utilization automatically means the cluster is overprovisioned.

That objection matters.

CPU may not be the resource driving node provisioning. Memory, local storage, pod density, network requirements, GPU capacity, topology constraints, architecture, or DaemonSet overhead may determine the node count.

A cluster can show low CPU usage while being tightly constrained elsewhere.

Burstable instance families also complicate the interpretation. A team using credit-based virtual machines may intentionally avoid sustained high CPU utilization.

Time aggregation matters too.

A fleet-wide average can hide short peaks, periodic batch jobs, weekly traffic events, failover requirements, or concentrated node-level pressure. A cluster may appear idle overall while several nodes operate close to saturation.

The correct question is not simply, “What is average CPU utilization?”

It is:

• Which resource caused each node to exist?
• What are p95 and p99 workload peaks?
• How much requested CPU is stranded by placement constraints?
• Are memory requests or usage the real bottleneck?
• How much capacity is reserved for failure and traffic spikes?
• Are certain instance types poorly matched to the workload shape?
• Can the autoscaler actually remove nodes after requests are reduced?

Low utilization is a signal to investigate. It is not proof that every request can be cut safely.

Historical data must cover a real traffic cycle

Several commenters recommended basing changes on actual usage over at least a meaningful workload cycle.

Two weeks of metrics may be enough for a stable service with regular traffic. It may be inadequate for monthly reporting jobs, seasonal workloads, backups, billing cycles, or unpredictable customer events.

A platform team should collect data that includes the relevant peaks.

That can include:

• CPU and memory usage percentiles
• Request-to-usage ratios
• Throttling metrics
• OOM events
• Application latency
• Queue depth
• Error rates
• Replica count changes
• Batch schedules
• Business traffic cycles

The discussion mentioned using p95 usage as a basis for lowering requests. That can be a practical starting point, but it should not be treated as a universal formula.

Some workloads need burst headroom. Others can rely on horizontal scaling. Some are latency sensitive. Some are asynchronous and tolerate slower completion.

Rightsizing should follow workload behavior, not one cluster-wide percentage.

The value of tools such as KRR, Goldilocks, or VPA is that they turn historical usage into recommendations. The limitation is that a recommendation still needs context.

VPA is useful evidence, not an automatic truth machine

The discussion included both support for VPA-style tooling and skepticism.

Some teams used Goldilocks to generate VPA resources and reported substantial improvements in utilization and cluster reduction. Others described VPA algorithms as naive and warned against treating them as a silver bullet.

Both views can be true.

VPA can identify workloads whose requests are far above observed usage. It can reduce the manual effort required to inspect hundreds of Deployments and CronJobs. In recommendation mode, it gives teams a concrete proposed value rather than another graph showing that waste exists.

But observed usage is not the same as required capacity.

A recommendation may miss rare peaks, cold-start behavior, dependency failures, future growth, or load patterns not present during the observation window. Automatic updates can also restart pods depending on the operating mode and version.

The safest adoption path is usually staged:

• Start with recommendations only
• Compare recommendations against known workload events
• Pilot on low-risk namespaces
• Change requests before limits
• Add clear rollback paths
• Monitor service-level indicators
• Move toward automation only after confidence is established

Tools remove analysis work. They do not remove accountability.

Node consolidation is where savings become real

One of the strongest technical corrections in the discussion was that lowering requests does not automatically lower the bill.

It only creates more schedulable space.

The organization saves money when workloads are repacked and empty nodes are terminated.

If Karpenter, Cluster Autoscaler, or another node-management system cannot consolidate the fleet, the requests may look cleaner while the same instances continue running.

This is where many cost dashboards stop too early. They calculate theoretical reclaimable resources but do not show whether the infrastructure can actually disappear.

A single blocking workload can pin an entire node.

Common blockers include:

• PodDisruptionBudgets that allow no eviction
• Do-not-evict annotations
• Local storage
• Strict affinity or anti-affinity
• Topology spread requirements
• Large pods that fit only on certain nodes
• DaemonSets
• Node selectors
• Taints and tolerations
• Minimum replica requirements
• Unavailable replacement capacity

The source discussion specifically raised the problem of a PDB or unevictable pod preventing Karpenter consolidation.

This is the distinction between efficiency and savings.

Rightsizing improves scheduler efficiency. Consolidation turns that efficiency into fewer billed nodes.

Both must be measured.

The descheduler can unlock stranded capacity

One operator managing around 15 clusters said the descheduler was one of the most helpful tools for improving utilization.

That makes sense because Kubernetes scheduling is primarily placement at creation time. The default scheduler does not continuously rearrange every running pod to produce the most cost-efficient layout.

Over time, clusters fragment.

Pods scale up and down. Nodes join and leave. Requests change. Temporary constraints disappear. The fleet can end up with enough aggregate free capacity to remove nodes, but the pods are distributed in a way that prevents it.

A descheduler can evict selected pods so they are placed again under current conditions.

This is especially valuable after rightsizing.

Lower requests create new packing possibilities, but existing pods may remain on their original nodes. Without rescheduling, the autoscaler may not see a node that can be removed.

Descheduling must still respect availability controls and workload behavior. Aggressive eviction can create churn or interact badly with fragile applications.

The correct sequence is:

• Lower requests carefully
• Identify nodes that could be consolidated
• Check for eviction blockers
• Reschedule or deschedule eligible workloads
• Confirm replacement pods become healthy
• Let the node autoscaler remove empty capacity
• Verify the cloud bill actually declines

That final verification is critical.

Chargeback changes the conversation immediately

The most supported organizational solution in the discussion was direct cost allocation.

One infrastructure team rebilled cloud and Kubernetes costs to the departments responsible for the workloads. Those teams budgeted their own compute. If they overspent, it affected their budget. If they saved, they received the credit.

The commenter said it worked.

This approach fixes the incentive mismatch.

Without chargeback, an application team owns the risk of rightsizing while the infrastructure organization receives the financial benefit. With chargeback, the same team owns both sides of the decision.

That does not guarantee that every team will optimize.

Some teams may reasonably choose to pay for extra headroom. If a service is critical and the cost is acceptable, that can be a valid business decision.

The important change is that the decision becomes visible and accountable.

The discussion described a model where teams budgeted compute units, defined as a CPU and memory combination. The infrastructure team calculated the price, added a shared-services allocation for observability, runners, networking, and related platform costs, and sent actual usage to finance for rebilling.

This also prevented the infrastructure department from becoming financially responsible for application behavior it did not control.

Chargeback is not just an accounting mechanism. It is a governance model.

Showback is weaker, but still useful

Not every organization can immediately rebill internal teams.

Finance systems, budgeting processes, executive sponsorship, and organizational culture may not support it.

In that case, showback can still improve behavior.

A useful report should show:

• Cost by team, namespace, application, and environment
• Requested versus used CPU and memory
• Theoretical savings
• Realized savings
• Nodes blocked from consolidation
• Workloads causing stranded capacity
• Trend over time
• Cost per business unit
• Shared platform tax

The report must be tied to ownership.

A generic cluster dashboard reinforces the idea that waste is the platform team’s problem. A team-specific bill makes the decision local.

The source discussion also included the view that once teams see the bill, rightsizing becomes their call. Platform teams can make recommendations, but they should not silently accept responsibility for every application’s reserved capacity.

That is a healthier boundary.

Small savings should be automated, not coordinated

One team described the difficulty of asking product engineers to spend time saving less than $50 per month per workload.

That is often the correct economic judgment at the individual service level.

An engineer’s time may cost more than the annual savings. Scheduling meetings, investigating, creating a change, reviewing it, and monitoring rollout can exceed the value recovered.

At platform scale, however, the total waste can still justify investment.

The answer is not to coordinate hundreds of tiny optimization projects. It is to automate the common case.

Platform teams can:

• Generate recommendations automatically
• Open pull requests with proposed request changes
• Preserve previous values in the diff
• Apply policy defaults to new workloads
• Use VPA recommendations or updates selectively
• Prioritize high-value offenders
• Ignore low-value exceptions
• Automate reporting and rollback
• Track realized node reduction

This is exactly the kind of problem platform engineering should solve once for many teams.

Manual coordination is appropriate for critical or unusual services. Routine waste should be removed through a standardized path.

A pilot creates political cover

The most repeated practical advice was to start with one service or one low-risk namespace.

Collect real usage. Lower requests modestly. Keep limits unchanged. Compare application latency, error rates, throttling, and resource behavior before and after. Preserve the old values so rollback is a one-line revert.

This does more than test the technology.

It creates organizational evidence.

The platform team can show:

• Nothing broke
• The workload remained within its SLO
• Scheduling density improved
• Nodes were consolidated
• The bill decreased
• Rollback was simple
• The monitoring process worked

One comment suggested reducing requests by 10 percent, waiting a couple of weeks, and repeating the process.

That exact percentage and target utilization should not be treated as universal. The principle is sound: reduce risk through incremental, observable changes instead of attempting a fleet-wide correction.

A successful pilot also creates shared responsibility. The application owner, platform team, and management can agree on the experiment and its success criteria before the change.

That reduces blame asymmetry.

Load testing is the missing confidence tool

One commenter asked why load testing had barely appeared in the discussion.

That question exposes a weakness in many rightsizing programs.

Historical metrics show what happened. Load testing helps estimate what happens under controlled stress.

For important services, teams should test:

• Normal peak load
• Expected growth
• Downstream slowdown
• Reduced replica count
• Node contention
• Burst behavior
• Autoscaling lag
• CPU throttling if limits exist
• Memory pressure
• Recovery after saturation

This is especially valuable when a service has a strange request justified by an anecdote such as “some requests can fetch an unbounded amount of data.”

That may represent a real risk. It may also be a permanent allocation based on one poorly understood edge case.

A load test turns the argument into evidence.

If the service truly needs a large amount of memory or CPU under a legitimate scenario, the request can be justified. If not, the oversized value can be corrected or the application behavior can be fixed.

Rightsizing should not mean guessing lower. It should mean replacing inherited guesses with measured requirements.

Policy prevents the waste from returning

Fixing existing requests is only half the job.

New workloads will reproduce the same problem unless the platform changes its defaults and admission rules.

The discussion mentioned Kyverno policies, generated VPA resources, namespace quotas, and controls that prevent unevictable workloads.

Useful policies can:

• Require requests on production workloads
• Prevent obviously extreme request-to-limit ratios
• Flag requests far above recent recommendations
• Require ownership labels
• Restrict PDB settings that allow zero voluntary disruption
• Prevent unreviewed do-not-evict annotations
• Require approved node selectors
• Apply default VPA recommendation resources
• Enforce namespace quotas
• Require cost-center metadata

Policies should not attempt to encode every application decision.

Their purpose is to prevent known failure patterns and make ownership visible.

A policy cannot determine the correct CPU request for every service. It can prevent teams from creating workloads that are impossible to consolidate or impossible to attribute.

That is a meaningful improvement.

The business must decide whether optimization matters

Several responses took a harder position: if management does not prioritize cost reduction, the platform engineer should document the issue and move on.

That is uncomfortable but correct.

Infrastructure efficiency is not automatically the highest business priority.

A startup focused on shipping may rationally accept a larger cloud bill. A product team may choose extra capacity over engineering effort. A critical service may retain expensive headroom because the downside risk is larger than the savings.

The platform team’s job is to make the tradeoff visible.

It should present the cost, risk, savings opportunity, operational effort, and implementation path. Management must then decide whether the work is worth doing.

What the platform team should not do is quietly accept all responsibility for fixing application allocations without authority, incentives, or sponsorship.

Optimization without a business mandate becomes unpaid risk ownership.

FAQ

Does 25 percent CPU utilization prove a Kubernetes cluster is overprovisioned?

No. CPU may not be the resource driving node count. Memory, topology, instance shape, storage, pod density, and availability requirements must also be examined.

What is the difference between CPU requests and limits?

Requests influence scheduling and guaranteed CPU share under contention. Limits cap CPU consumption and can cause throttling.

Can teams lower CPU requests without changing limits?

Yes. This can improve scheduling and bin packing while avoiding the direct throttling risk associated with lowering limits.

Why might lower requests fail to reduce the cloud bill?

The cluster must consolidate and terminate nodes. PDBs, affinity rules, local storage, do-not-evict settings, and other blockers may prevent node removal.

Are VPA recommendations enough for safe rightsizing?

No. They are useful evidence based on observed usage, but they may miss rare peaks, future growth, and workload-specific behavior.

How should a rightsizing program begin?

Start with a low-risk service or namespace, use a meaningful history window, lower requests gradually, keep rollback simple, and measure application and node-level outcomes.

What is the benefit of Kubernetes chargeback?

Chargeback gives application teams ownership of both the cost and the savings, reducing the incentive mismatch that causes permanent overprovisioning.

Should every small workload be manually optimized?

No. When individual savings are small, platform automation is usually more economical than asking every team to perform manual analysis and coordination.

CTA

Kubernetes cost optimization requires visibility into requests, real usage, node consolidation, and the workloads blocking savings. Request an online trial and explore how Sensaka helps teams connect infrastructure efficiency with operational risk.