💡 Originally published on devtocash.com — where this guide stays updated. I write hands-on DevOps/SRE deep-dives there weekly.
The 100-word answer
To allocate Kubernetes cost per namespace: run OpenCost against your existing Prometheus, query its allocation API with aggregate=namespace, and — this is the part everyone skips — make three explicit policy decisions before you show anyone a number: how idle capacity is charged, how shared namespaces (kube-system, monitoring) are split, and the fact that cost is computed from requests, not usage. Then enforce the preconditions with ResourceQuotas and a policy engine so teams can't game the math, and automate a weekly showback report. Skip the policy decisions and the first team that disputes their number will kill the whole initiative.
I run this setup on multi-team clusters, and the tooling was never the hard part. The hard part is producing a number an engineering manager can't argue with.
Why the cloud bill can't answer this
AWS Cost Explorer will happily tell you the EKS node group cost $14,200 last month. It cannot tell you that the checkout namespace burned 40% of it, because to the cloud provider your cluster is an opaque pile of EC2. Allocation tooling exists to bridge that gap: measure what each container was allocated on each node, multiply by that node's hourly rate, roll it up by namespace.
Namespace is the right allocation boundary for most orgs because it's usually already the team boundary — one namespace per service team or per product line. That makes it the natural unit for showback (here's what you spent) and later chargeback (it's coming out of your budget). Start with showback. Chargeback before teams trust the numbers is how FinOps initiatives die in the first quarter — the broader sequencing is covered in the FinOps for Kubernetes guide.
Step 1: get allocation data flowing
I use OpenCost here because it's free, CNCF-governed, and rides on the Prometheus you already run — if you're weighing it against the commercial option, I compared both hands-on in OpenCost vs Kubecost. Everything below (idle policy, shared cost, label enforcement) applies to either tool.
helm repo add opencost-charts https://opencost.github.io/opencost-helm-chart
helm install opencost opencost-charts/opencost \
--namespace opencost --create-namespace \
--set opencost.prometheus.internal.enabled=false \
--set opencost.prometheus.external.enabled=true \
--set opencost.prometheus.external.url="http://prometheus-server.monitoring.svc:80"
OpenCost assumes Prometheus is already scraping the cluster. On cloud, also configure the provider pricing integration (on AWS, an IAM role that can read the Pricing API and your Savings Plans) — otherwise OpenCost falls back to on-demand list prices and overstates cost on any cluster with reservations or spot.
The per-namespace query, accumulated over a week:
kubectl port-forward -n opencost svc/opencost 9003:9003
curl -sG http://localhost:9003/allocation/compute \
--data-urlencode "window=7d" \
--data-urlencode "aggregate=namespace" \
--data-urlencode "accumulate=true" \
--data-urlencode "shareIdle=true" | \
jq -r '.data[0] | to_entries[] |
[.key, (.value.totalCost*100|round/100), (.value.cpuEfficiency*100|round)] | @tsv' | \
sort -t$'\t' -k2 -rn | column -t
That gives you namespace, dollar cost, and CPU efficiency in one table. Note the shareIdle=true — that's not a formatting flag, it's a policy decision, and it's the first of three you need to make deliberately.
Step 2: the three decisions that make the numbers defensible
Requests, not usage, drive the bill
OpenCost (and Kubecost, same engine) charges each container max(request, usage) for CPU and memory. This is correct — a pod requesting 4 CPU reserves that capacity whether it uses it or not, and the scheduler can't give it to anyone else. But it has two consequences teams will discover the moment you publish numbers:
- Over-requesting is expensive even at 5% utilization. That's the feature: the report finally makes over-provisioning visible. Efficiency below ~0.4 is your rightsizing list.
- A namespace with no requests set looks nearly free while actually consuming burstable capacity. Its waste gets smeared across everyone else as idle cost. This is the gaming vector, and it's why enforcement (next section) is not optional.
Idle cost: distribute it or invoice it separately
Idle cost is provisioned-but-unallocated node capacity — you paid for it, no namespace requested it. You have two honest options:
-
shareIdle=true— distribute idle proportionally to each namespace's allocated cost. Simple, totals match the cloud bill, but teams get charged for capacity they never asked for. -
shareIdle=false— show idle as its own__idle__line item owned by the platform team.
I recommend the second for showback. Idle capacity is a platform problem — it means your nodes are poorly bin-packed or your autoscaler is lazy, and the fix is platform work like switching to Karpenter for better consolidation, not something the checkout team can act on. Putting __idle__ on the report with an owner turned it from an invisible tax into a tracked metric that we drove from 31% to 14% of cluster cost.
Shared namespaces: split by policy, in writing
kube-system, monitoring, ingress-nginx, opencost itself — every tenant benefits, nobody owns them. OpenCost handles this with sharing parameters:
curl -sG http://localhost:9003/allocation/compute \
--data-urlencode "window=7d" \
--data-urlencode "aggregate=namespace" \
--data-urlencode "accumulate=true" \
--data-urlencode "shareNamespaces=kube-system,monitoring,ingress-nginx,opencost" \
--data-urlencode "shareSplit=weighted"
shareSplit=weighted distributes shared cost proportionally to each namespace's direct cost; even splits it equally. Weighted is fairer for lopsided clusters; even is easier to explain. Either is fine — what matters is that the rule is written down before the first report goes out, because retroactive methodology changes are how you lose trust permanently.
Step 3: enforce the preconditions
The math is only as good as its inputs, and the inputs are requests and labels. Enforce both.
A ResourceQuota forces every pod in the namespace to declare requests (pods without requests are rejected outright once requests.cpu is quota-limited), and caps the namespace's reservable capacity so "just request 16 CPU to be safe" has a price:
apiVersion: v1
kind: ResourceQuota
metadata:
name: team-checkout-quota
namespace: checkout
spec:
hard:
requests.cpu: "24"
requests.memory: 96Gi
limits.memory: 128Gi
persistentvolumeclaims: "20"
Pair it with a LimitRange so pods that omit requests get a sane default instead of an admission error during migration. Then add a policy that requires an ownership label on every namespace, so cost rows always map to a team:
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-team-label
spec:
validationFailureAction: Enforce
rules:
- name: namespace-has-team
match:
any:
- resources:
kinds: ["Namespace"]
validate:
message: "Every namespace needs a team label for cost allocation."
pattern:
metadata:
labels:
team: "?*"
If you're not running a policy engine yet, the Kyverno policy-as-code guide covers installation and rollout in audit-then-enforce stages. With the team label in place, aggregate=label:team gives you per-team rollups even when teams own several namespaces.
Put it in Grafana and alert on growth, not level
OpenCost exports its pricing model as Prometheus metrics, which means per-namespace cost is just PromQL. Approximate monthly run rate per namespace (CPU + memory):
(
sum by (namespace) (container_cpu_allocation
* on(node) group_left() node_cpu_hourly_cost)
+
sum by (namespace) (container_memory_allocation_bytes / 1073741824
* on(node) group_left() node_ram_hourly_cost)
) * 730
The alert that actually catches incidents is week-over-week growth, because absolute cost levels are known and boring but a 60% jump means someone shipped a replica bump or a memory leak:
- alert: NamespaceCostSpike
expr: |
sum by (namespace) (container_cpu_allocation * on(node) group_left() node_cpu_hourly_cost)
/ (sum by (namespace) (container_cpu_allocation * on(node) group_left() node_cpu_hourly_cost) offset 7d)
> 1.5
for: 6h
labels:
severity: ticket
annotations:
summary: "Namespace {{ $labels.namespace }} CPU cost up >50% week-over-week"
The for: 6h matters — allocation metrics wobble during deploys and autoscaling, and a cost alert that pages on noise gets deleted within a month.
Automate the weekly showback report
Trust comes from cadence. A CronJob that posts the same table to Slack every Monday beats a dashboard nobody opens:
apiVersion: batch/v1
kind: CronJob
metadata:
name: weekly-showback
namespace: opencost
spec:
schedule: "0 8 * * 1"
jobTemplate:
spec:
template:
spec:
restartPolicy: Never
containers:
- name: report
image: badouralix/curl-jq:alpine
envFrom:
- secretRef:
name: slack-webhook
command: ["/bin/sh", "-c"]
args:
- |
REPORT=$(curl -sG http://opencost.opencost.svc:9003/allocation/compute \
--data-urlencode "window=7d" \
--data-urlencode "aggregate=namespace" \
--data-urlencode "accumulate=true" \
--data-urlencode "shareNamespaces=kube-system,monitoring" \
--data-urlencode "shareSplit=weighted" | \
jq -r '.data[0] | to_entries
| sort_by(-.value.totalCost) | .[:10][]
| "\(.key): $\(.value.totalCost|round) (cpu eff \((.value.cpuEfficiency*100)|round)%)"')
curl -s -X POST "$SLACK_WEBHOOK_URL" \
-H 'Content-Type: application/json' \
-d "{\"text\": \"*Top-10 namespace cost, last 7d*\n\`\`\`$REPORT\`\`\`\"}"
Ten lines of namespace, dollars, and efficiency. The efficiency column is what turns the report from finance trivia into an engineering work queue — every namespace under 40% has an obvious next action.
What the numbers are for
Showback is instrumentation, not the goal. Once the report is stable and undisputed, the sequence that actually cuts spend is: rightsize the worst-efficiency namespaces (the Kubernetes cost optimization guide walks through the full playbook), fix bin-packing so the __idle__ line shrinks, and only then talk about chargeback. And because the allocation API returns clean structured JSON, it's also ideal agent input — I've wired an autonomous FinOps agent to this exact endpoint to draft rightsizing PRs from the same efficiency data your Monday report surfaces.
Attribution is the unglamorous 20% of Kubernetes cost work that makes the other 80% possible. Get the policy decisions in writing, enforce requests and labels at admission, ship the report weekly — and the first time a team disputes a number, you'll have an answer instead of an argument.
📌 Read the latest version of this guide — plus the full library of DevOps, SRE, Kubernetes, observability & cloud-cost guides — on devtocash.com.
Top comments (0)