Kubernetes Cost Optimization: Where the Money Actually Goes and How to Cut It

Nobody sized this cluster on purpose

Every over-budget Kubernetes cluster I’ve been brought in to look at has the same origin story. Someone set CPU and memory requests once, early on, usually by guessing generously “to be safe,” and nobody has revisited them since. Traffic grew, more services got added the same way, and eighteen months later the cluster is requesting 3x the CPU it actually uses, on nodes sized for a peak that rarely happens, running on-demand pricing for workloads that have been steady for a year. None of this is anyone’s fault exactly. It’s just what happens when nobody’s job is to watch it.

The fix is not exotic. It’s four levers, applied roughly in this order because that’s also the order of impact per hour spent:

1. Rightsize requests and limits to real usage
2. Bin-pack and autoscale properly
3. Use spot and committed pricing for the workloads that fit
4. Kill the waste nobody’s looking at

Lever 1: rightsizing requests and limits

The Kubernetes scheduler places pods based on requests, not actual usage, not limits. A pod that requests 2 CPU and uses 200m is still reserving 2 CPU worth of node capacity that nothing else can use. Multiply that across a hundred services and you’re paying for nodes that exist almost entirely to hold reserved-but-unused capacity.

Start by comparing requested vs. actual:

kubectl top pods -A --sort-by=cpu
kubectl get pods -A -o json | jq -r '
  .items[] | .metadata.namespace + "/" + .metadata.name + " requests: " +
  (.spec.containers[0].resources.requests.cpu // "none")'

For anything running long enough to have a usage history, pull real percentile data rather than eyeballing kubectl top at one point in time — a Prometheus query against container_cpu_usage_seconds_total and container_memory_working_set_bytes over 30 days gives you p50/p95/p99, which is what you actually want to size against, not a single snapshot that might catch a quiet moment.

quantile_over_time(0.95,
  sum(rate(container_cpu_usage_seconds_total{namespace="prod"}[5m])) by (pod)[30d:5m]
)

Size CPU requests to roughly p95 usage with headroom, not p50 and not the theoretical peak. Memory is less forgiving than CPU because exceeding a memory limit gets your container OOMKilled, not throttled, so leave more headroom there and re-check after the change rather than assuming it’s fine.

Tools that make this faster: Goldilocks (VPA-based dashboard giving you request recommendations per workload without applying them automatically), and Kubecost or your cloud’s native cost tool for the per-namespace, per-workload cost breakdown that turns this from a guess into a prioritized list.

Lever 2: bin-packing and autoscaling, both ends

Rightsizing individual pods only pays off if the nodes are actually packed efficiently. This is where Karpenter (or Cluster Autoscaler if you’re not ready to migrate) earns its keep. Karpenter provisions nodes just-in-time based on the actual pending pod requirements instead of picking from a fixed, pre-defined node group, which typically means better bin-packing and less standing idle capacity:

apiVersion: karpenter.sh/v1
kind: NodePool
spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
      nodeClassRef:
        name: default
  limits:
    cpu: 1000
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized

consolidationPolicy: WhenEmptyOrUnderutilized is the line doing the cost work here — it actively consolidates workloads onto fewer nodes as usage drops, rather than leaving yesterday’s peak-sized fleet running overnight.

Don’t forget the pod side of autoscaling. HPA for request-driven services, VPA in recommendation mode to keep requests honest over time as usage shifts, and for GPU or batch workloads, scale-to-zero where the workload tolerates cold starts. A cluster that scales nodes but has every deployment pinned to a fixed replica count is only half-optimized.

Lever 3: spot and committed pricing, matched to the workload

Spot instances run 60-90% cheaper than on-demand and are a straightforward win for anything that tolerates interruption: stateless API replicas behind a load balancer, CI runners, batch and ETL jobs, dev/staging environments entirely. Karpenter handles spot interruption reasonably gracefully with its interruption handler, but you still need enough replicas and disruption budgets that losing one node doesn’t take a service down:

apiVersion: policy/v1
kind: PodDisruptionBudget
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: api

For the steady-state baseload that isn’t going anywhere (your always-on core services, databases, anything single-replica or stateful), spot is the wrong tool. That’s what Savings Plans or Reserved Instances are for. Model your steady 24/7 floor from the last 60-90 days of usage and commit against that floor, not against your peak, and not against 100% of current spend since some of that spend is workloads you’re about to rightsize away.

Lever 4: killing the waste nobody’s watching

This is the unglamorous part and it’s usually worth more than people expect:

• Idle namespaces: dev/test environments left running 24/7 that only get used during business hours. A scheduled scale-to-zero (or full teardown) outside working hours can cut that slice by 60-70% for free.
• Orphaned PersistentVolumes: PVCs left behind after a workload was deleted, still billing storage months later. kubectl get pv -A | grep Released finds the orphans.
• Unused LoadBalancers: every type: LoadBalancer Service provisions a cloud load balancer that bills whether or not it’s receiving traffic. Audit for services with zero recent connections.
• Log and egress costs hiding in observability: verbose debug logging left on in production, and cross-AZ or cross-region traffic that could be avoided with topology-aware routing, both show up as compute-adjacent line items that never get attributed back to “cost of running the cluster” until someone actually goes looking.

Put a floor under it so it doesn’t creep back

None of this stays fixed without something watching it. A budget alert per namespace, a weekly Kubecost or Cost Explorer report to the team that owns the workload (not just to finance), and a policy that new services start from a measured request, not a guess, are what keep the savings from eroding back to where they started in six months. I’ve seen more than one team do an excellent cost pass and quietly regress to the old spend within a year purely because nobody owned watching it afterward.

What this looks like end to end

On a recent engagement cutting a SaaS startup’s AWS bill ~42% in three weeks, the order was exactly this: pull real usage data first (Cost Explorer plus CUR, about 60% of spend was compute and a lot of it was idle), rightsize EKS node groups and workload requests/limits against 30 days of actual usage, bring in Karpenter for bin-packing, move fault-tolerant batch and staging workloads to spot, and only then look at savings-plan commitments for what was left running steady. Reliability didn’t move. The bill did, and it showed up on the very next invoice for the rightsizing and waste-cleanup pieces specifically, because those reduce actual resource consumption immediately rather than waiting on a structural migration to land.

If your bill has been climbing and nobody can point at exactly why, that’s usually a sign nobody’s measured it yet, not that the workload genuinely needs more. That’s cloud cost optimization work, and it usually pays for itself within weeks.