Early in a cost engagement this year I pulled one number for every cluster: requested CPU divided by CPU actually used. It came out between 7 and 20 on all four clusters. Not on one noisy namespace. On every cluster.
The second number was worse in a different way. About 40% of the workloads had no requests at all.
Those two numbers explained most of the AWS bill before I opened Cost Explorer.
A node autoscaler like Karpenter does not look at how busy your pods are. It looks at what they asked for. When a pod is pending, the autoscaler adds up the requests of everything that needs a home and rents nodes big enough to hold that sum. Your monitoring can show nodes sitting mostly idle all day, and the autoscaler will still be right by its own rules. Those nodes are full, on paper.
So the line I used with the team was simple: what you ask for is what you pay for. If a service requests one full CPU and uses a tenth of one, you are renting a full CPU. Do that across a whole cluster and the waste is most of the bill.
Usage matters for performance. Requests decide the invoice.
The 40% with no requests looked harmless. They were not costing anything on the scheduler's books, so why worry.
The problem is that those pods still use CPU and memory. The scheduler places them as if they were free, packs them next to pods that did declare requests, and then everyone competes on the node. You get noisy neighbours, odd latency spikes and, for memory, evictions that look random. The usual reaction is to raise requests on the services that complain, which buys more nodes. The pods that started it are still invisible.
Every workload needs a request. Even a rough one is better than none, because it gives the scheduler something true to work with.
When I lined up requests against 30 days of real usage, a few patterns came out.
Five event-driven services each used less than 2% of a 100m CPU request. They woke up for a message, did a few milliseconds of work and went back to sleep. Each one looked tiny on its own. Together they were still holding space on nodes all day.
One service went from 200m CPU and 256Mi memory down to 25m and 100Mi. After the change it showed zero CPU throttling and zero out-of-memory kills. Nobody noticed, which is what a good right-sizing change looks like.
Right-sizing is not only cutting. A document rendering service had a 30-day peak of about double its request. The fix was to raise its request so the scheduler could plan for its real peak.
This matters more than it sounds. If the team learns that "right-sizing" means "make the numbers smaller", the first time a sized-down service falls over they stop trusting the whole exercise. Showing early that sizing goes both ways kept the conversation about data, not about cost cutting.
I saw the same thing from the reliability side later in the engagement. A production web service had a CPU request well below what its pods used at idle, and that one number made its autoscaler flap. Raising the request fixed it. I wrote that one up separately.
You can right-size a cluster in a week. It will drift back in a quarter.
New services get created by copying an old manifest. Someone bumps a request during an incident and nobody lowers it again. An old load test still sets the memory limit for a service that now does half the work. None of this is anyone being careless. It is what happens when requests are set once and never looked at.
So the most useful thing I built was not a change to any workload. It was a right-sizing dashboard the whole team could read, showing requests next to real usage for every workload, sorted by the size of the gap. Then a short session with the engineers, which I called "Right-Sizing for Real", and a pre-merge checklist for anything that touches resources.
Keep a checklist like that short. It should make people look at real usage before they pick a number, and when the answer is "I don't know", the next step is to ask, not to guess high.
The point was that the bill should not depend on me. If I am the only person who understands why a service has the request it has, the savings leave when I do.
Right-sizing was one part of a larger cost effort, alongside removing duplicated tooling, an unused environment and a Kubernetes version in paid extended support. Monthly AWS spend went from about $20,800 in May to about $9,300 in August, 55% lower. I wrote up the full case in Cutting a SaaS Platform's Kubernetes Cloud Bill by 55%.
I am not going to give a sizing formula here, because the right one depends on how a service behaves under load and how much risk the team is willing to carry. The principle is enough to start: make requests match reality, in both directions, and make that a habit instead of a project.
If your node count keeps growing while your dashboards say the cluster is idle, I can help you find where the requests went. See Cloud Cost & FinOps.
Spot nodes were reclaimed, the on-demand fallback hit its own limit, and production was down for about 25 minutes. Then I moved production off spot.
AWS daily spend looked like it fell from $284 to $62 in two days. It was late billing data, and why you never report savings from the last 1 to 3 days.
A fintech's CloudWatch line went from about $2.4k to $6.4k a month in two months. One API operation explained most of it: PutLogEvents.
