Don't Size a New Platform on CPU

The legacy platform handled millions of transactions a day. Over 13 months of monitoring data, its monthly peak CPU was 69.4%. Its monthly averages sat between 1.6% and 6.7%.

That was the most useful number I found while assessing a new Kubernetes platform that was about to replace it. Not because it told me how many cores to buy. Because it told me CPU was the wrong thing to size on.

The question I was asked

The new platform was a few months from launch, with seasonal traffic peaks ahead. I was asked to assess whether it was ready and what it needed. The obvious capacity question is "how much compute do we need for launch day?", and the obvious way to answer it is to look at the old system's CPU and scale it up.

If I'd done that, I would have concluded the new platform had plenty of headroom. The old one barely used its CPUs. Two fixed nodes in the new cluster looked fine by that logic.

The trouble is that the old platform was never CPU-bound. Something else set its ceiling. A new platform sized on CPU inherits the wrong mental model, and it fails somewhere nobody is watching.

Where it would actually fail

I read the code before I had cloud access: the infrastructure repositories in full, plus what monitoring and the delivery tooling could show me. That gave me a different picture.

Every production service ran as a single replica. One small managed database instance held 9 databases behind a max_connections of 200, with no connection pooler in front of it.

Now do the arithmetic the CPU graph hides. Say you look at launch traffic and decide to scale every service from 1 replica to 3. Each replica opens its own connection pool. More than a dozen services, each with a pool of 10 or 20 connections, times 3. You're past 200 before the first real user arrives. The database starts refusing connections, the services start failing health checks, and the autoscaler, if there is one, adds more replicas that open more connections.

That's the failure mode a CPU-based plan walks straight into. The fix is in the wrong order if you're not careful: add the pooler first, then add replicas.

What I size on instead

For a web platform with a shared database, these tell me more than CPU does:

• Transactions per second at peak, not per day. Millions a day sounds big. Spread over business hours it's a manageable rate, but the peak minute is what matters, and seasonal peaks can be several times a normal day.
• Database connections per replica, times replicas, times services. Compare that to the connection limit. If it's close, nothing else matters until there's a pooler.
• Latency of the slowest dependency. If requests spend most of their time waiting on a database or an external API, more CPU does nothing.
• Memory per pod under real load. Out-of-memory kills look like crashes, not slowness, and they show up long before CPU does.
• Replica count per service. One replica means every deploy, node drain or crash is downtime for that service, no matter how big the node is.

CPU still matters. I just treat it as the last thing to check, not the first.

Read the code, not the slides

One more thing from this assessment. Some of what I'd been told verbally didn't match the code. One component described as self-hosted was a hosted service. The team said there was no observability stack; Terraform said there was a full one, just with no alerts or dashboards wired up.

Neither was anyone hiding anything. People describe the system they remember, and systems drift from that. When I assess a platform now, I mark each finding with where it came from. "Terraform declares this" is not the same as "this is running", and "the team said" is not the same as either.

The good parts

It wasn't all gaps, and an assessment that only lists problems isn't useful. Passwords never entered Terraform state, because the code used write-only arguments. There was a promotion gate that refused to deploy a version to production unless it had already been in test. Nothing deployed the latest tag. Those are the habits that make the rest fixable.

What I'd do differently

I'd ask for the old platform's monitoring data on day one, before reading any code. Thirteen months of real traffic is the best capacity document you'll ever get. Once I saw single-digit CPU averages next to millions of daily transactions, the rest of the assessment reorganised itself around the database.

I'd also write the connection arithmetic into the findings as a table: services, replicas, pool size, total, limit. A table like that ends the "just add replicas" conversation faster than any paragraph.

A short checklist before you size anything

1. Get at least a year of the old system's metrics, including its worst week.
2. Find the peak minute, not the daily total.
3. Add up database connections across every service at the replica count you plan for. Compare it to the limit.
4. Put a pooler in before you scale out.
5. Get every production service to at least two replicas.
6. Only then look at CPU.

If you're about to launch a new platform and want a second look at where it will break first, I do that kind of assessment. See Kubernetes & Platform.