High CPU gets throttled. High Memory gets killed.

If you've worked on anything at scale, you've seen this in an incident channel: a pod is unhappy, dashboards are red, and someone says "it's hitting its limits." The usual fix is to bump the resource limits and move on.

Sometimes that's the right call — traffic grew, infra needs to grow with it. But sometimes it's not an infra problem at all. It's a few lines of code doing more work, or holding onto more memory, than they need to. Scaling up just buys time before the same bug hits a bigger ceiling.

In this article, we'll look at why a CPU limit and a memory limit fail in completely different ways, then walk through three real versions of the same Ruby script — each one fixing the previous version's problem while quietly creating, or finally avoiding, the next one.

How CPU and Memory Limits Actually Work

A Kubernetes resource block looks symmetric:

resources:
  limits:
    cpu: "1"
    memory: "512Mi"

Two limits, same shape, same file. But they're enforced by two completely different mechanisms in the kernel.

CPU limits are enforced by the CFS quota (cpu.cfs_quota_us in the cgroup). If your process wants more CPU than its quota allows, the kernel scheduler just gives it fewer time slices. The process doesn't stop — it keeps running, keeps its state, keeps serving requests. It's just slower. Requests take longer, queues back up, p99 latency climbs. Nothing crashes.

Memory limits are enforced by the OOM killer. There's no equivalent "slow down." Memory is a hard ceiling — either you're under it or you're not. The instant your process's RSS crosses memory.max, the kernel sends SIGKILL. No warning, no graceful degradation, no "please free up some memory." The process is just gone, and whatever it was doing is gone with it. Kubernetes sees the container exit and restarts it — from zero.

Same YAML, same-looking limits, completely different failure modes:

	CPU limit exceeded	Memory limit exceeded
Mechanism	CFS quota (scheduler throttling)	OOM killer (SIGKILL)
What happens	Process keeps running, slower	Process is killed instantly
Symptom	Rising latency, timeouts	Restarts, OOMKilled, exit 137
Recoverable mid-request?	Yes	No

Summary:

• A CPU limit slows your process down. It keeps running, keeps its state, and recovers on its own once pressure eases.
• A memory limit kills your process outright, the instant it crosses the line. There's no mid-request recovery — it restarts from zero.
• You can engineer graceful degradation for CPU pressure (queue, retry, shed load). You can't do that for memory pressure after the fact — it has to be kept under control before it happens.

Three Attempts at the Same Problem

To make this concrete, let's look at a boring, realistic task — find duplicate order IDs in a 50,000-row CSV export — written three times, the way most of us actually arrive at a "good" solution: badly, then fast-but-wrong, then actually right.

The file looks like this (real columns, representative rows — ORD-1042 shows up twice, which is exactly the kind of duplicate we're hunting for):

order_id,customer,product,price,notes
ORD-1042,customer_1042,product_187,43.10,Deliver to front desk, ring bell twice.
ORD-1043,customer_1043,product_298,70.47,Leave with neighbor if not home.
ORD-1042,customer_2290,product_112,12.95,Gift wrap requested.
ORD-1044,customer_1044,product_044,6.97,No substitutions please.

Attempt 1: The Naive Version (CPU Problem)

# v1_naive.rb
rows = File.readlines(path, chomp: true)[1..]

duplicates = []
rows.each_with_index do |row, i|
  id = row.split(",", 2).first
  rows.each_with_index do |other_row, j|
    next if i == j
    other_id = other_row.split(",", 2).first
    duplicates << id if id == other_id
  end
end

Explanation:

• Every row gets compared against every other row — O(n²) comparisons, no shortcuts.
• The order_id is pulled straight out of the real CSV line on every single comparison, so there's no pre-processing step hiding the work elsewhere.
• This is pure computation. No file is reopened, no network call is made, no extra memory is allocated beyond the lines already in hand.

Measured on an 8,000-row slice with /usr/bin/time -l:

rows: 8000, duplicate hits: 387
19.81 real   19.62 user   0.16 sys
maximum resident set size: 26MB

19.62 of 19.81 seconds was CPU time — this is almost entirely CPU-bound, not I/O. Memory stays flat at Ruby's own baseline (~26MB). At the full 50,000-row file, this scales with n², so it would take roughly 13 minutes for a single dedup pass.

In a pod with a 1-core CPU limit, this doesn't crash. It just sits there grinding, request handlers backing up behind it, until something upstream times out.

Attempt 2: The "Fix" (CPU Solved, Memory Problem Introduced)

The instinct here is right: stop comparing every row to every row, use a hash for O(1) lookups.

# v2_hash_full_records.rb
require "csv"
records = {}
duplicates = 0

CSV.foreach(path, headers: true) do |row|
  id = row["order_id"]
  if records.key?(id)
    duplicates += 1
  else
    records[id] = row.to_h   # caches the WHOLE row, not just the id
  end
end

Explanation:

• A hash keyed by order_id turns the lookup into O(1), so the whole scan becomes O(n) instead of O(n²).
• Every row is now parsed properly as a CSV row, not a raw split line — which is more correct, but also more expensive per row.
• The bug is one line: records[id] = row.to_h caches every column of every unique row, not just the id. Nothing ever gets evicted, so the hash grows for the lifetime of the scan.

Measured on the full 50,000-row (23MB) file:

unique rows cached: 47480, duplicate hits: 2520
0.65 real   0.48 user   0.06 sys
maximum resident set size: 90MB

0.65 seconds. The CPU problem is completely solved — roughly 1,190x faster than the naive version would be at this scale. But look at memory: 90MB to process a 23MB file. Every unique row's full data is cached in the hash for the lifetime of the scan, so memory now scales with file size, not with what the job actually needs.

If this pod had a 75MB memory limit (a reasonable limit for a small background worker), this version would get OOMKilled partway through the file — probably somewhere around row 39,000 of 50,000 — and never finish. No error message, no stack trace. Just a SIGKILL and a restart.

Attempt 3: The Actual Fix (Both Problems Solved)

# v3_stream_set.rb
require "set"
seen = Set.new
duplicates = 0

File.foreach(path, chomp: true).with_index do |line, i|
  next if i == 0
  id = line.split(",", 2).first
  if seen.include?(id)
    duplicates += 1
  else
    seen << id   # only the id is ever kept in memory
  end
end

Explanation:

• Same O(1) hash-lookup trick as Attempt 2 (a Set is backed by a Hash), so CPU stays fast.
• The file is streamed line-by-line with File.foreach, instead of being parsed into row objects up front — nothing is ever held in memory beyond the current line.
• Only the order_id string is ever stored, in the Set — never the rest of the row.

Measured on the full 50,000-row file:

unique ids tracked: 47480, duplicate hits: 2520
0.08 real   0.06 user   0.01 sys
maximum resident set size: 29MB

0.08 seconds — about 8x faster than Attempt 2, and roughly 9,700x faster than Attempt 1 would be at this scale. 29MB peak memory — close to Ruby's own interpreter baseline, and about a third of what Attempt 2 used.

Summary:

	Rows	Time	Peak RSS	What happens in a real pod
1 · nested loop	8,000 (full file: ~13 min projected)	19.81s	26MB	CPU-bound — pod throttles, requests queue up
2 · hash, full rows	50,000	0.65s	90MB	Fast, but OOMKilled crossing a 75MB limit
3 · stream + set	50,000	0.08s	29MB	Fast and stays well under any reasonable limit

• Going from Attempt 1 to Attempt 2 is the fix most of us reach for first, and it's not wrong — the algorithmic complexity (O(n²) → O(n)) was the right call.
• The bug in Attempt 2 was in what got cached, not how the lookup worked.
• Attempt 3 fixes both at once not by trading CPU for memory, but by shrinking the footprint everywhere: stream the input, cache only the minimum.

Best Practices

A few practical signals for telling the two apart in production:

• Check CPU first: kubectl top pod shows CPU pegged near its limit while memory stays flat and boring. If you're scraping cgroup/cAdvisor metrics, container_cpu_cfs_throttled_seconds_total climbing confirms it. The symptom is latency, not crashes.
• Check for OOMKilled: kubectl describe pod shows OOMKilled in the last state, with exit code 137, and the restart count ticking up. Memory usually climbs steadily right up until the kill — often a sawtooth pattern if you're graphing RSS over time.
• Reproduce locally with /usr/bin/time: /usr/bin/time -l (macOS) or /usr/bin/time -v (Linux) on a local repro gives you both user/sys CPU time and maximum resident set size in one shot — it's usually the fastest way to tell the two apart before reaching for a profiler.
• Go deeper when needed: stackprof or rbtrace for CPU hot paths in Ruby, and memory_profiler or derailed_benchmarks for allocation and retention issues.

Conclusion

A CPU limit and a memory limit look identical in a YAML file, but they fail in opposite ways — one degrades gracefully, the other doesn't degrade at all. That asymmetry should shape how you design around each one: build retry and backoff for CPU pressure, but treat memory growth as something to prevent, not something to recover from.

The three versions of the dedup script above show how easy it is to fix one and create the other without noticing. Going from a nested loop to a hash lookup is the right instinct — the trap is in what the hash ends up holding onto. Once you're deliberate about caching only the smallest piece of data you actually need, and streaming rather than loading, both problems tend to disappear at the same time.

Next time a pod is unhappy and the instinct is to scale up, it's worth taking thirty seconds to check which limit it actually hit first. Sometimes it really is infra. Often, it's a line of code like records[id] = row.to_h, quietly doing more than it needs to.