DEV Community: Huzaifa Saifuddin

High CPU gets throttled. High Memory gets killed.

Huzaifa Saifuddin — Sat, 31 Oct 2026 17:34:45 +0000

If you've worked on anything at scale, you've seen this in an incident channel: a pod is unhappy, dashboards are red, and someone says "it's hitting its limits." The usual fix is to bump the resource limits and move on.

Sometimes that's the right call — traffic grew, infra needs to grow with it. But sometimes it's not an infra problem at all. It's a few lines of code doing more work, or holding onto more memory, than they need to. Scaling up just buys time before the same bug hits a bigger ceiling.

In this article, we'll look at why a CPU limit and a memory limit fail in completely different ways, then walk through three real versions of the same Ruby script — each one fixing the previous version's problem while quietly creating, or finally avoiding, the next one.

How CPU and Memory Limits Actually Work

A Kubernetes resource block looks symmetric:

resources:
  limits:
    cpu: "1"
    memory: "512Mi"

Two limits, same shape, same file. But they're enforced by two completely different mechanisms in the kernel.

CPU limits are enforced by the CFS quota (cpu.cfs_quota_us in the cgroup). If your process wants more CPU than its quota allows, the kernel scheduler just gives it fewer time slices. The process doesn't stop — it keeps running, keeps its state, keeps serving requests. It's just slower. Requests take longer, queues back up, p99 latency climbs. Nothing crashes.

Memory limits are enforced by the OOM killer. There's no equivalent "slow down." Memory is a hard ceiling — either you're under it or you're not. The instant your process's RSS crosses memory.max, the kernel sends SIGKILL. No warning, no graceful degradation, no "please free up some memory." The process is just gone, and whatever it was doing is gone with it. Kubernetes sees the container exit and restarts it — from zero.

Same YAML, same-looking limits, completely different failure modes:

	CPU limit exceeded	Memory limit exceeded
Mechanism	CFS quota (scheduler throttling)	OOM killer (SIGKILL)
What happens	Process keeps running, slower	Process is killed instantly
Symptom	Rising latency, timeouts	Restarts, `OOMKilled`, exit 137
Recoverable mid-request?	Yes	No

Summary:

A CPU limit slows your process down. It keeps running, keeps its state, and recovers on its own once pressure eases.
A memory limit kills your process outright, the instant it crosses the line. There's no mid-request recovery — it restarts from zero.
You can engineer graceful degradation for CPU pressure (queue, retry, shed load). You can't do that for memory pressure after the fact — it has to be kept under control before it happens.

Three Attempts at the Same Problem

To make this concrete, let's look at a boring, realistic task — find duplicate order IDs in a 50,000-row CSV export — written three times, the way most of us actually arrive at a "good" solution: badly, then fast-but-wrong, then actually right.

The file looks like this (real columns, representative rows — ORD-1042 shows up twice, which is exactly the kind of duplicate we're hunting for):

order_id,customer,product,price,notes
ORD-1042,customer_1042,product_187,43.10,Deliver to front desk, ring bell twice.
ORD-1043,customer_1043,product_298,70.47,Leave with neighbor if not home.
ORD-1042,customer_2290,product_112,12.95,Gift wrap requested.
ORD-1044,customer_1044,product_044,6.97,No substitutions please.

Attempt 1: The Naive Version (CPU Problem)

# v1_naive.rb
rows = File.readlines(path, chomp: true)[1..]

duplicates = []
rows.each_with_index do |row, i|
  id = row.split(",", 2).first
  rows.each_with_index do |other_row, j|
    next if i == j
    other_id = other_row.split(",", 2).first
    duplicates << id if id == other_id
  end
end

Explanation:

Every row gets compared against every other row — O(n²) comparisons, no shortcuts.
The order_id is pulled straight out of the real CSV line on every single comparison, so there's no pre-processing step hiding the work elsewhere.
This is pure computation. No file is reopened, no network call is made, no extra memory is allocated beyond the lines already in hand.

Measured on an 8,000-row slice with /usr/bin/time -l:

rows: 8000, duplicate hits: 387
19.81 real   19.62 user   0.16 sys
maximum resident set size: 26MB

19.62 of 19.81 seconds was CPU time — this is almost entirely CPU-bound, not I/O. Memory stays flat at Ruby's own baseline (~26MB). At the full 50,000-row file, this scales with n², so it would take roughly 13 minutes for a single dedup pass.

In a pod with a 1-core CPU limit, this doesn't crash. It just sits there grinding, request handlers backing up behind it, until something upstream times out.

Attempt 2: The "Fix" (CPU Solved, Memory Problem Introduced)

The instinct here is right: stop comparing every row to every row, use a hash for O(1) lookups.

# v2_hash_full_records.rb
require "csv"
records = {}
duplicates = 0

CSV.foreach(path, headers: true) do |row|
  id = row["order_id"]
  if records.key?(id)
    duplicates += 1
  else
    records[id] = row.to_h   # caches the WHOLE row, not just the id
  end
end

Explanation:

A hash keyed by order_id turns the lookup into O(1), so the whole scan becomes O(n) instead of O(n²).
Every row is now parsed properly as a CSV row, not a raw split line — which is more correct, but also more expensive per row.
The bug is one line: records[id] = row.to_h caches every column of every unique row, not just the id. Nothing ever gets evicted, so the hash grows for the lifetime of the scan.

Measured on the full 50,000-row (23MB) file:

unique rows cached: 47480, duplicate hits: 2520
0.65 real   0.48 user   0.06 sys
maximum resident set size: 90MB

0.65 seconds. The CPU problem is completely solved — roughly 1,190x faster than the naive version would be at this scale. But look at memory: 90MB to process a 23MB file. Every unique row's full data is cached in the hash for the lifetime of the scan, so memory now scales with file size, not with what the job actually needs.

If this pod had a 75MB memory limit (a reasonable limit for a small background worker), this version would get OOMKilled partway through the file — probably somewhere around row 39,000 of 50,000 — and never finish. No error message, no stack trace. Just a SIGKILL and a restart.

Attempt 3: The Actual Fix (Both Problems Solved)

# v3_stream_set.rb
require "set"
seen = Set.new
duplicates = 0

File.foreach(path, chomp: true).with_index do |line, i|
  next if i == 0
  id = line.split(",", 2).first
  if seen.include?(id)
    duplicates += 1
  else
    seen << id   # only the id is ever kept in memory
  end
end

Explanation:

Same O(1) hash-lookup trick as Attempt 2 (a Set is backed by a Hash), so CPU stays fast.
The file is streamed line-by-line with File.foreach, instead of being parsed into row objects up front — nothing is ever held in memory beyond the current line.
Only the order_id string is ever stored, in the Set — never the rest of the row.

Measured on the full 50,000-row file:

unique ids tracked: 47480, duplicate hits: 2520
0.08 real   0.06 user   0.01 sys
maximum resident set size: 29MB

0.08 seconds — about 8x faster than Attempt 2, and roughly 9,700x faster than Attempt 1 would be at this scale. 29MB peak memory — close to Ruby's own interpreter baseline, and about a third of what Attempt 2 used.

Summary:

	Rows	Time	Peak RSS	What happens in a real pod
1 · nested loop	8,000 (full file: ~13 min projected)	19.81s	26MB	CPU-bound — pod throttles, requests queue up
2 · hash, full rows	50,000	0.65s	90MB	Fast, but OOMKilled crossing a 75MB limit
3 · stream + set	50,000	0.08s	29MB	Fast and stays well under any reasonable limit

Going from Attempt 1 to Attempt 2 is the fix most of us reach for first, and it's not wrong — the algorithmic complexity (O(n²) → O(n)) was the right call.
The bug in Attempt 2 was in what got cached, not how the lookup worked.
Attempt 3 fixes both at once not by trading CPU for memory, but by shrinking the footprint everywhere: stream the input, cache only the minimum.

Best Practices

A few practical signals for telling the two apart in production:

Check CPU first: kubectl top pod shows CPU pegged near its limit while memory stays flat and boring. If you're scraping cgroup/cAdvisor metrics, container_cpu_cfs_throttled_seconds_total climbing confirms it. The symptom is latency, not crashes.
Check for OOMKilled: kubectl describe pod shows OOMKilled in the last state, with exit code 137, and the restart count ticking up. Memory usually climbs steadily right up until the kill — often a sawtooth pattern if you're graphing RSS over time.
Reproduce locally with /usr/bin/time: /usr/bin/time -l (macOS) or /usr/bin/time -v (Linux) on a local repro gives you both user/sys CPU time and maximum resident set size in one shot — it's usually the fastest way to tell the two apart before reaching for a profiler.
Go deeper when needed: stackprof or rbtrace for CPU hot paths in Ruby, and memory_profiler or derailed_benchmarks for allocation and retention issues.

Conclusion

A CPU limit and a memory limit look identical in a YAML file, but they fail in opposite ways — one degrades gracefully, the other doesn't degrade at all. That asymmetry should shape how you design around each one: build retry and backoff for CPU pressure, but treat memory growth as something to prevent, not something to recover from.

The three versions of the dedup script above show how easy it is to fix one and create the other without noticing. Going from a nested loop to a hash lookup is the right instinct — the trap is in what the hash ends up holding onto. Once you're deliberate about caching only the smallest piece of data you actually need, and streaming rather than loading, both problems tend to disappear at the same time.

Next time a pod is unhappy and the instinct is to scale up, it's worth taking thirty seconds to check which limit it actually hit first. Sometimes it really is infra. Often, it's a line of code like records[id] = row.to_h, quietly doing more than it needs to.

Thread vs Fiber vs Ractor: I Measured Ruby's Concurrency Model

Huzaifa Saifuddin — Tue, 20 Oct 2026 10:05:32 +0000

A few weeks back, at RailsConf Pune (Deccan Queen), Shopify engineers gave a talk titled "When Rails Met Fibers: Migrating Sidekiq from Puma to Falcon." Their challenge: an AI chatbot where every request was itself waiting on an external AI API call, and Puma's thread-per-request model was buckling under all that waiting.

I've used Thread for over a decade — Puma's thread pool, Sidekiq workers, the usual. But that talk made me want to go past the surface-level API and actually verify what's happening underneath: what the GVL really guarantees, where Fiber genuinely earns its place, and where Ractor fits in. So I sat down and did exactly that — measuring Thread, Fiber, and Ractor myself instead of taking any of it on faith.

Every number in this post came from a script I ran. The full, runnable repo — one commit per finding — is here: thread-fiber-ractor on GitHub. Clone it and run the scripts yourself; your numbers will differ from mine (different machine, different core count), and that's kind of the point — don't trust my numbers either, verify them.

Built and run on Ruby 4.0.7.

The GVL, the one idea everything else builds on

MRI's Global VM Lock (GVL) lets exactly one thread execute a Ruby instruction at any given instant, in one process — no matter how many threads you spawn. That's the single fact that explains almost everything below.

A thread blocked on I/O (a network call, a DB round-trip) releases the GVL, so another thread can run while it waits.
A thread doing pure CPU work won't release it voluntarily — but it's not quite "holds the lock forever" either. MRI's VM has an internal timer that can still force a switch between CPU-bound threads. The catch: that forced switch just means CPU-bound threads take turns. It doesn't make them any faster, because only one of them is ever actually executing at once.

That second point is the one worth sitting with, because it's exactly what separates Thread from Fiber later in this post.

Thread: concurrency on I/O, nothing on CPU

Here's the proof, straight from 01_gvl_thread_basics.rb:

require "benchmark"

def count_primes(upto)
  count = 0
  (2..upto).each do |n|
    is_prime = (2..Math.sqrt(n).to_i).none? { |d| n % d == 0 }
    count += 1 if is_prime
  end
  count
end

N = 1_000_000

# CPU-bound: does Thread give you parallelism?
serial = Benchmark.realtime { count_primes(N); count_primes(N) }
parallel = Benchmark.realtime do
  t1 = Thread.new { count_primes(N) }
  t2 = Thread.new { count_primes(N) }
  [t1, t2].each(&:join)
end

# I/O-bound: does Thread give you concurrency?
serial_io = Benchmark.realtime { sleep(1); sleep(1) }
parallel_io = Benchmark.realtime do
  t1 = Thread.new { sleep(1) }
  t2 = Thread.new { sleep(1) }
  [t1, t2].each(&:join)
end

Output:

---- CPU-bound: does Thread give you parallelism? ----
Two calls, one after another:   5.77s
Same two calls, on two Threads:  5.79s
speedup: 1.0x  (expect ~1.0x — the GVL serializes CPU-bound Ruby)

---- I/O-bound: does Thread give you concurrency? ----
Two sleeps, one after another:  2.0s
Same two sleeps, on two Threads: 1.01s
speedup: 1.98x  (expect ~2.0x — I/O releases the GVL)

Identical CPU-bound work: zero speedup across two threads — in fact marginally slower (5.79s vs 5.77s), which lines up with forced-switch overhead rather than one thread holding the lock uninterrupted. The same two threads doing I/O get a real ~2x speedup, because I/O actually releases the lock.

This is Thread's whole story: great concurrency primitive for I/O you already have (that's what Puma's thread pool and Sidekiq workers are doing for you), zero help for CPU-bound work.

The hidden ceiling

Threads map to real OS kernel threads, and the kernel hard-caps how many of those can exist. On my machine (02_thread_ceiling.rb):

Creating Threads until the OS refuses...
Hit a hard OS wall at 2046 threads -- can't create Thread: Resource temporarily unavailable

Creating 50000 Fibers (not an OS construct — no equivalent ceiling)...
Created all 50000 Fibers with zero errors (24.4x past where Threads hit their wall)

2,046 threads and the OS refuses to create more. 50,000 Fibers, in the same process, zero errors — because a Fiber isn't an OS-level construct at all. Which is a good place to introduce Fiber properly.

Fiber: cooperative, cheap, and not a speed trick

A Fiber is a coroutine: pause and resume control flow, entirely inside one Ruby process, with no OS involvement. Fiber.new creates one, Fiber.yield pauses it, #resume continues it (03_fiber_basics.rb):

fiber = Fiber.new do
  puts "fiber: step 1"
  Fiber.yield
  puts "fiber: step 2"
end

fiber.resume # => "fiber: step 1"
fiber.resume # => "fiber: step 2"

One detail that trips people up: a Fiber only ever runs on whichever OS thread calls #resume. Resuming it from a different thread raises FiberError: fiber called across threads.

Fiber alone is not fast

It's tempting to assume Fiber is a speed trick because it's a newer, fancier control-flow tool. It isn't. A Fiber switch has real overhead, and for a simple "stop when you hit a condition" loop, a plain loop wins every time (04_fiber_is_not_a_speed_trick.rb):

plain loop: 0.0824s
fiber:      0.5839s
fiber is 7.1x SLOWER than a plain loop for this

Fiber's value isn't raw speed. It's structural — which is exactly what Fiber::SchedulerInterface unlocks.

The real payoff: Fiber::SchedulerInterface

Registering a scheduler (via Fiber.set_scheduler) makes Ruby intercept blocking calls — sleep, socket I/O — and hand them to the scheduler instead of actually blocking the OS thread. The scheduler registers interest with the kernel's own I/O multiplexer (epoll/kqueue), and the thread is immediately free to run other ready fibers. This is the exact mechanism behind Falcon — the app server from the RailsConf talk that kicked this whole thing off.

The async gem is the reference implementation. Here's the proof (05_fiber_scheduler.rb):

require "async"

Async do |task|
  3_000.times { task.async { sleep 1 } }
end

---- Threads: 20 concurrent 'slow I/O calls' (sleep 1 each) ----
  wall time: 1.01s
  OS threads used: 20 (one per call)

---- Fibers + scheduler: same 20 calls, one task.async each ----
  wall time: 1.0s
  OS threads used: 1 (every task cooperatively shared the same thread)

---- Now push past where Threads hit their wall (~2,046) ----
Running 3000 concurrent 'slow calls' as Fibers -- Thread.new would have crashed by now.
  wall time: 1.11s
  3000 concurrent I/O-bound calls, 1 OS thread, no crash.

3,000 concurrent "I/O calls," one OS thread, 1.11 seconds. That's something Thread literally cannot attempt past its own ceiling.

The cost: no preemption safety net

Here's the part that doesn't make it into the pitch. Thread's GVL still has that forced timer switch we talked about earlier — even a pure CPU loop gets preempted eventually. Fiber has nothing like it. A fiber only ever gives up the thread when it decides to, by hitting a blocking call. A CPU-bound fiber with zero yield points starves every other fiber on that thread for its entire run — even ones whose I/O already finished (06_fiber_has_no_preemption.rb):

Async do |task|
  task.async { busy_loop(2) }        # pure CPU, no yield point at all
  task.async { sleep(0.1) }          # "ready" almost immediately
end

[CPU-bound fiber] finished after 2.0s
[I/O fiber]        finished after 2.1s (expected ~0.1s if it weren't starved)

The I/O fiber was ready in 0.1 seconds. It didn't actually finish until 2.1 seconds — starved the entire time by a CPU-bound fiber that never yielded. If that CPU-bound fiber ran forever, so would the wait. There's no timeout coming to rescue you.

That tradeoff — massive I/O concurrency, zero CPU-starvation protection — is almost certainly the exact thing Shopify had to reason about migrating to Falcon.

Ractor: actual parallelism

Thread gives you concurrency, never parallelism, on CPU-bound work — that's the GVL's whole deal. Ractor is the one primitive that breaks that rule entirely.

Ractor basics

07_ractor_basics.rb

r = Ractor.new { "hello from a ractor" }
r.value # => "hello from a ractor"

Worth noting: Ractor's API is still marked experimental and has changed across Ruby versions — an older #take method from some tutorials doesn't exist anymore; #value is the current way to get a Ractor's result. Don't trust a blog post's (including this one's) method names without checking against your own Ruby version.

The sharing rules

Since Ractors genuinely run at the same instant, Ruby has to prevent two of them from mutating the same object at once. It does this by making the setup physically impossible, not just "risky" (08_ractor_sharing_rules.rb):

# closures over a mutable outer variable are refused immediately
arr = [1, 2, 3]
Ractor.new { arr << 4 }
# => ArgumentError: can not isolate a Proc because it accesses outer variables (arr)

# #send of a mutable object copies it
r = Ractor.new { Ractor.receive << 4 }
r.send([1, 2, 3])
r.value.object_id == [1, 2, 3].object_id # => false, it's a copy

# a frozen object is shared by TRUE reference
frozen = [1, 2, 3].freeze
r2 = Ractor.new { Ractor.receive }
r2.send(frozen)
r2.value.object_id == frozen.object_id # => true, zero-copy

# Ractor.make_shareable deep-freezes recursively
data = { name: "x", roles: ["lead", "engineer"] }
Ractor.make_shareable(data)
data.frozen?          # => true
data[:roles].frozen?  # => true, the nested array too

Put together: there is no path to "two Ractors holding a live reference to the same mutable object." Not a risky one, not a rare one — none. If you need real shared mutable state across Ractors, the only sanctioned path is message-passing — each Ractor owns its own private state, and they coordinate by sending copies (or transferring ownership via move: true) across, never by two of them reaching into the same box at once.

The payoff: real CPU speedup

Same CPU benchmark as the Thread section, now split across Ractors instead (09_ractor_vs_thread_cpu.rb):

ractors = 4.times.map { Ractor.new { count_primes(600_000) } }
ractors.each(&:value)

---- Serial: 4 calls, one after another ----
  wall time: 5.79s

---- Threads: 4 concurrent, same GVL ----
  wall time: 5.82s
  speedup vs serial: 0.99x

---- Ractors: 4 concurrent, independent GVLs ----
  wall time: 1.73s
  speedup vs serial: 3.35x

Same workload, same machine, same core count available the whole time. Thread: flat. Ractor: 3.35x faster. The only variable that changed is whether each worker shares one GVL or gets its own.

Worth knowing before you reach for this in a Rails app: ActiveRecord isn't Ractor-safe yet. I tried calling Rails.application and ApplicationRecord.connection from inside a Ractor and both hard-failed — Rails.application raises Ractor::IsolationError because Rails itself holds the booted app in a mutable instance variable only the main Ractor can read. So the practical pattern is: pull data out of Rails/AR first, on the main Ractor, then hand off only plain, frozen data to a pool of Ractors for the CPU-heavy part.

Bonus: the Rails `Current.user` gotcha under Fiber concurrency

This one connects straight back to the Falcon migration. Most Rails apps keep request-scoped context — current user, request id — via ActiveSupport::CurrentAttributes. It's built on ActiveSupport::IsolatedExecutionState, which defaults to isolation_level = :thread — storing state as a plain attribute on the shared Thread object, not scoped per-fiber.

Under a Fiber-scheduler server like Falcon, multiple fibers share one OS thread. With the default :thread isolation, they'll clobber each other's Current state (10_thread_current_gotcha.rb):

class Current < ActiveSupport::CurrentAttributes
  attribute :request_id
end

Async do |task|
  task.async do
    Current.request_id = "req-A"
    sleep(0.3)
    puts Current.request_id # => ???
  end
  task.async do
    sleep(0.1)
    Current.request_id = "req-B"
  end
end

---- isolation_level = :thread (Rails' default) ----
  [A] woke back up. Current.request_id is now: "req-B"

---- isolation_level = :fiber ----
  [A] woke back up. Current.request_id is now: "req-A"

Under the default, fiber A's own request id got overwritten by fiber B's — a different request's data leaking into fiber A's execution. Rails actually ships a real fix for this: config.active_support.isolation_level = :fiber. If your app ever moves to a Fiber-scheduler server, this is a config line worth knowing exists.

Which one do you actually need?

Thread — concurrency on I/O you already have (Puma, Sidekiq). Never parallelism on CPU-bound Ruby. Hard OS ceiling.
Fiber — I/O concurrency at a scale Thread can't reach (thousands of fibers, one OS thread). Cooperative, with zero preemption safety net.
Ractor — compute parallelism Thread can never give you. The price is real isolation: no shared mutable state, enforced by the VM, and (for now) no ActiveRecord inside one.

None of these replace the others. Pick based on where your actual bottleneck is.

Try it yourself

All ten scripts, with real output and one commit per finding: github.com/HuzaifaSaifuddin/thread-fiber-ractor

git clone https://github.com/HuzaifaSaifuddin/thread-fiber-ractor
cd thread-fiber-ractor
bundle install
ruby 01_gvl_thread_basics.rb

Your numbers will come out different from mine — different machine, different core count, different load. Run them anyway. That's the whole point of this post: stop trusting the docs, go measure it.

If you've used Fiber or Ractor in a real app — or migrated a service to Falcon — I'd genuinely like to hear what you ran into. Drop it in the comments.

DEV Community: Huzaifa Saifuddin

High CPU gets throttled. High Memory gets killed.

How CPU and Memory Limits Actually Work

Three Attempts at the Same Problem

Attempt 1: The Naive Version (CPU Problem)

Attempt 2: The "Fix" (CPU Solved, Memory Problem Introduced)

Attempt 3: The Actual Fix (Both Problems Solved)

Best Practices

Conclusion

Thread vs Fiber vs Ractor: I Measured Ruby's Concurrency Model

The GVL, the one idea everything else builds on

Thread: concurrency on I/O, nothing on CPU

The hidden ceiling

Fiber: cooperative, cheap, and not a speed trick

Fiber alone is not fast

The real payoff: Fiber::SchedulerInterface

The cost: no preemption safety net

Ractor: actual parallelism

Ractor basics

The sharing rules

The payoff: real CPU speedup

Bonus: the Rails Current.user gotcha under Fiber concurrency

Which one do you actually need?

Try it yourself

Bonus: the Rails `Current.user` gotcha under Fiber concurrency