DEV Community: Dane Wu

You've Tuned the Queries and It's Still Slow — Now Change the Data Model

Dane Wu — Mon, 29 Jun 2026 08:58:49 +0000

Rails Performance: Lessons from Production — #9

The first eight posts optimized without touching the schema — fixing N+1, adding indexes, caching, moving work to the background. But sometimes you tune the query to its limit and it's still slow — because the root cause is the schema design itself: you normalized the data so cleanly that every query has to JOIN a pile of tables and recompute a derived value. This post takes a different axis: changing the data model. The core idea is the denormalization trade-off. Same example throughout (Courier has_many :shipments).

💥 Indexes are added, N+1 is fixed, the couriers page is still slow

A couriers list page. Each row shows "how many shipments this courier has" + "the status of the latest one." You did everything right:

@couriers = Courier.includes(:shipments).limit(50)
# in the view: courier.shipments.count, courier.shipments.last.status

N+1 is fixed (includes), shipments.courier_id is indexed. But the page is still slow — the APM shows every row running a COUNT and finding "the latest one."

The thing is: "shipment count" and "latest status" are values that, every page load, for every courier, get recomputed from millions of shipment rows. This isn't a query-style problem — your data model is forcing you to recompute derived values every time.

Query tricks have hit their ceiling. The next step is to change the data model: turn "compute every time" into "compute once and store it." That's denormalization.

🧮 Move 1: counter_cache — stop COUNTing the child table every time

The most common, built-in denormalization. Store "the parent's count" on the model:

class Shipment < ApplicationRecord
  belongs_to :courier, counter_cache: true   # adds a shipments_count column to couriers
end

Rails automatically bumps couriers.shipments_count ±1 on every shipment create/destroy. Reading it:

courier.shipments_count   # reads a column, zero COUNT queries

"Each courier runs its own SELECT COUNT(*)" becomes "read a ready-made column."

There's one cost you must know about: counter_cache only updates on ActiveRecord create / destroy. Use update_all, delete_all, import, or raw SQL and it won't move — the number drifts. When it drifts, fix it with Courier.reset_counters(courier.id, :shipments) (pass that courier's id).

Concurrency, though, you don't have to worry about — it's safe. counter_cache doesn't "read it out, +1, write it back" — it lets the DB do the increment:

UPDATE couriers SET shipments_count = shipments_count + 1 WHERE id = 7

Why it matters: with "read then add" (count = count + 1 computed in Ruby), two requests each creating a shipment at the same time could both read 100, both compute 101, both write 101 → one increment lost. But SET count = count + 1 is a single atomic operation in the DB; two concurrent ones get serialized (100 → 101 → 102) and nothing is miscounted.

📦 Move 2: store the "computed value" redundantly

counter_cache is a special case. The general form of denormalization is: any "frequently needed and expensive" derived value — precompute it and store it on the parent.

"Latest shipment status" is expensive to fetch every time (shipments.order(:created_at).last.status) → store a copy on the courier:

# couriers gets a latest_shipment_status column
class Shipment < ApplicationRecord
  belongs_to :courier

  # after a new shipment is saved, sync its status onto the courier's redundant column
  after_create_commit do
    courier.update_column(:latest_shipment_status, status)
  end
end

Breaking that callback down:

after_create_commit: an ActiveRecord callback that fires only after a new record is created and the transaction commits (the _commit variant guarantees it's really persisted, not mid-transaction).
courier.update_column(:latest_shipment_status, status): write this shipment's status into the courier's latest_shipment_status column (the redundant copy). update_column issues a single SQL UPDATE on one column, skipping validations/callbacks and not touching updated_at — right for "just sync a redundant value."

In plain terms: every time a new shipment is saved, copy its status onto the courier's "latest status." Reading it is just courier.latest_shipment_status — one column, no JOIN, no sort.

⚠️ But don't rush to denormalize — for WHERE courier_id = ? ORDER BY created_at DESC LIMIT 1, first add a (courier_id, created_at) composite index and each query becomes fast (the index is already sorted; grab one row). If an index solves it, don't touch the data model.

So when is redundancy worth it? ① Even if each query is fast, a list of 50 couriers is still 50 round-trips — too many at high volume/traffic; ② "the latest one per group" is genuinely hard to de-N+1 cleanly — includes loads all shipments into memory to pick from, and doing it in one SQL statement needs a window function / lateral join (not native in Rails, raw SQL). When the index isn't enough, or the query is too awkward to maintain, a single redundant column is the cheapest fix.

This is the key technique for "read-heavy, hot-data" products like stock or e-commerce: things like price change %, volume, total sales — expensive aggregates that must display in real time — are often stored redundantly, or even:

materialized view (Postgres CREATE MATERIALIZED VIEW): "materialize" one expensive aggregate query into a physical table, refreshed on a schedule (REFRESH). Good for "reports / leaderboards that are expensive but can tolerate a few minutes stale" (in Rails, the scenic gem).

⚖️ The core cost of denormalization: consistency

Denormalization isn't free — you stored an extra copy, and that copy can drift from the truth. This is the fundamental trade-off against normalization:

Normalized: data has a single source of truth, always correct, but queries JOIN/recompute (slow reads).
Denormalized: a redundant copy makes reads fast, but you're now responsible for keeping that copy in sync (writes get complex, and it can be inconsistent).

There are three ways to maintain consistency, from synchronous to asynchronous:

callback / touch (synchronous): update the redundant column inline on write (the after_create_commit above is exactly this). Simple, but easy to let get out of hand: callbacks that run queries, touch the parent, or enqueue jobs add hidden cost to every save (buried in the model); worse, they cascade — one callback triggers another save, which triggers its callback… one write fans out into a chain of updates and cache invalidations, and can deadlock under concurrency. This is why the community calls callbacks a code smell.
background job (asynchronous): enqueue a job to update the redundant value after the write. Accept brief inconsistency (eventual consistency) for fast writes — common for hot data. For maintaining redundant data, prefer a job (#6) over a synchronous callback — no write slowdown, no cascade.
periodic reconciliation job (safety net): batch-recompute and fix any drifted values (like counter_cache's reset_counters).

The point of denormalization isn't "store a copy" — it's this trade-off: not "store junk to go fast," but knowingly spending consistency complexity to buy read speed, with a reconciliation backstop in place. Denormalize without thinking that through and you'll eventually get bitten by "why don't these two numbers match?"

🔗 Other ways to change the data model

Beyond denormalization, a few common schema-level moves — reach for them when the situation calls for it:

Polymorphic associations perform poorly: commentable_type + commentable_id can't have a real foreign key and can't JOIN efficiently to a single table. At scale, switch to explicit associations (separate FK columns/tables) to win back integrity and performance.
has_many :through join tables: slow without indexes — the join table needs a composite index on (a_id, b_id) (and the reverse pair if you query both directions).
Data lifecycle on big tables: time-series data (stock candles, logs) explodes in size → partitioning (by time), hot/cold separation, archiving, keeping the main table lean so queries scan only the relevant partition.
Column types: money/prices in decimal, not float (float has rounding errors); the right time type — schema-design fundamentals.

These all change the schema structure; the cost and risk are far higher than rewriting a query — the last layer you reach for, after confirming the earlier moves aren't enough.

🏁 Wrap-up: the whole series as one map

"When should I change the data model?" is answered by a bigger question — how to work through a slow request, one layer at a time. That's the spine of this whole series. Performance optimization is really three axes:

① Make each request do less work (this axis has a light-to-heavy order)

Measure first (APM / EXPLAIN) — find the real bottleneck, don't guess.
Tune queries and indexes first: fix N+1, add composite indexes, don't drag data into Ruby (#1–#3). ~80% of problems end here.
Then cache: don't recompute what you already computed (#4, #5).
Trim the Ruby-layer waste: serialization, allocation, memoization (#7).
Only then change the data model: when the query pattern conflicts with the normalized schema, denormalize judiciously (counter_cache, redundant hot data, materialized views); changing schema structure (splitting tables, partitioning) is more deliberate still (#9, this post).

② Move slow work out of the request — email, third-party APIs, reports shouldn't block a request the user is waiting on; push them to the background (#6).

③ Let the machine serve more requests at once — no matter how fast the code, the wrong Puma / connection-pool / CDN setup still causes a traffic jam (#8).

One principle:

Performance optimization = make each request do less + move slow work out of the request + let the machine serve more at once — and each one starts the same way: measure first, then go from light to heavy.

Back to this post's layer: normalization is the right default; denormalization is a deliberate trade — spending consistency complexity to buy read speed — used only when measurement proves queries, indexes, and caching aren't enough, and always with a reconciliation backstop.

After these nine posts, facing a slow page you hold not just a pile of tricks but a sense of which layer to reach for first — and that judgment tends to decide the outcome more than any single technique.

The Code Is Fine, but Requests Queue Until They Time Out: Puma, Pools, CDN

Dane Wu — Sun, 28 Jun 2026 09:30:00 +0000

Rails Performance: Lessons from Production — #8

The previous six posts optimized the code — queries, caching, background work, the app layer. But sometimes every request is fast on its own, and yet under concurrency the whole batch slows down or times out. The problem isn't the code, it's how the machine is configured: how many Puma workers/threads, whether the connection pool is big enough, whether static assets go through a CDN. This post covers that last layer.

💥 A single request is 50ms, but 100 at once time out

Load testing turned up something odd: one request alone takes 50ms, but 100 arriving at once make the later ones queue, and some time out outright. The code didn't get slower — "how many it can handle at once" isn't enough.

That's the infrastructure layer — how many requests your app can serve in parallel is decided by Puma's config, not by your code.

🧵 Puma: workers (processes) × threads

Puma is Rails' default web server, and its concurrency comes from two dimensions:

workers (processes): controlled by WEB_CONCURRENCY. Each worker is a separate Ruby process and uses its own slice of memory.
threads: each worker runs N threads, handling N requests at once.

# config/puma.rb
workers Integer(ENV.fetch("WEB_CONCURRENCY", 2))   # number of processes
threads 5, 5                                        # 5 threads per worker

Requests handled in parallel ≈ workers × threads. Above that's 2 × 5 = 10. The "100 at once" from the opening far exceeds that, so the rest queue.

How to tune:

Worker count: each worker uses its own memory, so the real ceiling on workers is usually memory, not CPU cores. CPU-bound work uses core count as a reference; an IO-bound Rails app often runs more workers than cores. Load test to find it.
Thread count: MRI runs only one thread of Ruby at a time (the GVL, formerly "GIL"), so for pure CPU work more threads help little. But the moment a thread enters blocking IO (DB, external API) it releases the GVL, letting another thread run in that gap — so for a Rails app that spends lots of time waiting on IO, more threads help a lot. The more IO waiting, the bigger the win.

There's no universal number — load test while watching memory and CPU. The point is to know that "parallel ceiling = workers × threads," and to add capacity here when it's not enough.

🔌 Connection pool: align threads with DB connections

This is the most common hidden trap. Every Puma thread that queries the DB needs a DB connection, taken from the connection pool. If the pool is smaller than the thread count, threads can't get a connection, stall waiting, and eventually error:

ActiveRecord::ConnectionTimeoutError: could not obtain a connection from the pool

Rule: each worker's pool ≥ that worker's thread count.

# config/database.yml
production:
  pool: <%= ENV.fetch("RAILS_MAX_THREADS", 5) %>   # align with threads

And do the total math. The pool is per-process — all threads in a worker share that worker's single pool, so the real total of connections hitting the DB is workers × pool (not × threads; they happen to be equal when you set pool = threads, but pool is often set a bit larger to leave room for Active Storage, load_async, and other non-request paths). That total must not exceed the DB's max_connections (PostgreSQL defaults to 100 — but the usable number is lower after reserved connections and other services, and managed DBs often scale it by instance size).

When connections genuinely run short, put PgBouncer in front so many app connections share a few DB connections — but don't treat it as a free switch: the big savings come from its transaction mode, and transaction mode conflicts with the prepared statements Rails enables by default, so you'll need prepared_statements: false or a recent PgBouncer with prepared-statement support.

In one line: Puma threads → pool size → DB max_connections — align this chain top to bottom; any link too small and it stalls.

🌐 CDN: don't make your Rails server ship static assets

Every image, every JS/CSS file served by your Rails server means spending a precious Puma thread to ship a file — wasteful. Those static, unchanging assets should go to a CDN:

Static assets (JS/CSS/images): served from the CDN, fetched from a nearby edge node — fast, and off your server.
Active Storage uploads: configure them to go through the CDN too, instead of streaming from your app / object storage every time.

A CDN offloads "shipping files" entirely from your Rails server, leaving Puma threads for "requests that actually run Ruby."

🚀 Other production switches worth flipping on

A few low-cost, high-value settings:

bootsnap: caches compiled Ruby/YAML, speeding up boot (faster deploys and restarts).
Asset precompilation (assets:precompile): compile JS/CSS ahead of time, instead of compiling on a user's request.
Enable gzip / compression: compress responses before sending, saving bandwidth.

These are usually handled by Rails defaults or your deploy platform, but knowing they exist and confirming they're on is table stakes.

🏁 Wrap-up

When each request is fast on its own but concurrency collapses, the problem is the infrastructure layer:

setting	governs	key point
Puma workers × threads	how many requests at once	the parallel ceiling — tune by load testing
connection pool	whether threads can get a DB connection	pool ≥ threads; total (`workers × pool`) ≤ DB max_connections
CDN	who ships static assets	offload to the CDN, keep Puma for Ruby requests

One principle:

The first six posts were about "make each request faster"; this one is about "let the machine serve more requests at once." Different problems — no matter how fast the code, the wrong concurrency config still causes a traffic jam.

And this layer has a chain: Puma threads → connection pool → DB connection ceiling — align it top to bottom. Before bumping Puma, ask "can my DB handle this many connections?" That's why this post comes last: it's the bill you add up when you fit all the earlier optimizations into a real machine.

Your SQL Is Fast but the API Is Slow: It's the Ruby Layer

Dane Wu — Sun, 28 Jun 2026 09:26:47 +0000

Rails Performance: Lessons from Production — #7

By now the DB queries, caching, and background work are handled. But sometimes the SQL is genuinely fast and the API is still slow, the CPU still high — because the bottleneck moved into Ruby itself: heavy serialization, allocating too many objects, recomputing the same thing. This post is about the cost in the application layer, after the request reaches the controller. Same example throughout (a {% raw %}shipments table).

💥 SQL took 30ms, the API took 800ms

A shipments-list API. The APM showed SQL was only 30ms, but the whole request took 800ms. N+1 and indexes were already checked — the DB was fine. The slow part was "what Ruby does after the data comes back."

That remaining 770ms went to three common places: bloated serialization, too many intermediate objects, and the same computation running repeatedly. Let's take them one at a time.

📦 Trap 1: serializing a pile of fields nobody uses

The most common one — the API turns the whole model into JSON:

render json: @shipments   # every field of every shipment gets dumped

Problems:

A shipment has 40 columns, the frontend uses 5 — serializing the other 35 × thousands of rows is pure waste.
You might also leak sensitive fields (internal cost, API keys) by accident.

The fix: return only the fields you need. Spell them out:

render json: @shipments.as_json(only: [:id, :tracking_no, :status])

Serialization is pure CPU work — fewer fields × more rows, more saved.

Two things worth adding:

Safety should come from a structural whitelist, not from remembering only:. as_json dumps all fields by default, and only: is a manual whitelist the next person easily rewrites back to render json: @shipments. To make "new fields don't leak by default" a structural guarantee, use a serializer (Alba / blueprinter / AMS) that pins the visible schema.
as_json(only:) saves serialized fields, but doesn't save "instantiating every row into an AR object + building a Hash per row" — which is exactly the cost in Trap 2. When fields are few but rows are huge and you want to skip instantiation too, use pluck(:id, :tracking_no, :status) to fetch arrays and never build AR objects at all.

♻️ Trap 2: allocating too many intermediate objects in a hot path (GC pressure)

Every object Ruby creates costs memory; lots of allocation increases GC (garbage collection) work and grows the heap, which drags throughput once it scales. (MRI's GC is generational + incremental, so short-lived objects are actually relatively cheap — the truly expensive thing is "many, long-lived, growing the heap." So the point is: don't allocate pointlessly in a hot path.) It shows up most on a path that runs "for every row":

# ❌ each shipment builds a new hash, a new string, runs a format
@shipments.map do |s|
  { label: "#{s.tracking_no} - #{s.status.upcase} - #{format_date(s.created_at)}" }
end

Thousands of rows × several intermediate objects each = tens of thousands of short-lived objects thrown at the GC.

The fix: allocate fewer intermediates in the hot path — push what the DB can compute to the DB (back to #3), merge steps, and don't rebuild the same thing inside the loop.

# pull the invariant out of the loop, don't rebuild it each iteration
status_labels = { "delivered" => "Delivered", ... }   # built once
@shipments.map { |s| status_labels[s.status] }

Concept: object allocation has a cost. One or two don't matter, but amplified by "every row × every request" the GC becomes a hidden bottleneck.

🧮 Trap 3: the same computation, run many times in one request

def total_weight
  shipments.sum(&:weight)   # recomputes on every call
end
# in one request, the view, a helper, and the serializer each call it → computed three times

The fix: memoization — compute once, reuse within the same request.

def total_weight
  @total_weight ||= shipments.sum(&:weight)   # compute first time, then just return it
end

@x ||= ... means "compute only when @x is nil or false, otherwise return it." Within the same object, in the same request, this runs once.

⚠️ The ||= trap: it tests "is the value nil/false," not "has it been computed." If the result is legitimately nil or false (e.g. a lookup that returns nil), every call recomputes and memoization is effectively dead — a hidden perf bug on an expensive query. For that, write @x = compute unless defined?(@x). (sum returns a number so it dodges this, but you should know the general gotcha.)

Also: memoization works "within one request" because the object being memoized on is itself built per-request (GC'd when the request ends) — not because the mechanism is request-scoped. Memoize on a class-level / singleton / long-lived service object and it persists across requests, causing stale data or memory leaks. Be careful.

🏁 Wrap-up

When the DB is already fast but the API is still slow, the bottleneck is usually in the Ruby layer:

trap	symptom	fix
bloated serialization	returning fields nobody uses	return only what's needed (`only:` / a serializer / `pluck`)
too much allocation	hot path builds many intermediates, high GC	allocate fewer intermediates, push to the DB
repeated computation	same computation many times per request	memoization (`@x

One principle:

DB optimization is "ask the database for less"; application-layer optimization is "do less busywork after the data comes back."

Each of these looks like "saves a tiny bit" on its own, but they share one amplifier — "every row × every request." One field, one object, one computation, times thousands of rows and thousands of requests, is exactly those vanished hundred-odd milliseconds. Confirm with an APM that it's genuinely the Ruby layer (SQL fast, request slow) before digging in here.

One Notification Email Made Our Checkout API 3 Seconds Slower

Dane Wu — Sun, 28 Jun 2026 09:25:58 +0000

Rails Performance: Lessons from Production — #6

The earlier posts optimized what happens inside a request — fewer queries, indexes, caching. This one flips the angle: some slowness shouldn't happen in the request at all. Sending email, calling a third party, generating a report — push that slow work to the background and let the request return immediately. Same example throughout (a shipments table).

💥 The checkout was fast, but the user waited 3 seconds

Placing an order is fast in itself — writing one row to the DB takes 10ms. But after the user hits "checkout," the screen froze for 3 seconds before responding.

The controller looked like this:

def create
  order = Order.create!(order_params)         # 10ms
  CourierApi.create_shipment(order)            # call the courier API: 1.5s
  OrderMailer.confirmation(order).deliver_now  # send the confirmation email: 1.2s
  render json: order
end

The order was created long ago, but the user has to sit and wait for the courier API and the email before seeing a result. Worse: the moment the courier API slows down or goes down, the whole checkout request hangs or times out — the user thinks checkout failed, when in fact the order was created.

The root issue: cramming "slow, and not needed right now" work into a request the user is waiting on.

📤 The fix: push it to the background, return immediately

Turn that slow work into a background job — the request only enqueues the task and returns; the actual work runs on a background worker:

def create
  order = Order.create!(order_params)          # 10ms
  CreateShipmentJob.perform_later(order.id)    # enqueue, ~0ms
  OrderMailer.confirmation(order).deliver_later # email goes to the background too
  render json: order                           # return immediately, the user doesn't wait
end

What perform_later does: writes "what to do + arguments" into a queue (Redis or the database, depending on your engine) and returns at once. The request finishes in ~20ms.

A separate worker process pulls the task off the queue and runs it:

class CreateShipmentJob < ApplicationJob
  def perform(order_id)
    order = Order.find(order_id)
    CourierApi.create_shipment(order)   # this 1.5s runs in the background, nobody waits
  end
end

The user's perceived time drops from 3 seconds back to 20ms. If the courier API is slow or down, it only affects that background job (which can retry) — it doesn't drag down checkout.

What belongs in the background: email, third-party APIs, reports/exports, image processing, push notifications — the common thread is "slow, and the user doesn't need to see the result right now."

⚙️ The tools: Active Job + Sidekiq / Solid Queue

Rails' perform_later is Active Job — a unified interface; you can swap the underlying queue engine:

Sidekiq: the mainstream choice, uses Redis as the queue, fast and mature.
Solid Queue: the new Rails 8 default, uses the database as the queue, no separate Redis to run.

# config/application.rb
config.active_job.queue_adapter = :sidekiq

You don't rewrite your job code to switch engines — that's the value of Active Job.

🧨 The traps (the stuff you only hit after you've shipped it)

Pushing work to the background isn't "fire and forget." A few you must understand:

1. Idempotency — a job can run more than once
Background queues are almost always at-least-once: a job may finish successfully but the worker crashes — or gets SIGTERM'd during a deploy — before it reports completion (ack). The queue thinks it didn't finish, re-dispatches it, and it runs again. (Retries are another source: Sidekiq's own Worker API retries 25 times by default; but bare Active Job doesn't retry by default — you add retry_on yourself.) So a job must be designed so that running it twice gives the same result as running it once.

def perform(order_id)
  order = Order.find(order_id)
  return if order.shipment.present?   # catches most duplicates
  CourierApi.create_shipment(order)
end

Note this present? is "check-then-act": under concurrency two jobs can both pass the check and both create a shipment. Real protection needs a DB unique index, or an idempotency key sent to the third-party API so they dedupe. A guard clause only catches the common case — it isn't true idempotency.

2. Pass an id, not the whole object

CreateShipmentJob.perform_later(order)       # works, but not recommended
CreateShipmentJob.perform_later(order.id)    # ✅ pass the id

First, clear up a common misconception: passing the whole object does not stuff all its data into the queue — Active Job uses GlobalID, storing just a gid://app/Order/123 reference and re-finding it at execution time, so you get the latest data anyway (just as fresh as finding it yourself).

So why still pass the id? The real reasons: ① if the object is deleted before the job runs, GlobalID deserialization raises ActiveJob::DeserializationError and the whole job fails — passing an id lets you decide what to do when find misses; ② smaller payload, clearer intent; ③ a plain PORO (not ActiveRecord) has no GlobalID, so you must pass an id anyway.

A useful way to see it: pass an id when you cross your own queue boundary (re-fetched later by the same system). Inside perform, normal in-process calls pass the live object — CourierApi.create_shipment(order) hands over the object because the external API can't look up your internal id; it needs the actual fields to build its request.

3. Use queue priorities
Email can wait, but "payment succeeded" needs to be quick. Split jobs across queues with different priorities:

class CreateShipmentJob < ApplicationJob
  queue_as :critical   # important work on a high-priority queue
end

Otherwise a pile of low-priority report jobs clogs the queue and the urgent ones wait behind them.

4. Cap retries, and handle final failure
Auto-retry is great, but infinite retries on a "courier API permanently down" is just waste. Set a retry limit; past it, move to a dead letter queue or alert a human — don't let it vanish silently. (Terminology varies by engine: Sidekiq calls it the dead set; Solid Queue records failures in the failed_executions table.)

5. Keep jobs small and fast
A job that runs for 30 minutes ties up a worker for 30 minutes, and if it dies halfway it reruns from the start. Break big work up (e.g. find_each batch by batch, one job per batch).

🏁 Wrap-up

	in the request (❌)	background job (✅)
user experience	waits for the slow work	returns immediately
third party goes down	whole request hangs / times out	only that job is affected, retryable
web worker	held hostage by slow work	freed up to serve others

One principle:

The request does only what the user needs right now; slow work that can finish later goes to the background.

But the background buys a new class of responsibilities — jobs retry (be idempotent), run later (pass ids, not objects), and fail (cap retries and handle it). "Push it to the background" is easy; "push it correctly" is what makes you senior — the moment you enqueue it, think about what happens when it retries, runs late, or fails.

We Added a Cache; Three Days Later It Took the Database Down at Peak

Dane Wu — Sat, 27 Jun 2026 01:39:38 +0000

Rails Performance: Lessons from Production — #5 · Caching, part 2

The previous post covered how caching works (the four layers). This one is about how it bites — the failure modes you only meet after it ships. Adding a cache is one line of fetch, but what it buys you is a whole new class of problems: hammering the DB the instant a key expires, logic breaking when the cache vanishes, memory blowing up. Same example throughout (a shipments table).

🐎 Trap 1: the instant it expires, everyone hammers the DB at once (stampede)

We cached the homepage's 800ms ranking stat with expires_in: 5.minutes. The homepage flew. Felt great.

Three days later, at peak hour, DB CPU suddenly pegged at 100% and the whole site slowed down. The APM showed a spike every 5 minutes — like clockwork.

The cause was a cache stampede: the instant that key expired, all the requests arriving at that moment missed simultaneously and all ran that same 800ms query at once. The cache was supposed to protect the DB; instead, "everyone expiring at the same instant" made the DB take a huge spike in that one second.

How big a spike? It's not about raw RPS — it's RPS × recompute time. The query takes 800ms; during that 0.8s "miss window," every arriving request finds nothing and recomputes. Even a modest 50 RPS means ~40 requests running the same 800ms query in parallel — the DB starts gasping.

How to fix:

Don't give a pile of keys the same hard expiry. Switch to key-based (tied to the data version) and there's no "everyone expires at the same instant" time bomb.
If you genuinely need time-based expiry, use Rails' race_condition_ttl:

Rails.cache.fetch("courier_ranking", expires_in: 5.minutes, race_condition_ttl: 10.seconds) do
  Shipment.group(:courier_id).count
end

After expiry, only the first request that misses recomputes; the others get the just-expired old value immediately and return — they don't wait. Once the first finishes (~0.8s), the new value lands. race_condition_ttl: 10.seconds isn't "everyone waits 10s" — it's the maximum grace period the old value is kept alive in case the recompute drags.

Lesson: a cache protects the DB, but "caches expiring together" concentrates traffic onto the DB. When you add a cache, plan for the instant it expires.

🧱 Trap 2: treating the cache as the source of truth — Redis restarts and it crashes

Some logic was written like this — write a setting into the cache, then read straight back from the cache later:

Rails.cache.write("active_courier_ids", Courier.active.pluck(:id))
# ... elsewhere ...
ids = Rails.cache.read("active_courier_ids")
ids.each { |id| ... }   # one day this blew up: ids was nil

A Redis maintenance restart wiped the cache. That read returned nil, nil.each raised NoMethodError, and the feature died.

The root issue: we treated the cache as the "home" of the data. But a cache can disappear at any moment — Redis restarts, memory fills and entries get evicted, keys expire. It's not a database.

How to fix: a cache can only ever be a "recomputable copy," never the source of truth. If it's missing, you must be able to recompute it — use fetch (recomputes on a miss) instead of a bare read:

ids = Rails.cache.fetch("active_courier_ids") { Courier.active.pluck(:id) }

Lesson: always assume the cache will be gone the next second. Any code that "breaks if the cache is missing" is a landmine.

💣 Trap 3: putting something variable in the key — Redis memory blows up

To cache each user's search results, someone set the key like this:

Rails.cache.fetch("search/#{params[:q]}") { expensive_search(params[:q]) }

Looks reasonable. But params[:q] is free user input — every distinct search string spawns a new key. Once live, all sorts of bizarre query strings poured in, the number of keys grew without bound, Redis memory climbed until it blew up, and the whole cache service went down.

The problem: the key's cardinality is out of control — you think you're caching, but you're actually hoarding without limit.

How to fix:

Build keys only from bounded, controlled dimensions (category, id), never drop raw user input straight into the key.
Give Redis an eviction policy (maxmemory-policy), don't leave it at the default noeviction (which rejects writes and errors when full). For caches, allkeys-lru is common (drop the least-recently-used when full), but the point isn't which one — it's having a "drop the old when full" policy at all, so it doesn't blow up entirely.

Lesson: cache space is finite. Every key costs memory; a key that grows without bound is a memory leak.

☠️ Trap 4: caching the failed result

The ranking stat was changed to come from an internal API:

Rails.cache.fetch("courier_ranking", expires_in: 5.minutes) do
  RankingApi.fetch   # one time it returned nil (the API was briefly down)
end

That time the API briefly failed and returned nil, and fetch stored that nil as "the result" for 5 minutes. So the API was long since fixed, but for the next 5 minutes every user still got an empty ranking — a few-second outage amplified by the cache into a 5-minute site-wide failure.

How to fix: don't cache failed / empty results. If you get an abnormal value, don't store it (or let it expire immediately):

Rails.cache.fetch("courier_ranking", expires_in: 5.minutes) do
  result = RankingApi.fetch
  raise "empty" if result.blank?   # don't let an empty result get stored
  result
end

Lesson: a cache faithfully preserves your errors too. Before storing, confirm this is a result worth keeping for 5 minutes.

🏁 Wrap-up: the cost of caching

Caching makes a system faster, but it buys a whole new class of problems. These four traps are four faces of one sentence:

A cache is a copy that can vanish at any moment, will be faithfully preserved, and occupies finite space — so always think about the instant it expires, disappears, blows up, or stores the wrong thing.

the instant it expires → don't let everyone expire together (stampede)
the instant it vanishes → don't treat it as the source of truth
the matter of space → don't let keys grow without bound
the matter of correctness → don't cache a failed result

That one line of fetch makes caching look easy. But what actually makes you senior is, the moment you add a cache, anticipating every way it can go wrong — these four are the ones you only learn in production, and never forget once you do.

The Fastest Query Is the One You Never Run: The Four Layers of Rails Caching

Dane Wu — Sat, 27 Jun 2026 01:38:31 +0000

Rails Performance: Lessons from Production — #4

The first three posts were about making queries cheaper — fewer N+1s, indexes, not dragging data back into Ruby. This one flips the angle: the fastest query is the one you never run. Compute a result once, store it, and hand it to everyone after. Same example throughout (a shipments table), walking through the four layers of Rails caching: compute, invalidate, render, transfer.

🔥 The same 800ms stat, recomputed for every visitor

The homepage shows a "courier shipment ranking." It's heavy — scanning a few million shipments and a GROUP BY takes 800ms.

The thing is: that number looks the same to every user, and it doesn't need to be real-time (a few minutes stale is fine). But our code makes every visitor recompute it from scratch — 800ms each. Under traffic, the DB gets ground down by the same calculation over and over.

That's the sweet spot for caching — expensive (worth saving) and hit repeatedly (the same result is reused). Let's start from the most basic tool.

Continuing the four layers from #1: the first three posts optimized "how data is fetched"; caching is "don't compute what you already computed." One idea runs through the whole post — use updated_at as a "version," reuse while the data hasn't changed.

📦 Layer 1 (compute): `Rails.cache.fetch`

Wrap that 800ms calculation:

Rails.cache.fetch("courier_ranking", expires_in: 5.minutes) do
  Shipment.group(:courier_id).count   # the heavy calculation
end

What fetch does, in one line: if it's there, take it; if not, compute it and store it.

First time: look up key → miss → run the block (800ms) → store it with a 5-minute expiry → return.
Next 5 minutes: look up key → hit → return the stored value, the block never runs (milliseconds).

So the first user pays 800ms, everyone else for the next 5 minutes gets it free.

Two key parameters:

key ("courier_ranking"): the cache's name. If the result varies by condition, the key must encode that, e.g. "courier_stats/#{courier.id}".
expires_in: how long until it expires, i.e. "how stale can you tolerate." Shorten it for fresher data.

When is something worth caching? ① Expensive — high recompute cost, worth saving. ② Hit repeatedly — the same result is used many times (across users, or the same query requested over and over).

As for "freshness" — that's not a precondition, it's a problem to solve: tolerate some staleness → use expires_in; need it current → use key-based invalidation so it updates the moment data changes (the next layer is exactly this).

🔑 Layer 2 (invalidate): key-based expiration

expires_in only handles "time's up," not "the data changed." You set 5 minutes, but a new shipment lands in between, and users see a stale number. How do you make the cache "update the moment data changes"?

❌ The dumb way: manually delete the cache when data changes

after_commit { Rails.cache.delete("courier_ranking") }

Problem: you have to remember "which operations affect which caches," and clear each one by hand. With many caches it becomes a nightmare — miss one and you've got a stale-data bug. This is the origin of the famous line "cache invalidation is one of the two hard problems."

✅ The smart way: put the "version" in the key, so the key changes when the data does

Rails generates a key for each record that changes with updated_at:

courier.cache_key_with_version
# → "couriers/1-20260626120000"   ← the trailing part is the updated_at timestamp

Use it as the cache key:

Rails.cache.fetch(courier.cache_key_with_version) do
  expensive_render(courier)
end

courier unchanged → key unchanged → hit (old value) ✅
courier modified → updated_at changes → key changes → miss → recomputed automatically ✅

You never manually delete anything — when data changes, the old key is abandoned and the new key naturally computes the new value.

Mindset: don't "clear the old cache," let "a data change produce a new key." That notorious cache-invalidation problem mostly disappears.

🪆 Layer 3 (render): fragment + Russian Doll

So far we cached computed results. But sometimes computing the data is fast and the slow part is rendering it into HTML (lots of ERB, helpers). Here what you cache is the rendered HTML fragment.

Fragment caching: wrap a chunk with cache in the view:

<% @couriers.each do |courier| %>
  <% cache courier do %>          <%# automatically uses courier.cache_key_with_version as the key %>
    <div class="courier-card">
      <h3><%= courier.name %></h3>
      <p>Shipments: <%= courier.shipments_count %></p>
    </div>
  <% end %>
<% end %>

First time → miss → run ERB to produce HTML → store it.
After → hit → return the stored HTML directly, ERB doesn't run.
courier modified → key changes → only that chunk re-renders.

Russian Doll caching: nested — an outer cache wrapping inner caches, like a matryoshka:

<% cache courier do %>                  <%# outer: courier card %>
  <h3><%= courier.name %></h3>
  <% courier.shipments.each do |shipment| %>
    <% cache shipment do %>             <%# inner: each shipment %>
      <%= render shipment %>
    <% end %>
  <% end %>
<% end %>

The clever part is "change one, re-render only one." Change a shipment → only that inner card re-renders; the outer courier card also reassembles, but the unchanged shipments' inner cards come straight from cache, so reassembly is just stitching existing HTML — fast.

For "change a shipment → outer courier updates too" to work, the inner change must propagate up:

class Shipment < ApplicationRecord
  belongs_to :courier, touch: true   # a shipment change also touches the courier's updated_at
end

touch: true makes the inner change bump the outer key, which is what makes the whole Russian Doll work.

Why is touch needed? Because an outer cache hit returns the whole stored HTML wholesale — it never re-checks the inner caches. So if the outer key doesn't change, you'd serve stale inner content. touch forces the outer to miss and reassemble (pulling unchanged inners from cache).

🗄️ Where the cache lives (cache store)

Where Rails.cache stores things is decided by the cache store, and the production choice matters:

store	where	use
`:file_store` (the default when unset)	local disk files	single machine; not shared across machines
`:memory_store`	a single process's memory	lost on restart, not shared across workers — don't use in production
`:redis_cache_store`	Redis	⭐ the production mainstream, shared across servers
`:mem_cache_store`	Memcached	pure-cache scenarios
`:solid_cache_store`	the database (default in new Rails 8 apps)	when you don't want to run a separate Redis

The key point: production usually has multiple servers / workers. With memory_store, the cache server A computed isn't visible to server B — effectively not shared. So use Redis / Memcached, a centralized store, so the whole system shares one cache.

# config/environments/production.rb
config.cache_store = :redis_cache_store, { url: ENV["REDIS_URL"] }

🌐 Layer 4 (transfer): HTTP caching (ETag)

The earlier layers save "recompute," but the server still assembles and sends the response. HTTP caching goes further — when data hasn't changed, don't even send it:

def show
  @courier = Courier.find(params[:id])
  fresh_when(@courier)   # computes an ETag (version fingerprint) from updated_at
end

On response, the server includes an ETag header; the browser remembers it.
On the next request, the browser automatically sends "I have this version."
The server compares: unchanged → returns 304 Not Modified with an empty body, the browser reuses its own copy; changed → normal 200 + new content.

What it saves is bandwidth and transfer — when nothing changed, just a tiny 304 instead of re-sending the whole page.

🏁 Wrap-up: the four layers of caching

layer	tool	what it saves
compute	`Rails.cache.fetch`	recomputing expensive work
invalidate	key-based (`cache_key_with_version`)	the pain of manual cache clearing
render	fragment / Russian Doll	re-rendering HTML
transfer	HTTP caching (ETag)	even the send (304)

All four share one underlying idea:

Use updated_at as a "version" — reuse while data is unchanged, auto-update when it changes.

The biggest trap in caching isn't "how to store," it's "how to keep it from going stale." Remember key-based — don't clear caches by hand, let the key follow the data's version — and that infamous cache-invalidation problem is mostly solved. But don't swing the other way and cache everything either: first confirm the thing is genuinely "expensive + reused," then cache it, or you've just added complexity and a "why am I seeing stale data" debugging nightmare.

📌 This post is about how caching works. But caching's real difficulty is in the failure modes you hit after it ships — hammering the DB the instant a key expires (stampede is just the appetizer), a whole code path breaking when the cache disappears, an unbounded key set blowing up memory, even faithfully storing an error. Those "you only learn it by getting burned" traps are in the follow-up: We added a cache; three days later it took the database down at peak.

ActiveRecord: You Wanted a Number, but Loaded 500,000 Rows Into Memory

Dane Wu — Fri, 26 Jun 2026 01:18:02 +0000

Rails Performance: Lessons from Production — #3

The first two posts were about making queries cheaper — fewer N+1s, indexes, not dragging data back into Ruby. This one turns that into a more practical habit: let the database do what the database can do. A .count in the wrong place, a misused .present?, and you can pull an entire table into memory just to get a single number. Same example throughout (a shipments table).

💥 A report page that ate the server's memory

We had an admin report showing, per courier, the number of delivered shipments. It was fine — until someone tweaked the filtering logic. After that, opening the page spiked memory, and under traffic it OOM-killed the server.

The culprit was this line:

courier.shipments.select { |s| s.status == "delivered" }.size

Looks reasonable — "it's just select." But this .select { ... } (with a block) is Ruby's Array method, not ActiveRecord's select(:col). It loads all of that courier's shipments into memory first, then filters them one by one in Ruby. Fine on a small table; on a big one it eats all the RAM.

One word, select, two completely different meanings — one builds SQL, the other loads everything and filters in Ruby. It's one of the easiest traps to fall into and the hardest to spot. The fix is to let the DB filter and count:

courier.shipments.where(status: "delivered").count
# → SELECT COUNT(*) FROM shipments WHERE courier_id = ? AND status = 'delivered'

The DB filters, counts, and returns a single number. The memory problem vanishes.

Behind this is a recurring pattern: for a single number, boolean, or sum, you drag a whole batch of rows back into Ruby. This post walks through its common variants.

Continuing the four layers from series #1: moving data from ③ the DB back to ④ memory has a cost. The whole theme is — don't move it if you don't have to, and if you must, move less.

🔢 Counting: `count` vs `size` vs `length`

All three "count things," but they work differently — pick wrong and you hit the trap above.

count — always sends SELECT COUNT(*) to the DB and returns a number.

Shipment.where(status: "delivered").count
# → SELECT COUNT(*) FROM shipments WHERE status = 'delivered'   always queries the DB

Downside: even if the records are already loaded, it still queries the DB again.

length — can only count an in-memory array, so it force-loads everything then counts.

shipments = Shipment.where(status: "delivered")
shipments.length
# → SELECT * FROM shipments WHERE status = 'delivered'   pulls the whole batch into memory, then counts the array

Downside: if not loaded, it drags the whole batch into memory (the opening trap).

size — picks based on state: not loaded → acts like count; already loaded → acts like length. Always the cheapest option.

shipments = Shipment.where(status: "delivered")
shipments.size      # not loaded → runs count, SELECT COUNT(*), no rows moved
shipments.to_a      # (this loads the data into memory)
shipments.size      # loaded → counts the in-memory array, no DB query

	Behavior	Risk
`count`	always queries the DB	re-queries even when loaded
`length`	always uses memory	drags the whole batch in when not loaded
`size`	not loaded → DB, loaded → memory	almost none, safest

How to choose: only want a number, won't use the rows → count; data already (or about to be) loaded and you count while using it → size; length almost never.

This also explains why series #1 used c.shipments.size in the view, not .count — shipments were preloaded into memory, so size counts memory; .count would re-query the DB per courier, bringing the N+1 right back.

❓ Existence: `exists?` vs `present?`

You want to know whether a courier has any delivered shipments — just true / false.

❌ present? / any? — loads the matching rows into memory first, then checks if the array is empty:

courier.shipments.where(status: "delivered").present?
# → SELECT * FROM shipments WHERE courier_id = 1 AND status = 'delivered'
#   (loads all matches, just to decide "any?")

✅ exists? — asks the DB and stops at the first hit:

courier.shipments.where(status: "delivered").exists?
# → SELECT 1 FROM shipments WHERE courier_id = 1 AND status = 'delivered' LIMIT 1

LIMIT 1 is the key: the DB returns "yes" on the first match — no full scan, no rows moved.

Exception: if you'll use those rows right after the check, load them first with .to_a and use present? — the load isn't wasted:

shipments = courier.shipments.where(status: "delivered").to_a
if shipments.present?   # ✅ present? is right here, the data is already needed
  shipments.each { ... }
end

In one line: just checking existence, won't use the rows → exists?; using the rows right after → load first, then present?.

➕ Aggregation: sum, average, group — hand them to the DB

Compute "total weight of all delivered shipments."

❌ Pull rows back and loop in Ruby:

Shipment.where(status: "delivered").sum { |s| s.weight }   # block = Ruby method

sum with a block is Ruby's Array method — it loads 500k objects into memory, then loops and adds them one by one.

✅ Let the DB sum:

Shipment.where(status: "delivered").sum(:weight)   # column name = ActiveRecord method
# → SELECT SUM(weight) FROM shipments WHERE status = 'delivered'

The difference is the argument: sum(:weight) passes a column symbol → builds SQL for the DB ✅; sum { |s| s.weight } passes a block → loads then loops in Ruby ❌.

Other aggregates work the same — ActiveRecord wraps them:

Shipment.average(:weight)   # AVG
Shipment.maximum(:weight)   # MAX
Shipment.minimum(:weight)   # MIN

Grouped counts (how many each courier shipped) also go to the DB:

# ❌ pull everything, group in Ruby
Shipment.all.group_by(&:courier_id).transform_values(&:count)

# ✅ DB groups & counts, returns a tiny result
Shipment.group(:courier_id).count
# → SELECT courier_id, COUNT(*) FROM shipments GROUP BY courier_id
# → { 1 => 1200, 2 => 800, ... }

Rule of thumb: check whether sum / count is given a column name or a block — column name goes to the DB (right), block runs in Ruby (usually wrong).

In one line: sum, average, count, group — operations that "collapse a pile of rows into one result" are exactly what the DB does, right next to the data. Pulling rows back to compute = hauling a truckload home just to read one number off it.

📦 When you really must: `find_each` loads in batches

Sometimes you genuinely have to pull every row back into Ruby (sending emails, generating reports, exporting). For that unavoidable case, the point shifts to: don't load all at once — load in batches.

❌ .each — loads everything into memory at once:

Shipment.where(status: "delivered").each do |s|
  SendMailJob.perform_later(s)
end
# → SELECT * ... pulls all 500k rows; every object in RAM at once → possible OOM

✅ find_each — fetches batch by batch under the hood (1000 by default), processing one batch before loading the next:

Shipment.where(status: "delivered").find_each do |s|
  SendMailJob.perform_later(s)
end
# → SELECT * ... ORDER BY id LIMIT 1000
#   SELECT * ... AND id > 1000 ORDER BY id LIMIT 1000
#   ...uses the last id of the previous batch as the next starting point

At any moment only 1000 rows are in memory — 500k won't blow up. Batch size is tunable: find_each(batch_size: 500).

For operating on a whole batch (bulk updates), use in_batches:

Shipment.where(status: "pending").in_batches do |batch|
  batch.update_all(status: "processing")   # update a whole batch at once, not row by row
end

🏁 Wrap-up

You want	❌ pull into Ruby	✅ keep in the DB / batch
a count	`.to_a.count`	`.count` / `.size`
existence	`.present?`	`.exists?`
sum / avg / group	pull back and loop	`sum(:column)`, `group.count`
process many rows	`.each` (load all)	`find_each` (batch)

One principle runs through all of it:

Let the database do what the database can do; when you must pull into Ruby, do it in batches, not all at once.

This is the same thing as series #1 — moving data into memory has a cost. That opening OOM wasn't caused by "too much data," but by "moving too much data back for a single number." Before you write count, sum, or present?, ask: do I want a result, or do I actually need this whole batch of rows? It's usually the former — so let the DB hand you the result.

A 1000x Speedup From One Index — and Why It Sometimes Does Nothing

Dane Wu — Thu, 25 Jun 2026 15:55:02 +0000

Rails Performance: Lessons from Production — #2

"Slow query? Add an index" is something everyone has heard. But I once hit a more embarrassing situation: I added the index, and the query didn't get any faster. Debugging that forced me to actually understand how indexes work — when they're lightning fast, and when they simply won't be used. This post walks through it with one example: a shipments table with a few million rows.

🐌 I added an index and the query was still slow

The shipments table has 3 million rows. The front end looks up one row by tracking number: Shipment.where(tracking_no: "ABC123") — 12 seconds slow. I added an index and it dropped to 8 milliseconds. One line of CREATE INDEX, a thousand times faster. Felt great.

Until another query, where I added an index the same way, and EXPLAIN still showed a full table scan — no faster at all. That's when it clicked: an index isn't magic that makes things fast just by existing. It has its own rules — and if you don't understand them, adding one does nothing.

📖 Why an index is fast: start from the full table scan

Without an index, the DB can only find a row by scanning from the first row to the last, comparing tracking_no one by one. With 3 million rows that's 1.5 million on average — like a book with no index, where finding a word means flipping through the whole thing.

SELECT * FROM shipments WHERE tracking_no = 'ABC123';
-- No index: Seq Scan, scans all 3 million rows

With an index, it's like the index page at the back of a book: it keeps tracking_no pre-sorted in a separate copy, each value noting "where this row lives in the main table." The DB doesn't flip through the whole book — it locates the value directly in the sorted index, then jumps to that row.

CREATE INDEX index_shipments_on_tracking_no ON shipments (tracking_no);

One sentence: an index is fast because it's pre-sorted, so the DB can use "fast lookup on sorted data" instead of "scan every row."

🌳 Under the hood it's a B-tree (not a binary tree)

Common confusion: an index's B-tree is not a binary tree. A binary tree has at most 2 branches per node; a B-tree has hundreds of branches per node — a wide, shallow tree.

Why wide and shallow? Because reading from disk is slow. Every level the DB descends is one more disk read.

Binary tree: 3 million rows is ~21 levels deep → 21 disk reads.
B-tree: each level fans out hundreds of times, so 3 million rows is only 3–4 levels → 3–4 disk reads.

So the whole point of a B-tree is to minimize the number of disk reads. That's the physical reason an index is fast.

💰 Indexes aren't free, so you can't index every column

An index is "an extra sorted copy of the data." Two costs:

Writes get slower — every INSERT / UPDATE / DELETE doesn't just change the main table, it has to maintain every index (inserting the new value into the right spot in each B-tree). With 5 indexes, each write touches 6 places. For a table like shipments that constantly takes new orders, that cost is very real.

Space — an index really does store an extra copy on disk.

So an index is fundamentally a trade-off: slower writes and more storage in exchange for faster reads. The rule of thumb: only index a column that's actually queried (where / order / join). Read-heavy, write-light columns pay off the most; on write-heavy tables, be sparing.

🎯 So when do you actually add one?

Indexes come from two places:

1. The ones you add by design (predictable)

Foreign keys: shipments.courier_id — you'll definitely where(courier_id:) and join on it.
Unique lookup columns: tracking_no, email, slug — the ones you use to pinpoint a single row.
Obviously high-frequency filter/sort columns: ones you know at design time will be queried constantly.

Rails note: in a migration, t.references / add_reference builds an index on the foreign-key column by default; foreign_key: true adds a data-integrity constraint, which is a separate thing from an index — don't conflate them.

2. The ones you add after launch, driven by data (the important part)

Don't speculatively add the rest — you can't guess which will be slow. The real workflow is "measure first, then index where needed":

Find the slow query from APM or the DB's slow query log.
Use EXPLAIN to confirm it's actually doing a full table scan (Seq Scan in Postgres, type: ALL in MySQL).
Add an index on that column, then EXPLAIN again to confirm it became an Index Scan.
Confirm it's actually faster.

One thing often overlooked: small tables don't need indexes. A few thousand rows scans in milliseconds anyway; an index just adds write cost. The value of an index only shows up once the table is big enough.

In short: add the few predictable ones by design; for the rest, index against actual slow queries and verify with EXPLAIN; leave small tables alone.

🔑 Composite indexes and the leftmost prefix

Real queries are often multi-condition: where(courier_id: 1, status: "delivered"). For that, a single composite index can cover multiple columns:

CREATE INDEX idx ON shipments (courier_id, status, created_at);

The key is how it's sorted: first by courier_id, then by status within the same courier_id, then by created_at within the same status. Just like a phone book — sorted by last name, then first name within a last name.

The leftmost-prefix rule: because it sorts starting from the left column, you can only use it contiguously from the left.

Query	Uses the index?
`where(courier_id: 1)`	✅
`where(courier_id: 1, status: "x")`	✅
`where(status: "delivered")`	❌ skips the leftmost courier_id

Why doesn't where(status:) use it? The phone book is sorted by last name first. If you only know the first name "John" but not the last name, the Johns are scattered across every last name — not grouped by first name — so you still flip through the whole book. status is scattered in the index the same way.

So column order matters. Two principles:

Put the column most often queried alone on the left (so it works both alone and combined).
Equality (=) before range (>, <). A range "truncates" the index — created_at > x spans a big stretch across many different created_at values, and within that stretch the later columns are out of order, so the index can't continue.

⚠️ Why an index sometimes doesn't kick in (back to the opening trap)

EXPLAIN still shows a full table scan even though you added the index — usually one of these, all sharing one cause: they defeat the index's sort order.

1. Applying a function or operation to the column

where("DATE(created_at) = ?", "2026-06-25")          # ❌ DB must compute DATE() per row to compare
where(created_at: Date.parse("2026-06-25").all_day)  # ✅ use a range instead — touch the value, not the column

The index sorts by the column's raw value; the moment you compute on the column, that sort order is useless.

2. LIKE '%xxx' with a leading wildcard

where("tracking_no LIKE ?", "ABC%")   # ✅ knows the prefix, can locate
where("tracking_no LIKE ?", "%123")   # ❌ no prefix, scans everything

3. Type mismatch — tracking_no is a string column but you pass a number:

where(tracking_no: 123)    # ❌ DB casts every row's column to a number to compare = operating on the column → index defeated
where(tracking_no: "123")  # ✅ types match, uses the index directly

4. The condition matches too large a fraction — if a condition hits a big chunk of the table, the DB deliberately skips the index, because "locate + repeatedly jump back to the main table" ends up slower than just scanning once:

where(status: "active")    # ❌ if 90% of shipments are active → DB prefers a full scan
where(status: "returned")  # ✅ only 1%, a small slice → the index pays off

Indexes are for grabbing a small slice, not a big chunk. So a column like status with only a few, unevenly distributed values often can't use an index.

5. OR across different columns — courier_id and tracking_no each have an index, but a single OR query struggles to use both at once and often degrades to a full scan. Splitting into two queries, each using its own index, then merging, is faster:

# ❌ one OR, hard to use both indexes
Shipment.where("courier_id = ? OR tracking_no = ?", 1, "ABC123")

# ✅ split into two, each uses its own index, then merge + dedupe
a = Shipment.where(courier_id: 1)
b = Shipment.where(tracking_no: "ABC123")
(a + b).uniq

-- the SQL equivalent: UNION two queries that each use their index
SELECT * FROM shipments WHERE courier_id = 1
UNION
SELECT * FROM shipments WHERE tracking_no = 'ABC123';

The shared principle: an index works off the sort order of the column's raw value. Anything that makes the DB "compute/cast the column per row" or "lose the ability to locate by prefix" defeats it. So — don't touch the column, touch the value side instead.

🏁 Wrap-up

An index isn't a switch that makes things fast just by flipping it. It's fast because it's pre-sorted and the B-tree minimizes disk reads; it has a cost, so you only index columns that get queried; and it has rules — a composite index is used contiguously from the left, and operating on a column defeats it.

That opening query that "had an index but wasn't faster" turned out to have a function wrapped around the column. The question was never "should I add an index" — it's whether you understand how it sorts and when it gets used. Next time a query is slow, don't rush to CREATE INDEX — run EXPLAIN first and see what it's actually doing. The answer is usually right there.

🧩 Appendix: `(a + b).uniq` and UNION / DISTINCT

About that (a + b).uniq from the OR-split above, in three parts:

a = Shipment.where(courier_id: 1)          # query 1
b = Shipment.where(tracking_no: "ABC123")  # query 2
(a + b).uniq

① What a + b does — runs both queries, fetches each, and concatenates them into one array (not one combined SQL statement).

② What .uniq does — dedupes. A shipment might satisfy both conditions and appear on both sides, so concatenating creates duplicates; .uniq folds them into one.

③ Why + hits the DB — because a and b aren't arrays yet; they're "not-yet-executed queries" (ActiveRecord::Relation), lazy. (Back to the four layers from #1: .where only accumulates conditions, it hasn't touched the DB.) + is an Array method — it needs arrays, not relations — so it forces both to execute and become arrays before adding:

a + b
#  → a.to_a fires SELECT ... WHERE courier_id = 1
#  → b.to_a fires SELECT ... WHERE tracking_no = 'ABC123'
#  → then concatenates the two arrays

So hitting the DB isn't done by + — it's a side effect of + forcing the relations to execute so it can build the array.

The whole (a + b).uniq is just simulating SQL's UNION in Ruby: two separate queries, merged, deduped.

Aside: UNION vs DISTINCT — both dedupe; the difference is "whose duplicates":

	What it does
`DISTINCT`	dedupes within one result set (one query)
`UNION`	merges two queries + dedupes
`UNION ALL`	merges two queries but does not dedupe (faster)

Ruby on Rails activerecord

Dane Wu — Thu, 25 Jun 2026 13:16:55 +0000

From 8s to 1s: Truly Understanding Rails N+1 by Opening Up ActiveRecord

Dane Wu — Thu, 25 Jun 2026 09:59:05 +0000

Rails Performance: Lessons from Production — #1

Most people fix N+1 by "adding includes." But includes isn't a silver bullet — I've hit a page that already had includes, that Bullet never flagged, and that was still N+1. That forced me to actually understand what ActiveRecord does underneath, instead of memorizing a few rules. This post takes it apart one layer at a time, using one example throughout (Courier has_many :shipments), with the actual SQL at every step.

🐢 I added `includes` and the page was still N+1

We had an admin page listing couriers, each row showing how many shipments that courier had. As data grew, the page went from 1 second to 7–8 seconds.

I opened the APM (New Relic): SQL alone took 6+ seconds, and the query count had exploded — one page listing 200 couriers fired 200+ queries. Classic N+1. I added includes(:shipments), the query count dropped to 2, the page came back under 1 second, and I installed the Bullet gem in dev to catch N+1 automatically. Problem solved, I thought.

Until another page — which already had includes, and which Bullet never warned about — was still slow, still N+1. Debugging that one taught me the real lesson: whether includes works depends on how you use the data it preloads, and you can only see that if you understand what ActiveRecord does underneath.

So this post doesn't start with "add includes." It starts one level deeper.

Same example throughout: Courier has_many :shipments. Every step shows the actual SQL that gets sent.

🧱 Layer 0: every query passes through four stages

Every line of ActiveRecord you write runs through this pipeline:

① ActiveRecord syntax (Ruby)
     Courier.where(...).includes(...)
     just "accumulating conditions", lazy — hasn't touched the DB yet
         ↓ Rails translates
② SQL (a string)
     SELECT ... FROM couriers WHERE ...
     your conditions turned into something the DB understands
         ↓ sent
③ Database executes
     where the real work happens: JOIN, WHERE filter, ORDER, DISTINCT
     all of it here → returns a flat table of rows
         ↓ returned
④ Memory (Ruby objects)
     Rails turns each row back into an ActiveRecord object for you

Remember two sentences — everything below is a corollary of them:

Filtering, JOINs, sorting, deduping all happen in ③ the database. And the DB can only filter on columns that exist in that SQL statement — a column that isn't in the query can't be filtered on.
Objects, .size, association arrays all happen in ④ memory. The data is already back; nothing here touches the DB.

ActiveRecord's selling point is hiding ②③ so it feels like there's only ① and ④. Great most of the time — but every performance problem lives in the ②③ you stopped looking at.

🔁 Layer 1: what N+1 actually is

Let's grab three rows with limit(3). The naive version (no preloading):

couriers = Courier.limit(3)
couriers.each do |c|
  c.shipments.size   # every iteration fires another query against ③
end

The actual SQL sent:

SELECT * FROM couriers LIMIT 3;                  -- the "1": fetch 3 couriers
SELECT * FROM shipments WHERE courier_id = 1;    -- courier #1
SELECT * FROM shipments WHERE courier_id = 2;    -- courier #2
SELECT * FROM shipments WHERE courier_id = 3;    -- courier #3

Fetching 3 couriers is 1 query; each iteration adds 1 more for its shipments: 1 + 3 = 4. With 200 couriers, that's 201.

Why? Because c.shipments is lazy by default — touch it and it fires a fresh query against ③. Put that inside a loop and you get N queries.

📦 Layer 2: three ways to preload, collapsing those N queries

The only difference is "how many SQL statements ② produces, and whether ③ does a JOIN." Same three rows:

preload — fetch separately (no JOIN)

Courier.preload(:shipments).limit(3)

SELECT * FROM couriers LIMIT 3;                        -- parent table
SELECT * FROM shipments WHERE courier_id IN (1, 2, 3); -- the N queries collapsed into one IN

Afterwards, Rails matches the shipments back to their courier in ④ memory using courier_id. Always 2 queries.

eager_load — one query with a LEFT JOIN

Courier.eager_load(:shipments).limit(3)

SELECT couriers.id   AS t0_r0, couriers.name AS t0_r1,
       shipments.id  AS t1_r0, shipments.tracking_no AS t1_r1
FROM couriers
LEFT JOIN shipments ON shipments.courier_id = couriers.id;

One SQL statement, joining into a wide table. The columns get renamed to t0_* / t1_* because once the tables are joined, id and name collide — Rails numbers each column so it can split the row back into separate objects.

includes — let Rails choose

# A: no filtering on the association → runs as preload (the 2 queries above)
Courier.includes(:shipments).limit(3)

# B: filtering on a shipments column → auto-switches to eager_load (the JOIN above)
Courier.includes(:shipments).where(shipments: { status: "delivered" })
       .references(:shipments)

includes isn't a third fetching strategy — it auto-switches: no association filtering → preload; filtering on an association column (add references) → eager_load (JOIN). Use it by default and let Rails decide.

So now the N queries are collapsed to a fixed count. But real pages aren't that simple — which is the payoff for that "includes still N+1" teaser. Here's the core.

🔀 Part 2 — After preloading: are you filtering the child or the parent?

Real pages rarely "fetch everything and show it" — they have conditions. So first answer one question, because it decides which solution you use:

Filtering the child — every courier should only display a subset of its shipments (e.g. only delivered ones).
Filtering the parent — you only want to keep couriers matching a condition (e.g. only couriers that have a delivered shipment).

These are two different needs with two different solutions. Conflating them is exactly how you write the "added includes but still N+1" code. Let's split them.

🍃 Need A: filter the child — each courier shows only its delivered shipments

Keep every courier, but each one only displays its "delivered" shipments.

❌ The naive way: `.where` on the association in the view (this is the bug from the opening)

# controller: preloads ALL shipments
couriers = Courier.includes(:shipments).limit(3)

<%# view: but each courier only wants the delivered ones %>
<% couriers.each do |c| %>
  <%= c.shipments.where(status: "delivered").size %>   <%# ← blows up here %>
<% end %>

The actual SQL:

-- includes preloads ALL shipments first (these 2 are fine on their own)
SELECT * FROM couriers LIMIT 3;
SELECT * FROM shipments WHERE courier_id IN (1, 2, 3);

-- but .where(status:) in the view is a NEW conditional query; Rails ignores the in-memory set
-- and re-queries once per courier:
SELECT * FROM shipments WHERE courier_id = 1 AND status = 'delivered';
SELECT * FROM shipments WHERE courier_id = 2 AND status = 'delivered';
SELECT * FROM shipments WHERE courier_id = 3 AND status = 'delivered';

Double waste: you spent 2 queries loading all shipments into ④ memory (never used), then .where re-queries once per courier → N+1 is fully back. And Bullet stays silent, because :shipments was preloaded — it only checks "did you preload," not "did you actually use what you preloaded."

Root cause (back to Layer 0): it's the nature of .where.

In ActiveRecord, .where is always defined as "build a new SQL statement and send it to ③ the database." It is not "filter the array in ④ memory." Two different layers:

What you assumed: this courier's shipments are already in memory, so .where just picks the delivered ones from that array.
What actually happens: the moment .where is called, Rails builds a WHERE ... AND status = 'delivered' and re-queries ③, completely ignoring the in-memory set.

Why doesn't it filter in memory? Timing. The preload order is: ② builds and sends the "all shipments" SQL → ③ runs it → ④ loads it into memory. By the time you write .where in the view, that preload SQL was built, sent, and returned long ago — the condition arrives too late to be slipped into a statement that already ran. Rails can't go back and rewrite that SQL, so it builds a fresh one against the DB.

To filter the in-memory set without hitting the DB, use Ruby's select (not SQL's): c.shipments.select { |s| s.status == "delivered" } — that genuinely filters the ④ array. But it still has to load everything first, which is rough on memory at scale. The clean fix is the scoped association below, making the condition take effect back at ②.

✅ The fix: a scoped association (weld the condition into the association definition)

Make the condition take effect at ② when the SQL is built, not patched on at ④. Define an association that carries the condition:

class Courier < ApplicationRecord
  has_many :shipments
  # scoped association: this association only ever contains delivered ones
  has_many :delivered_shipments,
           -> { where(status: "delivered") },
           class_name: "Shipment"
end

# controller
couriers = Courier.includes(:delivered_shipments).limit(3)

<%# view: use it directly, no more .where %>
<% couriers.each do |c| %>
  <%= c.delivered_shipments.size %>
<% end %>

The actual SQL — back to a fixed 2 queries, with the condition merged into the preload's IN:

SELECT * FROM couriers LIMIT 3;
SELECT * FROM shipments WHERE courier_id IN (1, 2, 3) AND status = 'delivered';

The key difference: the naive condition shows up at ④ memory (too late, doesn't match the preload); the scoped association makes the condition part of the association definition, so Rails sees it at ② when building the preload SQL and welds it straight into the same IN — ③ fetches exactly what's needed in one shot.

Analogy: sending someone to the warehouse. The naive way is "bring back everything," and only when it's in front of you do you say "I just wanted the delivered ones" → a wasted trip, and a second one to filter. The scoped association writes "only the delivered ones" on the list before they leave → one correct trip.

(Detail: an association scope must be wrapped in a lambda -> { ... } — that's required Rails syntax. The lambda also means "run the condition on every query," which matters for time-based values like 7.days.ago: with the lambda it's recomputed each query as "7 days before today"; without it, the value is computed once when the class loads and frozen at the server's boot date, so "last 7 days" silently drifts wrong as time passes. Our status: "delivered" is a constant so it makes no difference here — but scopes are always written as lambdas regardless.)

🌳 Need B (advanced): filter the parent — keep only couriers that have a delivered shipment

List only couriers with at least one delivered shipment — drop the rest entirely. Note how this differs from Need A: A keeps all couriers and filters their shipments; B filters the couriers themselves.

⚠️ The naive way: eager_load + where (works, but "row explosion" on has_many)

To filter the parent by a child column (shipments.status), you must JOIN — preload can't do it (Layer 0: filtering happens in ③, so the column has to be in that SQL). The obvious tool is eager_load:

Courier.eager_load(:shipments).where(shipments: { status: "delivered" })

SELECT couriers.*, shipments.*
FROM couriers
LEFT JOIN shipments ON shipments.courier_id = couriers.id
WHERE shipments.status = 'delivered';

Correct result, but a performance trap: eager_load also pulls the shipment columns back to build objects, and on a has_many the parent gets duplicated. If a courier has 50 delivered shipments, that courier's columns appear in 50 rows:

courier_id | name      | ... | shipment_id
1          | BlackCat  | ... | 10
1          | BlackCat  | ... | 11   ← BlackCat's whole row copied again
1          | BlackCat  | ... | 12   ← and again (×50)

200 couriers × 50 each → 10,000 rows, parent columns copied over and over. Transfer size and memory both balloon, and Rails still has to dedupe it back into objects. At scale, it drags.

✅ The fix: joins to filter + distinct to dedupe + preload to load

The principle: "filtering" and "displaying" are two separate jobs — split them. Filter with joins (doesn't pull child columns, avoids the explosion), display with preload (a separate IN).

Two tools first:

joins: does an INNER JOIN but only SELECTs the parent's columns. The child is just a filter, never loaded into objects.
(Note: "doesn't pull the child back" isn't a property of JOIN — it's a consequence of joins only selecting the parent. JOIN connects tables; SELECT decides which columns come back. joins writes SELECT couriers.*; eager_load writes SELECT couriers.*, shipments.*. That's the whole difference.)

distinct: folds identical rows into one. Because joins on a has_many duplicates the parent, but the child columns aren't selected, those rows are "identical in every column" — so distinct collapses them into one.

Put together:

couriers = Courier.joins(:shipments)                          # INNER JOIN, only to filter
                  .where(shipments: { status: "delivered" })  # filter couriers by a child column
                  .distinct                                    # drop duplicate couriers from the JOIN
                  .preload(:shipments)                         # separately load shipments to display

The actual SQL — two queries, each with one job:

-- ① joins + distinct: only to find couriers WITH a delivered shipment; child columns not selected → no explosion
SELECT DISTINCT couriers.* FROM couriers
INNER JOIN shipments ON shipments.courier_id = couriers.id
WHERE shipments.status = 'delivered';
-- say this returns couriers 1 and 2

-- ② preload: a separate IN to load the shipments to display, into ④ memory
SELECT * FROM shipments WHERE courier_id IN (1, 2);

The view uses c.shipments (already preloaded):

<% couriers.each do |c| %>
  <%= c.name %>: <%= c.shipments.size %>
<% end %>

Filtering goes to the JOIN, loading goes to preload — so you get both "filter the parent by a child column" and "no row explosion."

Note that preload(:shipments) here loads all shipments (including non-delivered ones). If you want "keep only couriers with a delivered shipment and display only the delivered ones," combine A and B: joins(:delivered_shipments).distinct.preload(:delivered_shipments). Scoped associations and joins compose.

🧭 Wrap-up: every rule is a corollary of the same foundation

Looking back, this post started from one "added includes but still N+1" incident and ended up at a single thing: for the line of Ruby you wrote, what happens in which layer.

Once the four layers click, all the scattered "rules" connect:

preload can't filter by an association column — because filtering is in ③, and the child table isn't in the main SQL.
c.shipments.where(...) breaks the preload — because .where re-queries ③, ignoring the ④ in-memory set.
joins doesn't load the association, eager_load does — the difference is which columns SELECT pulls.
.size doesn't hit the DB, .count does — .size counts the ④ array, .count goes back to ③ with a SQL statement.

And "Need A vs Need B" is just two sides of the same question: are you filtering the child or the parent? Answer that and the solution falls out — filter the child with a scoped association, filter the parent with joins (and preload if row explosion is a concern).

🗺️ Appendix: the full N+1 map

"Filter child/parent" is one piece — don't mistake it for the whole. The full breakdown:

First cut: is it preloaded at all?
  No  → add includes (fixes ~80%, the basics)
  Yes but still slow ↓
Second cut: HOW are you using the preloaded data?
  ├ Filtering with a condition → filter child (scoped association) / filter parent (joins + preload)  ← this post
  ├ .count in a loop → use .size (counts memory) or counter_cache (skips preload entirely)
  └ Nested associations: c.shipments.each { s.packages } → includes(shipments: :packages)

The first cut is the basics — an includes away. The one that trips people up is the second cut: "I did preload, why is it still slow?" That's what this post is about.

Optimization isn't memorizing a checklist — it's being able to ask the right question about a slow query: is this step in the DB or in memory? Is the column in that SQL statement? Am I filtering the child or the parent? Answer those and the fix usually follows. This lens carries over to any language's ORM (Django ORM, Eloquent, Hibernate), because they all do the same thing — turn objects into SQL, across databases.

Diagnosing a slow Rails page, layer by layer

Dane Wu — Sun, 21 Jun 2026 11:09:13 +0000

Rails Performance: Lessons from Production — overview

Before optimizing individual layers — N+1, indexes, caching — you need a way to find which layer is actually slow. This is the layer-by-layer diagnostic model the rest of the series builds on, walked end to end on one real slow page.

"This page feels slow" is a vague bug report. Before changing any code, it helps to
have a fixed way to locate where the time goes. A Rails request passes through a few
predictable layers, and each layer has its own tools and its own typical failure mode.

Here is the mental model I use, and a real example of walking it end to end.

🗺️ The layers of a request

Middleware → Controller → SQL → View → external calls → browser

Most slowness lives in one of these:

Layer	Typical problem	How it shows up
Controller	heavy logic in the request	large "Executing" time
SQL (count)	N+1 — many tiny repeated queries	query count explodes
SQL (single)	a slow query, usually a missing index	one query dominates
View	rendering logic, or N+1 hiding in the template	large "Rendering" time
External	a synchronous API/email call	a gap that isn't SQL
Browser	large images, heavy JS	backend fast, page still slow

The point is not to guess. It's to read the numbers and let them point at the layer.

📊 Step 1 — read the numbers (development)

In development I lean on two tools:

rack-mini-profiler — a badge in the corner that breaks a request into controller / view / SQL time, and counts every query.
bullet — watches for N+1 and tells you exactly which association to preload.

On a storefront page that lists a page of 48 products, rack-mini-profiler showed:

Executing: stores#show     2 sql
Rendering: show.html.erb   49 sql   ← 49 queries just to render?
SQL Summary:               51 sql total

Fifty-one queries to render one page of products is a red flag, and the fact that 49 of
them happen during rendering points straight at the view. bullet confirmed it:

USE eager loading detected
Product => [:image_attachment]
Add to your query: .includes([:image_attachment])

🔁 Step 2 — understand the N+1

The products use Active Storage for their images:

class Product < ApplicationRecord
  has_one_attached :image
end

An attached image isn't a column on products. Active Storage spreads it across three
tables: active_storage_attachments (which record owns which file),
active_storage_blobs (the file's metadata + a storage key), and
active_storage_variant_records (generated thumbnails). The file bytes themselves live in a storage service — disk locally, object storage in production.

So every time the view touches product.image, Rails walks those tables. In a loop over N products, that's N extra round-trips: a textbook N+1.

🔧 Step 3 — fix and re-measure

The fix is to preload the attachment once, up front. Active Storage generates a scope
for exactly this:

# before
@products = Product.where("stock > 0")
# after
@products = Product.where("stock > 0").with_attached_image

(with_attached_image is just an Active Storage flavoured includes.)

Re-measured on the same page:

	Before	After
SQL queries	51	3
ActiveRecord time	~210 ms	~12 ms
bullet warnings	yes	none

The query count is now flat regardless of how many products are on the page — O(1) instead of O(n). That's the real win: N+1 isn't scary because of its cost on any single request, it's scary because it grows with your catalog and your traffic. The same page under a few hundred requests a minute turns a handful of extra queries into thousands of extra round-trips against the database.

🐌 The other kind of slow: a single heavy query

N+1 is about query count. The other common case is one query that is slow on its own — usually a missing index. Here EXPLAIN is the tool: it shows how Postgres plans to run a query without running it.

Looking up orders by a column with no index, over an orders table with ~800k rows:

Seq Scan on orders  (cost=0.00..21450.00 rows=1 width=93)
  Filter: (customer_email = 'someone@example.com')

Seq Scan means Postgres reads the whole table row by row to find one order — wasteful when there are hundreds of thousands of them. After adding an index on that column:

Index Scan using index_orders_on_customer_email on orders  (cost=0.42..8.44 rows=1 ...)
  Index Cond: (customer_email = 'someone@example.com')

Seq Scan → Index Scan, and the planner's cost estimate drops from ~21,000 to ~8 — the index turns "scan everything" into "jump straight to the row."

Two gotchas worth knowing:

Data volume matters. On a small table Postgres picks Seq Scan even when an index exists — scanning a few rows is cheaper than an index lookup. The index only earns its keep once the table is large, so test against production-scale data, not a dev seed.
Selectivity matters. An index only helps when the query matches a small slice. A query that returns most of the table will be a Seq Scan regardless.

🚀 How this plays out in production

Development tools (rack-mini-profiler, bullet) catch problems before they ship. But dev never fully mirrors production — you don't hit every page, your data is small, and some N+1s only appear with real data shapes. So production needs an APM (I use Scout) watching real traffic to catch what slipped through.

The end-to-end flow when something is slow in production:

APM flags a slow endpoint
  → reproduce locally with realistic data
  → EXPLAIN the suspect query
  → add the index / preload / cache
  → deploy, confirm in the APM that it actually got faster

bullet is the prevention, the APM is the evidence. They're not redundant — they're defense in depth, because dev can never be a perfect copy of prod.

🏁 Takeaways

Don't guess where a page is slow — read it layer by layer.
Two distinct DB problems: many queries (N+1, fix with includes/preload) vs one slow query (missing index, find with EXPLAIN).
N+1 matters because it scales with data, not because it's slow today.
EXPLAIN results depend on data volume and selectivity — test with realistic data.
Prevent in development, verify in production.

DEV Community: Dane Wu

You've Tuned the Queries and It's Still Slow — Now Change the Data Model

💥 Indexes are added, N+1 is fixed, the couriers page is still slow

🧮 Move 1: counter_cache — stop COUNTing the child table every time

📦 Move 2: store the "computed value" redundantly

⚖️ The core cost of denormalization: consistency

🔗 Other ways to change the data model

🏁 Wrap-up: the whole series as one map

The Code Is Fine, but Requests Queue Until They Time Out: Puma, Pools, CDN

💥 A single request is 50ms, but 100 at once time out

🧵 Puma: workers (processes) × threads

🔌 Connection pool: align threads with DB connections

🌐 CDN: don't make your Rails server ship static assets

🚀 Other production switches worth flipping on

🏁 Wrap-up

Your SQL Is Fast but the API Is Slow: It's the Ruby Layer

💥 SQL took 30ms, the API took 800ms

📦 Trap 1: serializing a pile of fields nobody uses

♻️ Trap 2: allocating too many intermediate objects in a hot path (GC pressure)

🧮 Trap 3: the same computation, run many times in one request

🏁 Wrap-up

One Notification Email Made Our Checkout API 3 Seconds Slower

💥 The checkout was fast, but the user waited 3 seconds

📤 The fix: push it to the background, return immediately

⚙️ The tools: Active Job + Sidekiq / Solid Queue

🧨 The traps (the stuff you only hit after you've shipped it)

🏁 Wrap-up

We Added a Cache; Three Days Later It Took the Database Down at Peak

🐎 Trap 1: the instant it expires, everyone hammers the DB at once (stampede)

🧱 Trap 2: treating the cache as the source of truth — Redis restarts and it crashes

💣 Trap 3: putting something variable in the key — Redis memory blows up

☠️ Trap 4: caching the failed result

🏁 Wrap-up: the cost of caching

The Fastest Query Is the One You Never Run: The Four Layers of Rails Caching

🔥 The same 800ms stat, recomputed for every visitor

📦 Layer 1 (compute): Rails.cache.fetch

🔑 Layer 2 (invalidate): key-based expiration

🪆 Layer 3 (render): fragment + Russian Doll

🗄️ Where the cache lives (cache store)

🌐 Layer 4 (transfer): HTTP caching (ETag)

🏁 Wrap-up: the four layers of caching

ActiveRecord: You Wanted a Number, but Loaded 500,000 Rows Into Memory

💥 A report page that ate the server's memory

🔢 Counting: count vs size vs length

❓ Existence: exists? vs present?

➕ Aggregation: sum, average, group — hand them to the DB

📦 When you really must: find_each loads in batches

🏁 Wrap-up

A 1000x Speedup From One Index — and Why It Sometimes Does Nothing

🐌 I added an index and the query was still slow

📖 Why an index is fast: start from the full table scan

🌳 Under the hood it's a B-tree (not a binary tree)

💰 Indexes aren't free, so you can't index every column

🎯 So when do you actually add one?

🔑 Composite indexes and the leftmost prefix

⚠️ Why an index sometimes doesn't kick in (back to the opening trap)

🏁 Wrap-up

🧩 Appendix: (a + b).uniq and UNION / DISTINCT

Ruby on Rails activerecord

From 8s to 1s: Truly Understanding Rails N+1 by Opening Up ActiveRecord

🐢 I added includes and the page was still N+1

🧱 Layer 0: every query passes through four stages

🔁 Layer 1: what N+1 actually is

📦 Layer 2: three ways to preload, collapsing those N queries

preload — fetch separately (no JOIN)

eager_load — one query with a LEFT JOIN

includes — let Rails choose

🔀 Part 2 — After preloading: are you filtering the child or the parent?

🍃 Need A: filter the child — each courier shows only its delivered shipments

❌ The naive way: .where on the association in the view (this is the bug from the opening)

✅ The fix: a scoped association (weld the condition into the association definition)

🌳 Need B (advanced): filter the parent — keep only couriers that have a delivered shipment

⚠️ The naive way: eager_load + where (works, but "row explosion" on has_many)

✅ The fix: joins to filter + distinct to dedupe + preload to load

🧭 Wrap-up: every rule is a corollary of the same foundation

🗺️ Appendix: the full N+1 map

Diagnosing a slow Rails page, layer by layer

🗺️ The layers of a request

📊 Step 1 — read the numbers (development)

🔁 Step 2 — understand the N+1

📦 Layer 1 (compute): `Rails.cache.fetch`

🔢 Counting: `count` vs `size` vs `length`

❓ Existence: `exists?` vs `present?`

📦 When you really must: `find_each` loads in batches

🧩 Appendix: `(a + b).uniq` and UNION / DISTINCT

🐢 I added `includes` and the page was still N+1

❌ The naive way: `.where` on the association in the view (this is the bug from the opening)