DEV Community

Cover image for The 15-Year Naming Bug: How Rails Finally Got `polynomially_longer` Right
David Teren
David Teren

Posted on

The 15-Year Naming Bug: How Rails Finally Got `polynomially_longer` Right

#rails #performance

A tale of mathematical misnomers, production outages, and why naming things correctly really does matter.

TL;DR

For 15 years, Rails (and nearly every major Ruby background-job library) shipped a retry strategy called :exponentially_longer that was never exponential — it was a quartic polynomial (executions**4).

This mismatch caused real queue floods and mis-planned capacity.

Rails 7.1 finally fixed the name:

wait: :polynomially_longer   # the honest one
Enter fullscreen mode Exit fullscreen mode

It Started With a SolidQueue Mystery

Our Rails 8 app processes retail audio files, forking SoX and LAME, hammering the database, and crashing with SIGSEGV under load. Deadlocks and connection-pool exhaustion were frequent.

We reached for ActiveJob’s retry mechanism and wrote:

class SpmProcessJob < ApplicationJob
  retry_on ActiveRecord::Deadlocked, wait: :polynomially_longer, attempts: 5
  retry_on PG::ConnectionBad,        wait: :polynomially_longer, attempts: 3
end
Enter fullscreen mode Exit fullscreen mode

Wait… polynomially_longer?

Wasn’t that always called exponentially_longer?

That single question unveiled one of the longest-running naming bugs in the Ruby ecosystem.

The Discovery (September 2023)

Victor Mours was investigating a partial production outage in a French social-services appointment system. Background jobs were timing out and retrying so fast they overflowed the queues.

During the post-mortem they discovered the supposed exponential backoff wasn’t exponential at all — it was polynomial. And the mistake was 15 years old.

→ Rails PR #49292

Exponential vs Polynomial – The Numbers

Rails’ real formula (including jitter + base offset):

delay = executions**4 + (rand * executions**4 * 0.15) + 2
Enter fullscreen mode Exit fullscreen mode
Retry # Polynomial (real observed) True exponential 4ˣ
1 ~3 sec 4 sec
2 ~18 sec 16 sec
3 ~1.4 min 1 min
5 ~10 min 17 min
7 ~40 min 5 hours
10 ~3 hours 12 days

It feels exponential for the first couple of retries… then diverges dramatically.

Where It All Began (2008)

Tobias Lütke’s original delayed_job:

# 2008 – the seed of the bug
time + (attempts ** 4) + 5
Enter fullscreen mode Exit fullscreen mode

A year later the docs called it “exponential.” From there it spread unchanged to Sidekiq, Resque-retry, Sneakers, and finally ActiveJob in Rails 4.2 (2016) under the name :exponentially_longer.

People Noticed – Years Earlier

  • Sidekiq #3569 (2017) – “This is quartic, not exponential.” Closed without fix.
  • GoodJob users (2023) – Started seeing deprecation warnings after Rails 7.1 and realized the same thing.

Why This Actually Hurt Production

  • Queues filled far faster than capacity plans predicted
  • Retries happened “too soon” during DB or API outages
  • Misleading mental models for incident response
  • Secondary failures (connection-pool exhaustion, thundering herds despite jitter)

The Fix – Rails 7.1 

# Rails 7.1+
retry_on MyError, wait: :polynomially_longer, attempts: 10

# Old name (still works, but warns)
retry_on MyError, wait: :exponentially_longer
Enter fullscreen mode Exit fullscreen mode

No behavior change — just truth in labeling.

Rafael França (Rails core): “The intent is really to be a polynomial backoff.”

When Polynomial Backoff Is Perfect

Great for:

  • ActiveRecord deadlocks
  • Connection-pool exhaustion
  • Transient network blips
  • Rate-limiting responses

Avoid for prolonged third-party outages — use a custom exponential proc instead.

Custom Strategies (when you need something else) 

# True exponential
retry_on ApiOutage, wait: ->(n) { (2 ** n).seconds }, attempts: 12

# Fixed interval
retry_on RateLimitError, wait: 30.seconds

# Time-aware (don’t retry at 3 a.m.)
retry_on BatchError, wait: ->(n) {
  delay = n**4
  delay += 8.hours if Time.current.hour.between?(2, 5)
  delay.seconds
}
Enter fullscreen mode Exit fullscreen mode

Takeaways

  1. Names matter — a bad one misled an entire ecosystem for a decade and a half.
  2. Polynomial ≠ exponential; the difference is enormous after ~5 retries.
  3. Jitter is your friend (Rails includes it by default).
  4. Audit your jobs when upgrading to Rails 7.1/8.0 — you’ll see deprecation warnings.
  5. If you ever thought Rails or Sidekiq gave you exponential backoff… you never actually had it.

Run this today and smile:

grep -r "exponentially_longer" app/jobs/
Enter fullscreen mode Exit fullscreen mode

You now know exactly what you’re getting — and why it’s called polynomially_longer.

References

Top comments (0)