DEV Community

developerz.ai
developerz.ai

Posted on

Scaling Rails Applications with Background Jobs: Practical Patterns

Scaling Rails Applications with Background Jobs: Practical Patterns

When a Rails app grows beyond a few thousand daily users, synchronous request‑response cycles start to choke. Offloading work to background jobs not only improves perceived performance but also protects your database from overload. Below are three battle‑tested patterns we use at developerz.ai to keep Rails services responsive at scale.

1. Use find_each for Batch Processing

Rails’ find_each loads records in batches (default 1000) and yields them one at a time, preventing memory bloat. Combine it with a background worker:

class BulkEmailWorker < ApplicationJob
  queue_as :default

  def perform(user_ids)
    User.where(id: user_ids).find_each(batch_size: 500) do |user|
      UserMailer.welcome(user).deliver_later
    end
  end
end
Enter fullscreen mode Exit fullscreen mode

This ensures only a handful of User objects live in memory at any moment.

2. Leverage ActiveJob’s set(wait:) for Rate‑Limited APIs

External APIs often impose rate limits. Scheduling jobs with a delay spreads the load:

User.where(active: true).find_each do |user|
  SyncExternalProfileJob.set(wait: rand(1..5).minutes).perform_later(user.id)
end
Enter fullscreen mode Exit fullscreen mode

The random wait avoids thundering‑herd spikes and keeps your API keys safe.

3. Partition Work with Sidekiq Queues

Separate critical from non‑critical work by assigning different queues:

:queues:
  - critical
  - default
  - low_priority
Enter fullscreen mode Exit fullscreen mode

Critical jobs (e.g., payment processing) go to the critical queue, guaranteeing they’re pulled first. Low‑priority tasks such as analytics aggregation can sit in low_priority and be processed when the system is idle.

Putting It All Together

A typical flow looks like this:

  1. Controller receives a request and enqueues a job.
  2. Job uses find_each to process records in batches.
  3. Sidekiq routes the job to the appropriate queue.
  4. Rate‑limited external calls get scheduled with set(wait:).

By following these patterns, you’ll see a measurable reduction in request latency and a more predictable memory footprint.

Ready to refactor your Rails stack? Reach out at developerz.ai for a code review or a custom implementation.

Top comments (0)