Mastering Background Processing in Rails 8: Sidekiq & Redis Optimization

In modern web applications, keeping the Request-Response cycle fast is crucial for user experience. If a user registers on your Rails 8 app and you trigger a welcome email directly inside the controller, the browser will remain in a loading state until the SMTP server responds.
​
To solve this, we offload heavy, non-blocking tasks (like sending emails, generating PDFs, or syncing third-party APIs) to background workers. While Rails 8 now introduces Solid Queue as its default database-backed adapter, Sidekiq remains the industry gold standard for high-throughput, multi-threaded asynchronous processing powered by Redis (an in-memory data store).

​1. System Requirements & Installation
​
First, ensure that Redis is installed and running on your production or local environment.
​
For Ubuntu/Linux:

sudo apt update
sudo apt install redis-server
sudo systemctl enable redis-server.service

Next, add the Sidekiq gem to your Rails 8 Gemfile:

gem 'sidekiq'

Execute the bundle command in your terminal:

bundle install

2. Configuring Rails 8 to use Sidekiq

​Even with Rails 8's new defaults, switching to Sidekiq is seamless. You need to instruct your Active Job framework to use the Sidekiq adapter. Update your config/application.rb or config/environments/production.rb:

module BlogApp
  class Application < Rails::Application
    # Initialize configuration defaults for originally generated Rails version.
    config.load_defaults 8.0

    # Setting Sidekiq as the backend queue adapter
    config.active_job.queue_adapter = :sidekiq
  end
end

3. Production-Ready Redis Connection Pooling

​A common issue in production is running out of Redis connections due to improper pool sizes. To handle this cleanly, create a dedicated initializer file at config/initializers/sidekiq.rb:

# config/initializers/sidekiq.rb

Sidekiq.configure_server do |config|
  # Server pool size should match or slightly exceed your concurrency limit
  config.redis = { 
    url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/0'), 
    size: 25 
  }
end

Sidekiq.configure_client do |config|
  # Client pool size scales with your web server (Puma) threads
  config.redis = { 
    url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/0'), 
    size: 5 
  }
end

4. Real-World Example: Writing a Rails 8 Job

​Let's create a concrete example. We will generate a job that processes a user subscription report and sends an email.
​Instead of using old workers, we leverage modern Rails 8 standard inheritance. Run the generator command:

bin/rails generate job ProcessSubscriptionReport

This generates a file in app/jobs/process_subscription_report_job.rb. Let's implement it with proper error handling and a custom queue:

# app/jobs/process_subscription_report_job.rb
class ProcessSubscriptionReportJob < ApplicationJob
  # Classifying the priority queue in Sidekiq
  queue_as :default

  # Sidekiq-specific retry configuration
  sidekiq_options retry: 3, backtrace: true

  # Best Practice: Avoid passing complex ActiveRecord objects. Pass IDs instead.
  def perform(user_id)
    user = User.find_by(id: user_id)
    return unless user

    # Simulating heavy reporting logic
    report_data = UserReportGenerator.generate_for(user)

    # Triggering Mailer
    UserMailer.report_ready_email(user, report_data).deliver_now
  rescue ActiveRecord::RecordNotFound => e
    Rails.logger.error "Job failed: User with ID #{user_id} no longer exists. Error: #{e.message}"
  end
end

5. Sidekiq Concurrency & Concurrency Tuning
​Create a configuration file at config/sidekiq.yml to manage your concurrency limits and queues systematically for production:

# config/sidekiq.yml
:concurrency: 10
:queues:
  - critical
  - default
  - low

To run Sidekiq in your terminal, simply execute:

bundle exec sidekiq