DEV Community: Abhinav Kushwaha

Designing a Bulletproof Webhook Ingestion System in Ruby on Rails

Abhinav Kushwaha — Mon, 08 Jun 2026 16:15:19 +0000

As your Rails application grows and begins integrating with external platforms—think Stripe, Shopify, or GitHub—handling incoming webhooks efficiently becomes critical.

It’s easy to spin up a quick controller action, parse some JSON, and update a database record. But what happens when an external service suddenly floods your server with thousands of concurrent requests? Or worse, what if your third-party provider experiences network instability and drops connection mid-flight?

If your webhook endpoint performs heavy database writes, executes API callbacks, or sends emails synchronously, you are asking for trouble. Today, we will build a resilient, decoupled, and production-ready webhook ingestion system using Rails 7/8, Solid Queue (or Sidekiq), and database-backed idling.

The Blueprint: Decouple Fast, Process Later

The absolute golden rule of webhooks is: Acknowledge receipt immediately, handle processing asynchronously. Your endpoint should do exactly three things:

Verify the request signature (security first!).
Persist the raw payload to an inbound webhooks table.
Enqueue a background job and instantly return a 200 OK.

By moving all business logic out of the request-response cycle, you keep your database locks brief, protect your web workers from timed-out connections, and ensure zero data loss.

Step 1: Data Architecture & Security

First, let's create a dedicated table to house our raw webhook data. This gives us an immutable audit log and allows for seamless job retries if background workers fail.

rails generate model InboundWebhook status:integer provider:string payload:jsonb error_message:text
rails db:migrate

We will use an enum to keep track of the webhook lifecycle:

# app/models/inbound_webhook.rb
class InboundWebhook < ApplicationRecord
  enum :status, { pending: 0, processing: 1, completed: 2, failed: 3 }, default: :pending

  validates :provider, :payload, presence: true
end

Step 2: The High-Speed Controller

Our controller needs to be incredibly lean. It should strictly handle verification and record creation, then pass the heavy lifting off to an ActiveJob.

# app/controllers/webhooks/stripe_controller.rb
module Webhooks
  class StripeController < ApplicationController
    skip_before_action :verify_authenticity_token # Webhooks don't have CSRF tokens
    before_action :verify_signature!

    def create
      webhook = InboundWebhook.create!(
        provider: 'stripe',
        payload: JSON.parse(request.body.read)
      )

      # Hand off to background workers immediately
      ProcessWebhookJob.perform_later(webhook)

      render json: { success: true }, status: :ok
    rescue JSON::ParserError => e
      render json: { error: "Malformed payload" }, status: :bad_request
    end

    private

    def verify_signature!
      endpoint_secret = Rails.application.credentials.dig(:stripe, :webhook_secret)
      signature = request.env['HTTP_STRIPE_SIGNATURE']
      payload = request.body.read

      # Use Stripe's official SDK tool to verify the event authenticity safely
      Stripe::Webhook.construct_event(payload, signature, endpoint_secret)
    rescue Stripe::SignatureVerificationError => e
      render json: { error: "Invalid signature" }, status: :unauthorized
    end
  end
end

Step 3: Resilient Background Processing

Now that the request has safely closed with a fast 200 OK, our background architecture takes over. If the underlying logic fails (due to a third-party API outage, a database deadlock, etc.), our system marks the webhook as failed and saves the stack trace instead of silently swallowing the error.

# app/jobs/process_webhook_job.rb
class ProcessWebhookJob < ApplicationJob
  queue_as :default

  # Automatically retry network or concurrency issues before marking as failed
  retry_on ActiveRecord::Deadlocked, wait: 2.seconds, attempts: 3

  def perform(webhook)
    return if webhook.completed?

    webhook.processing!

    # Routable processing based on payload types
    case webhook.provider
    when 'stripe'
      process_stripe_event(webhook.payload)
    else
      raise "Unknown webhook provider: #{webhook.provider}"
    end

    webhook.completed!
  rescue => e
    webhook.update!(status: :failed, error_message: "#{e.class}: #{e.message}")
    raise e # Re-raise to let your error tracker (Sentry/Honeybadger) catch it
  end

  private

  def process_stripe_event(payload)
    case payload['type']
    when 'charge.succeeded'
      # Implement your transaction tracking or ledger updates here
      # Invoice.payment_received!(payload['data']['object'])
    when 'customer.subscription.deleted'
      # Handle subscription cancellations gently
    end
  end
end

Step 4: The Superpower — Idempotency Guardrails

Webhooks are guaranteed to be delivered at least once. This means your application will eventually receive the exact same webhook payload twice. If you don't account for this, you risk double-charging clients or duplicating inventory data.

To make our processing layer strictly idempotent, we can utilize database uniqueness constraints or Redis locks based on the provider's unique event ID.

# Prevent duplicate processing using Stripe's unique event ID
def process_stripe_event(payload)
  event_id = payload['id']

  # Use an atomic lock or transaction mapping to prevent race conditions
  return if InboundWebhook.where(status: :completed)
                          .where("payload->>'id' = ?", event_id).exists?

  # Proceed with processing safely...
end

Wrapping Up
By decoupling webhook storage from execution, your Rails app can handle sudden traffic spikes without flinching.

The Win: Web servers spend a fraction of a millisecond handling external requests.

The Peace of Mind: If your worker crashes, you still have the full history of payloads sitting securely in your database ready to be rerun with a quick webhook.reload.process_later Rake task.

How are you currently scaling your webhook consumers? Are you using Redis-backed Sidekiq or looking into Rails 8's native Solid Queue? Let's discuss in the comments below!

Mastering Background Processing in Rails 8: Sidekiq & Redis Optimization

Abhinav Kushwaha — Sun, 31 May 2026 05:20:16 +0000

In modern web applications, keeping the Request-Response cycle fast is crucial for user experience. If a user registers on your Rails 8 app and you trigger a welcome email directly inside the controller, the browser will remain in a loading state until the SMTP server responds.

To solve this, we offload heavy, non-blocking tasks (like sending emails, generating PDFs, or syncing third-party APIs) to background workers. While Rails 8 now introduces Solid Queue as its default database-backed adapter, Sidekiq remains the industry gold standard for high-throughput, multi-threaded asynchronous processing powered by Redis (an in-memory data store).

1. System Requirements & Installation

First, ensure that Redis is installed and running on your production or local environment.

For Ubuntu/Linux:

sudo apt update
sudo apt install redis-server
sudo systemctl enable redis-server.service

Next, add the Sidekiq gem to your Rails 8 Gemfile:

gem 'sidekiq'

Execute the bundle command in your terminal:

bundle install

2. Configuring Rails 8 to use Sidekiq

Even with Rails 8's new defaults, switching to Sidekiq is seamless. You need to instruct your Active Job framework to use the Sidekiq adapter. Update your config/application.rb or config/environments/production.rb:

module BlogApp
  class Application < Rails::Application
    # Initialize configuration defaults for originally generated Rails version.
    config.load_defaults 8.0

    # Setting Sidekiq as the backend queue adapter
    config.active_job.queue_adapter = :sidekiq
  end
end

3. Production-Ready Redis Connection Pooling

A common issue in production is running out of Redis connections due to improper pool sizes. To handle this cleanly, create a dedicated initializer file at config/initializers/sidekiq.rb:

# config/initializers/sidekiq.rb

Sidekiq.configure_server do |config|
  # Server pool size should match or slightly exceed your concurrency limit
  config.redis = { 
    url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/0'), 
    size: 25 
  }
end

Sidekiq.configure_client do |config|
  # Client pool size scales with your web server (Puma) threads
  config.redis = { 
    url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/0'), 
    size: 5 
  }
end

4. Real-World Example: Writing a Rails 8 Job

Let's create a concrete example. We will generate a job that processes a user subscription report and sends an email.
Instead of using old workers, we leverage modern Rails 8 standard inheritance. Run the generator command:

bin/rails generate job ProcessSubscriptionReport

This generates a file in app/jobs/process_subscription_report_job.rb. Let's implement it with proper error handling and a custom queue:

# app/jobs/process_subscription_report_job.rb
class ProcessSubscriptionReportJob < ApplicationJob
  # Classifying the priority queue in Sidekiq
  queue_as :default

  # Sidekiq-specific retry configuration
  sidekiq_options retry: 3, backtrace: true

  # Best Practice: Avoid passing complex ActiveRecord objects. Pass IDs instead.
  def perform(user_id)
    user = User.find_by(id: user_id)
    return unless user

    # Simulating heavy reporting logic
    report_data = UserReportGenerator.generate_for(user)

    # Triggering Mailer
    UserMailer.report_ready_email(user, report_data).deliver_now
  rescue ActiveRecord::RecordNotFound => e
    Rails.logger.error "Job failed: User with ID #{user_id} no longer exists. Error: #{e.message}"
  end
end

5. Sidekiq Concurrency & Concurrency Tuning
Create a configuration file at config/sidekiq.yml to manage your concurrency limits and queues systematically for production:

# config/sidekiq.yml
:concurrency: 10
:queues:
  - critical
  - default
  - low

To run Sidekiq in your terminal, simply execute:

bundle exec sidekiq