How a State Machine (AASM) Ended Our Recurring Payment Failures in Rails

Payments have a lifecycle, and a lifecycle with scattered update(status: ...) calls will eventually corrupt itself. I used AASM to turn our payment status handling into an explicit state machine, then added row locking, idempotency, and a transition log to make it hold up under retries and duplicate events. Here is the problem, the approach, and the code.

The problem: failures that kept coming back

I worked on a Rails platform that handled recurring payments for renewals across multiple regions. Customers paid through bank (ACH) transfers and cards, and every payment had to stay consistent with a payment provider and an external CRM.
The symptoms were familiar to anyone who has maintained a payment flow:

• Payments stuck in states that made no sense
• The same payment processed twice after a retry
• Payment summaries that didn't match what had actually happened
• Bugs that we patched, only to see a close cousin of them appear a few weeks later Each fix addressed one symptom. None of them addressed the cause.

Why it kept breaking
When I traced the code, the pattern was clear. Payment status was just a string column, and anything could change it:

# in a controller
payment.update(status: "paid")

# in a background job
payment.update(status: "failed", retry_count: payment.retry_count + 1)

# in a webhook handler
payment.update(status: event.data.status)

Three problems followed from this:

1. Impossible transitions were possible. Nothing stopped a payment from going from failed straight to paid without a retry, or from refunded back to processing.
2. No single source of truth. The rules for what could happen to a payment were spread across controllers, jobs, callbacks, and webhook handlers. Each place made its own assumptions.
3. Retries and webhooks are inherently messy. Providers send events more than once and sometimes out of order. Without rules, whichever write arrived last won. The bug was not in any single line. The design allowed invalid states, so sooner or later one of them happened.

The idea: model the lifecycle explicitly

A state machine is a simple concept:
• A fixed set of states (pending, processing, succeeded, failed, refunded)
• A fixed set of events that move a record between states (start_processing, settle, mark_failed, refund)
• A list of allowed transitions. Anything not on the list is rejected
The lifecycle looks like this:

pending ----> processing ----> succeeded ----> refunded
                  |                ^
                  v                |
               failed -------------+
         (retry goes back to processing)

Once the rules are written down in one place, an invalid transition is no longer a silent data bug. It is an error that you can see, log, and handle.

Implementing it with AASM

AASM is a well-established Ruby gem for exactly this. Here is a simplified, generic version of the model:

class Payment < ApplicationRecord
  include AASM

  aasm column: :status, whiny_transitions: true, whiny_persistence: true do
    state :pending, initial: true
    state :processing
    state :succeeded
    state :failed
    state :refunded

    event :start_processing do
      transitions from: %i[pending failed], to: :processing
    end

    event :settle do
      transitions from: :processing, to: :succeeded, after: :record_settled_at
    end

    event :mark_failed do
      transitions from: :processing, to: :failed, after: :increment_attempts
    end

    event :refund do
      transitions from: :succeeded, to: :refunded, guard: :refundable?
    end
  end

  private

  def record_settled_at
    self.settled_at = Time.current
  end

  def increment_attempts
    self.attempts += 1
  end

  def refundable?
    settled_at.present? && settled_at > 90.days.ago
  end
end

A few things are worth calling out:
• Only declared transitions are allowed. Calling payment.settle! on a pending payment raises AASM::InvalidTransition instead of quietly writing bad data.
• Guards hold business rules. The refund event only fires if refundable? returns true, so the rule lives next to the transition it protects.
• may_settle? lets you check first. This is useful in jobs and webhook handlers, where you want to skip gracefully instead of raising.
• Name events carefully. I used mark_failed instead of fail, because fail collides with Ruby's built-in Kernel#fail.
• Bang methods persist. settle! saves the record and raises if the save fails. Plain settle only changes the in-memory state, which is a common source of confusion.
With that in place, the scattered updates become intentional calls:

payment.start_processing!   # instead of update(status: "processing")
payment.settle!             # instead of update(status: "paid")

A state machine alone is not enough

This is the part that most tutorials skip. AASM prevents invalid transitions, but it does not protect you from two valid transitions happening at the same time, or from the same event arriving twice. For payments, that gap matters, so I added three more layers.

1. Lock the row while transitioning Two workers can load the same payment, both see processing, and both try to settle it. Row-level locking serializes them:

payment.with_lock do
  payment.settle! if payment.may_settle?
end

with_lock reloads the record inside a transaction and holds a database lock, so the second worker waits, then sees succeeded, and may_settle? returns false.

1. Make event handling idempotent Providers deliver webhooks at least once, so duplicates are normal. Store each event's ID with a unique database constraint, and treat repeats as a no-op:

class ProcessPaymentEvent
  def call(event)
    ProcessedEvent.create!(provider_event_id: event.id)
  rescue ActiveRecord::RecordNotUnique
    return # already handled, do nothing
  else
    payment = Payment.find(event.payment_id)
    payment.with_lock do
      payment.settle! if payment.may_settle?
    end
  end
end

The unique index does the real work here. Application-level checks like exists? followed by create have a race window, but the database constraint does not.

1. Keep side effects out of the transition Sending an email or syncing to a CRM inside the transition means a slow or failing external call can break your payment state. Instead, let the transition commit first, then enqueue the side effect:

class Payment < ApplicationRecord
  after_commit :enqueue_crm_sync, if: :saved_change_to_status?

  private

  def enqueue_crm_sync
    CrmSyncJob.perform_later(id)
  end
end

If the CRM is down, the job retries on its own schedule. The payment's state is already correct and unaffected.

Keep a transition history

When something goes wrong in production, the first question is always "what happened to this payment, and in what order?" A small history table answers it:

aasm column: :status do
  after_all_transitions :log_transition
end

def log_transition
  PaymentTransition.create!(
    payment: self,
    from_state: aasm.from_state,
    to_state: aasm.to_state,
    event: aasm.current_event
  )
end

This gives you an audit trail for support, finance, and debugging, and it makes discrepancies far easier to trace.

Testing the state machine

State machines are easy to test, and the tests double as documentation of the rules:

RSpec.describe Payment do
  it "settles a processing payment" do
    payment = create(:payment, status: "processing")
    payment.settle!
    expect(payment).to be_succeeded
  end

  it "rejects settling a pending payment" do
    payment = create(:payment, status: "pending")
    expect { payment.settle! }.to raise_error(AASM::InvalidTransition)
  end

  it "ignores a duplicate settle event" do
    payment = create(:payment, status: "processing")
    2.times { ProcessPaymentEvent.new.call(event_for(payment)) }
    expect(payment.reload).to be_succeeded
    expect(PaymentTransition.where(payment: payment).count).to eq(1)
  end
end

I recommend testing the invalid transitions as carefully as the valid ones. Those tests are what keep the rules from eroding later.
Migrating existing data
If you adopt this on a system that already has data, expect some cleanup:
• Audit the distinct values currently in the status column. There are usually more than anyone remembers.
• Map each legacy value to one of your new states, and decide what to do with records in ambiguous states.
• Run the backfill in batches and verify counts before and after.
• Deploy the state machine in a way that old and new code paths don't fight over the same column.
This is the least glamorous part of the work, and probably the most important.

Results and lessons

After the move to an explicit state machine, the nature of our payment bugs changed. The recurring failures and mismatched summaries stopped coming back, because the invalid states that produced them could no longer be written. The bugs that remained were easier to diagnose, since the transition history showed exactly what happened and in what order. Code review also got easier, because any change to payment behavior now had to go through a named event.
The lessons I took away:

1. If a record has a lifecycle, model it as a state machine. A string column and a few update calls are not a lifecycle.
2. Make invalid states unrepresentable, or at least loud. Silent corruption is the expensive kind.
3. Combine the state machine with locking and idempotency. AASM handles the rules, and the database handles concurrency.
4. Keep side effects out of transitions. Commit state first, then enqueue the follow-up work.
5. Log every transition. You will need that history sooner than you think. There are trade-offs. You need more upfront design, a migration for existing data, and discipline to avoid stuffing heavy business logic into callbacks. For anything involving money, I think they are well worth it.

Further reading
• The AASM README on GitHub, which documents guards, callbacks, and persistence options
• The Rails guides on ActiveRecord::Locking for optimistic and pessimistic locking
If you've dealt with payment status bugs in your own Rails apps, I'd like to hear how you handled them. Find me on LinkedIn.