Rails CSRF Internals Uncovered — And a Real-World Bug We Faced

In a previous post, I explored the Fundamentals of CSRF — what it is, how it works, and the standard mechanisms used to prevent it. While these mechanisms often “just work” in most frameworks, things can get tricky when you’re building or debugging something custom.

At one of my Org, we encountered a puzzling issue: “Invalid CSRF token” errors appearing on two different pages, each caused by a different underlying reason. To debug it, I had to dive deep into how Rails handles CSRF tokens internally.

This post shares what I learned from that journey — including how Rails generates, masks, and verifies CSRF tokens, how we implemented similar logic in our codebase, and what ultimately caused the issue.

💎 How Rails Handles CSRF (Under the Hood)

Rails’ CSRF protection works out of the box, But once you peek under the hood, there’s some really clever stuff going on — especially around how tokens are generated, masked, and verified.

At its core, Rails generates a base token , stores it in the session, and then sends a masked version of that token in the forms or headers. When a request comes in, Rails unmasks the token from the request and simply compares it with the one stored in the session. If they match, the request is good to go. A little more breakdown for each step.

1. Generating the CSRF Token

When a new session starts, Rails creates a random string using SecureRandom and stores it in the session under :_csrf_token. This is the base token and stays the same for the entire session (unless something like a login/logout resets it).

File — request_forgery_protection.rb

session[:_csrf_token] ||= SecureRandom.base64(32)

This base token is what everything else is built on.

2. Masking the Token (The Cool Part)

Rails doesn’t just throw that raw token into your forms or headers. Instead, it masks the token every time it sends it to the client.

Here’s what that means:

• It creates a random one-time pad (same length as the token).
• It XORs the base token with the pad to “encrypt” it.
• It then sticks the pad and the encrypted result together.
• Finally, the combined result is Base64-encoded.

So even if the base token never changes, the token sent to the frontend looks different every time. This helps protect against attacks like BREACH.

3. Where the Token Shows Up

The masked token is:

• Embedded in forms via a hidden input field (authenticity_token)
• Sent in the X-CSRF-Token header for AJAX or fetch requests

This way, Rails knows to expect it during any state-changing request like POST, PUT, PATCH, or DELETE.

4. Verifying the Token

When a request comes in, Rails does the reverse dance:

• It grabs the token from the header or form field.
• Decodes it from Base64.
• If it’s a masked token (i.e., twice the expected length), it splits out the pad and encrypted part and XORs them to get back the original token.
• Compares this unmasked token to what’s stored in the session (:_csrf_token).

If anything’s off — wrong value, malformed format, etc. — Rails throws an InvalidAuthenticityToken error.

To keep things extra safe, Rails regenerates the CSRF token after login or sign-up. So even though your session is still active, the CSRF token is refreshed. This prevents old tokens from being reused in new sessions.

This behaviour is defined in the Devise initialiser: config/initializers/devise.rb:102

Now let’s get started with the issues I ran into.

To make debugging easier, I first built a few common utilities for decoding and comparing CSRF tokens.

Common Utils

To parse Session token

def decrypt_session(cookie_string, mode = 'json')
  serializer = case mode
  when 'json' then JSON
  when 'marshal' then ActiveSupport::MessageEncryptor::NullSerializer
  end
  cookie = CGI::unescape(cookie_string.strip)
  salt = Rails.configuration.action_dispatch.encrypted_cookie_salt
  signed_salt = Rails.configuration.action_dispatch.encrypted_signed_cookie_salt
  key_generator = ActiveSupport::KeyGenerator.new(
    Rails.application.secrets.secret_key_base,
    iterations: 1000
  )
  secret = key_generator.generate_key(salt)[0, 32]
  sign_secret = key_generator.generate_key(signed_salt)
  encryptor = ActiveSupport::MessageEncryptor.new(
    secret,
    sign_secret,
    serializer: serializer
  )
  result = encryptor.decrypt_and_verify(cookie)
  (mode == 'marshal') ? Marshal.load(result) : result
end

To parse CSRF token

def unmask_token(masked_token)
  token_length = masked_token.length / 2
  one_time_pad = masked_token[0...token_length]
  encrypted_token = masked_token[token_length..-1]
  xor_byte_strings(one_time_pad, encrypted_token)
end

def encode_token(token)
  Base64.strict_encode64(token)
end

def decode_token(token)
  Base64.strict_decode64(token)
end

def xor_byte_strings(s1, s2)
  s1.bytes.zip(s2.bytes).map { |(c1, c2)| c1 ^ c2 }.pack('c*')
end

def due(token)
  d = decode_token(token)
  u = unmask_token(d)
  encode_token(u)
end

First Issue: Invalid CSRF Token on Landing Page

Issue

There was a recurring problem where users would sometimes face an Invalid CSRF token error on the new landing

What I Found

When the error happened, the two tokens didn’t match:

• The session contained one _csrf_token
• The CSRF API returned another

But in Rails, there are really only two valid scenarios:

1. No token exists yet → Rails generates one, saves it in the session, and returns it (after masking).
2. Token already exists → Rails reuses it, masking it again before sending.

So how could they be different?

It turned out to be a race condition :

• Two requests landed almost at the same time.
• Both checked the session, saw no CSRF token, and decided to generate one.
• One request won the race and saved its token in the session.
• But the other request had already generated its own token, so it returned a value that was immediately outdated.

This explains why the issue appeared when users hit the new landing page directly without any cookies (for example, in incognito mode).

The Fix

The problem was that we were eagerly calling the CSRF API as soon as the page loaded. That increased the chance of this race condition.

The solution was simple:

👉 Remove the initial CSRF API call and instead fetch the token only when it’s actually needed — at the time of the first POST request.

This way, only one request is responsible for generating and storing the token, eliminating the race.

Second Issue: Invalid CSRF Token on Product Page

Issue

We noticed users sometimes got “Invalid CSRF token” errors when opening the Product landing page from Slack links. The issue was inconsistent — only happening when the page was opened in Chrome’s Slack preview.

Here’s the request flow that caused the issue:

1. User clicks a Slack link to open the Product landing page.
2. The page makes an initial API call to load content.
3. A subsequent API call (e.g., user/v2 for OTP) fails with an invalid CSRF token error.

It seemed like the session was being dropped between requests , causing CSRF verification to fail. This issue was particularly tricky to debug.

Debugging Process

I approached this step by step, adding logs and tracing the request flow.

1. Inspecting CSRF Verification

• Added logging inside Rails’ request_forgery_protection.rb where the CSRF token is compared with the session.
• Finding: The session was empty at this point, even though the cookie itself was present.

This suggested the session was being created correctly initially, but something later in the middleware chain was removing it.

2. Checking Session Creation

• Added loggers in ActionDispatch::Request::Session and CookieStore.
• Found that the session was properly populated from the cookie, including the CSRF token.

So the drop wasn’t during session creation — it happened later in the middleware stack.

3. Tracing Middleware Calls

• Added logging in SanitizeQueryStringMiddleware.
• Observation: The session was available before the middleware but gone after.
• Testing middleware order confirmed that session availability depended on execution order.
• Rack-protection middleware, especially session_hijacking.rb, ran after SanitizeQueryStringMiddleware.

4. Identifying the Root Cause

• In rack-protection/lib/rack/protection/session_hijacking.rb, I found a check comparing the session’s tracking hash with request environment values.
• Specifically, the HTTP_ACCEPT_LANGUAGE header differed between requests:

Request HTTP_ACCEPT_LANGUAGEInitial Product APIen-USOTP API (user/v2)en-US,en;q=0.9,hi;q=0.8

• This difference triggered rack-protection’s session-hijacking check , which dropped the session.
• Dropping the session invalidated the CSRF token, causing the failure.

5. Hypothesis Testing

Replicated the flow using Postman:

1. First, call the Product page to get the CSRF token.
2. Then, call user/v2 with the same cookies and CSRF token.

Results:

1. Same Accept-Language header → CSRF passed
2. Different header → CSRF failed

This confirmed that the mismatch in headers caused the session drop and CSRF failure.

Fix

The fix was straightforward:

• Remove HTTP_ACCEPT_LANGUAGE tracking in session-hijacking (already removed in newer rack-protection versions — Commit Link).
• Upgrade or patch the rack-protection middleware.

Now, minor differences in request headers no longer drop the session, and the CSRF token remains valid across the flow — even in Slack preview.

CSRF protection usually works behind the scenes, and you rarely notice it — until it breaks. Going through Rails’ internals and debugging these real issues reminded me that even small things, like request timing or headers, can trip it up. Understanding how the tokens are generated, masked, and verified made it much easier to find the root cause and fix it properly.

—

Refs:

• CSRF Protection Code
• Session Hijacking Code
• Removal of ACCEPT_LANGUAGE Commit

202509220350